2024-04-24T22:56:34.165 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T22:56:34.165 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179 --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --owner scheduled_teuthology@teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179/orig.config.yaml 2024-04-24T22:56:34.221 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T22:56:34.313 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179 branch: squid description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672179' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKEPt1CQtIMn+3Jrv0a3xOXikA/LbAg690wZGs99omoM+WdlNGKaFYGVS9y5IREivEiMAROXd7G2myZhSpDv16w= smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIMoj4yS0kEiJ2yYtBo/bpsisRHZUUsb3JQ9sjiZJvUWByBGJkDucqTyWlriyG+XUFJhAd2lx1OF/XYTKvf3UIw= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-24T22:56:34.314 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T22:56:34.314 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T22:56:34.314 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T22:56:34.316 INFO:teuthology.task.internal:Checking packages... 2024-04-24T22:56:34.340 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T22:56:34.340 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T22:56:34.340 INFO:teuthology.packaging:ref: None 2024-04-24T22:56:34.340 INFO:teuthology.packaging:tag: None 2024-04-24T22:56:34.340 INFO:teuthology.packaging:branch: squid 2024-04-24T22:56:34.340 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T22:56:34.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T22:56:34.628 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T22:56:34.630 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T22:56:34.637 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T22:56:34.637 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T22:56:34.729 INFO:teuthology.task.internal:Saving configuration 2024-04-24T22:56:34.752 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T22:56:34.758 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T22:56:34.783 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:51:43.944883', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKEPt1CQtIMn+3Jrv0a3xOXikA/LbAg690wZGs99omoM+WdlNGKaFYGVS9y5IREivEiMAROXd7G2myZhSpDv16w='} 2024-04-24T22:56:34.808 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:51:43.946852', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIMoj4yS0kEiJ2yYtBo/bpsisRHZUUsb3JQ9sjiZJvUWByBGJkDucqTyWlriyG+XUFJhAd2lx1OF/XYTKvf3UIw='} 2024-04-24T22:56:34.808 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T22:56:34.814 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-24T22:56:34.814 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-24T22:56:34.814 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T22:56:34.887 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f51d8d9d3a0>, signals=[15]) 2024-04-24T22:56:34.888 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T22:56:34.894 INFO:teuthology.task.internal:Opening connections... 2024-04-24T22:56:34.894 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T22:56:34.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:56:34.979 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2024-04-24T22:56:34.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T22:56:35.054 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T22:56:35.061 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-04-24T22:56:35.079 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-04-24T22:56:35.080 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-04-24T22:56:35.138 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-04-24T22:56:35.138 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-04-24T22:56:35.138 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-04-24T22:56:35.138 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:56:35.138 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:56:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:56:35.140 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-04-24T22:56:35.167 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2024-04-24T22:56:35.185 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2024-04-24T22:56:35.186 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2024-04-24T22:56:35.243 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="CentOS Stream" 2024-04-24T22:56:35.243 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="9" 2024-04-24T22:56:35.243 INFO:teuthology.orchestra.run.smithi040.stdout:ID="centos" 2024-04-24T22:56:35.243 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE="rhel fedora" 2024-04-24T22:56:35.243 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="9" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:PLATFORM_ID="platform:el9" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:ANSI_COLOR="0;31" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:LOGO="fedora-logo-icon" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://centos.org/" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T22:56:35.244 INFO:teuthology.orchestra.run.smithi040.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T22:56:35.244 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2024-04-24T22:56:35.266 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T22:56:35.276 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T22:56:35.281 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T22:56:35.281 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:56:35.284 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2024-04-24T22:56:35.301 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T22:56:35.307 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T22:56:35.307 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:56:35.342 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2024-04-24T22:56:35.369 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T22:56:35.418 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T22:56:35.433 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-24T22:56:35.433 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-24T22:56:35.433 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:56:35.434 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T22:56:35.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:56:35.451 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-04-24T22:56:35.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:56:35.454 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2024-04-24T22:56:35.509 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:56:35.509 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-437.el9.x86_64 2024-04-24T22:56:35.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-04-24T22:56:35.512 INFO:teuthology.orchestra.run.smithi040.stdout:5.14.0-437.el9.x86_64 2024-04-24T22:56:35.512 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.14.0-437.el9.x86_64 2024-04-24T22:56:35.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum install -y kernel 2024-04-24T22:56:37.792 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T22:56:39.061 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-24T22:56:39.684 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-24T22:56:40.804 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-04-24T22:56:45.197 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-04-24T22:56:47.564 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-24T22:56:47.826 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.2 MB 00:04 2024-04-24T22:56:49.535 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-24T22:56:50.690 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 21 MB 00:02 2024-04-24T22:56:50.720 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-24T22:56:55.940 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-24T22:56:56.186 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T22:56:57.859 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:56:57.859 INFO:teuthology.orchestra.run.smithi040.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:56:57.954 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T22:56:57.955 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T22:56:57.955 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T22:56:58.070 DEBUG:teuthology.orchestra.run.smithi040:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:56:58.142 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T22:56:58.142 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T22:56:58.222 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T22:56:58.223 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T22:56:58.223 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T22:56:58.325 DEBUG:teuthology.orchestra.run.smithi031:> echo no | sudo yum reinstall kernel || true 2024-04-24T22:56:58.629 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:56:55 PM UTC. 2024-04-24T22:56:58.758 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T22:56:58.766 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T22:56:58.766 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-24T22:56:58.767 INFO:teuthology.orchestra.run.smithi040.stderr:Operation aborted. 2024-04-24T22:56:58.828 INFO:teuthology.orchestra.run.smithi040.stdout:Is this ok [y/N]: 2024-04-24T22:56:58.829 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum reinstall -y kernel || true 2024-04-24T22:56:58.844 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 10:56:56 PM UTC. 2024-04-24T22:56:58.968 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T22:56:58.987 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T22:56:58.987 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-24T22:56:58.987 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T22:56:58.987 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalling: 2024-04-24T22:56:58.987 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:56:58.987 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T22:56:58.988 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T22:56:58.988 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T22:56:58.988 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T22:56:58.988 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 255 k 2024-04-24T22:56:58.988 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 0 2024-04-24T22:56:58.988 INFO:teuthology.orchestra.run.smithi031.stderr:Operation aborted. 2024-04-24T22:56:59.042 INFO:teuthology.orchestra.run.smithi031.stdout:Is this ok [y/N]: 2024-04-24T22:56:59.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum reinstall -y kernel || true 2024-04-24T22:56:59.341 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:04 ago on Wed 24 Apr 2024 10:56:55 PM UTC. 2024-04-24T22:56:59.462 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T22:56:59.462 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T22:56:59.462 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalling: 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 255 k 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 0 2024-04-24T22:56:59.463 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T22:56:59.557 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 10:56:56 PM UTC. 2024-04-24T22:56:59.677 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T22:56:59.678 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T22:56:59.678 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-24T22:56:59.678 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T22:56:59.678 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalling: 2024-04-24T22:56:59.678 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T22:56:59.678 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T22:56:59.679 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T22:56:59.679 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T22:56:59.679 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T22:56:59.679 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 255 k 2024-04-24T22:56:59.679 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 0 2024-04-24T22:56:59.679 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T22:56:59.876 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-24T22:56:59.876 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:56:59.876 INFO:teuthology.orchestra.run.smithi040.stdout:Total 617 kB/s | 255 kB 00:00 2024-04-24T22:56:59.893 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T22:57:00.078 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T22:57:00.078 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T22:57:00.083 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T22:57:00.084 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T22:57:00.217 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T22:57:00.229 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-437.el9.x86_64.rpm 635 kB/s | 255 kB 00:00 2024-04-24T22:57:00.230 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T22:57:00.230 INFO:teuthology.orchestra.run.smithi031.stdout:Total 463 kB/s | 255 kB 00:00 2024-04-24T22:57:00.250 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T22:57:00.310 INFO:teuthology.orchestra.run.smithi040.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:57:00.310 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:00.452 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T22:57:00.452 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T22:57:00.457 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T22:57:00.457 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T22:57:00.588 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T22:57:00.672 INFO:teuthology.orchestra.run.smithi031.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:57:00.672 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:01.023 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:01.024 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:57:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T22:57:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T22:57:01.750 INFO:teuthology.orchestra.run.smithi040.stdout:Reinstalled: 2024-04-24T22:57:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:57:01.750 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T22:57:01.750 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T22:57:01.931 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:57:01.979 INFO:teuthology.orchestra.run.smithi040.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:57:01.979 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:57:01.979 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:57:01.979 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-24T22:57:01.979 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T22:57:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T22:57:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T22:57:01.997 INFO:teuthology.orchestra.run.smithi031.stdout:Reinstalled: 2024-04-24T22:57:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T22:57:01.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T22:57:01.997 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T22:57:02.012 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2024-04-24T22:57:02.031 DEBUG:teuthology.parallel:result is None 2024-04-24T22:57:02.186 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T22:57:02.234 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T22:57:02.235 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T22:57:02.235 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T22:57:02.235 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-24T22:57:02.235 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T22:57:02.313 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-04-24T22:57:02.332 DEBUG:teuthology.parallel:result is None 2024-04-24T22:57:02.333 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T22:57:02.349 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T22:57:02.349 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:57:02.380 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T22:57:02.396 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T22:57:02.410 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T22:57:02.417 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T22:57:02.417 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:57:02.437 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T22:57:02.470 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T22:57:02.561 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T22:57:02.561 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:57:02.565 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T22:57:02.593 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:02.602 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:02.613 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:02.623 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T22:57:02.624 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T22:57:02.747 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T22:57:02.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:57:02.750 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T22:57:02.775 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T22:57:02.922 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T22:57:02.922 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:57:02.925 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T22:57:02.940 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:03.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:03.065 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:03.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:03.186 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T22:57:03.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:57:03.251 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:03.295 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T22:57:03.346 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:03.430 DEBUG:teuthology.orchestra.run.smithi040:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T22:57:03.472 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T22:57:03.473 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T22:57:03.539 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-04-24T22:57:03.541 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2024-04-24T22:57:03.573 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:57:03.618 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T22:57:04.041 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T22:57:04.049 INFO:teuthology.task.internal:Starting timer... 2024-04-24T22:57:04.049 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T22:57:04.060 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T22:57:04.067 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T22:57:04.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-04-24T22:57:04.131 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-04-24T22:57:04.133 DEBUG:teuthology.orchestra.run.smithi040:> sudo service auditd rotate 2024-04-24T22:57:04.187 INFO:teuthology.orchestra.run.smithi040.stdout:Rotating logs: 2024-04-24T22:57:04.188 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T22:57:04.189 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-04-24T22:57:04.227 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-04-24T22:57:04.227 DEBUG:teuthology.orchestra.run.smithi040:> /usr/sbin/getenforce 2024-04-24T22:57:04.253 INFO:teuthology.orchestra.run.smithi040.stdout:Permissive 2024-04-24T22:57:04.253 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi040.front.sepia.ceph.com': 'permissive'} 2024-04-24T22:57:04.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:57:04.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:04.292 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T22:57:04.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T22:57:04.318 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T22:57:04.318 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:57:04.359 DEBUG:teuthology.orchestra.run.smithi040:> sudo /usr/sbin/setenforce permissive 2024-04-24T22:57:04.384 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T22:57:04.394 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T22:57:04.413 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T22:57:04.414 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi040.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T23:00:41.170 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi040.front.sepia.ceph.com')] 2024-04-24T23:00:41.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-24T23:00:41.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:00:41.250 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-04-24T23:00:41.293 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-04-24T23:00:41.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-24T23:00:41.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:00:41.374 DEBUG:teuthology.orchestra.run.smithi040:> true 2024-04-24T23:00:41.460 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2024-04-24T23:00:41.460 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T23:00:41.476 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T23:00:41.477 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:00:41.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:00:41.480 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T23:00:41.481 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:00:41.508 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:00:41.523 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:00:41.533 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T23:00:41.548 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-04-24T23:00:41.549 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T23:00:41.560 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-04-24T23:00:41.574 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:00:41.576 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: ntpd: command not found 2024-04-24T23:00:41.588 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:00:41.588 INFO:teuthology.orchestra.run.smithi040.stdout:506 Cannot talk to daemon 2024-04-24T23:00:41.602 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T23:00:41.617 INFO:teuthology.orchestra.run.smithi040.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T23:00:41.649 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:00:41.653 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:00:41.653 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-24T23:00:41.653 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:00:41.653 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:00:41.653 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:00:41.653 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:00:41.673 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:00:41.676 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:00:41.676 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-24T23:00:41.676 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:00:41.676 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:00:41.676 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:00:41.676 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:00:41.677 INFO:teuthology.run_tasks:Running task install... 2024-04-24T23:00:41.685 DEBUG:teuthology.task.install:project ceph 2024-04-24T23:00:41.685 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-24T23:00:41.685 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T23:00:41.685 INFO:teuthology.task.install:Using flavor: default 2024-04-24T23:00:41.691 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T23:00:41.691 INFO:teuthology.task.install:extra packages: [] 2024-04-24T23:00:41.692 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:00:41.692 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:00:41.692 INFO:teuthology.packaging:ref: None 2024-04-24T23:00:41.692 INFO:teuthology.packaging:tag: None 2024-04-24T23:00:41.692 INFO:teuthology.packaging:branch: quincy 2024-04-24T23:00:41.692 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:00:41.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-24T23:00:41.694 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-24T23:00:41.695 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:00:41.695 INFO:teuthology.packaging:ref: None 2024-04-24T23:00:41.695 INFO:teuthology.packaging:tag: None 2024-04-24T23:00:41.695 INFO:teuthology.packaging:branch: quincy 2024-04-24T23:00:41.695 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:00:41.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-24T23:00:41.933 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/ 2024-04-24T23:00:41.934 INFO:teuthology.task.install.rpm:Package version is 17.2.7-902.gc61472c8 2024-04-24T23:00:41.940 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/ 2024-04-24T23:00:41.941 INFO:teuthology.task.install.rpm:Package version is 17.2.7-902.gc61472c8 2024-04-24T23:00:42.046 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:00:42.046 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:00:42.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:00:42.073 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T23:00:42.073 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:00:42.073 INFO:teuthology.packaging:ref: None 2024-04-24T23:00:42.073 INFO:teuthology.packaging:tag: None 2024-04-24T23:00:42.073 INFO:teuthology.packaging:branch: quincy 2024-04-24T23:00:42.073 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:00:42.073 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:00:42.110 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T23:00:42.110 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:00:42.110 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T23:00:42.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:00:42.141 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T23:00:42.141 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:00:42.141 INFO:teuthology.packaging:ref: None 2024-04-24T23:00:42.141 INFO:teuthology.packaging:tag: None 2024-04-24T23:00:42.141 INFO:teuthology.packaging:branch: quincy 2024-04-24T23:00:42.141 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:00:42.142 DEBUG:teuthology.orchestra.run.smithi040:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T23:00:42.210 DEBUG:teuthology.orchestra.run.smithi040:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T23:00:42.218 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:00:42.282 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-04-24T23:00:42.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-04-24T23:00:42.289 DEBUG:teuthology.orchestra.run.smithi040:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T23:00:42.356 INFO:teuthology.orchestra.run.smithi040.stdout:check_obsoletes = 1 2024-04-24T23:00:42.357 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum clean all 2024-04-24T23:00:42.518 INFO:teuthology.orchestra.run.smithi031.stdout:48 files removed 2024-04-24T23:00:42.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-04-24T23:00:42.599 INFO:teuthology.orchestra.run.smithi040.stdout:48 files removed 2024-04-24T23:00:42.626 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-radosgw 2024-04-24T23:00:43.154 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 229 kB/s | 77 kB 00:00 2024-04-24T23:00:43.320 INFO:teuthology.orchestra.run.smithi040.stdout:ceph packages for x86_64 183 kB/s | 77 kB 00:00 2024-04-24T23:00:43.579 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 28 kB/s | 11 kB 00:00 2024-04-24T23:00:43.729 INFO:teuthology.orchestra.run.smithi040.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-24T23:00:43.963 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 5.1 kB/s | 1.8 kB 00:00 2024-04-24T23:00:44.080 INFO:teuthology.orchestra.run.smithi040.stdout:ceph source packages 5.7 kB/s | 1.8 kB 00:00 2024-04-24T23:00:44.655 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T23:00:44.797 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-24T23:00:47.190 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-24T23:00:47.456 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-24T23:00:51.882 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.2 MB 00:00 2024-04-24T23:00:52.633 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.2 MB 00:01 2024-04-24T23:00:53.332 INFO:teuthology.orchestra.run.smithi040.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 16 kB 00:00 2024-04-24T23:00:54.662 INFO:teuthology.orchestra.run.smithi040.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-24T23:00:54.982 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-24T23:00:56.416 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-24T23:00:59.843 INFO:teuthology.orchestra.run.smithi040.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T23:01:01.733 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-24T23:01:02.046 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:01:02.048 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:02.048 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:01:02.048 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:02.048 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:01:02.048 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 14 M 2024-04-24T23:01:02.048 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrading: 2024-04-24T23:01:02.048 INFO:teuthology.orchestra.run.smithi040.stdout: librados2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.1 M 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base x86_64 2:17.2.7-902.gc61472c8.el9 ceph 5.0 M 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 22 M 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 637 k 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 457 k 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:01:02.049 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 4.9 M 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 45 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 117 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs x86_64 2:17.2.7-902.gc61472c8.el9 ceph 156 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados x86_64 2:17.2.7-902.gc61472c8.el9 ceph 320 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 293 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 99 k 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:02.050 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:01:02.051 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:02.051 INFO:teuthology.orchestra.run.smithi040.stdout:Install 22 Packages 2024-04-24T23:01:02.051 INFO:teuthology.orchestra.run.smithi040.stdout:Upgrade 2 Packages 2024-04-24T23:01:02.051 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:02.051 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 58 M 2024-04-24T23:01:02.051 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:01:03.929 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:01:03.931 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:01:03.931 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:03.931 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:01:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 14 M 2024-04-24T23:01:03.931 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-04-24T23:01:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.1 M 2024-04-24T23:01:03.931 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:17.2.7-902.gc61472c8.el9 ceph 5.0 M 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 22 M 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 637 k 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 457 k 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 4.9 M 2024-04-24T23:01:03.932 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-24T23:01:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-24T23:01:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 45 k 2024-04-24T23:01:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 117 k 2024-04-24T23:01:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:17.2.7-902.gc61472c8.el9 ceph 156 k 2024-04-24T23:01:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-24T23:01:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:17.2.7-902.gc61472c8.el9 ceph 320 k 2024-04-24T23:01:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 293 k 2024-04-24T23:01:03.933 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 99 k 2024-04-24T23:01:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-24T23:01:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:03.934 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:01:03.934 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:03.934 INFO:teuthology.orchestra.run.smithi031.stdout:Install 22 Packages 2024-04-24T23:01:03.934 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-04-24T23:01:03.934 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:03.935 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 58 M 2024-04-24T23:01:03.935 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:01:04.477 INFO:teuthology.orchestra.run.smithi040.stdout:(1/24): ceph-radosgw-17.2.7-902.gc61472c8.el9.x 9.0 MB/s | 14 MB 00:01 2024-04-24T23:01:04.530 INFO:teuthology.orchestra.run.smithi040.stdout:(2/24): ceph-selinux-17.2.7-902.gc61472c8.el9.x 507 kB/s | 25 kB 00:00 2024-04-24T23:01:04.661 INFO:teuthology.orchestra.run.smithi040.stdout:(3/24): libcephfs2-17.2.7-902.gc61472c8.el9.x86 4.7 MB/s | 637 kB 00:00 2024-04-24T23:01:04.745 INFO:teuthology.orchestra.run.smithi040.stdout:(4/24): libradosstriper1-17.2.7-902.gc61472c8.e 5.4 MB/s | 457 kB 00:00 2024-04-24T23:01:05.279 INFO:teuthology.orchestra.run.smithi040.stdout:(5/24): librgw2-17.2.7-902.gc61472c8.el9.x86_64 9.2 MB/s | 4.9 MB 00:00 2024-04-24T23:01:05.495 INFO:teuthology.orchestra.run.smithi040.stdout:(6/24): python3-ceph-argparse-17.2.7-902.gc6147 208 kB/s | 45 kB 00:00 2024-04-24T23:01:05.754 INFO:teuthology.orchestra.run.smithi040.stdout:(7/24): ceph-common-17.2.7-902.gc61472c8.el9.x8 7.9 MB/s | 22 MB 00:02 2024-04-24T23:01:05.779 INFO:teuthology.orchestra.run.smithi040.stdout:(8/24): python3-ceph-common-17.2.7-902.gc61472c 415 kB/s | 117 kB 00:00 2024-04-24T23:01:05.813 INFO:teuthology.orchestra.run.smithi040.stdout:(9/24): python3-cephfs-17.2.7-902.gc61472c8.el9 2.6 MB/s | 156 kB 00:00 2024-04-24T23:01:05.871 INFO:teuthology.orchestra.run.smithi040.stdout:(10/24): python3-rados-17.2.7-902.gc61472c8.el9 3.4 MB/s | 320 kB 00:00 2024-04-24T23:01:05.896 INFO:teuthology.orchestra.run.smithi040.stdout:(11/24): python3-rbd-17.2.7-902.gc61472c8.el9.x 3.4 MB/s | 293 kB 00:00 2024-04-24T23:01:05.922 INFO:teuthology.orchestra.run.smithi040.stdout:(12/24): python3-rgw-17.2.7-902.gc61472c8.el9.x 1.9 MB/s | 99 kB 00:00 2024-04-24T23:01:06.080 INFO:teuthology.orchestra.run.smithi040.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 286 kB/s | 45 kB 00:00 2024-04-24T23:01:06.114 INFO:teuthology.orchestra.run.smithi040.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 738 kB/s | 160 kB 00:00 2024-04-24T23:01:06.214 INFO:teuthology.orchestra.run.smithi040.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-04-24T23:01:06.273 INFO:teuthology.orchestra.run.smithi040.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-04-24T23:01:06.298 INFO:teuthology.orchestra.run.smithi040.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 506 kB/s | 42 kB 00:00 2024-04-24T23:01:06.548 INFO:teuthology.orchestra.run.smithi040.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 194 kB/s | 49 kB 00:00 2024-04-24T23:01:06.857 INFO:teuthology.orchestra.run.smithi040.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 218 kB/s | 67 kB 00:00 2024-04-24T23:01:06.882 INFO:teuthology.orchestra.run.smithi040.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 507 kB/s | 309 kB 00:00 2024-04-24T23:01:07.075 INFO:teuthology.orchestra.run.smithi040.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-24T23:01:07.209 INFO:teuthology.orchestra.run.smithi040.stdout:(22/24): librados2-17.2.7-902.gc61472c8.el9.x86 9.4 MB/s | 3.1 MB 00:00 2024-04-24T23:01:07.409 INFO:teuthology.orchestra.run.smithi040.stdout:(23/24): ceph-base-17.2.7-902.gc61472c8.el9.x86 1.1 MB/s | 5.0 MB 00:04 2024-04-24T23:01:07.476 INFO:teuthology.orchestra.run.smithi040.stdout:(24/24): librbd1-17.2.7-902.gc61472c8.el9.x86_6 7.3 MB/s | 2.9 MB 00:00 2024-04-24T23:01:07.479 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:01:07.479 INFO:teuthology.orchestra.run.smithi040.stdout:Total 11 MB/s | 58 MB 00:05 2024-04-24T23:01:07.543 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:01:07.581 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:01:07.581 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:01:07.867 INFO:teuthology.orchestra.run.smithi031.stdout:(1/24): ceph-base-17.2.7-902.gc61472c8.el9.x86_ 9.5 MB/s | 5.0 MB 00:00 2024-04-24T23:01:07.927 INFO:teuthology.orchestra.run.smithi031.stdout:(2/24): ceph-selinux-17.2.7-902.gc61472c8.el9.x 433 kB/s | 25 kB 00:00 2024-04-24T23:01:08.009 INFO:teuthology.orchestra.run.smithi031.stdout:(3/24): libcephfs2-17.2.7-902.gc61472c8.el9.x86 7.5 MB/s | 637 kB 00:00 2024-04-24T23:01:08.040 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:01:08.040 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:01:08.085 INFO:teuthology.orchestra.run.smithi031.stdout:(4/24): libradosstriper1-17.2.7-902.gc61472c8.e 6.0 MB/s | 457 kB 00:00 2024-04-24T23:01:08.460 INFO:teuthology.orchestra.run.smithi031.stdout:(5/24): librgw2-17.2.7-902.gc61472c8.el9.x86_64 13 MB/s | 4.9 MB 00:00 2024-04-24T23:01:08.527 INFO:teuthology.orchestra.run.smithi031.stdout:(6/24): python3-ceph-argparse-17.2.7-902.gc6147 681 kB/s | 45 kB 00:00 2024-04-24T23:01:08.602 INFO:teuthology.orchestra.run.smithi031.stdout:(7/24): python3-ceph-common-17.2.7-902.gc61472c 1.5 MB/s | 117 kB 00:00 2024-04-24T23:01:08.761 INFO:teuthology.orchestra.run.smithi031.stdout:(8/24): ceph-radosgw-17.2.7-902.gc61472c8.el9.x 9.9 MB/s | 14 MB 00:01 2024-04-24T23:01:08.786 INFO:teuthology.orchestra.run.smithi031.stdout:(9/24): python3-cephfs-17.2.7-902.gc61472c8.el9 850 kB/s | 156 kB 00:00 2024-04-24T23:01:08.811 INFO:teuthology.orchestra.run.smithi031.stdout:(10/24): python3-rados-17.2.7-902.gc61472c8.el9 6.3 MB/s | 320 kB 00:00 2024-04-24T23:01:08.844 INFO:teuthology.orchestra.run.smithi031.stdout:(11/24): python3-rbd-17.2.7-902.gc61472c8.el9.x 4.9 MB/s | 293 kB 00:00 2024-04-24T23:01:08.870 INFO:teuthology.orchestra.run.smithi031.stdout:(12/24): python3-rgw-17.2.7-902.gc61472c8.el9.x 1.7 MB/s | 99 kB 00:00 2024-04-24T23:01:08.929 INFO:teuthology.orchestra.run.smithi031.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-24T23:01:08.954 INFO:teuthology.orchestra.run.smithi031.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-24T23:01:09.088 INFO:teuthology.orchestra.run.smithi031.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-24T23:01:09.146 INFO:teuthology.orchestra.run.smithi031.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 727 kB/s | 42 kB 00:00 2024-04-24T23:01:09.180 INFO:teuthology.orchestra.run.smithi031.stdout:(17/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-04-24T23:01:09.296 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:01:09.563 INFO:teuthology.orchestra.run.smithi031.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 741 kB/s | 309 kB 00:00 2024-04-24T23:01:09.622 INFO:teuthology.orchestra.run.smithi031.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-24T23:01:09.689 INFO:teuthology.orchestra.run.smithi031.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 95 kB/s | 49 kB 00:00 2024-04-24T23:01:09.690 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-24T23:01:09.753 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T23:01:09.829 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T23:01:09.909 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-24T23:01:09.946 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T23:01:09.989 INFO:teuthology.orchestra.run.smithi031.stdout:(21/24): librados2-17.2.7-902.gc61472c8.el9.x86 10 MB/s | 3.1 MB 00:00 2024-04-24T23:01:10.017 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T23:01:10.056 INFO:teuthology.orchestra.run.smithi031.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-04-24T23:01:10.072 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-24T23:01:10.101 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-24T23:01:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:(23/24): ceph-common-17.2.7-902.gc61472c8.el9.x 7.4 MB/s | 22 MB 00:03 2024-04-24T23:01:10.423 INFO:teuthology.orchestra.run.smithi031.stdout:(24/24): librbd1-17.2.7-902.gc61472c8.el9.x86_6 6.8 MB/s | 2.9 MB 00:00 2024-04-24T23:01:10.425 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:01:10.426 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.9 MB/s | 58 MB 00:06 2024-04-24T23:01:10.487 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:01:10.524 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:01:10.524 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:01:10.557 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-24T23:01:10.582 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-24T23:01:10.645 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-24T23:01:10.682 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-24T23:01:10.726 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 10/26 2024-04-24T23:01:10.763 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-24T23:01:10.834 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-24T23:01:10.860 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T23:01:10.925 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T23:01:10.965 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-24T23:01:10.983 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:01:10.984 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:01:11.002 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-24T23:01:11.035 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-24T23:01:11.335 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-24T23:01:11.381 INFO:teuthology.orchestra.run.smithi040.stdout: Upgrading : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-24T23:01:11.471 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-24T23:01:11.547 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 19/26 2024-04-24T23:01:11.575 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 20/26 2024-04-24T23:01:12.325 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:01:12.677 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-24T23:01:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T23:01:12.851 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T23:01:12.931 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-24T23:01:12.960 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T23:01:13.048 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T23:01:13.103 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-24T23:01:13.131 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-24T23:01:13.588 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-24T23:01:13.621 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-24T23:01:13.683 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-24T23:01:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-24T23:01:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 10/26 2024-04-24T23:01:13.817 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-24T23:01:13.889 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-24T23:01:13.915 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T23:01:13.980 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T23:01:14.019 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-24T23:01:14.057 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-24T23:01:14.089 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-24T23:01:14.092 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T23:01:14.131 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T23:01:14.398 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-24T23:01:14.443 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-24T23:01:14.517 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-24T23:01:14.594 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 19/26 2024-04-24T23:01:14.620 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 20/26 2024-04-24T23:01:14.664 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T23:01:14.711 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-24T23:01:14.813 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-24T23:01:14.813 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:01:14.813 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:01:14.813 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:14.850 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T23:01:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T23:01:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T23:01:17.728 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-24T23:01:17.887 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-24T23:01:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-24T23:01:17.969 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-24T23:01:17.969 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-24T23:01:17.969 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:17.996 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T23:01:24.731 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T23:01:24.731 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /sys 2024-04-24T23:01:24.731 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /proc 2024-04-24T23:01:24.731 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /mnt 2024-04-24T23:01:24.731 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /var/tmp 2024-04-24T23:01:24.731 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /home 2024-04-24T23:01:24.732 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /root 2024-04-24T23:01:24.732 INFO:teuthology.orchestra.run.smithi040.stdout:skipping the directory /tmp 2024-04-24T23:01:24.732 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:26.036 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-24T23:01:26.114 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-24T23:01:26.114 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:26.114 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:01:26.114 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:01:26.115 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:01:26.115 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:26.116 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-24T23:01:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-24T23:01:26.205 INFO:teuthology.orchestra.run.smithi040.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T23:01:27.607 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T23:01:27.607 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 1/26 2024-04-24T23:01:27.607 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T23:01:27.607 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-24T23:01:27.607 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T23:01:27.607 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 5/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 6/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 7/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 8/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 9/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 10/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-24T23:01:27.608 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-24T23:01:27.609 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 25/26 2024-04-24T23:01:27.700 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T23:01:27.700 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-04-24T23:01:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-04-24T23:01:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-04-24T23:01:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-04-24T23:01:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-04-24T23:01:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-04-24T23:01:27.701 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-04-24T23:01:27.701 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout:Upgraded: 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:28.312 INFO:teuthology.orchestra.run.smithi040.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:01:28.313 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:28.313 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:01:28.604 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-test 2024-04-24T23:01:29.009 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-24T23:01:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-24T23:01:29.105 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:29.105 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-24T23:01:29.106 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:01:29.106 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-24T23:01:29.106 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:29.107 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-24T23:01:29.195 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-24T23:01:29.195 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T23:01:29.200 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:30 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:01:29.322 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:01:29.323 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:29.323 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:01:29.323 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:29.323 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:01:29.323 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test x86_64 2:17.2.7-902.gc61472c8.el9 ceph 48 M 2024-04-24T23:01:29.323 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-24T23:01:29.323 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite x86_64 2:17.2.7-902.gc61472c8.el9 ceph 163 k 2024-04-24T23:01:29.324 INFO:teuthology.orchestra.run.smithi040.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:01:29.324 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:01:29.324 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:29.324 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:01:29.324 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:29.324 INFO:teuthology.orchestra.run.smithi040.stdout:Install 4 Packages 2024-04-24T23:01:29.324 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:29.325 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 48 M 2024-04-24T23:01:29.325 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 192 M 2024-04-24T23:01:29.325 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:01:29.775 INFO:teuthology.orchestra.run.smithi040.stdout:(1/4): libcephsqlite-17.2.7-902.gc61472c8.el9.x 437 kB/s | 163 kB 00:00 2024-04-24T23:01:30.003 INFO:teuthology.orchestra.run.smithi040.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 508 kB/s | 305 kB 00:00 2024-04-24T23:01:30.129 INFO:teuthology.orchestra.run.smithi040.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 180 kB/s | 64 kB 00:00 2024-04-24T23:01:30.595 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 1/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 5/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 6/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 7/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 8/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 9/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 10/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-24T23:01:30.596 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-24T23:01:30.597 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 25/26 2024-04-24T23:01:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-24T23:01:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:31.297 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-04-24T23:01:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:31.299 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:01:31.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-04-24T23:01:32.028 INFO:teuthology.orchestra.run.smithi040.stdout:(4/4): ceph-test-17.2.7-902.gc61472c8.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-24T23:01:32.030 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:01:32.030 INFO:teuthology.orchestra.run.smithi040.stdout:Total 18 MB/s | 48 MB 00:02 2024-04-24T23:01:32.057 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:01:32.078 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:01:32.078 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:01:32.128 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:31 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:01:32.250 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:01:32.251 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:01:32.251 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:32.251 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:01:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:17.2.7-902.gc61472c8.el9 ceph 48 M 2024-04-24T23:01:32.251 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-24T23:01:32.251 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:17.2.7-902.gc61472c8.el9 ceph 163 k 2024-04-24T23:01:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-24T23:01:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-24T23:01:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:32.252 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:01:32.252 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:32.252 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-04-24T23:01:32.252 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:32.252 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 48 M 2024-04-24T23:01:32.253 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 192 M 2024-04-24T23:01:32.253 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:01:32.461 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:01:32.461 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:01:32.773 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): libcephsqlite-17.2.7-902.gc61472c8.el9.x 669 kB/s | 163 kB 00:00 2024-04-24T23:01:32.807 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 305 kB 00:00 2024-04-24T23:01:32.949 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 362 kB/s | 64 kB 00:00 2024-04-24T23:01:32.996 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:01:33.079 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-24T23:01:33.119 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-24T23:01:33.146 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-24T23:01:34.882 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): ceph-test-17.2.7-902.gc61472c8.el9.x86_6 20 MB/s | 48 MB 00:02 2024-04-24T23:01:34.884 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:01:34.884 INFO:teuthology.orchestra.run.smithi031.stdout:Total 18 MB/s | 48 MB 00:02 2024-04-24T23:01:34.911 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:01:34.932 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:01:34.932 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:01:35.314 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:01:35.315 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:01:35.850 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:01:35.933 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-24T23:01:35.973 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-24T23:01:36.000 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-24T23:01:37.756 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-24T23:01:37.837 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-24T23:01:40.020 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-24T23:01:40.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 1/4 2024-04-24T23:01:40.020 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2/4 2024-04-24T23:01:40.021 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:40.284 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:01:40.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph 2024-04-24T23:01:40.621 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-24T23:01:40.699 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-24T23:01:41.067 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:00:42 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:01:41.194 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:01:41.196 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: ceph x86_64 2:17.2.7-902.gc61472c8.el9 ceph 6.5 k 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.1 M 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr x86_64 2:17.2.7-902.gc61472c8.el9 ceph 1.4 M 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 240 k 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.9 M 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 15 M 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:01:41.197 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:01:41.198 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:01:41.202 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:01:41.203 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:01:41.203 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:01:41.203 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:01:41.203 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:41.203 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:01:41.203 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:01:41.203 INFO:teuthology.orchestra.run.smithi040.stdout:Install 33 Packages 2024-04-24T23:01:41.203 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:41.204 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 27 M 2024-04-24T23:01:41.204 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 96 M 2024-04-24T23:01:41.204 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:01:42.908 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-24T23:01:42.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 1/4 2024-04-24T23:01:42.908 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2/4 2024-04-24T23:01:42.909 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-24T23:01:43.181 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-24T23:01:43.182 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:43.182 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:01:43.182 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:43.182 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:43.182 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-24T23:01:43.182 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-24T23:01:43.182 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:43.182 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:01:43.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-04-24T23:01:43.956 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:42 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:01:44.082 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:01:44.084 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:44.084 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:01:44.084 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:44.084 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:01:44.084 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:17.2.7-902.gc61472c8.el9 ceph 6.5 k 2024-04-24T23:01:44.084 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-24T23:01:44.084 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.1 M 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:17.2.7-902.gc61472c8.el9 ceph 1.4 M 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 240 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.9 M 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 15 M 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-24T23:01:44.085 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-24T23:01:44.086 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-24T23:01:44.087 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-24T23:01:44.087 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-24T23:01:44.087 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-24T23:01:44.087 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:44.087 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:01:44.087 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:44.087 INFO:teuthology.orchestra.run.smithi031.stdout:Install 33 Packages 2024-04-24T23:01:44.087 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:44.088 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 27 M 2024-04-24T23:01:44.088 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 96 M 2024-04-24T23:01:44.088 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:01:45.664 INFO:teuthology.orchestra.run.smithi031.stdout:(1/33): ceph-17.2.7-902.gc61472c8.el9.x86_64.rp 37 kB/s | 6.5 kB 00:00 2024-04-24T23:01:45.906 INFO:teuthology.orchestra.run.smithi031.stdout:(2/33): ceph-mds-17.2.7-902.gc61472c8.el9.x86_6 5.0 MB/s | 2.1 MB 00:00 2024-04-24T23:01:46.140 INFO:teuthology.orchestra.run.smithi031.stdout:(3/33): ceph-mon-17.2.7-902.gc61472c8.el9.x86_6 8.3 MB/s | 3.9 MB 00:00 2024-04-24T23:01:46.223 INFO:teuthology.orchestra.run.smithi031.stdout:(4/33): ceph-mgr-modules-core-17.2.7-902.gc6147 2.8 MB/s | 240 kB 00:00 2024-04-24T23:01:46.357 INFO:teuthology.orchestra.run.smithi031.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-04-24T23:01:46.399 INFO:teuthology.orchestra.run.smithi031.stdout:(6/33): ceph-mgr-17.2.7-902.gc61472c8.el9.x86_6 1.5 MB/s | 1.4 MB 00:00 2024-04-24T23:01:46.457 INFO:teuthology.orchestra.run.smithi031.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 12 MB/s | 1.2 MB 00:00 2024-04-24T23:01:46.499 INFO:teuthology.orchestra.run.smithi031.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-04-24T23:01:46.524 INFO:teuthology.orchestra.run.smithi031.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 851 kB/s | 106 kB 00:00 2024-04-24T23:01:46.550 INFO:teuthology.orchestra.run.smithi031.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-04-24T23:01:46.716 INFO:teuthology.orchestra.run.smithi031.stdout:(11/33): ceph-osd-17.2.7-902.gc61472c8.el9.x86_ 19 MB/s | 15 MB 00:00 2024-04-24T23:01:46.742 INFO:teuthology.orchestra.run.smithi031.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 987 kB/s | 215 kB 00:00 2024-04-24T23:01:46.875 INFO:teuthology.orchestra.run.smithi031.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 219 kB/s | 35 kB 00:00 2024-04-24T23:01:46.917 INFO:teuthology.orchestra.run.smithi031.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 291 kB/s | 51 kB 00:00 2024-04-24T23:01:46.942 INFO:teuthology.orchestra.run.smithi031.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 439 kB/s | 172 kB 00:00 2024-04-24T23:01:47.051 INFO:teuthology.orchestra.run.smithi031.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 237 kB/s | 42 kB 00:00 2024-04-24T23:01:47.177 INFO:teuthology.orchestra.run.smithi031.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 168 kB/s | 43 kB 00:00 2024-04-24T23:01:47.251 INFO:teuthology.orchestra.run.smithi031.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 556 kB/s | 172 kB 00:00 2024-04-24T23:01:47.277 INFO:teuthology.orchestra.run.smithi031.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-04-24T23:01:47.327 INFO:teuthology.orchestra.run.smithi031.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 235 kB/s | 18 kB 00:00 2024-04-24T23:01:47.352 INFO:teuthology.orchestra.run.smithi031.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-04-24T23:01:47.403 INFO:teuthology.orchestra.run.smithi031.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-04-24T23:01:47.428 INFO:teuthology.orchestra.run.smithi031.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-04-24T23:01:47.444 INFO:teuthology.orchestra.run.smithi031.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-04-24T23:01:47.478 INFO:teuthology.orchestra.run.smithi031.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 615 kB/s | 46 kB 00:00 2024-04-24T23:01:47.562 INFO:teuthology.orchestra.run.smithi031.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 197 kB/s | 16 kB 00:00 2024-04-24T23:01:47.595 INFO:teuthology.orchestra.run.smithi031.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-24T23:01:47.621 INFO:teuthology.orchestra.run.smithi031.stdout:(28/33): python3-more-itertools-8.12.0-2.el9.no 448 kB/s | 79 kB 00:00 2024-04-24T23:01:47.646 INFO:teuthology.orchestra.run.smithi031.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-04-24T23:01:47.663 INFO:teuthology.orchestra.run.smithi031.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 532 kB/s | 36 kB 00:00 2024-04-24T23:01:47.730 INFO:teuthology.orchestra.run.smithi031.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 299 kB/s | 20 kB 00:00 2024-04-24T23:01:47.755 INFO:teuthology.orchestra.run.smithi031.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-24T23:01:47.855 INFO:teuthology.orchestra.run.smithi031.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.0 MB/s | 427 kB 00:00 2024-04-24T23:01:47.858 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:01:47.858 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.2 MB/s | 27 MB 00:03 2024-04-24T23:01:47.981 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:01:48.022 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:01:48.022 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:01:48.309 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:01:48.309 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:01:49.477 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:01:49.541 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-24T23:01:49.574 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-24T23:01:49.615 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-24T23:01:49.712 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-24T23:01:49.797 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-24T23:01:49.843 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-24T23:01:49.877 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-24T23:01:49.907 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-24T23:01:49.942 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-24T23:01:49.975 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-24T23:01:50.016 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-24T23:01:50.045 INFO:teuthology.orchestra.run.smithi040.stdout:(1/33): ceph-17.2.7-902.gc61472c8.el9.x86_64.rp 27 kB/s | 6.5 kB 00:00 2024-04-24T23:01:50.059 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-24T23:01:50.093 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-24T23:01:50.123 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-24T23:01:50.201 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-24T23:01:50.212 INFO:teuthology.orchestra.run.smithi040.stdout:(2/33): ceph-mds-17.2.7-902.gc61472c8.el9.x86_6 5.1 MB/s | 2.1 MB 00:00 2024-04-24T23:01:50.253 INFO:teuthology.orchestra.run.smithi040.stdout:(3/33): ceph-mgr-17.2.7-902.gc61472c8.el9.x86_6 3.1 MB/s | 1.4 MB 00:00 2024-04-24T23:01:50.337 INFO:teuthology.orchestra.run.smithi040.stdout:(4/33): ceph-mgr-modules-core-17.2.7-902.gc6147 2.8 MB/s | 240 kB 00:00 2024-04-24T23:01:50.353 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-24T23:01:50.435 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-24T23:01:50.489 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-24T23:01:50.531 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-24T23:01:50.590 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-24T23:01:50.621 INFO:teuthology.orchestra.run.smithi040.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 895 kB/s | 253 kB 00:00 2024-04-24T23:01:50.653 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-24T23:01:50.883 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-24T23:01:50.935 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-24T23:01:50.946 INFO:teuthology.orchestra.run.smithi040.stdout:(6/33): ceph-osd-17.2.7-902.gc61472c8.el9.x86_6 21 MB/s | 15 MB 00:00 2024-04-24T23:01:51.015 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-24T23:01:51.071 INFO:teuthology.orchestra.run.smithi040.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 2.7 MB/s | 1.2 MB 00:00 2024-04-24T23:01:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-24T23:01:51.158 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-24T23:01:51.163 INFO:teuthology.orchestra.run.smithi040.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 492 kB/s | 106 kB 00:00 2024-04-24T23:01:51.280 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-24T23:01:51.321 INFO:teuthology.orchestra.run.smithi040.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 540 kB/s | 135 kB 00:00 2024-04-24T23:01:51.437 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 28/33 2024-04-24T23:01:51.438 INFO:teuthology.orchestra.run.smithi040.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 454 kB/s | 125 kB 00:00 2024-04-24T23:01:51.460 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-24T23:01:51.487 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-24T23:01:51.487 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:51.487 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:01:51.487 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:01:51.487 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:01:51.487 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:51.572 INFO:teuthology.orchestra.run.smithi040.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 858 kB/s | 215 kB 00:00 2024-04-24T23:01:51.672 INFO:teuthology.orchestra.run.smithi040.stdout:(12/33): ceph-mon-17.2.7-902.gc61472c8.el9.x86_ 2.4 MB/s | 3.9 MB 00:01 2024-04-24T23:01:51.698 INFO:teuthology.orchestra.run.smithi040.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 278 kB/s | 35 kB 00:00 2024-04-24T23:01:51.714 INFO:teuthology.orchestra.run.smithi040.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 624 kB/s | 172 kB 00:00 2024-04-24T23:01:51.815 INFO:teuthology.orchestra.run.smithi040.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-04-24T23:01:51.842 INFO:teuthology.orchestra.run.smithi040.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 304 kB/s | 51 kB 00:00 2024-04-24T23:01:51.865 INFO:teuthology.orchestra.run.smithi040.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 289 kB/s | 43 kB 00:00 2024-04-24T23:01:51.949 INFO:teuthology.orchestra.run.smithi040.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 128 kB/s | 11 kB 00:00 2024-04-24T23:01:51.999 INFO:teuthology.orchestra.run.smithi040.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-04-24T23:01:52.016 INFO:teuthology.orchestra.run.smithi040.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 265 kB/s | 18 kB 00:00 2024-04-24T23:01:52.041 INFO:teuthology.orchestra.run.smithi040.stdout:(21/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 758 kB/s | 172 kB 00:00 2024-04-24T23:01:52.058 INFO:teuthology.orchestra.run.smithi040.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 394 kB/s | 23 kB 00:00 2024-04-24T23:01:52.076 INFO:teuthology.orchestra.run.smithi040.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 331 kB/s | 19 kB 00:00 2024-04-24T23:01:52.092 INFO:teuthology.orchestra.run.smithi040.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 389 kB/s | 20 kB 00:00 2024-04-24T23:01:52.109 INFO:teuthology.orchestra.run.smithi040.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 915 kB/s | 46 kB 00:00 2024-04-24T23:01:52.160 INFO:teuthology.orchestra.run.smithi040.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 329 kB/s | 16 kB 00:00 2024-04-24T23:01:52.185 INFO:teuthology.orchestra.run.smithi040.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 723 kB/s | 79 kB 00:00 2024-04-24T23:01:52.210 INFO:teuthology.orchestra.run.smithi040.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-24T23:01:52.235 INFO:teuthology.orchestra.run.smithi040.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-04-24T23:01:52.252 INFO:teuthology.orchestra.run.smithi040.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 533 kB/s | 36 kB 00:00 2024-04-24T23:01:52.278 INFO:teuthology.orchestra.run.smithi040.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-24T23:01:52.303 INFO:teuthology.orchestra.run.smithi040.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-04-24T23:01:52.336 INFO:teuthology.orchestra.run.smithi040.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-04-24T23:01:52.338 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:01:52.338 INFO:teuthology.orchestra.run.smithi040.stdout:Total 2.4 MB/s | 27 MB 00:11 2024-04-24T23:01:52.454 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:01:52.494 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:01:52.494 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:01:52.778 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:01:52.779 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:01:52.877 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-24T23:01:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-24T23:01:52.905 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:52.905 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:01:52.905 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:01:52.905 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:01:52.905 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:53.294 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-24T23:01:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-24T23:01:53.319 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:53.319 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:01:53.319 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:01:53.319 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:01:53.319 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-24T23:01:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-24T23:01:53.553 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:53.553 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:01:53.553 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:01:53.553 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:01:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:53.618 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-24T23:01:53.926 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:01:53.998 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-24T23:01:54.030 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-24T23:01:54.063 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-24T23:01:54.159 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-24T23:01:54.243 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-24T23:01:54.300 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-24T23:01:54.334 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-24T23:01:54.364 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-24T23:01:54.398 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-24T23:01:54.432 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-24T23:01:54.473 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-24T23:01:54.516 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-24T23:01:54.550 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-24T23:01:54.589 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-24T23:01:54.665 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-24T23:01:54.816 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-24T23:01:54.899 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-24T23:01:54.951 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-24T23:01:54.989 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-24T23:01:55.047 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 1/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 3/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 4/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 5/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 6/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-24T23:01:55.111 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-24T23:01:55.112 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-24T23:01:55.113 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-24T23:01:55.305 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-24T23:01:55.359 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-24T23:01:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-24T23:01:55.529 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-24T23:01:55.566 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-24T23:01:55.687 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:01:55.827 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:01:55.828 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:55.829 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:01:55.853 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 28/33 2024-04-24T23:01:55.886 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-24T23:01:55.911 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-24T23:01:55.911 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:55.912 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-24T23:01:55.912 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:01:55.912 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-24T23:01:55.912 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:56.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-04-24T23:01:56.726 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:55 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:01:56.806 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:01:56.849 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:01:56.850 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:01:56.850 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:01:56.916 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-04-24T23:01:57.312 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-24T23:01:57.340 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-24T23:01:57.340 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:57.340 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-24T23:01:57.340 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:01:57.340 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-24T23:01:57.340 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:57.441 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:56 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:01:57.565 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:01:57.565 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 82 k 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 82 k 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 372 k 2024-04-24T23:01:57.566 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:01:57.730 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-24T23:01:57.755 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-24T23:01:57.755 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:57.755 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-24T23:01:57.755 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:01:57.755 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-24T23:01:57.755 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:57.776 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-17.2.7-902.gc61472c8.el9.noarch.rpm 392 kB/s | 82 kB 00:00 2024-04-24T23:01:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:01:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:Total 389 kB/s | 82 kB 00:00 2024-04-24T23:01:57.777 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:01:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:01:57.782 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:01:57.784 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:01:57.785 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:01:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:01:57.963 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-24T23:01:57.987 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-24T23:01:57.988 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:01:57.988 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-24T23:01:57.988 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:01:57.988 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-24T23:01:57.988 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:01:58.053 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-24T23:01:58.292 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T23:01:58.389 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T23:01:58.716 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T23:01:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T23:01:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:58.931 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:01:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:01:58.931 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:58.931 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:01:59.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T23:01:59.649 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:58 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:01:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-24T23:01:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 1/33 2024-04-24T23:01:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2/33 2024-04-24T23:01:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 3/33 2024-04-24T23:01:59.700 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 4/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 5/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 6/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-24T23:01:59.701 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-24T23:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-24T23:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-24T23:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-24T23:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-24T23:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-24T23:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-24T23:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-24T23:01:59.702 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-24T23:01:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-24T23:01:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-24T23:01:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-24T23:01:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-24T23:01:59.703 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-24T23:01:59.778 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:01:59.778 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-902.gc61472c8.el9 ceph 137 k 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 137 k 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 379 k 2024-04-24T23:01:59.779 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:00.115 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-17.2.7-902.gc61472c 409 kB/s | 137 kB 00:00 2024-04-24T23:02:00.116 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:00.116 INFO:teuthology.orchestra.run.smithi031.stdout:Total 407 kB/s | 137 kB 00:00 2024-04-24T23:02:00.116 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:00.125 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:00.125 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:00.184 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:00.184 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:00.350 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:00.424 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T23:02:00.450 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T23:02:00.451 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:02:00.451 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:02:00.451 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:00.511 INFO:teuthology.orchestra.run.smithi040.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-24T23:02:00.512 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ply-3.11-14.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:00.513 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:00.841 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-base 2024-04-24T23:02:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T23:02:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:01.310 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:01.311 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:01.374 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:02 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:01.457 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:01.497 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:01.498 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:01.498 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:01.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-04-24T23:02:01.572 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install cephadm 2024-04-24T23:02:02.044 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:01 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:02.097 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:03 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:02.125 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:02.169 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:02.170 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:02:02.170 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:02.223 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:02.224 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:02.224 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:02:02.224 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:02.224 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:02.224 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 82 k 2024-04-24T23:02:02.225 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:02.225 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:02.225 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:02.225 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-04-24T23:02:02.225 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:02.225 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 82 k 2024-04-24T23:02:02.225 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 372 k 2024-04-24T23:02:02.225 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:02.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T23:02:02.437 INFO:teuthology.orchestra.run.smithi040.stdout:cephadm-17.2.7-902.gc61472c8.el9.noarch.rpm 387 kB/s | 82 kB 00:00 2024-04-24T23:02:02.438 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:02.439 INFO:teuthology.orchestra.run.smithi040.stdout:Total 383 kB/s | 82 kB 00:00 2024-04-24T23:02:02.439 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:02.444 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:02.444 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:02.446 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:02.446 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:02.520 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:02.769 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:01 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:02.895 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:02.896 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:02.896 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:02:02.896 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:02.896 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:02.896 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 3.4 M 2024-04-24T23:02:02.896 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-24T23:02:02.896 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 24 k 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 15 k 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout:Install 7 Packages 2024-04-24T23:02:02.897 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:02.898 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.7 M 2024-04-24T23:02:02.898 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 80 M 2024-04-24T23:02:02.898 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:02.979 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T23:02:03.084 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T23:02:03.276 INFO:teuthology.orchestra.run.smithi031.stdout:(1/7): ceph-grafana-dashboards-17.2.7-902.gc614 127 kB/s | 24 kB 00:00 2024-04-24T23:02:03.302 INFO:teuthology.orchestra.run.smithi031.stdout:(2/7): ceph-prometheus-alerts-17.2.7-902.gc6147 67 kB/s | 15 kB 00:00 2024-04-24T23:02:03.369 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T23:02:03.385 INFO:teuthology.orchestra.run.smithi031.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 83 kB/s | 9.0 kB 00:00 2024-04-24T23:02:03.436 INFO:teuthology.orchestra.run.smithi031.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 306 kB/s | 41 kB 00:00 2024-04-24T23:02:03.469 INFO:teuthology.orchestra.run.smithi031.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 368 kB/s | 31 kB 00:00 2024-04-24T23:02:03.553 INFO:teuthology.orchestra.run.smithi031.stdout:(6/7): ceph-mgr-dashboard-17.2.7-902.gc61472c8. 7.2 MB/s | 3.4 MB 00:00 2024-04-24T23:02:03.594 INFO:teuthology.orchestra.run.smithi031.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2024-04-24T23:02:03.596 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:03.596 INFO:teuthology.orchestra.run.smithi031.stdout:Total 5.2 MB/s | 3.7 MB 00:00 2024-04-24T23:02:03.597 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:03.605 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:03.606 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:03.642 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-24T23:02:03.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:03.643 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:03.643 INFO:teuthology.orchestra.run.smithi040.stdout: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:03.643 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:03.643 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:03.671 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:03.671 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:03.832 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T23:02:03.979 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:04.029 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-24T23:02:04.064 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-24T23:02:04.111 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-24T23:02:04.141 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-24T23:02:04.162 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 5/7 2024-04-24T23:02:04.365 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:05 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:04.489 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:04.490 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:04.490 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-04-24T23:02:04.490 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:04.490 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:04.490 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-902.gc61472c8.el9 ceph 137 k 2024-04-24T23:02:04.490 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:04.490 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:04.491 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:04.491 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-04-24T23:02:04.491 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:04.491 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 137 k 2024-04-24T23:02:04.491 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 379 k 2024-04-24T23:02:04.491 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:04.694 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-immutable-object-cache-17.2.7-902.gc61472c 678 kB/s | 137 kB 00:00 2024-04-24T23:02:04.695 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:04.695 INFO:teuthology.orchestra.run.smithi040.stdout:Total 672 kB/s | 137 kB 00:00 2024-04-24T23:02:04.695 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:04.703 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:04.704 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:04.763 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:04.763 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:04.904 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:04.987 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T23:02:05.012 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T23:02:05.012 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:02:05.012 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-24T23:02:05.013 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:05.207 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 6/7 2024-04-24T23:02:05.241 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-24T23:02:05.915 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-24T23:02:05.915 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:05.915 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:05.915 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:05.915 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:05.915 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:06.138 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr 2024-04-24T23:02:06.286 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-24T23:02:06.286 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 1/7 2024-04-24T23:02:06.286 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2/7 2024-04-24T23:02:06.286 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 3/7 2024-04-24T23:02:06.286 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-24T23:02:06.287 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-24T23:02:06.287 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-24T23:02:06.538 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-24T23:02:06.538 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:06.538 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:06.538 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:06.538 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:06.538 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:06.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-24T23:02:06.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-24T23:02:06.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:02:06.539 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:02:06.539 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:06.539 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:06.668 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:07 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:06.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T23:02:06.749 INFO:teuthology.orchestra.run.smithi040.stdout:Package ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:06.792 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:06.793 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:06.794 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:06.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T23:02:07.256 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:06 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:07.383 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:07.384 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-04-24T23:02:07.384 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:02:07.384 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-04-24T23:02:07.384 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:07.384 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 7.4 M 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:02:07.385 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:07.386 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:07.386 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-04-24T23:02:07.386 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-04-24T23:02:07.386 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:07.386 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 38 M 2024-04-24T23:02:07.386 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 209 M 2024-04-24T23:02:07.386 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:07.437 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:08 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:07.562 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 3.4 M 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 24 k 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 15 k 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-24T23:02:07.563 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-24T23:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout:Installing weak dependencies: 2024-04-24T23:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-24T23:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout:Install 7 Packages 2024-04-24T23:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:07.564 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 3.7 M 2024-04-24T23:02:07.565 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 80 M 2024-04-24T23:02:07.565 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:08.060 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 611 kB/s | 195 kB 00:00 2024-04-24T23:02:08.093 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 803 kB 00:00 2024-04-24T23:02:08.319 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 126 kB/s | 33 kB 00:00 2024-04-24T23:02:08.452 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): ceph-mgr-diskprediction-local-17.2.7-90 10 MB/s | 7.4 MB 00:00 2024-04-24T23:02:08.478 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 115 kB/s | 18 kB 00:00 2024-04-24T23:02:08.711 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 136 kB/s | 35 kB 00:00 2024-04-24T23:02:09.020 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.2 MB/s | 3.0 MB 00:00 2024-04-24T23:02:09.154 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 552 kB/s | 244 kB 00:00 2024-04-24T23:02:09.345 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.3 MB/s | 250 kB 00:00 2024-04-24T23:02:09.437 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.9 MB/s | 4.7 MB 00:00 2024-04-24T23:02:10.921 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-04-24T23:02:11.265 INFO:teuthology.orchestra.run.smithi040.stdout:(1/7): ceph-grafana-dashboards-17.2.7-902.gc614 103 kB/s | 24 kB 00:00 2024-04-24T23:02:11.290 INFO:teuthology.orchestra.run.smithi040.stdout:(2/7): ceph-prometheus-alerts-17.2.7-902.gc6147 56 kB/s | 15 kB 00:00 2024-04-24T23:02:11.395 INFO:teuthology.orchestra.run.smithi040.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 72 kB/s | 9.0 kB 00:00 2024-04-24T23:02:11.449 INFO:teuthology.orchestra.run.smithi040.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 257 kB/s | 41 kB 00:00 2024-04-24T23:02:11.490 INFO:teuthology.orchestra.run.smithi040.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 334 kB/s | 31 kB 00:00 2024-04-24T23:02:11.541 INFO:teuthology.orchestra.run.smithi040.stdout:(6/7): ceph-mgr-dashboard-17.2.7-902.gc61472c8. 6.5 MB/s | 3.4 MB 00:00 2024-04-24T23:02:11.694 INFO:teuthology.orchestra.run.smithi040.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 779 kB/s | 188 kB 00:00 2024-04-24T23:02:11.698 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:11.698 INFO:teuthology.orchestra.run.smithi040.stdout:Total 906 kB/s | 3.7 MB 00:04 2024-04-24T23:02:11.698 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:11.705 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:11.706 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:11.780 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:11.780 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:12.090 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:12.151 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-24T23:02:12.186 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-24T23:02:12.242 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-24T23:02:12.289 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-24T23:02:12.318 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 5/7 2024-04-24T23:02:13.374 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 6/7 2024-04-24T23:02:13.421 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-24T23:02:13.837 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.1 MB/s | 5.1 MB 00:04 2024-04-24T23:02:13.838 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:13.838 INFO:teuthology.orchestra.run.smithi031.stdout:Total 5.9 MB/s | 38 MB 00:06 2024-04-24T23:02:14.203 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:14.231 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:14.231 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:14.502 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-24T23:02:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 1/7 2024-04-24T23:02:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2/7 2024-04-24T23:02:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 3/7 2024-04-24T23:02:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-24T23:02:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-24T23:02:14.503 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-24T23:02:14.662 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:14.662 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:14.747 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-24T23:02:14.747 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:14.747 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:14.747 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:14.747 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:14.747 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:14.747 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-24T23:02:14.747 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-24T23:02:14.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-24T23:02:14.748 INFO:teuthology.orchestra.run.smithi040.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-24T23:02:14.748 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:14.748 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:14.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T23:02:15.439 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:15.472 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:16 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:15.500 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-24T23:02:15.536 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-24T23:02:15.599 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:15.600 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 7.4 M 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-24T23:02:15.601 INFO:teuthology.orchestra.run.smithi040.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout:====================================================================================== 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout:Install 12 Packages 2024-04-24T23:02:15.602 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:15.603 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 38 M 2024-04-24T23:02:15.603 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 209 M 2024-04-24T23:02:15.603 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:15.640 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-24T23:02:15.678 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-24T23:02:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-24T23:02:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-24T23:02:15.973 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-24T23:02:16.274 INFO:teuthology.orchestra.run.smithi040.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 808 kB/s | 195 kB 00:00 2024-04-24T23:02:16.392 INFO:teuthology.orchestra.run.smithi040.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-04-24T23:02:16.433 INFO:teuthology.orchestra.run.smithi040.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-04-24T23:02:16.575 INFO:teuthology.orchestra.run.smithi040.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 128 kB/s | 18 kB 00:00 2024-04-24T23:02:16.659 INFO:teuthology.orchestra.run.smithi040.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-04-24T23:02:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-24T23:02:16.768 INFO:teuthology.orchestra.run.smithi040.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-90 10 MB/s | 7.4 MB 00:00 2024-04-24T23:02:16.777 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-24T23:02:16.985 INFO:teuthology.orchestra.run.smithi040.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-24T23:02:17.185 INFO:teuthology.orchestra.run.smithi040.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.8 MB/s | 3.0 MB 00:00 2024-04-24T23:02:17.244 INFO:teuthology.orchestra.run.smithi040.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-04-24T23:02:17.661 INFO:teuthology.orchestra.run.smithi040.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.7 MB/s | 4.7 MB 00:01 2024-04-24T23:02:17.927 INFO:teuthology.orchestra.run.smithi040.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.4 MB/s | 5.1 MB 00:00 2024-04-24T23:02:18.094 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T23:02:18.444 INFO:teuthology.orchestra.run.smithi040.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-24T23:02:18.445 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:18.445 INFO:teuthology.orchestra.run.smithi040.stdout:Total 13 MB/s | 38 MB 00:02 2024-04-24T23:02:18.811 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:18.838 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:18.838 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:19.171 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-24T23:02:19.200 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-24T23:02:19.272 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:19.272 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:20.058 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:20.124 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-24T23:02:20.176 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-24T23:02:20.283 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-24T23:02:20.327 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-24T23:02:20.514 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-24T23:02:20.543 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-24T23:02:20.622 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-24T23:02:21.314 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-24T23:02:21.436 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 1/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-24T23:02:21.704 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-24T23:02:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T23:02:21.705 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-24T23:02:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-24T23:02:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:22.099 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:22.099 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:22.100 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:22.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-04-24T23:02:22.797 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T23:02:23.050 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:22 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:23.176 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:23.177 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:02:23.177 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:23.177 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:23.177 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 47 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:23.178 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:23.179 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:23.179 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-04-24T23:02:23.179 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:23.182 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.2 M 2024-04-24T23:02:23.182 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-04-24T23:02:23.182 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:23.891 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 72 kB/s | 19 kB 00:00 2024-04-24T23:02:23.899 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-24T23:02:23.908 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): ceph-mgr-rook-17.2.7-902.gc61472c8.el9. 168 kB/s | 47 kB 00:00 2024-04-24T23:02:23.925 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 86 kB/s | 26 kB 00:00 2024-04-24T23:02:23.925 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-24T23:02:24.267 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 465 kB/s | 159 kB 00:00 2024-04-24T23:02:24.317 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 522 kB/s | 222 kB 00:00 2024-04-24T23:02:24.359 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 584 kB/s | 54 kB 00:00 2024-04-24T23:02:24.501 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 484 kB/s | 279 kB 00:00 2024-04-24T23:02:24.602 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 113 kB/s | 32 kB 00:00 2024-04-24T23:02:24.627 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 56 kB/s | 15 kB 00:00 2024-04-24T23:02:24.852 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 262 kB/s | 59 kB 00:00 2024-04-24T23:02:25.002 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 410 kB/s | 206 kB 00:00 2024-04-24T23:02:25.027 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 512 kB/s | 90 kB 00:00 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.9 MB/s | 1.0 MB 00:00 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:25.128 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.1 MB/s | 2.2 MB 00:01 2024-04-24T23:02:25.199 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:25.216 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:25.216 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:25.310 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:25.310 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:25.901 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:26.099 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-24T23:02:26.189 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-24T23:02:26.244 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-24T23:02:26.280 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-24T23:02:26.322 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-24T23:02:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 1/12 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-24T23:02:26.510 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-24T23:02:26.511 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-24T23:02:26.511 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-24T23:02:26.511 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-24T23:02:26.531 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-24T23:02:26.583 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-24T23:02:26.881 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-24T23:02:26.881 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:26.881 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:26.881 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:26.881 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-24T23:02:26.881 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-24T23:02:26.881 INFO:teuthology.orchestra.run.smithi040.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:26.882 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:27.285 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-rook 2024-04-24T23:02:27.347 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-24T23:02:27.422 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-24T23:02:27.456 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-24T23:02:27.502 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-24T23:02:27.525 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-24T23:02:27.834 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:28 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:27.961 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:27.962 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:27.962 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:02:27.962 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:27.962 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 47 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-24T23:02:27.963 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-24T23:02:27.964 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:27.964 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:27.964 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:27.964 INFO:teuthology.orchestra.run.smithi040.stdout:Install 13 Packages 2024-04-24T23:02:27.964 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:27.964 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.2 M 2024-04-24T23:02:27.964 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 26 M 2024-04-24T23:02:27.965 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:28.195 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-24T23:02:28.195 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 1/13 2024-04-24T23:02:28.195 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-24T23:02:28.196 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-24T23:02:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-24T23:02:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:28.553 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:02:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:02:28.553 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:28.554 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:28.610 INFO:teuthology.orchestra.run.smithi040.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 193 kB/s | 26 kB 00:00 2024-04-24T23:02:28.627 INFO:teuthology.orchestra.run.smithi040.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 128 kB/s | 19 kB 00:00 2024-04-24T23:02:28.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T23:02:28.844 INFO:teuthology.orchestra.run.smithi040.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 736 kB/s | 159 kB 00:00 2024-04-24T23:02:28.877 INFO:teuthology.orchestra.run.smithi040.stdout:(4/13): ceph-mgr-rook-17.2.7-902.gc61472c8.el9. 118 kB/s | 47 kB 00:00 2024-04-24T23:02:28.928 INFO:teuthology.orchestra.run.smithi040.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 741 kB/s | 222 kB 00:00 2024-04-24T23:02:29.128 INFO:teuthology.orchestra.run.smithi040.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 214 kB/s | 54 kB 00:00 2024-04-24T23:02:29.170 INFO:teuthology.orchestra.run.smithi040.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 855 kB/s | 279 kB 00:00 2024-04-24T23:02:29.195 INFO:teuthology.orchestra.run.smithi040.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 120 kB/s | 32 kB 00:00 2024-04-24T23:02:29.318 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:28 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:29.395 INFO:teuthology.orchestra.run.smithi040.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 56 kB/s | 15 kB 00:00 2024-04-24T23:02:29.440 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:29.441 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:02:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:29.441 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 123 k 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-04-24T23:02:29.442 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 7.0 M 2024-04-24T23:02:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-04-24T23:02:29.443 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:29.687 INFO:teuthology.orchestra.run.smithi040.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 398 kB/s | 206 kB 00:00 2024-04-24T23:02:29.738 INFO:teuthology.orchestra.run.smithi040.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-04-24T23:02:29.880 INFO:teuthology.orchestra.run.smithi040.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 1.5 MB/s | 1.0 MB 00:00 2024-04-24T23:02:29.931 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): ceph-mgr-cephadm-17.2.7-902.gc61472c8.el 573 kB/s | 123 kB 00:00 2024-04-24T23:02:29.956 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-04-24T23:02:30.038 INFO:teuthology.orchestra.run.smithi040.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 256 kB/s | 90 kB 00:00 2024-04-24T23:02:30.039 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:30.039 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.1 MB/s | 2.2 MB 00:02 2024-04-24T23:02:30.108 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:30.125 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:30.125 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:30.134 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-04-24T23:02:30.218 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:30.218 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:30.573 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 93 kB/s | 58 kB 00:00 2024-04-24T23:02:30.657 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 108 kB/s | 51 kB 00:00 2024-04-24T23:02:30.807 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 626 kB/s | 548 kB 00:00 2024-04-24T23:02:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:30.811 INFO:teuthology.orchestra.run.smithi031.stdout:Total 5.1 MB/s | 7.0 MB 00:01 2024-04-24T23:02:30.811 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:30.898 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:30.898 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:30.986 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-24T23:02:31.000 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:31.000 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:31.049 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-24T23:02:31.114 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-24T23:02:31.150 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-24T23:02:31.191 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-24T23:02:31.289 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-24T23:02:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:31.399 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-24T23:02:31.452 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-24T23:02:31.490 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-24T23:02:31.552 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-24T23:02:32.112 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-24T23:02:32.191 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-24T23:02:32.213 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-24T23:02:32.261 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-24T23:02:32.265 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-24T23:02:32.288 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-24T23:02:32.299 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-24T23:02:32.345 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-24T23:02:32.368 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-24T23:02:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-24T23:02:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/6 2024-04-24T23:02:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-24T23:02:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-24T23:02:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-24T23:02:32.958 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-24T23:02:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-24T23:02:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 1/13 2024-04-24T23:02:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-24T23:02:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-24T23:02:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-24T23:02:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-24T23:02:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-24T23:02:33.124 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-24T23:02:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-24T23:02:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-24T23:02:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-24T23:02:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-24T23:02:33.125 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-24T23:02:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-24T23:02:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:33.207 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:02:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:02:33.207 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:02:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:02:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:02:33.208 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:33.208 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:33.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-04-24T23:02:33.438 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-24T23:02:33.439 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-24T23:02:33.440 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-24T23:02:33.440 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-24T23:02:33.440 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-24T23:02:33.440 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:33.440 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:33.664 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T23:02:33.962 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:32 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:34.086 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:34.086 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 814 k 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 894 k 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 2.6 M 2024-04-24T23:02:34.087 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:34.202 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:35 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:34.327 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:34.328 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:34.328 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:02:34.328 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:34.328 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 123 k 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout:Install 6 Packages 2024-04-24T23:02:34.329 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:34.330 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 7.0 M 2024-04-24T23:02:34.330 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 33 M 2024-04-24T23:02:34.330 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:34.434 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 360 kB/s | 80 kB 00:00 2024-04-24T23:02:34.638 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-fuse-17.2.7-902.gc61472c8.el9.x86_6 1.9 MB/s | 814 kB 00:00 2024-04-24T23:02:34.639 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:34.639 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.6 MB/s | 894 kB 00:00 2024-04-24T23:02:34.647 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:34.671 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:34.672 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:34.753 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:34.754 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:34.791 INFO:teuthology.orchestra.run.smithi040.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-04-24T23:02:34.891 INFO:teuthology.orchestra.run.smithi040.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.4 MB/s | 548 kB 00:00 2024-04-24T23:02:34.916 INFO:teuthology.orchestra.run.smithi040.stdout:(3/6): ceph-mgr-cephadm-17.2.7-902.gc61472c8.el 348 kB/s | 123 kB 00:00 2024-04-24T23:02:34.941 INFO:teuthology.orchestra.run.smithi040.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-04-24T23:02:34.942 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:34.991 INFO:teuthology.orchestra.run.smithi040.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 678 kB/s | 51 kB 00:00 2024-04-24T23:02:35.075 INFO:teuthology.orchestra.run.smithi040.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-04-24T23:02:35.076 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:35.076 INFO:teuthology.orchestra.run.smithi040.stdout:Total 9.3 MB/s | 7.0 MB 00:00 2024-04-24T23:02:35.084 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-24T23:02:35.149 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:35.151 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-24T23:02:35.161 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:35.161 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:35.263 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:35.263 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:35.570 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:35.748 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-24T23:02:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-24T23:02:35.758 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/2 2024-04-24T23:02:35.804 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-24T23:02:36.016 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-24T23:02:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:36.017 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-24T23:02:36.017 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:36.017 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:36.272 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-04-24T23:02:36.364 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-24T23:02:36.441 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-24T23:02:36.512 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-24T23:02:36.539 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-24T23:02:36.805 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:35 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:36.929 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 126 k 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:36.930 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-24T23:02:36.931 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:36.931 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 126 k 2024-04-24T23:02:36.931 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 448 k 2024-04-24T23:02:36.931 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:37.260 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-17.2.7-902.gc61472c8.el9.x86_64. 383 kB/s | 126 kB 00:00 2024-04-24T23:02:37.261 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:37.261 INFO:teuthology.orchestra.run.smithi031.stdout:Total 381 kB/s | 126 kB 00:00 2024-04-24T23:02:37.261 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:37.267 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:37.268 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:37.295 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-24T23:02:37.295 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/6 2024-04-24T23:02:37.295 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-24T23:02:37.295 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-24T23:02:37.295 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-24T23:02:37.295 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-24T23:02:37.333 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:37.333 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:37.490 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:37.543 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-24T23:02:37.543 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:37.543 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:37.543 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-24T23:02:37.543 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-24T23:02:37.543 INFO:teuthology.orchestra.run.smithi040.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-24T23:02:37.543 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-24T23:02:37.543 INFO:teuthology.orchestra.run.smithi040.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-24T23:02:37.544 INFO:teuthology.orchestra.run.smithi040.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-24T23:02:37.544 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:37.544 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:37.790 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install ceph-fuse 2024-04-24T23:02:37.975 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:38.256 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:38.256 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:38.322 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:39 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:38.452 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repository Size 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 814 k 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout:Installing dependencies: 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:38.453 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:38.454 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:38.454 INFO:teuthology.orchestra.run.smithi040.stdout:Install 2 Packages 2024-04-24T23:02:38.454 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:38.454 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 894 k 2024-04-24T23:02:38.454 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 2.6 M 2024-04-24T23:02:38.454 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:38.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-04-24T23:02:38.702 INFO:teuthology.orchestra.run.smithi040.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 434 kB/s | 80 kB 00:00 2024-04-24T23:02:38.944 INFO:teuthology.orchestra.run.smithi040.stdout:(2/2): ceph-fuse-17.2.7-902.gc61472c8.el9.x86_6 1.9 MB/s | 814 kB 00:00 2024-04-24T23:02:38.945 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:38.945 INFO:teuthology.orchestra.run.smithi040.stdout:Total 1.8 MB/s | 894 kB 00:00 2024-04-24T23:02:38.957 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:38.980 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:38.980 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:39.029 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:38 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:39.061 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:39.061 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:39.110 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:39.153 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:39.154 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:02:39.154 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:39.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-04-24T23:02:39.254 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:39.377 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-24T23:02:39.452 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-24T23:02:39.768 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:38 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:39.891 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-24T23:02:39.892 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:39.893 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 25 k 2024-04-24T23:02:39.893 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 110 k 2024-04-24T23:02:39.893 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:40.065 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-24T23:02:40.065 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/2 2024-04-24T23:02:40.127 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-17.2.7-902.gc61472c8.el9.x86_64 107 kB/s | 25 kB 00:00 2024-04-24T23:02:40.128 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:40.128 INFO:teuthology.orchestra.run.smithi031.stdout:Total 106 kB/s | 25 kB 00:00 2024-04-24T23:02:40.128 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:40.131 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:40.132 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:40.133 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:40.133 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:40.217 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:40.310 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-24T23:02:40.310 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:40.310 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:40.311 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-24T23:02:40.311 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:40.311 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:40.324 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:40.547 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados-devel 2024-04-24T23:02:40.655 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:40.874 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:40.875 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:40.875 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:40.875 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:41.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-04-24T23:02:41.080 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:42 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:41.206 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:41.206 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:41.206 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 126 k 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 126 k 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 448 k 2024-04-24T23:02:41.207 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:41.407 INFO:teuthology.orchestra.run.smithi040.stdout:librados-devel-17.2.7-902.gc61472c8.el9.x86_64. 633 kB/s | 126 kB 00:00 2024-04-24T23:02:41.409 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:41.409 INFO:teuthology.orchestra.run.smithi040.stdout:Total 627 kB/s | 126 kB 00:00 2024-04-24T23:02:41.409 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:41.414 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:41.414 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:41.481 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:41.481 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:41.572 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:40 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:41.637 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:41.655 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:41.698 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:41.699 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:02:41.699 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:41.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-04-24T23:02:41.756 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:42.119 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:42.291 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:41 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:42.358 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:42.359 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:42.359 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:42.359 INFO:teuthology.orchestra.run.smithi040.stdout: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:42.359 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:42.359 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:42.375 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:42.415 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:42.416 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:02:42.416 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:42.472 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-04-24T23:02:42.551 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs2 2024-04-24T23:02:43.006 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:42 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:43.087 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:43.091 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:44 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:43.130 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:43.131 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:02:43.131 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:43.173 INFO:teuthology.orchestra.run.smithi040.stdout:Package libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:43.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-04-24T23:02:43.216 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:43.217 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:43.217 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:43.282 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install libcephfs-devel 2024-04-24T23:02:43.720 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:42 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:43.801 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:43.823 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:44 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:43.846 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:43.847 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:02:43.848 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:43.905 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-04-24T23:02:43.948 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:43.949 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:43.949 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-04-24T23:02:43.949 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 25 k 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 110 k 2024-04-24T23:02:43.950 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:44.196 INFO:teuthology.orchestra.run.smithi040.stdout:libcephfs-devel-17.2.7-902.gc61472c8.el9.x86_64 101 kB/s | 25 kB 00:00 2024-04-24T23:02:44.197 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:44.197 INFO:teuthology.orchestra.run.smithi040.stdout:Total 101 kB/s | 25 kB 00:00 2024-04-24T23:02:44.197 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:44.201 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:44.201 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:44.202 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:44.202 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:44.279 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:44.386 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:44.434 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:43 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:44.517 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:44.560 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:02:44.561 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:44.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-04-24T23:02:44.720 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:44.925 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:44.925 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:44.925 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:44.925 INFO:teuthology.orchestra.run.smithi040.stdout: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:44.925 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:44.926 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:45.068 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librados2 2024-04-24T23:02:45.163 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:44 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:45.244 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:45.286 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:45.287 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-04-24T23:02:45.287 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:45.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-04-24T23:02:45.610 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:46 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:45.692 INFO:teuthology.orchestra.run.smithi040.stdout:Package librados2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:45.734 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:45.736 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:45.736 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:45.797 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install librbd1 2024-04-24T23:02:45.895 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:44 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:46.018 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 82 k 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-24T23:02:46.019 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:46.020 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 82 k 2024-04-24T23:02:46.020 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 222 k 2024-04-24T23:02:46.020 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:46.230 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-17.2.7-902.gc61472c8.el9.x86_64.rpm 391 kB/s | 82 kB 00:00 2024-04-24T23:02:46.231 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:46.231 INFO:teuthology.orchestra.run.smithi031.stdout:Total 389 kB/s | 82 kB 00:00 2024-04-24T23:02:46.231 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:46.238 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:46.238 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:46.307 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:46.307 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:46.327 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:47 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:46.409 INFO:teuthology.orchestra.run.smithi040.stdout:Package librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:46.452 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:46.453 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:46.454 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:46.455 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:46.514 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rados 2024-04-24T23:02:46.562 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:46.896 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:47.046 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:48 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:47.120 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:47.120 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:47.121 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:47.121 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:47.128 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:47.172 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:47.173 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:47.173 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:47.230 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rgw 2024-04-24T23:02:47.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-04-24T23:02:47.763 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:48 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:47.844 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:47.887 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:47.888 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:47.889 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:47.935 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:46 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:47.946 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-cephfs 2024-04-24T23:02:48.060 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.9 M 2024-04-24T23:02:48.061 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 12 M 2024-04-24T23:02:48.062 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:48.458 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-17.2.7-902.gc61472c8.el9.x86_64.rpm 7.4 MB/s | 2.9 MB 00:00 2024-04-24T23:02:48.459 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:48.459 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.4 MB/s | 2.9 MB 00:00 2024-04-24T23:02:48.459 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:48.470 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:48.470 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:48.484 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:49 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:48.553 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:48.553 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:48.566 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:48.608 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:48.609 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:48.610 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:48.671 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install python3-rbd 2024-04-24T23:02:48.985 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:49.043 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:49.068 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:02:49.068 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:02:49.068 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:02:49.068 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:02:49.068 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:49.205 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:50 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:49.284 INFO:teuthology.orchestra.run.smithi040.stdout:Package python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-24T23:02:49.328 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:49.329 INFO:teuthology.orchestra.run.smithi040.stdout:Nothing to do. 2024-04-24T23:02:49.329 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:49.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-fuse 2024-04-24T23:02:49.928 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:50 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:50.052 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 82 k 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 82 k 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 222 k 2024-04-24T23:02:50.053 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:50.059 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:50.060 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:50.060 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:50.060 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:50.060 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:50.060 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:50.274 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-fuse-17.2.7-902.gc61472c8.el9.x86_64.rpm 371 kB/s | 82 kB 00:00 2024-04-24T23:02:50.275 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:50.275 INFO:teuthology.orchestra.run.smithi040.stdout:Total 369 kB/s | 82 kB 00:00 2024-04-24T23:02:50.275 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:50.283 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:50.283 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:50.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-04-24T23:02:50.349 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:50.349 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:50.508 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:50.599 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:50.878 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:49 ago on Wed 24 Apr 2024 11:01:01 PM UTC. 2024-04-24T23:02:50.946 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:51.003 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-04-24T23:02:51.005 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:51.005 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-04-24T23:02:51.005 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:51.005 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-04-24T23:02:51.005 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 165 k 2024-04-24T23:02:51.006 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:51.006 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-04-24T23:02:51.006 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-04-24T23:02:51.006 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-04-24T23:02:51.006 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:51.006 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 165 k 2024-04-24T23:02:51.007 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 477 k 2024-04-24T23:02:51.007 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-04-24T23:02:51.199 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:51.199 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:51.199 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:51.199 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:51.199 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:51.199 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:51.230 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-17.2.7-902.gc61472c8.el9.x86_64.rpm 736 kB/s | 165 kB 00:00 2024-04-24T23:02:51.230 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:51.230 INFO:teuthology.orchestra.run.smithi031.stdout:Total 731 kB/s | 165 kB 00:00 2024-04-24T23:02:51.231 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-04-24T23:02:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-04-24T23:02:51.239 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-04-24T23:02:51.307 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-04-24T23:02:51.307 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-04-24T23:02:51.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-mirror 2024-04-24T23:02:51.460 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-04-24T23:02:51.561 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:51.944 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:52 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:52.068 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:52.068 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:52.068 INFO:teuthology.orchestra.run.smithi040.stdout: Package Arch Version Repo Size 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:52.069 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 2.9 M 2024-04-24T23:02:52.070 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 12 M 2024-04-24T23:02:52.070 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:52.222 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:52.223 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-04-24T23:02:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:52.223 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:02:52.223 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-04-24T23:02:52.428 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-mirror-17.2.7-902.gc61472c8.el9.x86_64.rpm 8.2 MB/s | 2.9 MB 00:00 2024-04-24T23:02:52.429 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:52.429 INFO:teuthology.orchestra.run.smithi040.stdout:Total 8.2 MB/s | 2.9 MB 00:00 2024-04-24T23:02:52.429 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:52.440 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:52.440 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:52.493 DEBUG:teuthology.parallel:result is None 2024-04-24T23:02:52.525 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:52.525 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:52.959 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:53.012 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:53.038 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:53.038 INFO:teuthology.orchestra.run.smithi040.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-24T23:02:53.039 INFO:teuthology.orchestra.run.smithi040.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-24T23:02:53.039 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:02:53.039 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-24T23:02:53.039 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:54.090 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:54.091 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:54.091 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:54.091 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:54.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo yum -y install rbd-nbd 2024-04-24T23:02:54.859 INFO:teuthology.orchestra.run.smithi040.stdout:Last metadata expiration check: 0:01:55 ago on Wed 24 Apr 2024 11:00:59 PM UTC. 2024-04-24T23:02:54.983 INFO:teuthology.orchestra.run.smithi040.stdout:Dependencies resolved. 2024-04-24T23:02:54.983 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:54.983 INFO:teuthology.orchestra.run.smithi040.stdout: Package Architecture Version Repository Size 2024-04-24T23:02:54.983 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:54.983 INFO:teuthology.orchestra.run.smithi040.stdout:Installing: 2024-04-24T23:02:54.984 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 165 k 2024-04-24T23:02:54.984 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:54.984 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction Summary 2024-04-24T23:02:54.984 INFO:teuthology.orchestra.run.smithi040.stdout:================================================================================ 2024-04-24T23:02:54.984 INFO:teuthology.orchestra.run.smithi040.stdout:Install 1 Package 2024-04-24T23:02:54.984 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:54.985 INFO:teuthology.orchestra.run.smithi040.stdout:Total download size: 165 k 2024-04-24T23:02:54.985 INFO:teuthology.orchestra.run.smithi040.stdout:Installed size: 477 k 2024-04-24T23:02:54.985 INFO:teuthology.orchestra.run.smithi040.stdout:Downloading Packages: 2024-04-24T23:02:55.207 INFO:teuthology.orchestra.run.smithi040.stdout:rbd-nbd-17.2.7-902.gc61472c8.el9.x86_64.rpm 743 kB/s | 165 kB 00:00 2024-04-24T23:02:55.208 INFO:teuthology.orchestra.run.smithi040.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:02:55.208 INFO:teuthology.orchestra.run.smithi040.stdout:Total 737 kB/s | 165 kB 00:00 2024-04-24T23:02:55.208 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction check 2024-04-24T23:02:55.216 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction check succeeded. 2024-04-24T23:02:55.217 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction test 2024-04-24T23:02:55.284 INFO:teuthology.orchestra.run.smithi040.stdout:Transaction test succeeded. 2024-04-24T23:02:55.284 INFO:teuthology.orchestra.run.smithi040.stdout:Running transaction 2024-04-24T23:02:55.440 INFO:teuthology.orchestra.run.smithi040.stdout: Preparing : 1/1 2024-04-24T23:02:55.539 INFO:teuthology.orchestra.run.smithi040.stdout: Installing : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:55.916 INFO:teuthology.orchestra.run.smithi040.stdout: Running scriptlet: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: Verifying : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-24T23:02:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:56.181 INFO:teuthology.orchestra.run.smithi040.stdout:Installed: 2024-04-24T23:02:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-24T23:02:56.181 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:02:56.182 INFO:teuthology.orchestra.run.smithi040.stdout:Complete! 2024-04-24T23:02:56.387 DEBUG:teuthology.parallel:result is None 2024-04-24T23:02:56.387 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:02:56.387 INFO:teuthology.packaging:ref: None 2024-04-24T23:02:56.388 INFO:teuthology.packaging:tag: None 2024-04-24T23:02:56.388 INFO:teuthology.packaging:branch: quincy 2024-04-24T23:02:56.388 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:02:56.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-24T23:02:56.610 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:02:56.636 INFO:teuthology.orchestra.run.smithi031.stdout:17.2.7-902.gc61472c8.el9 2024-04-24T23:02:56.637 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-902.gc61472c8.el9 2024-04-24T23:02:56.637 INFO:teuthology.task.install:The correct ceph version 17.2.7-902.gc61472c8 is installed. 2024-04-24T23:02:56.639 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:02:56.639 INFO:teuthology.packaging:ref: None 2024-04-24T23:02:56.639 INFO:teuthology.packaging:tag: None 2024-04-24T23:02:56.639 INFO:teuthology.packaging:branch: quincy 2024-04-24T23:02:56.640 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:02:56.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-24T23:02:56.944 DEBUG:teuthology.orchestra.run.smithi040:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T23:02:56.966 INFO:teuthology.orchestra.run.smithi040.stdout:17.2.7-902.gc61472c8.el9 2024-04-24T23:02:56.967 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-902.gc61472c8.el9 2024-04-24T23:02:56.967 INFO:teuthology.task.install:The correct ceph version 17.2.7-902.gc61472c8 is installed. 2024-04-24T23:02:56.969 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T23:02:56.969 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:02:56.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:02:57.000 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:02:57.000 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T23:02:57.033 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T23:02:57.033 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:02:57.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:02:57.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:02:57.138 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:02:57.138 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T23:02:57.180 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T23:02:57.246 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T23:02:57.247 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:02:57.247 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:02:57.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:02:57.346 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:02:57.346 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T23:02:57.372 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T23:02:57.438 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T23:02:57.438 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:02:57.438 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:02:57.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:02:57.537 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:02:57.537 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T23:02:57.561 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T23:02:57.626 INFO:teuthology.run_tasks:Running task print... 2024-04-24T23:02:57.636 INFO:teuthology.task.print:**** done install task... 2024-04-24T23:02:57.636 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-24T23:02:57.744 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-24T23:02:57.744 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-24T23:02:57.745 INFO:tasks.cephadm:Cluster fsid is ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:02:57.745 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-24T23:02:57.745 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-24T23:02:57.746 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi031': '172.21.15.31', 'mon.smithi040': '172.21.15.40'} 2024-04-24T23:02:57.746 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-24T23:02:57.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-04-24T23:02:57.776 DEBUG:teuthology.orchestra.run.smithi040:> sudo hostname $(hostname -s) 2024-04-24T23:02:57.801 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-24T23:02:57.801 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:02:58.056 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 24 23:02 /home/ubuntu/cephtest/cephadm 2024-04-24T23:02:58.056 DEBUG:teuthology.orchestra.run.smithi040:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-24T23:02:58.249 INFO:teuthology.orchestra.run.smithi040.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 24 23:02 /home/ubuntu/cephtest/cephadm 2024-04-24T23:02:58.250 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:02:58.270 DEBUG:teuthology.orchestra.run.smithi040:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-24T23:02:58.299 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-24T23:02:58.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-24T23:02:58.312 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-24T23:02:59.793 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-24T23:02:59.961 INFO:teuthology.orchestra.run.smithi040.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-24T23:03:30.885 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-24T23:03:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-24T23:03:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-24T23:03:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "repo_digests": [ 2024-04-24T23:03:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-24T23:03:30.886 INFO:teuthology.orchestra.run.smithi040.stdout: ] 2024-04-24T23:03:30.886 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-24T23:03:32.113 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:03:32.113 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-24T23:03:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-24T23:03:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-04-24T23:03:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-24T23:03:32.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-04-24T23:03:32.114 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:03:32.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-04-24T23:03:32.189 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph 2024-04-24T23:03:32.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-04-24T23:03:32.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /etc/ceph 2024-04-24T23:03:32.292 INFO:tasks.cephadm:Writing seed config... 2024-04-24T23:03:32.293 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-24T23:03:32.293 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-24T23:03:32.293 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-24T23:03:32.293 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-24T23:03:32.293 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-24T23:03:32.294 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-24T23:03:32.295 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-24T23:03:32.296 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-24T23:03:32.296 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-24T23:03:32.296 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-24T23:03:32.296 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-24T23:03:32.296 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-24T23:03:32.296 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-24T23:03:32.296 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-24T23:03:32.296 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-24T23:03:32.297 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:03:32.297 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-24T23:03:32.312 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ca7b26e4-028e-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-24T23:03:32.313 DEBUG:teuthology.orchestra.run.smithi031:mon.smithi031> sudo journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031.service 2024-04-24T23:03:32.354 INFO:tasks.cephadm:Bootstrapping... 2024-04-24T23:03:32.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:32.563 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:03:32.563 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'ca7b26e4-028e-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-04-24T23:03:32.590 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:03:32.590 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-24T23:03:32.591 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-04-24T23:03:32.615 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:03:32.615 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-04-24T23:03:32.615 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-04-24T23:03:32.623 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:03:32.623 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:03:32.630 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:03:32.630 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-24T23:03:32.637 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-24T23:03:32.643 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-24T23:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:03:32.644 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-04-24T23:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-24T23:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-24T23:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-04-24T23:03:32.668 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-04-24T23:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-24T23:03:32.675 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-24T23:03:32.681 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-24T23:03:32.682 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-24T23:03:32.688 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-04-24T23:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-04-24T23:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-04-24T23:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-04-24T23:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 139859015877968 on /run/cephadm/ca7b26e4-028e-11ef-bc93-c7b262605968.lock 2024-04-24T23:03:32.695 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 139859015877968 acquired on /run/cephadm/ca7b26e4-028e-11ef-bc93-c7b262605968.lock 2024-04-24T23:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-04-24T23:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-04-24T23:03:32.696 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-24T23:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-04-24T23:03:32.700 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-04-24T23:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-24T23:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-24T23:03:32.703 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-24T23:03:32.705 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-24T23:03:32.706 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-24T23:03:32.706 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:03:32.706 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-24T23:03:32.706 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-24T23:03:32.706 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-24T23:03:32.707 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-24T23:03:32.707 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-04-24T23:03:32.707 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-24T23:03:32.707 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-24T23:03:32.708 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-24T23:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-24T23:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-24T23:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-04-24T23:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:0e2af62137028b073c6ccdb8d4d7e0ccccc475ce5f6e225cb983e90813813045 2024-04-24T23:03:33.197 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-24T23:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-24T23:03:33.198 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-24T23:03:34.144 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-24T23:03:34.144 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-24T23:03:34.144 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-04-24T23:03:35.074 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-04-24T23:03:35.074 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-04-24T23:03:36.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQDHjylmeCyxFxAAyPuuXREFgYxtWlH47Q1JVw== 2024-04-24T23:03:36.931 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQDIjylmIu2oExAAzk1YBpmJ8GXror4ojqL6Uw== 2024-04-24T23:03:37.830 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQDJjylma67iDRAAN5n7H2By+g7SI15CmvYRkA== 2024-04-24T23:03:37.831 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-04-24T23:03:38.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:03:38.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-24T23:03:38.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:38.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:03:38.762 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for smithi031 [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-24T23:03:38.762 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = octopus 2024-04-24T23:03:38.762 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:38.763 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-24T23:03:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:03:38.763 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-04-24T23:03:39.659 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.smithi031 on 2024-04-24T23:03:39.883 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-24T23:03:40.029 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-24T23:03:40.192 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968.target. 2024-04-24T23:03:40.192 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968.target. 2024-04-24T23:03:40.377 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031 2024-04-24T23:03:40.377 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031.service: Unit ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031.service not loaded. 2024-04-24T23:03:40.547 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968.target.wants/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031.service → /etc/systemd/system/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@.service. 2024-04-24T23:03:41.143 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:03:41.144 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-24T23:03:41.152 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:03:41.152 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-24T23:03:41.152 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-24T23:03:41.152 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:03:41.153 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-04-24T23:03:41.153 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-04-24T23:03:41.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:41 smithi031 systemd[1]: Started Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:03:42.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-04-24T23:03:42.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:42.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-24T23:03:42.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:42.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-04-24T23:03:42.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi031 (age 0.575901s) 2024-04-24T23:03:42.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-24T23:03:42.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-24T23:03:42.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.561+0000 7f6ae0b51700 1 Processor -- start 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.561+0000 7f6ae0b51700 1 -- start start 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.561+0000 7f6ae0b51700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.561+0000 7f6ae0b51700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc079090 con 0x7f6adc078c80 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.561+0000 7f6ada59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.561+0000 7f6ada59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36980/0 (socket says 172.21.15.31:36980) 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.562+0000 7f6ada59c700 1 -- 172.21.15.31:0/2752024156 learned_addr learned my addr 172.21.15.31:0/2752024156 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.562+0000 7f6ada59c700 1 -- 172.21.15.31:0/2752024156 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc077770 con 0x7f6adc078c80 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.562+0000 7f6ada59c700 1 --2- 172.21.15.31:0/2752024156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc077230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009e50 tx=0x7f6acc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f64397f5c6645cd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.562+0000 7f6ad959a700 1 -- 172.21.15.31:0/2752024156 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6acc0092e0 con 0x7f6adc078c80 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.562+0000 7f6ad959a700 1 -- 172.21.15.31:0/2752024156 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc00ba20 con 0x7f6adc078c80 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.562+0000 7f6ad959a700 1 -- 172.21.15.31:0/2752024156 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6acc00bd30 con 0x7f6adc078c80 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 -- 172.21.15.31:0/2752024156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 msgr2=0x7f6adc077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 --2- 172.21.15.31:0/2752024156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc077230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009e50 tx=0x7f6acc009840 comp rx=0 tx=0).stop 2024-04-24T23:03:42.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 -- 172.21.15.31:0/2752024156 shutdown_connections 2024-04-24T23:03:42.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 --2- 172.21.15.31:0/2752024156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc077230 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 -- 172.21.15.31:0/2752024156 >> 172.21.15.31:0/2752024156 conn(0x7f6adc0fe060 msgr2=0x7f6adc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 -- 172.21.15.31:0/2752024156 shutdown_connections 2024-04-24T23:03:42.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 -- 172.21.15.31:0/2752024156 wait complete. 2024-04-24T23:03:42.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 Processor -- start 2024-04-24T23:03:42.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.563+0000 7f6ae0b51700 1 -- start start 2024-04-24T23:03:42.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ae0b51700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc113030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ae0b51700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc113570 con 0x7f6adc078c80 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ada59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc113030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ada59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc113030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36992/0 (socket says 172.21.15.31:36992) 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ada59c700 1 -- 172.21.15.31:0/1704365721 learned_addr learned my addr 172.21.15.31:0/1704365721 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ada59c700 1 -- 172.21.15.31:0/1704365721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc009ae0 con 0x7f6adc078c80 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ada59c700 1 --2- 172.21.15.31:0/1704365721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc113030 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6acc003000 tx=0x7f6acc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/1704365721 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6acc012b90 con 0x7f6adc078c80 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/1704365721 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc012d10 con 0x7f6adc078c80 2024-04-24T23:03:42.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc113770 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/1704365721 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6acc01e400 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1142e0 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/1704365721 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f6acc019030 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.564+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/1704365721 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f6acc026400 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.565+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc05f5c0 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.567+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/1704365721 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f6acc017070 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.639+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f6adc04bed0 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.640+0000 7f6ad2ffd700 1 -- 172.21.15.31:0/1704365721 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f6acc01ea00 con 0x7f6adc078c80 2024-04-24T23:03:42.208 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.640+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 msgr2=0x7f6adc113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:42.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.640+0000 7f6ae0b51700 1 --2- 172.21.15.31:0/1704365721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc113030 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6acc003000 tx=0x7f6acc017000 comp rx=0 tx=0).stop 2024-04-24T23:03:42.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.640+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 shutdown_connections 2024-04-24T23:03:42.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.640+0000 7f6ae0b51700 1 --2- 172.21.15.31:0/1704365721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6adc078c80 0x7f6adc113030 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:42.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.640+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 >> 172.21.15.31:0/1704365721 conn(0x7f6adc0fe060 msgr2=0x7f6adc109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:42.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.640+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 shutdown_connections 2024-04-24T23:03:42.209 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:41.641+0000 7f6ae0b51700 1 -- 172.21.15.31:0/1704365721 wait complete. 2024-04-24T23:03:42.209 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-04-24T23:03:42.209 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-04-24T23:03:42.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:42 smithi031 ceph-mon[26040]: mon.smithi031 is new leader, mons smithi031 in quorum (ranks 0) 2024-04-24T23:03:42.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:42 smithi031 ceph-mon[26040]: monmap e1: 1 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-24T23:03:42.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:42 smithi031 ceph-mon[26040]: fsmap 2024-04-24T23:03:42.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:42 smithi031 ceph-mon[26040]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:03:42.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:42 smithi031 ceph-mon[26040]: mgrmap e1: no daemons active 2024-04-24T23:03:42.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:42 smithi031 ceph-mon[26040]: from='client.? 172.21.15.31:0/1704365721' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-24T23:03:43.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:43.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:03:43.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:43.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-04-24T23:03:43.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:03:43.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.621+0000 7f535cb06700 1 Processor -- start 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.621+0000 7f535cb06700 1 -- start start 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.621+0000 7f535cb06700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358102c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.621+0000 7f535cb06700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53581031a0 con 0x7f5358102850 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.622+0000 7f535659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.622+0000 7f535659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37000/0 (socket says 172.21.15.31:37000) 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.622+0000 7f535659c700 1 -- 172.21.15.31:0/4099361432 learned_addr learned my addr 172.21.15.31:0/4099361432 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.622+0000 7f535659c700 1 -- 172.21.15.31:0/4099361432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53581032e0 con 0x7f5358102850 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.622+0000 7f535659c700 1 --2- 172.21.15.31:0/4099361432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358102c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5350009e50 tx=0x7f5350009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2ea0588d8b656e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.622+0000 7f535559a700 1 -- 172.21.15.31:0/4099361432 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53500092e0 con 0x7f5358102850 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.622+0000 7f535559a700 1 -- 172.21.15.31:0/4099361432 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535000ba20 con 0x7f5358102850 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.622+0000 7f535559a700 1 -- 172.21.15.31:0/4099361432 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f535000bd30 con 0x7f5358102850 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.623+0000 7f535cb06700 1 -- 172.21.15.31:0/4099361432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 msgr2=0x7f5358102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.623+0000 7f535cb06700 1 --2- 172.21.15.31:0/4099361432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358102c60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5350009e50 tx=0x7f5350009840 comp rx=0 tx=0).stop 2024-04-24T23:03:43.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.623+0000 7f535cb06700 1 -- 172.21.15.31:0/4099361432 shutdown_connections 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.623+0000 7f535cb06700 1 --2- 172.21.15.31:0/4099361432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358102c60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.623+0000 7f535cb06700 1 -- 172.21.15.31:0/4099361432 >> 172.21.15.31:0/4099361432 conn(0x7f53580fdff0 msgr2=0x7f5358100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.623+0000 7f535cb06700 1 -- 172.21.15.31:0/4099361432 shutdown_connections 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.623+0000 7f535cb06700 1 -- 172.21.15.31:0/4099361432 wait complete. 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.623+0000 7f535cb06700 1 Processor -- start 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535cb06700 1 -- start start 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535cb06700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535cb06700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358078050 con 0x7f5358102850 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37004/0 (socket says 172.21.15.31:37004) 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535659c700 1 -- 172.21.15.31:0/1785140397 learned_addr learned my addr 172.21.15.31:0/1785140397 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535659c700 1 -- 172.21.15.31:0/1785140397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5350009ae0 con 0x7f5358102850 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535659c700 1 --2- 172.21.15.31:0/1785140397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358077b10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5358103590 tx=0x7f5350017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:43.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f534effd700 1 -- 172.21.15.31:0/1785140397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5350012b90 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f534effd700 1 -- 172.21.15.31:0/1785140397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350012d10 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f534effd700 1 -- 172.21.15.31:0/1785140397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f535001e400 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.624+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5358076120 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.625+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5358076610 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.625+0000 7f534effd700 1 -- 172.21.15.31:0/1785140397 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5350019030 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.625+0000 7f534effd700 1 -- 172.21.15.31:0/1785140397 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5350026400 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.626+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535804bed0 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.628+0000 7f534effd700 1 -- 172.21.15.31:0/1785140397 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f5350017070 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.697+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5358103b70 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.701+0000 7f534effd700 1 -- 172.21.15.31:0/1785140397 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f535001e9a0 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.701+0000 7f534effd700 1 -- 172.21.15.31:0/1785140397 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f5350026610 con 0x7f5358102850 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.702+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 msgr2=0x7f5358077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:43.206 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.702+0000 7f535cb06700 1 --2- 172.21.15.31:0/1785140397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358077b10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5358103590 tx=0x7f5350017000 comp rx=0 tx=0).stop 2024-04-24T23:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.702+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 shutdown_connections 2024-04-24T23:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.702+0000 7f535cb06700 1 --2- 172.21.15.31:0/1785140397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5358102850 0x7f5358077b10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.702+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 >> 172.21.15.31:0/1785140397 conn(0x7f53580fdff0 msgr2=0x7f53581052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.702+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 shutdown_connections 2024-04-24T23:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:42.702+0000 7f535cb06700 1 -- 172.21.15.31:0/1785140397 wait complete. 2024-04-24T23:03:43.207 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-04-24T23:03:43.318 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:43 smithi031 ceph-mon[26040]: from='client.? 172.21.15.31:0/1785140397' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:03:43.318 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:43 smithi031 ceph-mon[26040]: from='client.? 172.21.15.31:0/1785140397' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-24T23:03:44.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.644+0000 7fe667842700 1 Processor -- start 2024-04-24T23:03:44.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.644+0000 7fe667842700 1 -- start start 2024-04-24T23:03:44.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.644+0000 7fe667842700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.644+0000 7fe667842700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660105980 con 0x7fe660076c10 2024-04-24T23:03:44.259 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.644+0000 7fe6655de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.644+0000 7fe6655de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37014/0 (socket says 172.21.15.31:37014) 2024-04-24T23:03:44.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.644+0000 7fe6655de700 1 -- 172.21.15.31:0/4000598545 learned_addr learned my addr 172.21.15.31:0/4000598545 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:44.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.645+0000 7fe6655de700 1 -- 172.21.15.31:0/4000598545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe660105ac0 con 0x7fe660076c10 2024-04-24T23:03:44.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.645+0000 7fe6655de700 1 --2- 172.21.15.31:0/4000598545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660105440 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0098e0 tx=0x7fe65c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c0d846ba6d4c03 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.645+0000 7fe657fff700 1 -- 172.21.15.31:0/4000598545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe65c00b860 con 0x7fe660076c10 2024-04-24T23:03:44.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.645+0000 7fe657fff700 1 -- 172.21.15.31:0/4000598545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe65c00be50 con 0x7fe660076c10 2024-04-24T23:03:44.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.645+0000 7fe657fff700 1 -- 172.21.15.31:0/4000598545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe65c010380 con 0x7fe660076c10 2024-04-24T23:03:44.260 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.645+0000 7fe667842700 1 -- 172.21.15.31:0/4000598545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 msgr2=0x7fe660105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.645+0000 7fe667842700 1 --2- 172.21.15.31:0/4000598545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660105440 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0098e0 tx=0x7fe65c009250 comp rx=0 tx=0).stop 2024-04-24T23:03:44.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.645+0000 7fe667842700 1 -- 172.21.15.31:0/4000598545 shutdown_connections 2024-04-24T23:03:44.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.646+0000 7fe667842700 1 --2- 172.21.15.31:0/4000598545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660105440 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.646+0000 7fe667842700 1 -- 172.21.15.31:0/4000598545 >> 172.21.15.31:0/4000598545 conn(0x7fe6600fcef0 msgr2=0x7fe6600ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:44.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.646+0000 7fe667842700 1 -- 172.21.15.31:0/4000598545 shutdown_connections 2024-04-24T23:03:44.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.646+0000 7fe667842700 1 -- 172.21.15.31:0/4000598545 wait complete. 2024-04-24T23:03:44.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.646+0000 7fe667842700 1 Processor -- start 2024-04-24T23:03:44.261 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.646+0000 7fe667842700 1 -- start start 2024-04-24T23:03:44.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe667842700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660113790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:44.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe667842700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660113cd0 con 0x7fe660076c10 2024-04-24T23:03:44.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe6655de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660113790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:44.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe6655de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660113790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37026/0 (socket says 172.21.15.31:37026) 2024-04-24T23:03:44.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe6655de700 1 -- 172.21.15.31:0/1417910068 learned_addr learned my addr 172.21.15.31:0/1417910068 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:44.262 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe6655de700 1 -- 172.21.15.31:0/1417910068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0094f0 con 0x7fe660076c10 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe6655de700 1 --2- 172.21.15.31:0/1417910068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660113790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006b00 tx=0x7fe65c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe655ffb700 1 -- 172.21.15.31:0/1417910068 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe65c0107c0 con 0x7fe660076c10 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe655ffb700 1 -- 172.21.15.31:0/1417910068 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fe65c010db0 con 0x7fe660076c10 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe660113f30 con 0x7fe660076c10 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.647+0000 7fe655ffb700 1 -- 172.21.15.31:0/1417910068 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe65c01acf0 con 0x7fe660076c10 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.648+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe660114a40 con 0x7fe660076c10 2024-04-24T23:03:44.263 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.648+0000 7fe655ffb700 1 -- 172.21.15.31:0/1417910068 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fe65c024440 con 0x7fe660076c10 2024-04-24T23:03:44.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.648+0000 7fe655ffb700 1 -- 172.21.15.31:0/1417910068 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe65c019710 con 0x7fe660076c10 2024-04-24T23:03:44.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.649+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66005f5c0 con 0x7fe660076c10 2024-04-24T23:03:44.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.651+0000 7fe655ffb700 1 -- 172.21.15.31:0/1417910068 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fe65c042790 con 0x7fe660076c10 2024-04-24T23:03:44.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe660106810 con 0x7fe660076c10 2024-04-24T23:03:44.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.721+0000 7fe655ffb700 1 -- 172.21.15.31:0/1417910068 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fe65c031030 con 0x7fe660076c10 2024-04-24T23:03:44.264 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 msgr2=0x7fe660113790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:44.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7fe667842700 1 --2- 172.21.15.31:0/1417910068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660113790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe65c006b00 tx=0x7fe65c009250 comp rx=0 tx=0).stop 2024-04-24T23:03:44.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 shutdown_connections 2024-04-24T23:03:44.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7fe667842700 1 --2- 172.21.15.31:0/1417910068 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe660076c10 0x7fe660113790 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:44.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 >> 172.21.15.31:0/1417910068 conn(0x7fe6600fcef0 msgr2=0x7fe6600fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:44.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 shutdown_connections 2024-04-24T23:03:44.265 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:43.722+0000 7fe667842700 1 -- 172.21.15.31:0/1417910068 wait complete. 2024-04-24T23:03:44.265 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-04-24T23:03:44.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:44 smithi031 ceph-mon[26040]: from='client.? 172.21.15.31:0/1417910068' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:03:44.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:44 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:03:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:44 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[26036]: 2024-04-24T23:03:44.536+0000 7f0f6ed73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:03:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:44 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[26036]: 2024-04-24T23:03:44.536+0000 7f0f6ed73700 -1 mon.smithi031@0(leader) e1 *** Got Signal Terminated *** 2024-04-24T23:03:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:44 smithi031 podman[26285]: 2024-04-24 23:03:44.762305277 +0000 UTC m=+0.341684900 container died 604cd8a90dc3c0e99832fef0abd170564b18e1183a0f080f7b6c9f00bf338eff (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=quincy-c61472c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:03:45.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 podman[26285]: 2024-04-24 23:03:45.111355235 +0000 UTC m=+0.690734861 container cleanup 604cd8a90dc3c0e99832fef0abd170564b18e1183a0f080f7b6c9f00bf338eff (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-24T23:03:45.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 bash[26285]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031 2024-04-24T23:03:45.739 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 podman[26299]: 2024-04-24 23:03:45.370489148 +0000 UTC m=+0.610916492 container remove 604cd8a90dc3c0e99832fef0abd170564b18e1183a0f080f7b6c9f00bf338eff (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-24T23:03:45.739 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031.service: Deactivated successfully. 2024-04-24T23:03:45.739 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 systemd[1]: Stopped Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:03:45.739 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 systemd[1]: Starting Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:03:45.739 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 podman[26393]: 2024-04-24 23:03:45.738115171 +0000 UTC m=+0.093240909 container create 8572f02a7ab3ac176a810a34fd394c693afdf17d552f3a87f49b7d919f5e1957 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, RELEASE=quincy-c61472c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:03:46.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 podman[26393]: 2024-04-24 23:03:45.66347769 +0000 UTC m=+0.018603447 image pull 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-24T23:03:46.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 podman[26393]: 2024-04-24 23:03:45.932175288 +0000 UTC m=+0.287301026 container init 8572f02a7ab3ac176a810a34fd394c693afdf17d552f3a87f49b7d919f5e1957 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-24T23:03:46.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 podman[26393]: 2024-04-24 23:03:45.935081099 +0000 UTC m=+0.290206837 container start 8572f02a7ab3ac176a810a34fd394c693afdf17d552f3a87f49b7d919f5e1957 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable), process ceph-mon, pid 2 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: pidfile_write: ignore empty --pid-file 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: load: jerasure load: lrc 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: RocksDB version: 6.15.5 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Compile date Apr 24 2024 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: DB SUMMARY 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: DB Session ID: OVC32E7RDS0SE7K1M9MC 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi031/store.db dir, Total Num: 1, files: 000008.sst 2024-04-24T23:03:46.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi031/store.db: 000010.log size: 98164 ; 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.env: 0x562f22fe9b00 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.fs: Posix File System 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.info_log: 0x562f25745620 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.statistics: (nil) 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.use_fsync: 0 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:03:46.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.db_log_dir: 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi031/store.db 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:03:46.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.write_buffer_manager: 0x562f26438a20 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:03:46.004 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:03:46.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.unordered_write: 0 2024-04-24T23:03:46.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:03:46.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:03:46.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:03:46.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:03:46.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.row_cache: None 2024-04-24T23:03:46.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.wal_filter: None 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.preserve_deletes: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:03:46.007 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:03:46.008 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_open_files: -1 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Compression algorithms supported: 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: kZSTD supported: 0 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:03:46.009 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000009 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.merge_operator: 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_filter: None 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:03:46.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562f25718f58) 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_type: 0 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_index_type: 0 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_shortening: 1 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: hash_index_allow_collision: 1 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: checksum: 1 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: no_block_cache: 0 2024-04-24T23:03:46.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache: 0x562f2574b090 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_options: 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: capacity : 536870912 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_shard_bits : 4 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: strict_capacity_limit : 0 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_compressed: (nil) 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: persistent_cache: (nil) 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size: 4096 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size_deviation: 10 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_restart_interval: 16 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_block_restart_interval: 1 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_block_size: 4096 2024-04-24T23:03:46.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: partition_filters: 0 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: use_delta_encoding: 1 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: whole_key_filtering: 1 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: verify_compression: 0 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: format_version: 4 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_index_compression: 1 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_align: 0 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compression: NoCompression 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:03:46.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.num_levels: 7 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:03:46.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:03:46.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.arena_block_size: 4194304 2024-04-24T23:03:46.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:03:46.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:03:46.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-24T23:03:46.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:03:46.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:03:46.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:03:46.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:03:46.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:03:46.017 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:03:46.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:03:46.019 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:03:46.019 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:03:46.019 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:03:46.019 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.ttl: 2592000 2024-04-24T23:03:46.019 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:03:46.019 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999825961802, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-24T23:03:46.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999825962740, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93370, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91482, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84046, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1713999825, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "b0de33d7-6780-4b15-8538-06e0d36070d3", "db_session_id": "OVC32E7RDS0SE7K1M9MC"}} 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999825963074, "job": 1, "event": "recovery_finished"} 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi031/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x562f256df180 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: DB pointer 0x562f257b4000 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** DB Stats ** 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:03:46.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.1 0.00 0.00 1 0.001 0 0 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.1 0.00 0.00 1 0.001 0 0 2024-04-24T23:03:46.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.1 0.00 0.00 1 0.001 0 0 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.1 0.00 0.00 1 0.001 0 0 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:03:46.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative compaction: 0.00 GB write, 27.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval compaction: 0.00 GB write, 27.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.1 0.00 0.00 1 0.001 0 0 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.1 0.00 0.00 1 0.001 0 0 2024-04-24T23:03:46.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-24T23:03:46.025 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:03:46.025 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-24T23:03:46.025 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T23:03:46.025 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.1 0.00 0.00 1 0.001 0 0 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative compaction: 0.00 GB write, 27.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:03:46.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: starting mon.smithi031 rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi031 fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: mon.smithi031@-1(???) e1 preinit fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: mon.smithi031@-1(???).mds e1 new map 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: mon.smithi031@-1(???).mds e1 print_map 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: e1 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:03:46.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: legacy client fscid: -1 2024-04-24T23:03:46.028 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:03:46.028 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: No filesystems configured 2024-04-24T23:03:46.028 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: mon.smithi031@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:03:46.028 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:03:46.028 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:03:46.028 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:03:46.028 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:45 smithi031 ceph-mon[26411]: mon.smithi031@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-24T23:03:46.029 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-24T23:03:46.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:46 smithi031 bash[26393]: 8572f02a7ab3ac176a810a34fd394c693afdf17d552f3a87f49b7d919f5e1957 2024-04-24T23:03:46.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:46 smithi031 systemd[1]: Started Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:03:46.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:46 smithi031 ceph-mon[26411]: mon.smithi031 is new leader, mons smithi031 in quorum (ranks 0) 2024-04-24T23:03:46.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:46 smithi031 ceph-mon[26411]: monmap e1: 1 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-24T23:03:46.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:46 smithi031 ceph-mon[26411]: fsmap 2024-04-24T23:03:46.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:46 smithi031 ceph-mon[26411]: osdmap e1: 0 total, 0 up, 0 in 2024-04-24T23:03:46.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:46 smithi031 ceph-mon[26411]: mgrmap e1: no daemons active 2024-04-24T23:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.461+0000 7f5a1b740700 1 Processor -- start 2024-04-24T23:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.461+0000 7f5a1b740700 1 -- start start 2024-04-24T23:03:47.030 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.462+0000 7f5a1b740700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a14105410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.462+0000 7f5a1b740700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14105950 con 0x7f5a14071310 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.462+0000 7f5a194dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a14105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.462+0000 7f5a194dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a14105410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37036/0 (socket says 172.21.15.31:37036) 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.462+0000 7f5a194dc700 1 -- 172.21.15.31:0/3432700422 learned_addr learned my addr 172.21.15.31:0/3432700422 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.462+0000 7f5a194dc700 1 -- 172.21.15.31:0/3432700422 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a14105a90 con 0x7f5a14071310 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.462+0000 7f5a194dc700 1 --2- 172.21.15.31:0/3432700422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a14105410 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5a100098e0 tx=0x7f5a10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0416885cb683f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a0bfff700 1 -- 172.21.15.31:0/3432700422 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a1000b860 con 0x7f5a14071310 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a0bfff700 1 -- 172.21.15.31:0/3432700422 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5a1000be50 con 0x7f5a14071310 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a0bfff700 1 -- 172.21.15.31:0/3432700422 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a10010380 con 0x7f5a14071310 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a1b740700 1 -- 172.21.15.31:0/3432700422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 msgr2=0x7f5a14105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a1b740700 1 --2- 172.21.15.31:0/3432700422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a14105410 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5a100098e0 tx=0x7f5a10009250 comp rx=0 tx=0).stop 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a1b740700 1 -- 172.21.15.31:0/3432700422 shutdown_connections 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a1b740700 1 --2- 172.21.15.31:0/3432700422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a14105410 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.031 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a1b740700 1 -- 172.21.15.31:0/3432700422 >> 172.21.15.31:0/3432700422 conn(0x7f5a140fcec0 msgr2=0x7f5a140ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a1b740700 1 -- 172.21.15.31:0/3432700422 shutdown_connections 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.463+0000 7f5a1b740700 1 -- 172.21.15.31:0/3432700422 wait complete. 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.464+0000 7f5a1b740700 1 Processor -- start 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.464+0000 7f5a1b740700 1 -- start start 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.464+0000 7f5a1b740700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a141115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.464+0000 7f5a1b740700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14111b20 con 0x7f5a14071310 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.464+0000 7f5a194dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a141115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.464+0000 7f5a194dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a141115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37046/0 (socket says 172.21.15.31:37046) 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.464+0000 7f5a194dc700 1 -- 172.21.15.31:0/4215847153 learned_addr learned my addr 172.21.15.31:0/4215847153 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a194dc700 1 -- 172.21.15.31:0/4215847153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a100094f0 con 0x7f5a14071310 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a194dc700 1 --2- 172.21.15.31:0/4215847153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a141115e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5a10006b00 tx=0x7f5a10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a09ffb700 1 -- 172.21.15.31:0/4215847153 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a100107c0 con 0x7f5a14071310 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a09ffb700 1 -- 172.21.15.31:0/4215847153 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f5a10010db0 con 0x7f5a14071310 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a14111d20 con 0x7f5a14071310 2024-04-24T23:03:47.032 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a09ffb700 1 -- 172.21.15.31:0/4215847153 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a1001acf0 con 0x7f5a14071310 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a14112890 con 0x7f5a14071310 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a09ffb700 1 -- 172.21.15.31:0/4215847153 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5a10024440 con 0x7f5a14071310 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.465+0000 7f5a09ffb700 1 -- 172.21.15.31:0/4215847153 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5a10019710 con 0x7f5a14071310 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.466+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1405f5c0 con 0x7f5a14071310 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.468+0000 7f5a09ffb700 1 -- 172.21.15.31:0/4215847153 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f5a1001a530 con 0x7f5a14071310 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.537+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f5a14106840 con 0x7f5a14071310 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.539+0000 7f5a09ffb700 1 -- 172.21.15.31:0/4215847153 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5a10010920 con 0x7f5a14071310 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.540+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 msgr2=0x7f5a141115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.540+0000 7f5a1b740700 1 --2- 172.21.15.31:0/4215847153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a141115e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5a10006b00 tx=0x7f5a10009250 comp rx=0 tx=0).stop 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.540+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 shutdown_connections 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.540+0000 7f5a1b740700 1 --2- 172.21.15.31:0/4215847153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a14071310 0x7f5a141115e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.540+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 >> 172.21.15.31:0/4215847153 conn(0x7f5a140fcec0 msgr2=0x7f5a140fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.540+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 shutdown_connections 2024-04-24T23:03:47.033 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:46.540+0000 7f5a1b740700 1 -- 172.21.15.31:0/4215847153 wait complete. 2024-04-24T23:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-24T23:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-04-24T23:03:47.034 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 9283 ... 2024-04-24T23:03:47.211 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mgr.smithi031.rrnmwr 2024-04-24T23:03:47.212 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mgr.smithi031.rrnmwr.service: Unit ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mgr.smithi031.rrnmwr.service not loaded. 2024-04-24T23:03:47.364 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968.target.wants/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mgr.smithi031.rrnmwr.service → /etc/systemd/system/ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@.service. 2024-04-24T23:03:47.718 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:47 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/4215847153' entity='client.admin' 2024-04-24T23:03:47.923 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:03:47.923 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-24T23:03:47.932 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:03:47.932 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-24T23:03:47.932 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-24T23:03:47.932 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-04-24T23:03:47.941 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:03:47.942 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-24T23:03:47.950 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:03:47.950 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-24T23:03:47.950 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-24T23:03:47.950 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-24T23:03:47.950 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-04-24T23:03:47.950 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-04-24T23:03:48.680 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:48 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/383370664' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:03:48.985 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "ca7b26e4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:03:48.988 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:03:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:03:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:03:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-04-24T23:03:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-24T23:03:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:03:48.989 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:48.990 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-24T23:03:48.990 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:03:48.990 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:48.990 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:03:48.990 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:48.990 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:03:48.990 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:03:48.990 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:03:48.998 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:03:48.998 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:03:48.998 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:03:48.998 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:48.998 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:03:48.999 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:03:41.066240+0000", 2024-04-24T23:03:49.000 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.394+0000 7fa99dc9c700 1 Processor -- start 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.394+0000 7fa99dc9c700 1 -- start start 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.394+0000 7fa99dc9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.395+0000 7fa99dc9c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa998102090 con 0x7fa998101740 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.395+0000 7fa9977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.395+0000 7fa9977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37076/0 (socket says 172.21.15.31:37076) 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.395+0000 7fa9977fe700 1 -- 172.21.15.31:0/794318390 learned_addr learned my addr 172.21.15.31:0/794318390 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.395+0000 7fa9977fe700 1 -- 172.21.15.31:0/794318390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9981021d0 con 0x7fa998101740 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.395+0000 7fa9977fe700 1 --2- 172.21.15.31:0/794318390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998101b50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098e0 tx=0x7fa988009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7757362f9ff1bae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa996ffd700 1 -- 172.21.15.31:0/794318390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa98800b860 con 0x7fa998101740 2024-04-24T23:03:49.001 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa996ffd700 1 -- 172.21.15.31:0/794318390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa98800be50 con 0x7fa998101740 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa996ffd700 1 -- 172.21.15.31:0/794318390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa988010380 con 0x7fa998101740 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa99dc9c700 1 -- 172.21.15.31:0/794318390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 msgr2=0x7fa998101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa99dc9c700 1 --2- 172.21.15.31:0/794318390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998101b50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa9880098e0 tx=0x7fa988009250 comp rx=0 tx=0).stop 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa99dc9c700 1 -- 172.21.15.31:0/794318390 shutdown_connections 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa99dc9c700 1 --2- 172.21.15.31:0/794318390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998101b50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa99dc9c700 1 -- 172.21.15.31:0/794318390 >> 172.21.15.31:0/794318390 conn(0x7fa9980fcec0 msgr2=0x7fa9980ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa99dc9c700 1 -- 172.21.15.31:0/794318390 shutdown_connections 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.396+0000 7fa99dc9c700 1 -- 172.21.15.31:0/794318390 wait complete. 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.397+0000 7fa99dc9c700 1 Processor -- start 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.397+0000 7fa99dc9c700 1 -- start start 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.397+0000 7fa99dc9c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.397+0000 7fa99dc9c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa998113cc0 con 0x7fa998101740 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.397+0000 7fa9977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:49.002 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.397+0000 7fa9977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37088/0 (socket says 172.21.15.31:37088) 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.397+0000 7fa9977fe700 1 -- 172.21.15.31:0/383370664 learned_addr learned my addr 172.21.15.31:0/383370664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.398+0000 7fa9977fe700 1 -- 172.21.15.31:0/383370664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9880094f0 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.398+0000 7fa9977fe700 1 --2- 172.21.15.31:0/383370664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998113780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa988000c00 tx=0x7fa988009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.398+0000 7fa994ff9700 1 -- 172.21.15.31:0/383370664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9880107c0 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.398+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa998113ec0 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.398+0000 7fa994ff9700 1 -- 172.21.15.31:0/383370664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fa988010db0 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.398+0000 7fa994ff9700 1 -- 172.21.15.31:0/383370664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa98801ad00 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.398+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa998114a30 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.399+0000 7fa994ff9700 1 -- 172.21.15.31:0/383370664 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa988024440 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.399+0000 7fa994ff9700 1 -- 172.21.15.31:0/383370664 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa988019710 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.399+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99805f5c0 con 0x7fa998101740 2024-04-24T23:03:49.003 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.401+0000 7fa994ff9700 1 -- 172.21.15.31:0/383370664 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fa988042790 con 0x7fa998101740 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.474+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa998103510 con 0x7fa998101740 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.475+0000 7fa994ff9700 1 -- 172.21.15.31:0/383370664 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fa988031020 con 0x7fa998101740 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.476+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 msgr2=0x7fa998113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.476+0000 7fa99dc9c700 1 --2- 172.21.15.31:0/383370664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998113780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa988000c00 tx=0x7fa988009250 comp rx=0 tx=0).stop 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.476+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 shutdown_connections 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.476+0000 7fa99dc9c700 1 --2- 172.21.15.31:0/383370664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa998101740 0x7fa998113780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.476+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 >> 172.21.15.31:0/383370664 conn(0x7fa9980fcec0 msgr2=0x7fa998105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.476+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 shutdown_connections 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:48.476+0000 7fa99dc9c700 1 -- 172.21.15.31:0/383370664 wait complete. 2024-04-24T23:03:49.004 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-04-24T23:03:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:51 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/4006974123' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:03:52.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:52.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:52.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "ca7b26e4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:03:52.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:03:52.034 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:52.035 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-24T23:03:52.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:03:52.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:52.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:03:52.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:52.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:03:52.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:03:52.036 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:03:52.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:03:52.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:03:41.066240+0000", 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:52.046 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.414+0000 7f41ee02e700 1 Processor -- start 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.414+0000 7f41ee02e700 1 -- start start 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.414+0000 7f41ee02e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.414+0000 7f41ee02e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8071660 con 0x7f41e8071250 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.415+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.415+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81054d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39324/0 (socket says 172.21.15.31:39324) 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.415+0000 7f41e77fe700 1 -- 172.21.15.31:0/3308116110 learned_addr learned my addr 172.21.15.31:0/3308116110 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.415+0000 7f41e77fe700 1 -- 172.21.15.31:0/3308116110 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e8105a10 con 0x7f41e8071250 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.415+0000 7f41e77fe700 1 --2- 172.21.15.31:0/3308116110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81054d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41d80098e0 tx=0x7f41d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd15cdab908feafd server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.415+0000 7f41e67fc700 1 -- 172.21.15.31:0/3308116110 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41d800b860 con 0x7f41e8071250 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.415+0000 7f41e67fc700 1 -- 172.21.15.31:0/3308116110 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f41d800be50 con 0x7f41e8071250 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.415+0000 7f41e67fc700 1 -- 172.21.15.31:0/3308116110 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41d8010380 con 0x7f41e8071250 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.416+0000 7f41ee02e700 1 -- 172.21.15.31:0/3308116110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 msgr2=0x7f41e81054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.047 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.416+0000 7f41ee02e700 1 --2- 172.21.15.31:0/3308116110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81054d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f41d80098e0 tx=0x7f41d8009250 comp rx=0 tx=0).stop 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.416+0000 7f41ee02e700 1 -- 172.21.15.31:0/3308116110 shutdown_connections 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.416+0000 7f41ee02e700 1 --2- 172.21.15.31:0/3308116110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81054d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.416+0000 7f41ee02e700 1 -- 172.21.15.31:0/3308116110 >> 172.21.15.31:0/3308116110 conn(0x7f41e80fcec0 msgr2=0x7f41e80ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.416+0000 7f41ee02e700 1 -- 172.21.15.31:0/3308116110 shutdown_connections 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.416+0000 7f41ee02e700 1 -- 172.21.15.31:0/3308116110 wait complete. 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.417+0000 7f41ee02e700 1 Processor -- start 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.417+0000 7f41ee02e700 1 -- start start 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.417+0000 7f41ee02e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.417+0000 7f41ee02e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e8111b00 con 0x7f41e8071250 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.417+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.417+0000 7f41e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39326/0 (socket says 172.21.15.31:39326) 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.417+0000 7f41e77fe700 1 -- 172.21.15.31:0/4006974123 learned_addr learned my addr 172.21.15.31:0/4006974123 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41e77fe700 1 -- 172.21.15.31:0/4006974123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80094f0 con 0x7f41e8071250 2024-04-24T23:03:52.048 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41e77fe700 1 --2- 172.21.15.31:0/4006974123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81115c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f41d8006b00 tx=0x7f41d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41cffff700 1 -- 172.21.15.31:0/4006974123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41d80107c0 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41cffff700 1 -- 172.21.15.31:0/4006974123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f41d8010db0 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e8111d00 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41cffff700 1 -- 172.21.15.31:0/4006974123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41d801acf0 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e8112870 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41cffff700 1 -- 172.21.15.31:0/4006974123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f41d8024440 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.418+0000 7f41cffff700 1 -- 172.21.15.31:0/4006974123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f41d8019760 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.419+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e805f5c0 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.422+0000 7f41cffff700 1 -- 172.21.15.31:0/4006974123 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f41d8042790 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.496+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f41e804bed0 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.497+0000 7f41cffff700 1 -- 172.21.15.31:0/4006974123 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f41d8031020 con 0x7f41e8071250 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.498+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 msgr2=0x7f41e81115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.498+0000 7f41ee02e700 1 --2- 172.21.15.31:0/4006974123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81115c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f41d8006b00 tx=0x7f41d8009250 comp rx=0 tx=0).stop 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.498+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 shutdown_connections 2024-04-24T23:03:52.049 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.498+0000 7f41ee02e700 1 --2- 172.21.15.31:0/4006974123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41e8071250 0x7f41e81115c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:52.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.498+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 >> 172.21.15.31:0/4006974123 conn(0x7f41e80fcec0 msgr2=0x7f41e80fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:52.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.499+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 shutdown_connections 2024-04-24T23:03:52.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:51.499+0000 7f41ee02e700 1 -- 172.21.15.31:0/4006974123 wait complete. 2024-04-24T23:03:52.050 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-04-24T23:03:54.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:54 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1767163583' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:03:55.126 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "ca7b26e4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:55.127 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:03:55.128 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:03:55.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:03:55.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:03:55.136 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:55.137 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:03:55.138 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:03:41.066240+0000", 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.411+0000 7fbfb96b2700 1 Processor -- start 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.411+0000 7fbfb96b2700 1 -- start start 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.411+0000 7fbfb96b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb4102170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.411+0000 7fbfb96b2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb41026b0 con 0x7fbfb4101d60 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.411+0000 7fbfb2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb4102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.411+0000 7fbfb2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb4102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39342/0 (socket says 172.21.15.31:39342) 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.411+0000 7fbfb2ffd700 1 -- 172.21.15.31:0/1986500766 learned_addr learned my addr 172.21.15.31:0/1986500766 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.412+0000 7fbfb2ffd700 1 -- 172.21.15.31:0/1986500766 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb41027f0 con 0x7fbfb4101d60 2024-04-24T23:03:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.412+0000 7fbfb2ffd700 1 --2- 172.21.15.31:0/1986500766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb4102170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80098e0 tx=0x7fbfa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f3cacf0f414673 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.412+0000 7fbfb1ffb700 1 -- 172.21.15.31:0/1986500766 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfa800b860 con 0x7fbfb4101d60 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.412+0000 7fbfb1ffb700 1 -- 172.21.15.31:0/1986500766 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbfa800be50 con 0x7fbfb4101d60 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.412+0000 7fbfb1ffb700 1 -- 172.21.15.31:0/1986500766 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfa8010380 con 0x7fbfb4101d60 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1986500766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 msgr2=0x7fbfb4102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 --2- 172.21.15.31:0/1986500766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb4102170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80098e0 tx=0x7fbfa8009250 comp rx=0 tx=0).stop 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1986500766 shutdown_connections 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 --2- 172.21.15.31:0/1986500766 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb4102170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1986500766 >> 172.21.15.31:0/1986500766 conn(0x7fbfb40fc6f0 msgr2=0x7fbfb40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1986500766 shutdown_connections 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1986500766 wait complete. 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 Processor -- start 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.413+0000 7fbfb96b2700 1 -- start start 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbfb96b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb410f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbfb96b2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb410f910 con 0x7fbfb4101d60 2024-04-24T23:03:55.140 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbfb2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb410f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbfb2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb410f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39348/0 (socket says 172.21.15.31:39348) 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbfb2ffd700 1 -- 172.21.15.31:0/1767163583 learned_addr learned my addr 172.21.15.31:0/1767163583 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbfb2ffd700 1 -- 172.21.15.31:0/1767163583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80094f0 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbfb2ffd700 1 --2- 172.21.15.31:0/1767163583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb410f3d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8000c00 tx=0x7fbfa8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbf9b7fe700 1 -- 172.21.15.31:0/1767163583 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfa8010920 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb410fb10 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.414+0000 7fbf9b7fe700 1 -- 172.21.15.31:0/1767163583 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbfa801a430 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.415+0000 7fbf9b7fe700 1 -- 172.21.15.31:0/1767163583 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbfa8022420 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.415+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb410ff40 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.415+0000 7fbf9b7fe700 1 -- 172.21.15.31:0/1767163583 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fbfa8022620 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.415+0000 7fbf9b7fe700 1 -- 172.21.15.31:0/1767163583 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fbfa801e070 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.416+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb404bed0 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.419+0000 7fbf9b7fe700 1 -- 172.21.15.31:0/1767163583 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fbfa8030060 con 0x7fbfb4101d60 2024-04-24T23:03:55.141 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.492+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbfb4103570 con 0x7fbfb4101d60 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.493+0000 7fbf9b7fe700 1 -- 172.21.15.31:0/1767163583 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fbfa801aa10 con 0x7fbfb4101d60 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.494+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 msgr2=0x7fbfb410f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.494+0000 7fbfb96b2700 1 --2- 172.21.15.31:0/1767163583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb410f3d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8000c00 tx=0x7fbfa8009250 comp rx=0 tx=0).stop 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.494+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 shutdown_connections 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.494+0000 7fbfb96b2700 1 --2- 172.21.15.31:0/1767163583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfb4101d60 0x7fbfb410f3d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.494+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 >> 172.21.15.31:0/1767163583 conn(0x7fbfb40fc6f0 msgr2=0x7fbfb4105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.494+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 shutdown_connections 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:03:54.494+0000 7fbfb96b2700 1 -- 172.21.15.31:0/1767163583 wait complete. 2024-04-24T23:03:55.142 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (3/15)... 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: mgrmap e2: smithi031.rrnmwr(active, starting, since 0.00311783s) 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:03:56.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:03:56.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:03:56.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:55 smithi031 ceph-mon[26411]: from='mgr.14100 172.21.15.31:0/1913719857' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:03:56.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:56 smithi031 ceph-mon[26411]: mgrmap e3: smithi031.rrnmwr(active, since 1.00659s) 2024-04-24T23:03:57.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:03:57 smithi031 ceph-mon[26411]: mgrmap e4: smithi031.rrnmwr(active, since 2s) 2024-04-24T23:04:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:00 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/896328590' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "ca7b26e4-028e-11ef-bc93-c7b262605968", 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-04-24T23:04:06.567 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-24T23:04:06.568 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:06.569 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:04:06.570 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-04-24T23:03:41.066240+0000", 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.301+0000 7f304b8a7700 1 Processor -- start 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.301+0000 7f304b8a7700 1 -- start start 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.301+0000 7f304b8a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f3044101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.301+0000 7f304b8a7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3044102080 con 0x7f3044101730 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.301+0000 7f3049643700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f3044101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.301+0000 7f3049643700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f3044101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37958/0 (socket says 172.21.15.31:37958) 2024-04-24T23:04:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.301+0000 7f3049643700 1 -- 172.21.15.31:0/3255952128 learned_addr learned my addr 172.21.15.31:0/3255952128 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.302+0000 7f3049643700 1 -- 172.21.15.31:0/3255952128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30441021c0 con 0x7f3044101730 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.302+0000 7f3049643700 1 --2- 172.21.15.31:0/3255952128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f3044101b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f30400098e0 tx=0x7f3040009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a7bb35f94b840e server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.302+0000 7f303bfff700 1 -- 172.21.15.31:0/3255952128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f304000b860 con 0x7f3044101730 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.302+0000 7f303bfff700 1 -- 172.21.15.31:0/3255952128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f304000be50 con 0x7f3044101730 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.302+0000 7f303bfff700 1 -- 172.21.15.31:0/3255952128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3040010380 con 0x7f3044101730 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 -- 172.21.15.31:0/3255952128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 msgr2=0x7f3044101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 --2- 172.21.15.31:0/3255952128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f3044101b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f30400098e0 tx=0x7f3040009250 comp rx=0 tx=0).stop 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 -- 172.21.15.31:0/3255952128 shutdown_connections 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 --2- 172.21.15.31:0/3255952128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f3044101b40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 -- 172.21.15.31:0/3255952128 >> 172.21.15.31:0/3255952128 conn(0x7f30440fcef0 msgr2=0x7f30440ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 -- 172.21.15.31:0/3255952128 shutdown_connections 2024-04-24T23:04:06.572 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 -- 172.21.15.31:0/3255952128 wait complete. 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 Processor -- start 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.303+0000 7f304b8a7700 1 -- start start 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f304b8a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f30441115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f304b8a7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3044111b00 con 0x7f3044101730 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f3049643700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f30441115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f3049643700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f30441115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37972/0 (socket says 172.21.15.31:37972) 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f3049643700 1 -- 172.21.15.31:0/896328590 learned_addr learned my addr 172.21.15.31:0/896328590 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f3049643700 1 -- 172.21.15.31:0/896328590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30400094f0 con 0x7f3044101730 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f3049643700 1 --2- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f30441115c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f304000b440 tx=0x7f3040009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:06.573 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f3039ffb700 1 -- 172.21.15.31:0/896328590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3040010760 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f3039ffb700 1 -- 172.21.15.31:0/896328590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3040010d50 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3044111d00 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.304+0000 7f3039ffb700 1 -- 172.21.15.31:0/896328590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f304001ae20 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.305+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3044112870 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.305+0000 7f3039ffb700 1 -- 172.21.15.31:0/896328590 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f304002e440 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.305+0000 7f3039ffb700 1 --2- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f3030036850 0x7f3030038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.305+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304405f5c0 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.306+0000 7f3039ffb700 1 -- 172.21.15.31:0/896328590 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f304004a760 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.306+0000 7f3048e42700 1 --2- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f3030036850 0x7f3030038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.306+0000 7f3048e42700 1 --2- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f3030036850 0x7f3030038d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3034009990 tx=0x7f3034006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.311+0000 7f3039ffb700 1 -- 172.21.15.31:0/896328590 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3040018710 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.584+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f304404bed0 con 0x7f3044101730 2024-04-24T23:04:06.574 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.585+0000 7f3039ffb700 1 -- 172.21.15.31:0/896328590 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f304002d3a0 con 0x7f3044101730 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f3030036850 msgr2=0x7f3030038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 --2- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f3030036850 0x7f3030038d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3034009990 tx=0x7f3034006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 msgr2=0x7f30441115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 --2- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f30441115c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f304000b440 tx=0x7f3040009250 comp rx=0 tx=0).stop 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 shutdown_connections 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 --2- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f3030036850 0x7f3030038d00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 --2- 172.21.15.31:0/896328590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3044101730 0x7f30441115c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 >> 172.21.15.31:0/896328590 conn(0x7f30440fcef0 msgr2=0x7f30440fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 shutdown_connections 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:00.588+0000 7f304b8a7700 1 -- 172.21.15.31:0/896328590 wait complete. 2024-04-24T23:04:06.575 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-04-24T23:04:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:07 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/267563114' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-24T23:04:07.851 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-04-24T23:04:07.858 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.112+0000 7f56b8e40700 1 Processor -- start 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.112+0000 7f56b8e40700 1 -- start start 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.112+0000 7f56b8e40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.112+0000 7f56b8e40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4105590 con 0x7f56b4104bb0 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.113+0000 7f56b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.113+0000 7f56b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4104fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37980/0 (socket says 172.21.15.31:37980) 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.113+0000 7f56b259c700 1 -- 172.21.15.31:0/131874435 learned_addr learned my addr 172.21.15.31:0/131874435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.113+0000 7f56b259c700 1 -- 172.21.15.31:0/131874435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b41056d0 con 0x7f56b4104bb0 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.113+0000 7f56b259c700 1 --2- 172.21.15.31:0/131874435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4104fc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098e0 tx=0x7f56a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d190b965a2c0e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.859 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.113+0000 7f56b159a700 1 -- 172.21.15.31:0/131874435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56a400b860 con 0x7f56b4104bb0 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b159a700 1 -- 172.21.15.31:0/131874435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f56a400be50 con 0x7f56b4104bb0 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b159a700 1 -- 172.21.15.31:0/131874435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56a4010380 con 0x7f56b4104bb0 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b8e40700 1 -- 172.21.15.31:0/131874435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 msgr2=0x7f56b4104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b8e40700 1 --2- 172.21.15.31:0/131874435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4104fc0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098e0 tx=0x7f56a4009250 comp rx=0 tx=0).stop 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b8e40700 1 -- 172.21.15.31:0/131874435 shutdown_connections 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b8e40700 1 --2- 172.21.15.31:0/131874435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4104fc0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b8e40700 1 -- 172.21.15.31:0/131874435 >> 172.21.15.31:0/131874435 conn(0x7f56b4078300 msgr2=0x7f56b4078700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b8e40700 1 -- 172.21.15.31:0/131874435 shutdown_connections 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.114+0000 7f56b8e40700 1 -- 172.21.15.31:0/131874435 wait complete. 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.115+0000 7f56b8e40700 1 Processor -- start 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.115+0000 7f56b8e40700 1 -- start start 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.115+0000 7f56b8e40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.860 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.115+0000 7f56b8e40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4113cc0 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.115+0000 7f56b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.115+0000 7f56b259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37988/0 (socket says 172.21.15.31:37988) 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.115+0000 7f56b259c700 1 -- 172.21.15.31:0/267563114 learned_addr learned my addr 172.21.15.31:0/267563114 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.115+0000 7f56b259c700 1 -- 172.21.15.31:0/267563114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a40094f0 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.116+0000 7f56b259c700 1 --2- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4113780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56a4003050 tx=0x7f56a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.116+0000 7f56a2ffd700 1 -- 172.21.15.31:0/267563114 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56a4010760 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.116+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b4113ec0 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.116+0000 7f56a2ffd700 1 -- 172.21.15.31:0/267563114 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f56a4010d50 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.116+0000 7f56a2ffd700 1 -- 172.21.15.31:0/267563114 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f56a401ae20 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.116+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b4114a30 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.117+0000 7f56a2ffd700 1 -- 172.21.15.31:0/267563114 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f56a402e440 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.117+0000 7f56a2ffd700 1 --2- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f569c0367e0 0x7f569c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.117+0000 7f56a2ffd700 1 -- 172.21.15.31:0/267563114 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f56a404a500 con 0x7f56b4104bb0 2024-04-24T23:04:07.861 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.117+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b405f5c0 con 0x7f56b4104bb0 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.117+0000 7f56b1d9b700 1 --2- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f569c0367e0 0x7f569c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.118+0000 7f56b1d9b700 1 --2- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f569c0367e0 0x7f569c038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009990 tx=0x7f56a8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.122+0000 7f56a2ffd700 1 -- 172.21.15.31:0/267563114 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f56a40243f0 con 0x7f56b4104bb0 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.317+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f56b4106850 con 0x7f56b4104bb0 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.317+0000 7f56a2ffd700 1 -- 172.21.15.31:0/267563114 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f56a4048030 con 0x7f56b4104bb0 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.319+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f569c0367e0 msgr2=0x7f569c038c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.319+0000 7f56b8e40700 1 --2- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f569c0367e0 0x7f569c038c90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009990 tx=0x7f56a8006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.319+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 msgr2=0x7f56b4113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.319+0000 7f56b8e40700 1 --2- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4113780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f56a4003050 tx=0x7f56a4009250 comp rx=0 tx=0).stop 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.319+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 shutdown_connections 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.319+0000 7f56b8e40700 1 --2- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f569c0367e0 0x7f569c038c90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.319+0000 7f56b8e40700 1 --2- 172.21.15.31:0/267563114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56b4104bb0 0x7f56b4113780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:07.862 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.319+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 >> 172.21.15.31:0/267563114 conn(0x7f56b4078300 msgr2=0x7f56b4102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:07.863 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.320+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 shutdown_connections 2024-04-24T23:04:07.863 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:07.320+0000 7f56b8e40700 1 -- 172.21.15.31:0/267563114 wait complete. 2024-04-24T23:04:07.863 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-04-24T23:04:08.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:08 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3860431814' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.256+0000 7fcb55f94700 1 Processor -- start 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.256+0000 7fcb55f94700 1 -- start start 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.256+0000 7fcb55f94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.256+0000 7fcb55f94700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb501031d0 con 0x7fcb50102880 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.256+0000 7fcb4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.256+0000 7fcb4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38004/0 (socket says 172.21.15.31:38004) 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.256+0000 7fcb4f7fe700 1 -- 172.21.15.31:0/3247792755 learned_addr learned my addr 172.21.15.31:0/3247792755 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.257+0000 7fcb4f7fe700 1 -- 172.21.15.31:0/3247792755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50103310 con 0x7fcb50102880 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.257+0000 7fcb4f7fe700 1 --2- 172.21.15.31:0/3247792755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50102c90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009e50 tx=0x7fcb48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc46f2453ab9621 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:09.131 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.257+0000 7fcb4e7fc700 1 -- 172.21.15.31:0/3247792755 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb480092e0 con 0x7fcb50102880 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.257+0000 7fcb4e7fc700 1 -- 172.21.15.31:0/3247792755 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb48012430 con 0x7fcb50102880 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.257+0000 7fcb4e7fc700 1 -- 172.21.15.31:0/3247792755 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb480075c0 con 0x7fcb50102880 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.258+0000 7fcb55f94700 1 -- 172.21.15.31:0/3247792755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 msgr2=0x7fcb50102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.258+0000 7fcb55f94700 1 --2- 172.21.15.31:0/3247792755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50102c90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009e50 tx=0x7fcb48009840 comp rx=0 tx=0).stop 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.258+0000 7fcb55f94700 1 -- 172.21.15.31:0/3247792755 shutdown_connections 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.258+0000 7fcb55f94700 1 --2- 172.21.15.31:0/3247792755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50102c90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.258+0000 7fcb55f94700 1 -- 172.21.15.31:0/3247792755 >> 172.21.15.31:0/3247792755 conn(0x7fcb500fe060 msgr2=0x7fcb50100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.258+0000 7fcb55f94700 1 -- 172.21.15.31:0/3247792755 shutdown_connections 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.258+0000 7fcb55f94700 1 -- 172.21.15.31:0/3247792755 wait complete. 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.258+0000 7fcb55f94700 1 Processor -- start 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb55f94700 1 -- start start 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb55f94700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50111630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb55f94700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50111b70 con 0x7fcb50102880 2024-04-24T23:04:09.132 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50111630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb4f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50111630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38014/0 (socket says 172.21.15.31:38014) 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb4f7fe700 1 -- 172.21.15.31:0/3860431814 learned_addr learned my addr 172.21.15.31:0/3860431814 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb4f7fe700 1 -- 172.21.15.31:0/3860431814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb48009ae0 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb4f7fe700 1 --2- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50111630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009ab0 tx=0x7fcb48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.259+0000 7fcb47fff700 1 -- 172.21.15.31:0/3860431814 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb480092e0 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.260+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb50111d70 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.260+0000 7fcb47fff700 1 -- 172.21.15.31:0/3860431814 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb48007d10 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.260+0000 7fcb47fff700 1 -- 172.21.15.31:0/3860431814 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb4801c990 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.260+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb501128e0 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.260+0000 7fcb47fff700 1 -- 172.21.15.31:0/3860431814 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7fcb48019070 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.261+0000 7fcb47fff700 1 --2- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7fcb3c036850 0x7fcb3c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.261+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5005f5c0 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.261+0000 7fcb47fff700 1 -- 172.21.15.31:0/3860431814 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcb48049dd0 con 0x7fcb50102880 2024-04-24T23:04:09.133 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.261+0000 7fcb4effd700 1 --2- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7fcb3c036850 0x7fcb3c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.261+0000 7fcb4effd700 1 --2- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7fcb3c036850 0x7fcb3c038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009990 tx=0x7fcb38006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.265+0000 7fcb47fff700 1 -- 172.21.15.31:0/3860431814 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcb48019910 con 0x7fcb50102880 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.514+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fcb50112c60 con 0x7fcb50102880 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.567+0000 7fcb47fff700 1 -- 172.21.15.31:0/3860431814 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fcb48041090 con 0x7fcb50102880 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.568+0000 7fcb47fff700 1 -- 172.21.15.31:0/3860431814 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fcb480497e0 con 0x7fcb50102880 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7fcb3c036850 msgr2=0x7fcb3c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 --2- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7fcb3c036850 0x7fcb3c038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009990 tx=0x7fcb38006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 msgr2=0x7fcb50111630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 --2- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50111630 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009ab0 tx=0x7fcb48009840 comp rx=0 tx=0).stop 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 shutdown_connections 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 --2- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7fcb3c036850 0x7fcb3c038d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 --2- 172.21.15.31:0/3860431814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb50102880 0x7fcb50111630 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 >> 172.21.15.31:0/3860431814 conn(0x7fcb500fe060 msgr2=0x7fcb500feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:09.134 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 shutdown_connections 2024-04-24T23:04:09.135 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:08.571+0000 7fcb55f94700 1 -- 172.21.15.31:0/3860431814 wait complete. 2024-04-24T23:04:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:09 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3860431814' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-24T23:04:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:09 smithi031 ceph-mon[26411]: mgrmap e5: smithi031.rrnmwr(active, since 13s) 2024-04-24T23:04:10.430 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "smithi031.rrnmwr", 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.547+0000 7f9c75fb4700 1 Processor -- start 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.547+0000 7f9c75fb4700 1 -- start start 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.547+0000 7f9c75fb4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.547+0000 7f9c75fb4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70079060 con 0x7f9c70078c50 2024-04-24T23:04:10.437 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.547+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:10.437 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.547+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42612/0 (socket says 172.21.15.31:42612) 2024-04-24T23:04:10.437 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.547+0000 7f9c6f7fe700 1 -- 172.21.15.31:0/2173204626 learned_addr learned my addr 172.21.15.31:0/2173204626 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:10.437 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.548+0000 7f9c6f7fe700 1 -- 172.21.15.31:0/2173204626 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70077790 con 0x7f9c70078c50 2024-04-24T23:04:10.437 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.548+0000 7f9c6f7fe700 1 --2- 172.21.15.31:0/2173204626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70077250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c600098e0 tx=0x7f9c60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d4fc248e993dac server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:10.437 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.548+0000 7f9c6e7fc700 1 -- 172.21.15.31:0/2173204626 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c6000b860 con 0x7f9c70078c50 2024-04-24T23:04:10.437 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.548+0000 7f9c6e7fc700 1 -- 172.21.15.31:0/2173204626 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9c6000be50 con 0x7f9c70078c50 2024-04-24T23:04:10.437 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.548+0000 7f9c6e7fc700 1 -- 172.21.15.31:0/2173204626 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c60010380 con 0x7f9c70078c50 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2173204626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 msgr2=0x7f9c70077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 --2- 172.21.15.31:0/2173204626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70077250 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9c600098e0 tx=0x7f9c60009250 comp rx=0 tx=0).stop 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2173204626 shutdown_connections 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 --2- 172.21.15.31:0/2173204626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70077250 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2173204626 >> 172.21.15.31:0/2173204626 conn(0x7f9c700fe090 msgr2=0x7f9c701004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2173204626 shutdown_connections 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2173204626 wait complete. 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 Processor -- start 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.549+0000 7f9c75fb4700 1 -- start start 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c75fb4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c75fb4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70113d80 con 0x7f9c70078c50 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42614/0 (socket says 172.21.15.31:42614) 2024-04-24T23:04:10.438 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c6f7fe700 1 -- 172.21.15.31:0/2390699155 learned_addr learned my addr 172.21.15.31:0/2390699155 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c6f7fe700 1 -- 172.21.15.31:0/2390699155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600094f0 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c6f7fe700 1 --2- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70113840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000b440 tx=0x7f9c60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c57fff700 1 -- 172.21.15.31:0/2390699155 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c60010760 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c57fff700 1 -- 172.21.15.31:0/2390699155 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9c60010d50 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c70113f80 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c57fff700 1 -- 172.21.15.31:0/2390699155 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c6001adc0 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.550+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c70114af0 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.551+0000 7f9c57fff700 1 -- 172.21.15.31:0/2390699155 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f9c60019410 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.551+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7005f5c0 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.551+0000 7f9c57fff700 1 --2- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f9c58036830 0x7f9c58038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.552+0000 7f9c57fff700 1 -- 172.21.15.31:0/2390699155 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f9c60049fe0 con 0x7f9c70078c50 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.552+0000 7f9c6effd700 1 -- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f9c58036830 msgr2=0x7f9c58038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4106752127 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.552+0000 7f9c6effd700 1 --2- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f9c58036830 0x7f9c58038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:10.439 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.556+0000 7f9c57fff700 1 -- 172.21.15.31:0/2390699155 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9c60023940 con 0x7f9c70078c50 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.753+0000 7f9c6effd700 1 -- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f9c58036830 msgr2=0x7f9c58038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4106752127 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.753+0000 7f9c6effd700 1 --2- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f9c58036830 0x7f9c58038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.818+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9c70114e30 con 0x7f9c70078c50 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.822+0000 7f9c57fff700 1 -- 172.21.15.31:0/2390699155 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f9c60023350 con 0x7f9c70078c50 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.826+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f9c58036830 msgr2=0x7f9c58038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.826+0000 7f9c75fb4700 1 --2- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f9c58036830 0x7f9c58038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.826+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 msgr2=0x7f9c70113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.826+0000 7f9c75fb4700 1 --2- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70113840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000b440 tx=0x7f9c60009250 comp rx=0 tx=0).stop 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.826+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 shutdown_connections 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.826+0000 7f9c75fb4700 1 --2- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f9c58036830 0x7f9c58038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.826+0000 7f9c75fb4700 1 --2- 172.21.15.31:0/2390699155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c70078c50 0x7f9c70113840 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.826+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 >> 172.21.15.31:0/2390699155 conn(0x7f9c700fe090 msgr2=0x7f9c700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.827+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 shutdown_connections 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:09.827+0000 7f9c75fb4700 1 -- 172.21.15.31:0/2390699155 wait complete. 2024-04-24T23:04:10.440 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-24T23:04:10.441 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-04-24T23:04:10.679 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:10 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2390699155' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:04:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: Active manager daemon smithi031.rrnmwr restarted 2024-04-24T23:04:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:04:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: osdmap e2: 0 total, 0 up, 0 in 2024-04-24T23:04:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: mgrmap e6: smithi031.rrnmwr(active, starting, since 0.00376532s) 2024-04-24T23:04:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:04:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:04:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:04:15.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:15 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:04:16.648 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:16 smithi031 ceph-mon[26411]: Found migration_current of "None". Setting to last migration. 2024-04-24T23:04:16.648 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:16 smithi031 ceph-mon[26411]: mgrmap e7: smithi031.rrnmwr(active, since 1.00503s) 2024-04-24T23:04:16.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:16.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-24T23:04:16.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:04:16.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.860+0000 7f1275d72700 1 Processor -- start 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.860+0000 7f1275d72700 1 -- start start 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.860+0000 7f1275d72700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.860+0000 7f1275d72700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270103200 con 0x7f12701028b0 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.861+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.861+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42616/0 (socket says 172.21.15.31:42616) 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.861+0000 7f126f7fe700 1 -- 172.21.15.31:0/3247046081 learned_addr learned my addr 172.21.15.31:0/3247046081 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.861+0000 7f126f7fe700 1 -- 172.21.15.31:0/3247046081 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1270103340 con 0x7f12701028b0 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.861+0000 7f126f7fe700 1 --2- 172.21.15.31:0/3247046081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270102cc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f12640098e0 tx=0x7f1264009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf217c66e2597c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.861+0000 7f126e7fc700 1 -- 172.21.15.31:0/3247046081 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f126400b860 con 0x7f12701028b0 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.861+0000 7f126e7fc700 1 -- 172.21.15.31:0/3247046081 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f126400be50 con 0x7f12701028b0 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.861+0000 7f126e7fc700 1 -- 172.21.15.31:0/3247046081 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1264010380 con 0x7f12701028b0 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.862+0000 7f1275d72700 1 -- 172.21.15.31:0/3247046081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 msgr2=0x7f1270102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:16.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.862+0000 7f1275d72700 1 --2- 172.21.15.31:0/3247046081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270102cc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f12640098e0 tx=0x7f1264009250 comp rx=0 tx=0).stop 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.862+0000 7f1275d72700 1 -- 172.21.15.31:0/3247046081 shutdown_connections 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.862+0000 7f1275d72700 1 --2- 172.21.15.31:0/3247046081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270102cc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.862+0000 7f1275d72700 1 -- 172.21.15.31:0/3247046081 >> 172.21.15.31:0/3247046081 conn(0x7f12700fdff0 msgr2=0x7f1270100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.862+0000 7f1275d72700 1 -- 172.21.15.31:0/3247046081 shutdown_connections 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.862+0000 7f1275d72700 1 -- 172.21.15.31:0/3247046081 wait complete. 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.863+0000 7f1275d72700 1 Processor -- start 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.863+0000 7f1275d72700 1 -- start start 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.863+0000 7f1275d72700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.863+0000 7f1275d72700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1270078050 con 0x7f12701028b0 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.863+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.863+0000 7f126f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42632/0 (socket says 172.21.15.31:42632) 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.863+0000 7f126f7fe700 1 -- 172.21.15.31:0/1858946303 learned_addr learned my addr 172.21.15.31:0/1858946303 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.864+0000 7f126f7fe700 1 -- 172.21.15.31:0/1858946303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12640094f0 con 0x7f12701028b0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.864+0000 7f126f7fe700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270077b10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1264000c00 tx=0x7f1264009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.864+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f12640107c0 con 0x7f12701028b0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.864+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1264010db0 con 0x7f12701028b0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.864+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12700760c0 con 0x7f12701028b0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.864+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12700765b0 con 0x7f12701028b0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.864+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f126401ad00 con 0x7f12701028b0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.865+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f12640193c0 con 0x7f12701028b0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.865+0000 7f1257fff700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 0x7f1258038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.865+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 --> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1258039410 con 0x7f1258036850 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.865+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f126404a5d0 con 0x7f12701028b0 2024-04-24T23:04:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.865+0000 7f126effd700 1 -- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 msgr2=0x7f1258038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4106752127 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:10.865+0000 7f126effd700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 0x7f1258038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:11.066+0000 7f126effd700 1 -- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 msgr2=0x7f1258038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4106752127 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:11.066+0000 7f126effd700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 0x7f1258038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:11.467+0000 7f126effd700 1 -- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 msgr2=0x7f1258038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4106752127 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:11.467+0000 7f126effd700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 0x7f1258038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:12.268+0000 7f126effd700 1 -- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 msgr2=0x7f1258038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4106752127 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:12.268+0000 7f126effd700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 0x7f1258038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:13.869+0000 7f126effd700 1 -- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 msgr2=0x7f1258038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/4106752127 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:13.869+0000 7f126effd700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 0x7f1258038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:15.328+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7f126402c3a0 con 0x7f12701028b0 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:15.328+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 msgr2=0x7f1258038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:15.328+0000 7f1257fff700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 0x7f1258038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.329+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f1264019be0 con 0x7f12701028b0 2024-04-24T23:04:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.329+0000 7f1257fff700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1258039ff0 0x7f125803c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.329+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1258039410 con 0x7f1258039ff0 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.329+0000 7f126effd700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1258039ff0 0x7f125803c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.330+0000 7f126effd700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1258039ff0 0x7f125803c3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f12600039d0 tx=0x7f1260009350 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.331+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f1258039410 con 0x7f1258039ff0 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.341+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f125c0025f0 con 0x7f1258039ff0 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.341+0000 7f1257fff700 1 -- 172.21.15.31:0/1858946303 <== mgr.14120 v2:172.21.15.31:6800/2092175909 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f125c0025f0 con 0x7f1258039ff0 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1258039ff0 msgr2=0x7f125803c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1258039ff0 0x7f125803c3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f12600039d0 tx=0x7f1260009350 comp rx=0 tx=0).stop 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 msgr2=0x7f1270077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270077b10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1264000c00 tx=0x7f1264009250 comp rx=0 tx=0).stop 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 shutdown_connections 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1258039ff0 0x7f125803c3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12701028b0 0x7f1270077b10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:16.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 --2- 172.21.15.31:0/1858946303 >> [v2:172.21.15.31:6800/4106752127,v1:172.21.15.31:6801/4106752127] conn(0x7f1258036850 0x7f1258038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:16.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 >> 172.21.15.31:0/1858946303 conn(0x7f12700fdff0 msgr2=0x7f12700feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:16.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 shutdown_connections 2024-04-24T23:04:16.964 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:16.342+0000 7f1275d72700 1 -- 172.21.15.31:0/1858946303 wait complete. 2024-04-24T23:04:16.964 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-04-24T23:04:16.964 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-04-24T23:04:17.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:17 smithi031 ceph-mon[26411]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:04:17.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:17 smithi031 ceph-mon[26411]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:04:17.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:17 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:17.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:17 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:18.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.342+0000 7f887a1f6700 1 Processor -- start 2024-04-24T23:04:18.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.342+0000 7f887a1f6700 1 -- start start 2024-04-24T23:04:18.121 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.342+0000 7f887a1f6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f8874101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.342+0000 7f887a1f6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8874102080 con 0x7f8874101730 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.342+0000 7f88737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f8874101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.342+0000 7f88737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f8874101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42728/0 (socket says 172.21.15.31:42728) 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.342+0000 7f88737fe700 1 -- 172.21.15.31:0/815365502 learned_addr learned my addr 172.21.15.31:0/815365502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.342+0000 7f88737fe700 1 -- 172.21.15.31:0/815365502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88741021c0 con 0x7f8874101730 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f88737fe700 1 --2- 172.21.15.31:0/815365502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f8874101b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88640098e0 tx=0x7f8864009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8812571637dfc57a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f88727fc700 1 -- 172.21.15.31:0/815365502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f886400b860 con 0x7f8874101730 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f88727fc700 1 -- 172.21.15.31:0/815365502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f886400be50 con 0x7f8874101730 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f88727fc700 1 -- 172.21.15.31:0/815365502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8864010380 con 0x7f8874101730 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f887a1f6700 1 -- 172.21.15.31:0/815365502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 msgr2=0x7f8874101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f887a1f6700 1 --2- 172.21.15.31:0/815365502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f8874101b40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88640098e0 tx=0x7f8864009250 comp rx=0 tx=0).stop 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f887a1f6700 1 -- 172.21.15.31:0/815365502 shutdown_connections 2024-04-24T23:04:18.122 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f887a1f6700 1 --2- 172.21.15.31:0/815365502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f8874101b40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f887a1f6700 1 -- 172.21.15.31:0/815365502 >> 172.21.15.31:0/815365502 conn(0x7f88740fcef0 msgr2=0x7f88740ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f887a1f6700 1 -- 172.21.15.31:0/815365502 shutdown_connections 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.343+0000 7f887a1f6700 1 -- 172.21.15.31:0/815365502 wait complete. 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.344+0000 7f887a1f6700 1 Processor -- start 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.344+0000 7f887a1f6700 1 -- start start 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.344+0000 7f887a1f6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f88741115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.344+0000 7f887a1f6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8874111b00 con 0x7f8874101730 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.344+0000 7f88737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f88741115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.344+0000 7f88737fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f88741115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42732/0 (socket says 172.21.15.31:42732) 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.344+0000 7f88737fe700 1 -- 172.21.15.31:0/2267211134 learned_addr learned my addr 172.21.15.31:0/2267211134 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.345+0000 7f88737fe700 1 -- 172.21.15.31:0/2267211134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88640094f0 con 0x7f8874101730 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.345+0000 7f88737fe700 1 --2- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f88741115c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8864000c00 tx=0x7f8864009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.345+0000 7f885bfff700 1 -- 172.21.15.31:0/2267211134 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88640107c0 con 0x7f8874101730 2024-04-24T23:04:18.123 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.345+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8874111d00 con 0x7f8874101730 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.345+0000 7f885bfff700 1 -- 172.21.15.31:0/2267211134 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8864010db0 con 0x7f8874101730 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.345+0000 7f885bfff700 1 -- 172.21.15.31:0/2267211134 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f886401ad00 con 0x7f8874101730 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.345+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8874112870 con 0x7f8874101730 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.346+0000 7f885bfff700 1 -- 172.21.15.31:0/2267211134 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f88640193c0 con 0x7f8874101730 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.346+0000 7f885bfff700 1 --2- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f885c036720 0x7f885c038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.346+0000 7f885bfff700 1 -- 172.21.15.31:0/2267211134 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f886404a680 con 0x7f8874101730 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.346+0000 7f8872ffd700 1 --2- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f885c036720 0x7f885c038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.346+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887405f5c0 con 0x7f8874101730 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.347+0000 7f8872ffd700 1 --2- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f885c036720 0x7f885c038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8868009990 tx=0x7f8868006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.351+0000 7f885bfff700 1 -- 172.21.15.31:0/2267211134 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f88640196c0 con 0x7f8874101730 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.558+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f88741132f0 con 0x7f885c036720 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.567+0000 7f885bfff700 1 -- 172.21.15.31:0/2267211134 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f88741132f0 con 0x7f885c036720 2024-04-24T23:04:18.124 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f885c036720 msgr2=0x7f885c038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 --2- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f885c036720 0x7f885c038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8868009990 tx=0x7f8868006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 msgr2=0x7f88741115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 --2- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f88741115c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8864000c00 tx=0x7f8864009250 comp rx=0 tx=0).stop 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 shutdown_connections 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 --2- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f885c036720 0x7f885c038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 --2- 172.21.15.31:0/2267211134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8874101730 0x7f88741115c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 >> 172.21.15.31:0/2267211134 conn(0x7f88740fcef0 msgr2=0x7f88740fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 shutdown_connections 2024-04-24T23:04:18.125 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:17.569+0000 7f887a1f6700 1 -- 172.21.15.31:0/2267211134 wait complete. 2024-04-24T23:04:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:18 smithi031 ceph-mon[26411]: [24/Apr/2024:23:04:16] ENGINE Bus STARTING 2024-04-24T23:04:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:18 smithi031 ceph-mon[26411]: [24/Apr/2024:23:04:16] ENGINE Serving on https://172.21.15.31:7150 2024-04-24T23:04:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:18 smithi031 ceph-mon[26411]: [24/Apr/2024:23:04:16] ENGINE Bus STARTED 2024-04-24T23:04:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:18 smithi031 ceph-mon[26411]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:18 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:18 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:18 smithi031 ceph-mon[26411]: mgrmap e8: smithi031.rrnmwr(active, since 2s) 2024-04-24T23:04:19.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-24T23:04:19.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.588+0000 7fcb90411700 1 Processor -- start 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.588+0000 7fcb90411700 1 -- start start 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.588+0000 7fcb90411700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.588+0000 7fcb90411700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88103160 con 0x7fcb88102810 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.588+0000 7fcb8e1ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.588+0000 7fcb8e1ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42748/0 (socket says 172.21.15.31:42748) 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.588+0000 7fcb8e1ad700 1 -- 172.21.15.31:0/2932346177 learned_addr learned my addr 172.21.15.31:0/2932346177 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.588+0000 7fcb8e1ad700 1 -- 172.21.15.31:0/2932346177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb881032a0 con 0x7fcb88102810 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.589+0000 7fcb8e1ad700 1 --2- 172.21.15.31:0/2932346177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88102c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb840098e0 tx=0x7fcb84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f823791fe4c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.589+0000 7fcb8d1ab700 1 -- 172.21.15.31:0/2932346177 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb8400b860 con 0x7fcb88102810 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.589+0000 7fcb8d1ab700 1 -- 172.21.15.31:0/2932346177 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb8400be50 con 0x7fcb88102810 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.589+0000 7fcb8d1ab700 1 -- 172.21.15.31:0/2932346177 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb84010380 con 0x7fcb88102810 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.589+0000 7fcb90411700 1 -- 172.21.15.31:0/2932346177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 msgr2=0x7fcb88102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.589+0000 7fcb90411700 1 --2- 172.21.15.31:0/2932346177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88102c20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcb840098e0 tx=0x7fcb84009250 comp rx=0 tx=0).stop 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.589+0000 7fcb90411700 1 -- 172.21.15.31:0/2932346177 shutdown_connections 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.589+0000 7fcb90411700 1 --2- 172.21.15.31:0/2932346177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88102c20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb90411700 1 -- 172.21.15.31:0/2932346177 >> 172.21.15.31:0/2932346177 conn(0x7fcb880fdfd0 msgr2=0x7fcb881003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb90411700 1 -- 172.21.15.31:0/2932346177 shutdown_connections 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb90411700 1 -- 172.21.15.31:0/2932346177 wait complete. 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb90411700 1 Processor -- start 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb90411700 1 -- start start 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb90411700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb90411700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88079160 con 0x7fcb88102810 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb8e1ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb8e1ad700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42764/0 (socket says 172.21.15.31:42764) 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.590+0000 7fcb8e1ad700 1 -- 172.21.15.31:0/2895341157 learned_addr learned my addr 172.21.15.31:0/2895341157 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.591+0000 7fcb8e1ad700 1 -- 172.21.15.31:0/2895341157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840094f0 con 0x7fcb88102810 2024-04-24T23:04:19.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.591+0000 7fcb8e1ad700 1 --2- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88078c20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ba90 tx=0x7fcb84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.591+0000 7fcb7effd700 1 -- 172.21.15.31:0/2895341157 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb840107c0 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.591+0000 7fcb7effd700 1 -- 172.21.15.31:0/2895341157 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcb84010db0 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.591+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb880771d0 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.591+0000 7fcb7effd700 1 -- 172.21.15.31:0/2895341157 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb8401add0 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.591+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb880776c0 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.592+0000 7fcb7effd700 1 -- 172.21.15.31:0/2895341157 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fcb8402e440 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.592+0000 7fcb7effd700 1 --2- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fcb740368a0 0x7fcb74038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.592+0000 7fcb7effd700 1 -- 172.21.15.31:0/2895341157 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fcb8404a6a0 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.592+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8804bed0 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.592+0000 7fcb8d9ac700 1 --2- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fcb740368a0 0x7fcb74038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.593+0000 7fcb8d9ac700 1 --2- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fcb740368a0 0x7fcb74038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009990 tx=0x7fcb78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.597+0000 7fcb7effd700 1 -- 172.21.15.31:0/2895341157 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcb84018350 con 0x7fcb88102810 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.794+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb88077dd0 con 0x7fcb740368a0 2024-04-24T23:04:19.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.794+0000 7fcb7effd700 1 -- 172.21.15.31:0/2895341157 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fcb88077dd0 con 0x7fcb740368a0 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fcb740368a0 msgr2=0x7fcb74038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 --2- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fcb740368a0 0x7fcb74038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009990 tx=0x7fcb78006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 msgr2=0x7fcb88078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 --2- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88078c20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400ba90 tx=0x7fcb84009250 comp rx=0 tx=0).stop 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 shutdown_connections 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 --2- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fcb740368a0 0x7fcb74038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 --2- 172.21.15.31:0/2895341157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb88102810 0x7fcb88078c20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 >> 172.21.15.31:0/2895341157 conn(0x7fcb880fdfd0 msgr2=0x7fcb880feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 shutdown_connections 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:18.796+0000 7fcb90411700 1 -- 172.21.15.31:0/2895341157 wait complete. 2024-04-24T23:04:19.356 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-04-24T23:04:19.713 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:19 smithi031 ceph-mon[26411]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:20.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.736+0000 7fbb8dd66700 1 Processor -- start 2024-04-24T23:04:20.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.736+0000 7fbb8dd66700 1 -- start start 2024-04-24T23:04:20.717 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.736+0000 7fbb8dd66700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.736+0000 7fbb8dd66700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb881031d0 con 0x7fbb88102880 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.736+0000 7fbb877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.737+0000 7fbb877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41366/0 (socket says 172.21.15.31:41366) 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.737+0000 7fbb877fe700 1 -- 172.21.15.31:0/3288027103 learned_addr learned my addr 172.21.15.31:0/3288027103 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.737+0000 7fbb877fe700 1 -- 172.21.15.31:0/3288027103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88103310 con 0x7fbb88102880 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.737+0000 7fbb877fe700 1 --2- 172.21.15.31:0/3288027103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88102c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb780098e0 tx=0x7fbb78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2f4af12f06b2da server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.737+0000 7fbb86ffd700 1 -- 172.21.15.31:0/3288027103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb7800b860 con 0x7fbb88102880 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.737+0000 7fbb86ffd700 1 -- 172.21.15.31:0/3288027103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbb7800be50 con 0x7fbb88102880 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.737+0000 7fbb86ffd700 1 -- 172.21.15.31:0/3288027103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb78010380 con 0x7fbb88102880 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.738+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3288027103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 msgr2=0x7fbb88102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.738+0000 7fbb8dd66700 1 --2- 172.21.15.31:0/3288027103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88102c90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb780098e0 tx=0x7fbb78009250 comp rx=0 tx=0).stop 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.738+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3288027103 shutdown_connections 2024-04-24T23:04:20.718 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.738+0000 7fbb8dd66700 1 --2- 172.21.15.31:0/3288027103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88102c90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.738+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3288027103 >> 172.21.15.31:0/3288027103 conn(0x7fbb880fe060 msgr2=0x7fbb88100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.738+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3288027103 shutdown_connections 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.738+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3288027103 wait complete. 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.738+0000 7fbb8dd66700 1 Processor -- start 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb8dd66700 1 -- start start 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb8dd66700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb8dd66700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb88079160 con 0x7fbb88102880 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb877fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41378/0 (socket says 172.21.15.31:41378) 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb877fe700 1 -- 172.21.15.31:0/3132387399 learned_addr learned my addr 172.21.15.31:0/3132387399 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb877fe700 1 -- 172.21.15.31:0/3132387399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780094f0 con 0x7fbb88102880 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb877fe700 1 --2- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88078c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800b440 tx=0x7fbb78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb84ff9700 1 -- 172.21.15.31:0/3132387399 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb78010760 con 0x7fbb88102880 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb84ff9700 1 -- 172.21.15.31:0/3132387399 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbb78010d50 con 0x7fbb88102880 2024-04-24T23:04:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb84ff9700 1 -- 172.21.15.31:0/3132387399 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbb7801ae20 con 0x7fbb88102880 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.739+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb880771d0 con 0x7fbb88102880 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.740+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb880776c0 con 0x7fbb88102880 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.740+0000 7fbb84ff9700 1 -- 172.21.15.31:0/3132387399 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fbb7802e440 con 0x7fbb88102880 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.741+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8804bed0 con 0x7fbb88102880 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.741+0000 7fbb84ff9700 1 --2- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbb740368a0 0x7fbb74038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.741+0000 7fbb84ff9700 1 -- 172.21.15.31:0/3132387399 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbb7804a6d0 con 0x7fbb88102880 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.741+0000 7fbb7ffff700 1 --2- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbb740368a0 0x7fbb74038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.741+0000 7fbb7ffff700 1 --2- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbb740368a0 0x7fbb74038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009990 tx=0x7fbb70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.746+0000 7fbb84ff9700 1 -- 172.21.15.31:0/3132387399 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbb78018350 con 0x7fbb88102880 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:19.943+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb88077dd0 con 0x7fbb740368a0 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.082+0000 7fbb84ff9700 1 -- 172.21.15.31:0/3132387399 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbb88077dd0 con 0x7fbb740368a0 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbb740368a0 msgr2=0x7fbb74038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:20.720 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 --2- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbb740368a0 0x7fbb74038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009990 tx=0x7fbb70006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 msgr2=0x7fbb88078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 --2- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88078c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800b440 tx=0x7fbb78009250 comp rx=0 tx=0).stop 2024-04-24T23:04:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 shutdown_connections 2024-04-24T23:04:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 --2- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbb740368a0 0x7fbb74038d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 --2- 172.21.15.31:0/3132387399 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb88102880 0x7fbb88078c20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 >> 172.21.15.31:0/3132387399 conn(0x7fbb880fe060 msgr2=0x7fbb880feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 shutdown_connections 2024-04-24T23:04:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:20.084+0000 7fbb8dd66700 1 -- 172.21.15.31:0/3132387399 wait complete. 2024-04-24T23:04:21.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:21 smithi031 ceph-mon[26411]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:21.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:21 smithi031 ceph-mon[26411]: Generating ssh key... 2024-04-24T23:04:21.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:21 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:21.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:21 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYAxz0iE9pYj1EToztyOSmF0ydR1oBKqBGWN0/729C31TCxcZ1DsvEYLInoDGYnzTwuH9u6b7U5Z36k0UZL57H6fSxGDdha2uBECQmeWdeQYgFyVdEuZ4JbwhIwzuYpYoX7xdCIwb/eKsQTmQ/sohw5MwwjE+D1MUCo6bvM+crKjXCMKIYxttbXVuNC0f1CVuR6KSVKrExpkEApSIbhys5uqQ6ruyfApcvv1msIVJQoQPe2VPeMWaTWRR+jXiNqqa5uLUHO7+7Iiqo+cwKZEN40oEsCTpfS+nAta8dOBr3YeQqlU+IQCO2K+0T/Bytko7hnYGW5WyO/EkvN7nYiL5S63Px6TuncEJVAaiw7X91mdWMrO7Nn7vMUplI9Nq9B4+dO2TMXLJUHgv+W7PFb0u9L9tH51p6NmLMSp7W51huoGiYHGQPdatSf3BA2GXYQwFZb2/R6FeWFoMu5pdJyqZ3I05C4SDxhk+rpyOaEkPkshjIzZsQfiXMoHSmV5dsKqU= ceph-ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.162+0000 7f165a2e5700 1 Processor -- start 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.162+0000 7f165a2e5700 1 -- start start 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.163+0000 7f165a2e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f1654101b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.163+0000 7f165a2e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654102040 con 0x7f16541016f0 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.163+0000 7f1653fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f1654101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.163+0000 7f1653fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f1654101b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41386/0 (socket says 172.21.15.31:41386) 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.163+0000 7f1653fff700 1 -- 172.21.15.31:0/3875544994 learned_addr learned my addr 172.21.15.31:0/3875544994 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.163+0000 7f1653fff700 1 -- 172.21.15.31:0/3875544994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1654102180 con 0x7f16541016f0 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.163+0000 7f1653fff700 1 --2- 172.21.15.31:0/3875544994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f1654101b00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16440098e0 tx=0x7f1644009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab42ed64c0f65ad server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:21.958 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f16537fe700 1 -- 172.21.15.31:0/3875544994 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f164400b860 con 0x7f16541016f0 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f16537fe700 1 -- 172.21.15.31:0/3875544994 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f164400be50 con 0x7f16541016f0 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f16537fe700 1 -- 172.21.15.31:0/3875544994 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1644010380 con 0x7f16541016f0 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f165a2e5700 1 -- 172.21.15.31:0/3875544994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 msgr2=0x7f1654101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f165a2e5700 1 --2- 172.21.15.31:0/3875544994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f1654101b00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f16440098e0 tx=0x7f1644009250 comp rx=0 tx=0).stop 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f165a2e5700 1 -- 172.21.15.31:0/3875544994 shutdown_connections 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f165a2e5700 1 --2- 172.21.15.31:0/3875544994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f1654101b00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f165a2e5700 1 -- 172.21.15.31:0/3875544994 >> 172.21.15.31:0/3875544994 conn(0x7f16540fced0 msgr2=0x7f16540ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f165a2e5700 1 -- 172.21.15.31:0/3875544994 shutdown_connections 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.164+0000 7f165a2e5700 1 -- 172.21.15.31:0/3875544994 wait complete. 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.165+0000 7f165a2e5700 1 Processor -- start 2024-04-24T23:04:21.959 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.165+0000 7f165a2e5700 1 -- start start 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.165+0000 7f165a2e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f16541079e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.165+0000 7f165a2e5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654107f20 con 0x7f16541016f0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.165+0000 7f1653fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f16541079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.165+0000 7f1653fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f16541079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41388/0 (socket says 172.21.15.31:41388) 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.165+0000 7f1653fff700 1 -- 172.21.15.31:0/1729699842 learned_addr learned my addr 172.21.15.31:0/1729699842 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.165+0000 7f1653fff700 1 -- 172.21.15.31:0/1729699842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440094f0 con 0x7f16541016f0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.166+0000 7f1653fff700 1 --2- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f16541079e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1644000c00 tx=0x7f1644009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.166+0000 7f16517fa700 1 -- 172.21.15.31:0/1729699842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16440107c0 con 0x7f16541016f0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.166+0000 7f16517fa700 1 -- 172.21.15.31:0/1729699842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1644010db0 con 0x7f16541016f0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.166+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1654105f90 con 0x7f16541016f0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.166+0000 7f16517fa700 1 -- 172.21.15.31:0/1729699842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f164401ad00 con 0x7f16541016f0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.166+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1654106480 con 0x7f16541016f0 2024-04-24T23:04:21.960 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.167+0000 7f16517fa700 1 -- 172.21.15.31:0/1729699842 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f16440193c0 con 0x7f16541016f0 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.167+0000 7f16517fa700 1 --2- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f16400368a0 0x7f1640038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.167+0000 7f16517fa700 1 -- 172.21.15.31:0/1729699842 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f164404a620 con 0x7f16541016f0 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.167+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165404bed0 con 0x7f16541016f0 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.167+0000 7f164bfff700 1 --2- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f16400368a0 0x7f1640038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.168+0000 7f164bfff700 1 --2- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f16400368a0 0x7f1640038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f163c009990 tx=0x7f163c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.172+0000 7f16517fa700 1 -- 172.21.15.31:0/1729699842 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f16440196c0 con 0x7f16541016f0 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.369+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f16541069b0 con 0x7f16400368a0 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.369+0000 7f16517fa700 1 -- 172.21.15.31:0/1729699842 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f16541069b0 con 0x7f16400368a0 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.371+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f16400368a0 msgr2=0x7f1640038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.371+0000 7f165a2e5700 1 --2- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f16400368a0 0x7f1640038d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f163c009990 tx=0x7f163c006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.371+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 msgr2=0x7f16541079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:21.961 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.371+0000 7f165a2e5700 1 --2- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f16541079e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1644000c00 tx=0x7f1644009250 comp rx=0 tx=0).stop 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.372+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 shutdown_connections 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.372+0000 7f165a2e5700 1 --2- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f16400368a0 0x7f1640038d50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.372+0000 7f165a2e5700 1 --2- 172.21.15.31:0/1729699842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16541016f0 0x7f16541079e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.372+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 >> 172.21.15.31:0/1729699842 conn(0x7f16540fced0 msgr2=0x7f16540fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.372+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 shutdown_connections 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:21.372+0000 7f165a2e5700 1 -- 172.21.15.31:0/1729699842 wait complete. 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-04-24T23:04:21.962 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-04-24T23:04:22.159 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:22 smithi031 ceph-mon[26411]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:23.189 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:23 smithi031 ceph-mon[26411]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:24.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:24 smithi031 ceph-mon[26411]: Deploying cephadm binary to smithi031 2024-04-24T23:04:25.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-04-24T23:04:25.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.354+0000 7f855e392700 1 Processor -- start 2024-04-24T23:04:25.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.354+0000 7f855e392700 1 -- start start 2024-04-24T23:04:25.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.354+0000 7f855e392700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f8558102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:25.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.354+0000 7f855e392700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85581031d0 con 0x7f8558102880 2024-04-24T23:04:25.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.354+0000 7f8557fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f8558102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:25.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.354+0000 7f8557fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f8558102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41390/0 (socket says 172.21.15.31:41390) 2024-04-24T23:04:25.022 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.354+0000 7f8557fff700 1 -- 172.21.15.31:0/1460276395 learned_addr learned my addr 172.21.15.31:0/1460276395 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f8557fff700 1 -- 172.21.15.31:0/1460276395 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8558103310 con 0x7f8558102880 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f8557fff700 1 --2- 172.21.15.31:0/1460276395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f8558102c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85480098e0 tx=0x7f8548009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6636329f47be001 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f8556ffd700 1 -- 172.21.15.31:0/1460276395 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f854800b860 con 0x7f8558102880 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f8556ffd700 1 -- 172.21.15.31:0/1460276395 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f854800be50 con 0x7f8558102880 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f8556ffd700 1 -- 172.21.15.31:0/1460276395 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8548010380 con 0x7f8558102880 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f855e392700 1 -- 172.21.15.31:0/1460276395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 msgr2=0x7f8558102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f855e392700 1 --2- 172.21.15.31:0/1460276395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f8558102c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f85480098e0 tx=0x7f8548009250 comp rx=0 tx=0).stop 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f855e392700 1 -- 172.21.15.31:0/1460276395 shutdown_connections 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f855e392700 1 --2- 172.21.15.31:0/1460276395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f8558102c90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.355+0000 7f855e392700 1 -- 172.21.15.31:0/1460276395 >> 172.21.15.31:0/1460276395 conn(0x7f85580fe060 msgr2=0x7f8558100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.356+0000 7f855e392700 1 -- 172.21.15.31:0/1460276395 shutdown_connections 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.356+0000 7f855e392700 1 -- 172.21.15.31:0/1460276395 wait complete. 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.356+0000 7f855e392700 1 Processor -- start 2024-04-24T23:04:25.023 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.356+0000 7f855e392700 1 -- start start 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.356+0000 7f855e392700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f85581115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.356+0000 7f855e392700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558111b30 con 0x7f8558102880 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f8557fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f85581115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f8557fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f85581115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41396/0 (socket says 172.21.15.31:41396) 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f8557fff700 1 -- 172.21.15.31:0/2065690277 learned_addr learned my addr 172.21.15.31:0/2065690277 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f8557fff700 1 -- 172.21.15.31:0/2065690277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480094f0 con 0x7f8558102880 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f8557fff700 1 --2- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f85581115f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f854800b440 tx=0x7f8548009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f8554ff9700 1 -- 172.21.15.31:0/2065690277 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8548010760 con 0x7f8558102880 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f8554ff9700 1 -- 172.21.15.31:0/2065690277 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f8548010d50 con 0x7f8558102880 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8558111d30 con 0x7f8558102880 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f8554ff9700 1 -- 172.21.15.31:0/2065690277 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f854801ae20 con 0x7f8558102880 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.357+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85581128a0 con 0x7f8558102880 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.358+0000 7f8554ff9700 1 -- 172.21.15.31:0/2065690277 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f854802e440 con 0x7f8558102880 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.358+0000 7f8554ff9700 1 --2- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8540036830 0x7f8540038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:25.024 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.358+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855805f5c0 con 0x7f8558102880 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.358+0000 7f8554ff9700 1 -- 172.21.15.31:0/2065690277 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f854804a5d0 con 0x7f8558102880 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.358+0000 7f85577fe700 1 --2- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8540036830 0x7f8540038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.359+0000 7f85577fe700 1 --2- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8540036830 0x7f8540038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f854c009990 tx=0x7f854c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.363+0000 7f8554ff9700 1 -- 172.21.15.31:0/2065690277 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8548018940 con 0x7f8558102880 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:22.565+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}) v1 -- 0x7f85580ff200 con 0x7f8540036830 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.503+0000 7f8554ff9700 1 -- 172.21.15.31:0/2065690277 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f85580ff200 con 0x7f8540036830 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8540036830 msgr2=0x7f8540038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 --2- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8540036830 0x7f8540038ce0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f854c009990 tx=0x7f854c006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 msgr2=0x7f85581115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 --2- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f85581115f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f854800b440 tx=0x7f8548009250 comp rx=0 tx=0).stop 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 shutdown_connections 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 --2- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8540036830 0x7f8540038ce0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 --2- 172.21.15.31:0/2065690277 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8558102880 0x7f85581115f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:25.025 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 >> 172.21.15.31:0/2065690277 conn(0x7f85580fe060 msgr2=0x7f85580feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:25.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 shutdown_connections 2024-04-24T23:04:25.026 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:24.506+0000 7f855e392700 1 -- 172.21.15.31:0/2065690277 wait complete. 2024-04-24T23:04:25.026 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying mon service with default placement... 2024-04-24T23:04:25.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:25 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:25.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:25 smithi031 ceph-mon[26411]: Added host smithi031 2024-04-24T23:04:25.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:25 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:26.281 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-24T23:04:26.281 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.420+0000 7f29f38d1700 1 Processor -- start 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.420+0000 7f29f38d1700 1 -- start start 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.420+0000 7f29f38d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.420+0000 7f29f38d1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec105970 con 0x7f29ec071310 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.421+0000 7f29f166d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.421+0000 7f29f166d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41398/0 (socket says 172.21.15.31:41398) 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.421+0000 7f29f166d700 1 -- 172.21.15.31:0/1545815777 learned_addr learned my addr 172.21.15.31:0/1545815777 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.421+0000 7f29f166d700 1 -- 172.21.15.31:0/1545815777 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec105ab0 con 0x7f29ec071310 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.421+0000 7f29f166d700 1 --2- 172.21.15.31:0/1545815777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec105430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009e50 tx=0x7f29e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea69cf7467c257c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.421+0000 7f29ebfff700 1 -- 172.21.15.31:0/1545815777 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29e40092e0 con 0x7f29ec071310 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.421+0000 7f29ebfff700 1 -- 172.21.15.31:0/1545815777 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f29e4012430 con 0x7f29ec071310 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.421+0000 7f29ebfff700 1 -- 172.21.15.31:0/1545815777 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29e40075c0 con 0x7f29ec071310 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.422+0000 7f29f38d1700 1 -- 172.21.15.31:0/1545815777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 msgr2=0x7f29ec105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.422+0000 7f29f38d1700 1 --2- 172.21.15.31:0/1545815777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec105430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009e50 tx=0x7f29e4009840 comp rx=0 tx=0).stop 2024-04-24T23:04:26.282 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.422+0000 7f29f38d1700 1 -- 172.21.15.31:0/1545815777 shutdown_connections 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.422+0000 7f29f38d1700 1 --2- 172.21.15.31:0/1545815777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec105430 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.422+0000 7f29f38d1700 1 -- 172.21.15.31:0/1545815777 >> 172.21.15.31:0/1545815777 conn(0x7f29ec0fcea0 msgr2=0x7f29ec0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.422+0000 7f29f38d1700 1 -- 172.21.15.31:0/1545815777 shutdown_connections 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.422+0000 7f29f38d1700 1 -- 172.21.15.31:0/1545815777 wait complete. 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f38d1700 1 Processor -- start 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f38d1700 1 -- start start 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f38d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f38d1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec1032c0 con 0x7f29ec071310 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f166d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f166d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41404/0 (socket says 172.21.15.31:41404) 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f166d700 1 -- 172.21.15.31:0/3175366628 learned_addr learned my addr 172.21.15.31:0/3175366628 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f166d700 1 -- 172.21.15.31:0/3175366628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e4009ae0 con 0x7f29ec071310 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.423+0000 7f29f166d700 1 --2- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec102d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009fd0 tx=0x7f29e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:26.283 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.424+0000 7f29e9ffb700 1 -- 172.21.15.31:0/3175366628 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29e40092e0 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.424+0000 7f29e9ffb700 1 -- 172.21.15.31:0/3175366628 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f29e4007d10 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.424+0000 7f29e9ffb700 1 -- 172.21.15.31:0/3175366628 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f29e401c880 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.424+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec101390 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.424+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec101880 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.425+0000 7f29e9ffb700 1 -- 172.21.15.31:0/3175366628 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f29e4019070 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.425+0000 7f29e9ffb700 1 --2- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f29d4036850 0x7f29d4038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.425+0000 7f29e9ffb700 1 -- 172.21.15.31:0/3175366628 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f29e4049df0 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.425+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec04bed0 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.425+0000 7f29f0e6c700 1 --2- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f29d4036850 0x7f29d4038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.425+0000 7f29f0e6c700 1 --2- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f29d4036850 0x7f29d4038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009990 tx=0x7f29e0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.430+0000 7f29e9ffb700 1 -- 172.21.15.31:0/3175366628 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f29e4017080 con 0x7f29ec071310 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.638+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f29ec102040 con 0x7f29d4036850 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.645+0000 7f29e9ffb700 1 -- 172.21.15.31:0/3175366628 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f29ec102040 con 0x7f29d4036850 2024-04-24T23:04:26.284 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f29d4036850 msgr2=0x7f29d4038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 --2- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f29d4036850 0x7f29d4038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f29e0009990 tx=0x7f29e0006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 msgr2=0x7f29ec102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 --2- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec102d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009fd0 tx=0x7f29e4009840 comp rx=0 tx=0).stop 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 shutdown_connections 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 --2- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f29d4036850 0x7f29d4038d00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 --2- 172.21.15.31:0/3175366628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29ec071310 0x7f29ec102d80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 >> 172.21.15.31:0/3175366628 conn(0x7f29ec0fcea0 msgr2=0x7f29ec0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 shutdown_connections 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:25.647+0000 7f29f38d1700 1 -- 172.21.15.31:0/3175366628 wait complete. 2024-04-24T23:04:26.285 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying mgr service with default placement... 2024-04-24T23:04:26.850 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:26 smithi031 ceph-mon[26411]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:26.851 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:26 smithi031 ceph-mon[26411]: Saving service mon spec with placement count:5 2024-04-24T23:04:26.851 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:26 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:27.741 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-24T23:04:27.742 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.849+0000 7fdfd91ce700 1 Processor -- start 2024-04-24T23:04:27.742 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.849+0000 7fdfd91ce700 1 -- start start 2024-04-24T23:04:27.742 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.849+0000 7fdfd91ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd4102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:27.742 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.849+0000 7fdfd91ce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4103160 con 0x7fdfd4102810 2024-04-24T23:04:27.742 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.849+0000 7fdfd2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd4102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:27.742 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.849+0000 7fdfd2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd4102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41420/0 (socket says 172.21.15.31:41420) 2024-04-24T23:04:27.742 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.849+0000 7fdfd2d9d700 1 -- 172.21.15.31:0/4177897884 learned_addr learned my addr 172.21.15.31:0/4177897884 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.849+0000 7fdfd2d9d700 1 -- 172.21.15.31:0/4177897884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd41032a0 con 0x7fdfd4102810 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.850+0000 7fdfd2d9d700 1 --2- 172.21.15.31:0/4177897884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd4102c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80098e0 tx=0x7fdfc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdcfde463cf18d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.850+0000 7fdfd1d9b700 1 -- 172.21.15.31:0/4177897884 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfc800b860 con 0x7fdfd4102810 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.850+0000 7fdfd1d9b700 1 -- 172.21.15.31:0/4177897884 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdfc800be50 con 0x7fdfd4102810 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.850+0000 7fdfd1d9b700 1 -- 172.21.15.31:0/4177897884 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfc8010380 con 0x7fdfd4102810 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 -- 172.21.15.31:0/4177897884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 msgr2=0x7fdfd4102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 --2- 172.21.15.31:0/4177897884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd4102c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdfc80098e0 tx=0x7fdfc8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 -- 172.21.15.31:0/4177897884 shutdown_connections 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 --2- 172.21.15.31:0/4177897884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd4102c20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 -- 172.21.15.31:0/4177897884 >> 172.21.15.31:0/4177897884 conn(0x7fdfd40fdfd0 msgr2=0x7fdfd41003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 -- 172.21.15.31:0/4177897884 shutdown_connections 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 -- 172.21.15.31:0/4177897884 wait complete. 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 Processor -- start 2024-04-24T23:04:27.743 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.851+0000 7fdfd91ce700 1 -- start start 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd91ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd41116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd91ce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4111be0 con 0x7fdfd4102810 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd41116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd41116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41434/0 (socket says 172.21.15.31:41434) 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd2d9d700 1 -- 172.21.15.31:0/3062545791 learned_addr learned my addr 172.21.15.31:0/3062545791 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd2d9d700 1 -- 172.21.15.31:0/3062545791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc80094f0 con 0x7fdfd4102810 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd2d9d700 1 --2- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd41116a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8006b00 tx=0x7fdfc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfbb7fe700 1 -- 172.21.15.31:0/3062545791 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfc8010760 con 0x7fdfd4102810 2024-04-24T23:04:27.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd4111de0 con 0x7fdfd4102810 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfbb7fe700 1 -- 172.21.15.31:0/3062545791 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fdfc8010d50 con 0x7fdfd4102810 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfbb7fe700 1 -- 172.21.15.31:0/3062545791 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdfc801ad40 con 0x7fdfd4102810 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.852+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd4112950 con 0x7fdfd4102810 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.853+0000 7fdfbb7fe700 1 -- 172.21.15.31:0/3062545791 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fdfc802e440 con 0x7fdfd4102810 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.854+0000 7fdfbb7fe700 1 --2- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fdfbc036850 0x7fdfbc038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.854+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd405f5c0 con 0x7fdfd4102810 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.854+0000 7fdfbb7fe700 1 -- 172.21.15.31:0/3062545791 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fdfc804a580 con 0x7fdfd4102810 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.854+0000 7fdfd259c700 1 --2- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fdfbc036850 0x7fdfbc038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.854+0000 7fdfd259c700 1 --2- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fdfbc036850 0x7fdfbc038d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009990 tx=0x7fdfc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:26.859+0000 7fdfbb7fe700 1 -- 172.21.15.31:0/3062545791 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdfc80108c0 con 0x7fdfd4102810 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.069+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfd40ff200 con 0x7fdfbc036850 2024-04-24T23:04:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.077+0000 7fdfbb7fe700 1 -- 172.21.15.31:0/3062545791 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdfd40ff200 con 0x7fdfbc036850 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fdfbc036850 msgr2=0x7fdfbc038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 --2- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fdfbc036850 0x7fdfbc038d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009990 tx=0x7fdfc4006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 msgr2=0x7fdfd41116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 --2- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd41116a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8006b00 tx=0x7fdfc8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 shutdown_connections 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 --2- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fdfbc036850 0x7fdfbc038d00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 --2- 172.21.15.31:0/3062545791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfd4102810 0x7fdfd41116a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 >> 172.21.15.31:0/3062545791 conn(0x7fdfd40fdfd0 msgr2=0x7fdfd40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:27.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 shutdown_connections 2024-04-24T23:04:27.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:27.079+0000 7fdfd91ce700 1 -- 172.21.15.31:0/3062545791 wait complete. 2024-04-24T23:04:27.747 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying crash service with default placement... 2024-04-24T23:04:28.344 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:28 smithi031 ceph-mon[26411]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:28.344 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:28 smithi031 ceph-mon[26411]: Saving service mgr spec with placement count:2 2024-04-24T23:04:28.344 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:28 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.123+0000 7fbc062a1700 1 Processor -- start 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.123+0000 7fbc062a1700 1 -- start start 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbc062a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc00102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbc062a1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00103200 con 0x7fbc001028b0 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbbfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc00102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbbfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc00102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41450/0 (socket says 172.21.15.31:41450) 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbbfffff700 1 -- 172.21.15.31:0/244812032 learned_addr learned my addr 172.21.15.31:0/244812032 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:29.217 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbbfffff700 1 -- 172.21.15.31:0/244812032 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc00103340 con 0x7fbc001028b0 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbbfffff700 1 --2- 172.21.15.31:0/244812032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc00102cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098e0 tx=0x7fbbf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f582926948ae1cf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbbfeffd700 1 -- 172.21.15.31:0/244812032 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbf000b860 con 0x7fbc001028b0 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.124+0000 7fbbfeffd700 1 -- 172.21.15.31:0/244812032 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbbf000be50 con 0x7fbc001028b0 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.125+0000 7fbbfeffd700 1 -- 172.21.15.31:0/244812032 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbf0010380 con 0x7fbc001028b0 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.125+0000 7fbc062a1700 1 -- 172.21.15.31:0/244812032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 msgr2=0x7fbc00102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.125+0000 7fbc062a1700 1 --2- 172.21.15.31:0/244812032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc00102cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098e0 tx=0x7fbbf0009250 comp rx=0 tx=0).stop 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.125+0000 7fbc062a1700 1 -- 172.21.15.31:0/244812032 shutdown_connections 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.125+0000 7fbc062a1700 1 --2- 172.21.15.31:0/244812032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc00102cc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.125+0000 7fbc062a1700 1 -- 172.21.15.31:0/244812032 >> 172.21.15.31:0/244812032 conn(0x7fbc000fdff0 msgr2=0x7fbc00100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.125+0000 7fbc062a1700 1 -- 172.21.15.31:0/244812032 shutdown_connections 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.125+0000 7fbc062a1700 1 -- 172.21.15.31:0/244812032 wait complete. 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.126+0000 7fbc062a1700 1 Processor -- start 2024-04-24T23:04:29.218 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.126+0000 7fbc062a1700 1 -- start start 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.126+0000 7fbc062a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc001115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.126+0000 7fbc062a1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc00111b20 con 0x7fbc001028b0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.126+0000 7fbbfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc001115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.126+0000 7fbbfffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc001115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41452/0 (socket says 172.21.15.31:41452) 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.126+0000 7fbbfffff700 1 -- 172.21.15.31:0/3780704163 learned_addr learned my addr 172.21.15.31:0/3780704163 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.126+0000 7fbbfffff700 1 -- 172.21.15.31:0/3780704163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00094f0 con 0x7fbc001028b0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.127+0000 7fbbfffff700 1 --2- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc001115e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0000c00 tx=0x7fbbf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.127+0000 7fbbfcff9700 1 -- 172.21.15.31:0/3780704163 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbf00107c0 con 0x7fbc001028b0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.127+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc00111d20 con 0x7fbc001028b0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.127+0000 7fbbfcff9700 1 -- 172.21.15.31:0/3780704163 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fbbf0010db0 con 0x7fbc001028b0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.127+0000 7fbbfcff9700 1 -- 172.21.15.31:0/3780704163 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbf001ad00 con 0x7fbc001028b0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.127+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc00112890 con 0x7fbc001028b0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.128+0000 7fbbfcff9700 1 -- 172.21.15.31:0/3780704163 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fbbf00193c0 con 0x7fbc001028b0 2024-04-24T23:04:29.219 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.128+0000 7fbbfcff9700 1 --2- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbbe8036850 0x7fbbe8038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.128+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc001121b0 con 0x7fbc001028b0 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.128+0000 7fbbfcff9700 1 -- 172.21.15.31:0/3780704163 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbbf004a820 con 0x7fbc001028b0 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.128+0000 7fbbff7fe700 1 --2- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbbe8036850 0x7fbbe8038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.129+0000 7fbbff7fe700 1 --2- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbbe8036850 0x7fbbe8038d00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009990 tx=0x7fbbf4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.133+0000 7fbbfcff9700 1 -- 172.21.15.31:0/3780704163 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbbf0022930 con 0x7fbc001028b0 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.342+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc000ff200 con 0x7fbbe8036850 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.346+0000 7fbbfcff9700 1 -- 172.21.15.31:0/3780704163 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fbc000ff200 con 0x7fbbe8036850 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.348+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbbe8036850 msgr2=0x7fbbe8038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.348+0000 7fbc062a1700 1 --2- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbbe8036850 0x7fbbe8038d00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009990 tx=0x7fbbf4006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.348+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 msgr2=0x7fbc001115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.348+0000 7fbc062a1700 1 --2- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc001115e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0000c00 tx=0x7fbbf0009250 comp rx=0 tx=0).stop 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.349+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 shutdown_connections 2024-04-24T23:04:29.220 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.349+0000 7fbc062a1700 1 --2- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbbe8036850 0x7fbbe8038d00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:29.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.349+0000 7fbc062a1700 1 --2- 172.21.15.31:0/3780704163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc001028b0 0x7fbc001115e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:29.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.349+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 >> 172.21.15.31:0/3780704163 conn(0x7fbc000fdff0 msgr2=0x7fbc000feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:29.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.349+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 shutdown_connections 2024-04-24T23:04:29.221 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:28.349+0000 7fbc062a1700 1 -- 172.21.15.31:0/3780704163 wait complete. 2024-04-24T23:04:29.221 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying prometheus service with default placement... 2024-04-24T23:04:29.459 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:29 smithi031 ceph-mon[26411]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:29.459 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:29 smithi031 ceph-mon[26411]: Saving service crash spec with placement * 2024-04-24T23:04:29.459 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:29 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:29.459 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:29 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:29.460 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:29 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:30.873 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-24T23:04:30.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.639+0000 7f05b3921700 1 Processor -- start 2024-04-24T23:04:30.878 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.639+0000 7f05b3921700 1 -- start start 2024-04-24T23:04:30.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.639+0000 7f05b3921700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac101b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.639+0000 7f05b3921700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1020b0 con 0x7f05ac101760 2024-04-24T23:04:30.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.639+0000 7f05b16bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.639+0000 7f05b16bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55092/0 (socket says 172.21.15.31:55092) 2024-04-24T23:04:30.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.639+0000 7f05b16bd700 1 -- 172.21.15.31:0/1190287890 learned_addr learned my addr 172.21.15.31:0/1190287890 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:30.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.640+0000 7f05b16bd700 1 -- 172.21.15.31:0/1190287890 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac1021f0 con 0x7f05ac101760 2024-04-24T23:04:30.879 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.640+0000 7f05b16bd700 1 --2- 172.21.15.31:0/1190287890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac101b70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05a800bb80 tx=0x7f05a800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4a0ea1bce8423f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:30.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.640+0000 7f05a3fff700 1 -- 172.21.15.31:0/1190287890 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a800dd00 con 0x7f05ac101760 2024-04-24T23:04:30.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.640+0000 7f05a3fff700 1 -- 172.21.15.31:0/1190287890 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05a800de60 con 0x7f05ac101760 2024-04-24T23:04:30.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.640+0000 7f05a3fff700 1 -- 172.21.15.31:0/1190287890 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a8018920 con 0x7f05ac101760 2024-04-24T23:04:30.883 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.640+0000 7f05b3921700 1 -- 172.21.15.31:0/1190287890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 msgr2=0x7f05ac101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.641+0000 7f05b3921700 1 --2- 172.21.15.31:0/1190287890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac101b70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f05a800bb80 tx=0x7f05a800b4f0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.641+0000 7f05b3921700 1 -- 172.21.15.31:0/1190287890 shutdown_connections 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.641+0000 7f05b3921700 1 --2- 172.21.15.31:0/1190287890 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac101b70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.641+0000 7f05b3921700 1 -- 172.21.15.31:0/1190287890 >> 172.21.15.31:0/1190287890 conn(0x7f05ac0fcea0 msgr2=0x7f05ac0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.641+0000 7f05b3921700 1 -- 172.21.15.31:0/1190287890 shutdown_connections 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.641+0000 7f05b3921700 1 -- 172.21.15.31:0/1190287890 wait complete. 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.641+0000 7f05b3921700 1 Processor -- start 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.641+0000 7f05b3921700 1 -- start start 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05b3921700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac10f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05b3921700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac10f8b0 con 0x7f05ac101760 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05b16bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac10f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05b16bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac10f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55104/0 (socket says 172.21.15.31:55104) 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05b16bd700 1 -- 172.21.15.31:0/3877125146 learned_addr learned my addr 172.21.15.31:0/3877125146 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:30.884 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05b16bd700 1 -- 172.21.15.31:0/3877125146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a800b790 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05b16bd700 1 --2- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac10f370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05a8006d40 tx=0x7f05a800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05a1ffb700 1 -- 172.21.15.31:0/3877125146 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a800dc70 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05a1ffb700 1 -- 172.21.15.31:0/3877125146 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05a801d050 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac10fab0 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.642+0000 7f05a1ffb700 1 -- 172.21.15.31:0/3877125146 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05a80223d0 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.643+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac110620 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.643+0000 7f05a1ffb700 1 -- 172.21.15.31:0/3877125146 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f05a8022650 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.644+0000 7f05a1ffb700 1 --2- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f0598036850 0x7f0598038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.644+0000 7f05a1ffb700 1 -- 172.21.15.31:0/3877125146 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f05a8014070 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.644+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac05f5c0 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.644+0000 7f05b0ebc700 1 --2- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f0598036850 0x7f0598038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.644+0000 7f05b0ebc700 1 --2- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f0598036850 0x7f0598038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f059c009990 tx=0x7f059c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.653+0000 7f05a1ffb700 1 -- 172.21.15.31:0/3877125146 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f05a800eb70 con 0x7f05ac101760 2024-04-24T23:04:30.885 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.867+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f05ac107380 con 0x7f0598036850 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.872+0000 7f05a1ffb700 1 -- 172.21.15.31:0/3877125146 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f05ac107380 con 0x7f0598036850 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f0598036850 msgr2=0x7f0598038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 --2- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f0598036850 0x7f0598038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f059c009990 tx=0x7f059c006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 msgr2=0x7f05ac10f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 --2- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac10f370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05a8006d40 tx=0x7f05a800b4f0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 shutdown_connections 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 --2- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f0598036850 0x7f0598038d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 --2- 172.21.15.31:0/3877125146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05ac101760 0x7f05ac10f370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 >> 172.21.15.31:0/3877125146 conn(0x7f05ac0fcea0 msgr2=0x7f05ac105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 shutdown_connections 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:29.874+0000 7f05b3921700 1 -- 172.21.15.31:0/3877125146 wait complete. 2024-04-24T23:04:30.886 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying grafana service with default placement... 2024-04-24T23:04:31.235 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:30 smithi031 ceph-mon[26411]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:31.235 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:30 smithi031 ceph-mon[26411]: Saving service prometheus spec with placement count:1 2024-04-24T23:04:31.235 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:30 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:31.235 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:30 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.340+0000 7ff4d6810700 1 Processor -- start 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.340+0000 7ff4d6810700 1 -- start start 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.340+0000 7ff4d6810700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d01021f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.340+0000 7ff4d6810700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0102730 con 0x7ff4d0101de0 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.341+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d01021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.341+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d01021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55110/0 (socket says 172.21.15.31:55110) 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.341+0000 7ff4cffff700 1 -- 172.21.15.31:0/1923642605 learned_addr learned my addr 172.21.15.31:0/1923642605 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.341+0000 7ff4cffff700 1 -- 172.21.15.31:0/1923642605 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d0102870 con 0x7ff4d0101de0 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.341+0000 7ff4cffff700 1 --2- 172.21.15.31:0/1923642605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d01021f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00098e0 tx=0x7ff4c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e035f5786c4084a server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.341+0000 7ff4ceffd700 1 -- 172.21.15.31:0/1923642605 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4c000b860 con 0x7ff4d0101de0 2024-04-24T23:04:32.145 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.341+0000 7ff4ceffd700 1 -- 172.21.15.31:0/1923642605 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4c000be90 con 0x7ff4d0101de0 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.341+0000 7ff4ceffd700 1 -- 172.21.15.31:0/1923642605 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4c00103c0 con 0x7ff4d0101de0 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.342+0000 7ff4d6810700 1 -- 172.21.15.31:0/1923642605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 msgr2=0x7ff4d01021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.342+0000 7ff4d6810700 1 --2- 172.21.15.31:0/1923642605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d01021f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00098e0 tx=0x7ff4c0009250 comp rx=0 tx=0).stop 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.342+0000 7ff4d6810700 1 -- 172.21.15.31:0/1923642605 shutdown_connections 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.342+0000 7ff4d6810700 1 --2- 172.21.15.31:0/1923642605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d01021f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.342+0000 7ff4d6810700 1 -- 172.21.15.31:0/1923642605 >> 172.21.15.31:0/1923642605 conn(0x7ff4d00fc690 msgr2=0x7ff4d00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.342+0000 7ff4d6810700 1 -- 172.21.15.31:0/1923642605 shutdown_connections 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.342+0000 7ff4d6810700 1 -- 172.21.15.31:0/1923642605 wait complete. 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.343+0000 7ff4d6810700 1 Processor -- start 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.343+0000 7ff4d6810700 1 -- start start 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.343+0000 7ff4d6810700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d010f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.343+0000 7ff4d6810700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d010f9a0 con 0x7ff4d0101de0 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.343+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d010f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:32.146 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.343+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d010f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55114/0 (socket says 172.21.15.31:55114) 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.343+0000 7ff4cffff700 1 -- 172.21.15.31:0/2158303107 learned_addr learned my addr 172.21.15.31:0/2158303107 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.344+0000 7ff4cffff700 1 -- 172.21.15.31:0/2158303107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c00094f0 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.344+0000 7ff4cffff700 1 --2- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d010f460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0006d40 tx=0x7ff4c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.344+0000 7ff4ccff9700 1 -- 172.21.15.31:0/2158303107 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4c0010840 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.344+0000 7ff4ccff9700 1 -- 172.21.15.31:0/2158303107 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4c0010e70 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.344+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d010fba0 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.344+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d0110710 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.344+0000 7ff4ccff9700 1 -- 172.21.15.31:0/2158303107 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff4c0024ea0 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.345+0000 7ff4ccff9700 1 -- 172.21.15.31:0/2158303107 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ff4c001a500 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.345+0000 7ff4ccff9700 1 --2- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7ff4b8038a90 0x7ff4b803af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.345+0000 7ff4ccff9700 1 -- 172.21.15.31:0/2158303107 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff4c004b710 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.345+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4bc0052e0 con 0x7ff4d0101de0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.345+0000 7ff4cf7fe700 1 --2- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7ff4b8038a90 0x7ff4b803af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:32.147 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.346+0000 7ff4cf7fe700 1 --2- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7ff4b8038a90 0x7ff4b803af40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009990 tx=0x7ff4c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.351+0000 7ff4ccff9700 1 -- 172.21.15.31:0/2158303107 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff4c0018940 con 0x7ff4d0101de0 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.567+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4bc000bf0 con 0x7ff4b8038a90 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.572+0000 7ff4ccff9700 1 -- 172.21.15.31:0/2158303107 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff4bc000bf0 con 0x7ff4b8038a90 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7ff4b8038a90 msgr2=0x7ff4b803af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 --2- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7ff4b8038a90 0x7ff4b803af40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009990 tx=0x7ff4c4006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 msgr2=0x7ff4d010f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 --2- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d010f460 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0006d40 tx=0x7ff4c0009250 comp rx=0 tx=0).stop 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 shutdown_connections 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 --2- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7ff4b8038a90 0x7ff4b803af40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 --2- 172.21.15.31:0/2158303107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4d0101de0 0x7ff4d010f460 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 >> 172.21.15.31:0/2158303107 conn(0x7ff4d00fc690 msgr2=0x7ff4d00fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 shutdown_connections 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:31.575+0000 7ff4d6810700 1 -- 172.21.15.31:0/2158303107 wait complete. 2024-04-24T23:04:32.148 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying node-exporter service with default placement... 2024-04-24T23:04:32.517 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:32 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:32.518 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:32 smithi031 ceph-mon[26411]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:32.518 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:32 smithi031 ceph-mon[26411]: Saving service grafana spec with placement count:1 2024-04-24T23:04:32.518 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:32 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-24T23:04:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.548+0000 7f1d47a48700 1 Processor -- start 2024-04-24T23:04:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.549+0000 7f1d47a48700 1 -- start start 2024-04-24T23:04:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.549+0000 7f1d47a48700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.549+0000 7f1d47a48700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401031e0 con 0x7f1d40102890 2024-04-24T23:04:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.549+0000 7f1d457e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:33.453 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.549+0000 7f1d457e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55118/0 (socket says 172.21.15.31:55118) 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.549+0000 7f1d457e4700 1 -- 172.21.15.31:0/3451528226 learned_addr learned my addr 172.21.15.31:0/3451528226 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.549+0000 7f1d457e4700 1 -- 172.21.15.31:0/3451528226 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d40103320 con 0x7f1d40102890 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.549+0000 7f1d457e4700 1 --2- 172.21.15.31:0/3451528226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40102ca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d340098e0 tx=0x7f1d34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb0e151ac70870d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d33fff700 1 -- 172.21.15.31:0/3451528226 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d3400b860 con 0x7f1d40102890 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d33fff700 1 -- 172.21.15.31:0/3451528226 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d3400be90 con 0x7f1d40102890 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d33fff700 1 -- 172.21.15.31:0/3451528226 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d340103c0 con 0x7f1d40102890 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d47a48700 1 -- 172.21.15.31:0/3451528226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 msgr2=0x7f1d40102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d47a48700 1 --2- 172.21.15.31:0/3451528226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40102ca0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d340098e0 tx=0x7f1d34009250 comp rx=0 tx=0).stop 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d47a48700 1 -- 172.21.15.31:0/3451528226 shutdown_connections 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d47a48700 1 --2- 172.21.15.31:0/3451528226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40102ca0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:33.454 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d47a48700 1 -- 172.21.15.31:0/3451528226 >> 172.21.15.31:0/3451528226 conn(0x7f1d400fe010 msgr2=0x7f1d40100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d47a48700 1 -- 172.21.15.31:0/3451528226 shutdown_connections 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.550+0000 7f1d47a48700 1 -- 172.21.15.31:0/3451528226 wait complete. 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.551+0000 7f1d47a48700 1 Processor -- start 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.551+0000 7f1d47a48700 1 -- start start 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.551+0000 7f1d47a48700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.551+0000 7f1d47a48700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40111b40 con 0x7f1d40102890 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.551+0000 7f1d457e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.551+0000 7f1d457e4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55130/0 (socket says 172.21.15.31:55130) 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.551+0000 7f1d457e4700 1 -- 172.21.15.31:0/3055736984 learned_addr learned my addr 172.21.15.31:0/3055736984 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.552+0000 7f1d457e4700 1 -- 172.21.15.31:0/3055736984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d340094f0 con 0x7f1d40102890 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.552+0000 7f1d457e4700 1 --2- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40111600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d340090b0 tx=0x7f1d34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.552+0000 7f1d31ffb700 1 -- 172.21.15.31:0/3055736984 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d34010840 con 0x7f1d40102890 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.552+0000 7f1d31ffb700 1 -- 172.21.15.31:0/3055736984 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d34010e70 con 0x7f1d40102890 2024-04-24T23:04:33.455 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.552+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d40111d40 con 0x7f1d40102890 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.552+0000 7f1d31ffb700 1 -- 172.21.15.31:0/3055736984 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1d34024e40 con 0x7f1d40102890 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.552+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d401128b0 con 0x7f1d40102890 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.553+0000 7f1d31ffb700 1 -- 172.21.15.31:0/3055736984 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f1d3401a530 con 0x7f1d40102890 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.553+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4005f5c0 con 0x7f1d40102890 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.553+0000 7f1d31ffb700 1 --2- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1d2c036800 0x7f1d2c038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.553+0000 7f1d31ffb700 1 -- 172.21.15.31:0/3055736984 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f1d3404a8b0 con 0x7f1d40102890 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.554+0000 7f1d44fe3700 1 --2- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1d2c036800 0x7f1d2c038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.554+0000 7f1d44fe3700 1 --2- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1d2c036800 0x7f1d2c038cb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009990 tx=0x7f1d3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.558+0000 7f1d31ffb700 1 -- 172.21.15.31:0/3055736984 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1d34018350 con 0x7f1d40102890 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.776+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d400ff1b0 con 0x7f1d2c036800 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.781+0000 7f1d31ffb700 1 -- 172.21.15.31:0/3055736984 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f1d400ff1b0 con 0x7f1d2c036800 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1d2c036800 msgr2=0x7f1d2c038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 --2- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1d2c036800 0x7f1d2c038cb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009990 tx=0x7f1d3c006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:33.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 msgr2=0x7f1d40111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:33.457 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 --2- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40111600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d340090b0 tx=0x7f1d34009250 comp rx=0 tx=0).stop 2024-04-24T23:04:33.457 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 shutdown_connections 2024-04-24T23:04:33.457 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 --2- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f1d2c036800 0x7f1d2c038cb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:33.457 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 --2- 172.21.15.31:0/3055736984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d40102890 0x7f1d40111600 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:33.457 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 >> 172.21.15.31:0/3055736984 conn(0x7f1d400fe010 msgr2=0x7f1d400feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:33.457 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 shutdown_connections 2024-04-24T23:04:33.457 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:32.783+0000 7f1d47a48700 1 -- 172.21.15.31:0/3055736984 wait complete. 2024-04-24T23:04:33.457 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying alertmanager service with default placement... 2024-04-24T23:04:34.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:33 smithi031 ceph-mon[26411]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:34.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:33 smithi031 ceph-mon[26411]: Saving service node-exporter spec with placement * 2024-04-24T23:04:34.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:33 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:35.426 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-24T23:04:35.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.601+0000 7fab2f9fe700 1 Processor -- start 2024-04-24T23:04:35.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.601+0000 7fab2f9fe700 1 -- start start 2024-04-24T23:04:35.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.601+0000 7fab2f9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28101b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.601+0000 7fab2f9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab281020b0 con 0x7fab28101760 2024-04-24T23:04:35.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.602+0000 7fab2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.431 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.602+0000 7fab2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55142/0 (socket says 172.21.15.31:55142) 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.602+0000 7fab2d79a700 1 -- 172.21.15.31:0/3985973258 learned_addr learned my addr 172.21.15.31:0/3985973258 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.602+0000 7fab2d79a700 1 -- 172.21.15.31:0/3985973258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab281021f0 con 0x7fab28101760 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.602+0000 7fab2d79a700 1 --2- 172.21.15.31:0/3985973258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28101b70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab2400bb80 tx=0x7fab2400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c17305b3c738de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.602+0000 7fab1ffff700 1 -- 172.21.15.31:0/3985973258 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab2400dd00 con 0x7fab28101760 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab1ffff700 1 -- 172.21.15.31:0/3985973258 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab2400de60 con 0x7fab28101760 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab1ffff700 1 -- 172.21.15.31:0/3985973258 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab24018920 con 0x7fab28101760 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab2f9fe700 1 -- 172.21.15.31:0/3985973258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 msgr2=0x7fab28101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab2f9fe700 1 --2- 172.21.15.31:0/3985973258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28101b70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab2400bb80 tx=0x7fab2400b4f0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab2f9fe700 1 -- 172.21.15.31:0/3985973258 shutdown_connections 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab2f9fe700 1 --2- 172.21.15.31:0/3985973258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28101b70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab2f9fe700 1 -- 172.21.15.31:0/3985973258 >> 172.21.15.31:0/3985973258 conn(0x7fab280fcea0 msgr2=0x7fab280ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab2f9fe700 1 -- 172.21.15.31:0/3985973258 shutdown_connections 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.603+0000 7fab2f9fe700 1 -- 172.21.15.31:0/3985973258 wait complete. 2024-04-24T23:04:35.432 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.604+0000 7fab2f9fe700 1 Processor -- start 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.604+0000 7fab2f9fe700 1 -- start start 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.604+0000 7fab2f9fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28113780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.604+0000 7fab2f9fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab28113cc0 con 0x7fab28101760 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.604+0000 7fab2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28113780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.604+0000 7fab2d79a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28113780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55144/0 (socket says 172.21.15.31:55144) 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.604+0000 7fab2d79a700 1 -- 172.21.15.31:0/1679742707 learned_addr learned my addr 172.21.15.31:0/1679742707 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.604+0000 7fab2d79a700 1 -- 172.21.15.31:0/1679742707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2400b790 con 0x7fab28101760 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.605+0000 7fab2d79a700 1 --2- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28113780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab240106a0 tx=0x7fab2400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.605+0000 7fab1dffb700 1 -- 172.21.15.31:0/1679742707 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab2400dc70 con 0x7fab28101760 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.605+0000 7fab1dffb700 1 -- 172.21.15.31:0/1679742707 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab24010b80 con 0x7fab28101760 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.605+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab28113ec0 con 0x7fab28101760 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.605+0000 7fab1dffb700 1 -- 172.21.15.31:0/1679742707 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab24021550 con 0x7fab28101760 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.605+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab28114a30 con 0x7fab28101760 2024-04-24T23:04:35.433 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.606+0000 7fab1dffb700 1 -- 172.21.15.31:0/1679742707 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fab240217c0 con 0x7fab28101760 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.606+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2805f5c0 con 0x7fab28101760 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.606+0000 7fab1dffb700 1 --2- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fab140368a0 0x7fab14038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.606+0000 7fab1dffb700 1 -- 172.21.15.31:0/1679742707 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fab24014070 con 0x7fab28101760 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.606+0000 7fab2cf99700 1 --2- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fab140368a0 0x7fab14038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.607+0000 7fab2cf99700 1 --2- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fab140368a0 0x7fab14038d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fab18009990 tx=0x7fab18006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.611+0000 7fab1dffb700 1 -- 172.21.15.31:0/1679742707 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fab2402b920 con 0x7fab28101760 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.817+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fab28114de0 con 0x7fab140368a0 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.821+0000 7fab1dffb700 1 -- 172.21.15.31:0/1679742707 <== mgr.14120 v2:172.21.15.31:6800/2092175909 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fab28114de0 con 0x7fab140368a0 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fab140368a0 msgr2=0x7fab14038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 --2- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fab140368a0 0x7fab14038d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fab18009990 tx=0x7fab18006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 msgr2=0x7fab28113780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 --2- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28113780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fab240106a0 tx=0x7fab2400b4f0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.434 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 shutdown_connections 2024-04-24T23:04:35.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 --2- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fab140368a0 0x7fab14038d50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 --2- 172.21.15.31:0/1679742707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab28101760 0x7fab28113780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:35.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 >> 172.21.15.31:0/1679742707 conn(0x7fab280fcea0 msgr2=0x7fab28105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:35.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 shutdown_connections 2024-04-24T23:04:35.435 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:34.823+0000 7fab2f9fe700 1 -- 172.21.15.31:0/1679742707 wait complete. 2024-04-24T23:04:35.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:35 smithi031 ceph-mon[26411]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:35 smithi031 ceph-mon[26411]: Saving service alertmanager spec with placement count:1 2024-04-24T23:04:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:35 smithi031 ceph-mon[26411]: from='mgr.14120 172.21.15.31:0/1771444548' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:35 smithi031 ceph-mon[26411]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.817+0000 7f965754e700 1 Processor -- start 2024-04-24T23:04:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.817+0000 7f965754e700 1 -- start start 2024-04-24T23:04:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.817+0000 7f965754e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.817+0000 7f965754e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96501031d0 con 0x7f9650102880 2024-04-24T23:04:36.587 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.818+0000 7f96552ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.818+0000 7f96552ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55158/0 (socket says 172.21.15.31:55158) 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.818+0000 7f96552ea700 1 -- 172.21.15.31:0/1883649054 learned_addr learned my addr 172.21.15.31:0/1883649054 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.818+0000 7f96552ea700 1 -- 172.21.15.31:0/1883649054 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650103310 con 0x7f9650102880 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.818+0000 7f96552ea700 1 --2- 172.21.15.31:0/1883649054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650102c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9648009e50 tx=0x7f9648009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76704a51b2a7f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.818+0000 7f964ffff700 1 -- 172.21.15.31:0/1883649054 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96480092e0 con 0x7f9650102880 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.818+0000 7f964ffff700 1 -- 172.21.15.31:0/1883649054 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9648012430 con 0x7f9650102880 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.818+0000 7f964ffff700 1 -- 172.21.15.31:0/1883649054 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96480075a0 con 0x7f9650102880 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 -- 172.21.15.31:0/1883649054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 msgr2=0x7f9650102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 --2- 172.21.15.31:0/1883649054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650102c90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9648009e50 tx=0x7f9648009840 comp rx=0 tx=0).stop 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 -- 172.21.15.31:0/1883649054 shutdown_connections 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 --2- 172.21.15.31:0/1883649054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650102c90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 -- 172.21.15.31:0/1883649054 >> 172.21.15.31:0/1883649054 conn(0x7f96500fe060 msgr2=0x7f9650100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 -- 172.21.15.31:0/1883649054 shutdown_connections 2024-04-24T23:04:36.588 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 -- 172.21.15.31:0/1883649054 wait complete. 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 Processor -- start 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.819+0000 7f965754e700 1 -- start start 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f965754e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f965754e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9650079160 con 0x7f9650102880 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f96552ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f96552ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55162/0 (socket says 172.21.15.31:55162) 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f96552ea700 1 -- 172.21.15.31:0/1753221992 learned_addr learned my addr 172.21.15.31:0/1753221992 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f96552ea700 1 -- 172.21.15.31:0/1753221992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648009ae0 con 0x7f9650102880 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f96552ea700 1 --2- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650078c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9648009fd0 tx=0x7f9648009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f964dffb700 1 -- 172.21.15.31:0/1753221992 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96480092e0 con 0x7f9650102880 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f964dffb700 1 -- 172.21.15.31:0/1753221992 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9648007ce0 con 0x7f9650102880 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96500771d0 con 0x7f9650102880 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.820+0000 7f964dffb700 1 -- 172.21.15.31:0/1753221992 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9648025990 con 0x7f9650102880 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.821+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96500776c0 con 0x7f9650102880 2024-04-24T23:04:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.821+0000 7f964dffb700 1 -- 172.21.15.31:0/1753221992 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f9648019050 con 0x7f9650102880 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.822+0000 7f964dffb700 1 --2- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f96380368a0 0x7f9638038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.822+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965004bed0 con 0x7f9650102880 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.822+0000 7f964dffb700 1 -- 172.21.15.31:0/1753221992 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f964804a0d0 con 0x7f9650102880 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.822+0000 7f9654ae9700 1 --2- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f96380368a0 0x7f9638038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.822+0000 7f9654ae9700 1 --2- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f96380368a0 0x7f9638038d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9644009990 tx=0x7f9644006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:35.826+0000 7f964dffb700 1 -- 172.21.15.31:0/1753221992 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9648017070 con 0x7f9650102880 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.021+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f9650104620 con 0x7f9650102880 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.026+0000 7f964dffb700 1 -- 172.21.15.31:0/1753221992 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f9648020030 con 0x7f9650102880 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.028+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f96380368a0 msgr2=0x7f9638038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.028+0000 7f965754e700 1 --2- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f96380368a0 0x7f9638038d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9644009990 tx=0x7f9644006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.028+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 msgr2=0x7f9650078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.028+0000 7f965754e700 1 --2- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650078c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9648009fd0 tx=0x7f9648009840 comp rx=0 tx=0).stop 2024-04-24T23:04:36.590 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.029+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 shutdown_connections 2024-04-24T23:04:36.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.029+0000 7f965754e700 1 --2- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f96380368a0 0x7f9638038d50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.029+0000 7f965754e700 1 --2- 172.21.15.31:0/1753221992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9650102880 0x7f9650078c20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:36.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.029+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 >> 172.21.15.31:0/1753221992 conn(0x7f96500fe060 msgr2=0x7f96500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:36.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.029+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 shutdown_connections 2024-04-24T23:04:36.591 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:36.029+0000 7f965754e700 1 -- 172.21.15.31:0/1753221992 wait complete. 2024-04-24T23:04:37.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:37 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1753221992' entity='client.admin' 2024-04-24T23:04:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.041+0000 7f63c41d9700 1 Processor -- start 2024-04-24T23:04:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.041+0000 7f63c41d9700 1 -- start start 2024-04-24T23:04:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.041+0000 7f63c41d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.041+0000 7f63c41d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc102610 con 0x7f63bc101cc0 2024-04-24T23:04:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.041+0000 7f63c1f75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.041+0000 7f63c1f75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc1020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55170/0 (socket says 172.21.15.31:55170) 2024-04-24T23:04:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.041+0000 7f63c1f75700 1 -- 172.21.15.31:0/1104173892 learned_addr learned my addr 172.21.15.31:0/1104173892 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.042+0000 7f63c1f75700 1 -- 172.21.15.31:0/1104173892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc102750 con 0x7f63bc101cc0 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.042+0000 7f63c1f75700 1 --2- 172.21.15.31:0/1104173892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc1020d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63b80098e0 tx=0x7f63b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e75d73adb67b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.042+0000 7f63c0f73700 1 -- 172.21.15.31:0/1104173892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63b800b860 con 0x7f63bc101cc0 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.042+0000 7f63c0f73700 1 -- 172.21.15.31:0/1104173892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63b800be90 con 0x7f63bc101cc0 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.042+0000 7f63c0f73700 1 -- 172.21.15.31:0/1104173892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63b80103c0 con 0x7f63bc101cc0 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.042+0000 7f63c41d9700 1 -- 172.21.15.31:0/1104173892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 msgr2=0x7f63bc1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.042+0000 7f63c41d9700 1 --2- 172.21.15.31:0/1104173892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc1020d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f63b80098e0 tx=0x7f63b8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 -- 172.21.15.31:0/1104173892 shutdown_connections 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 --2- 172.21.15.31:0/1104173892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc1020d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 -- 172.21.15.31:0/1104173892 >> 172.21.15.31:0/1104173892 conn(0x7f63bc0fc6f0 msgr2=0x7f63bc0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 -- 172.21.15.31:0/1104173892 shutdown_connections 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 -- 172.21.15.31:0/1104173892 wait complete. 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 Processor -- start 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 -- start start 2024-04-24T23:04:37.911 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.043+0000 7f63c41d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc1142e0 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63c1f75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63c1f75700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc101520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55180/0 (socket says 172.21.15.31:55180) 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63c1f75700 1 -- 172.21.15.31:0/1960208431 learned_addr learned my addr 172.21.15.31:0/1960208431 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63c1f75700 1 -- 172.21.15.31:0/1960208431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b80094f0 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63c1f75700 1 --2- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc101520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63b80094c0 tx=0x7f63b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63b27fc700 1 -- 172.21.15.31:0/1960208431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63b8010890 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63b27fc700 1 -- 172.21.15.31:0/1960208431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63b8010ec0 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc114420 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63b27fc700 1 -- 172.21.15.31:0/1960208431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63b80243f0 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.044+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc114910 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.045+0000 7f63b27fc700 1 -- 172.21.15.31:0/1960208431 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f63b802d3f0 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.045+0000 7f63b27fc700 1 --2- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f63a80368a0 0x7f63a8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.045+0000 7f63b27fc700 1 -- 172.21.15.31:0/1960208431 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f63b804a230 con 0x7f63bc101cc0 2024-04-24T23:04:37.912 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.045+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc04bed0 con 0x7f63bc101cc0 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.045+0000 7f63c1774700 1 --2- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f63a80368a0 0x7f63a8038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.046+0000 7f63c1774700 1 --2- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f63a80368a0 0x7f63a8038d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009990 tx=0x7f63ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.050+0000 7f63b27fc700 1 -- 172.21.15.31:0/1960208431 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f63b802b8e0 con 0x7f63bc101cc0 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.245+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f63bc103530 con 0x7f63bc101cc0 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.251+0000 7f63b27fc700 1 -- 172.21.15.31:0/1960208431 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f63b802b2f0 con 0x7f63bc101cc0 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f63a80368a0 msgr2=0x7f63a8038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 --2- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f63a80368a0 0x7f63a8038d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009990 tx=0x7f63ac006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 msgr2=0x7f63bc101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 --2- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc101520 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f63b80094c0 tx=0x7f63b8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 shutdown_connections 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 --2- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f63a80368a0 0x7f63a8038d50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 --2- 172.21.15.31:0/1960208431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63bc101cc0 0x7f63bc101520 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 >> 172.21.15.31:0/1960208431 conn(0x7f63bc0fc6f0 msgr2=0x7f63bc0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 shutdown_connections 2024-04-24T23:04:37.913 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:37.253+0000 7f63c41d9700 1 -- 172.21.15.31:0/1960208431 wait complete. 2024-04-24T23:04:37.914 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-04-24T23:04:38.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:38 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1960208431' entity='client.admin' 2024-04-24T23:04:38.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:38 smithi031 ceph-mon[26411]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:04:39.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:39 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1955774114' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-24T23:04:39.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.302+0000 7f166560e700 1 Processor -- start 2024-04-24T23:04:39.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.302+0000 7f166560e700 1 -- start start 2024-04-24T23:04:39.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.302+0000 7f166560e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.302+0000 7f166560e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16601031c0 con 0x7f1660102870 2024-04-24T23:04:39.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.302+0000 7f165effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.303+0000 7f165effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55186/0 (socket says 172.21.15.31:55186) 2024-04-24T23:04:39.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.303+0000 7f165effd700 1 -- 172.21.15.31:0/795323503 learned_addr learned my addr 172.21.15.31:0/795323503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:39.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.303+0000 7f165effd700 1 -- 172.21.15.31:0/795323503 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1660103300 con 0x7f1660102870 2024-04-24T23:04:39.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.303+0000 7f165effd700 1 --2- 172.21.15.31:0/795323503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660102c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1658009e50 tx=0x7f1658009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2d278d804da59d server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.303+0000 7f165dffb700 1 -- 172.21.15.31:0/795323503 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16580092e0 con 0x7f1660102870 2024-04-24T23:04:39.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.303+0000 7f165dffb700 1 -- 172.21.15.31:0/795323503 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1658012430 con 0x7f1660102870 2024-04-24T23:04:39.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.303+0000 7f165dffb700 1 -- 172.21.15.31:0/795323503 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16580075a0 con 0x7f1660102870 2024-04-24T23:04:39.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.304+0000 7f166560e700 1 -- 172.21.15.31:0/795323503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 msgr2=0x7f1660102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.304+0000 7f166560e700 1 --2- 172.21.15.31:0/795323503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660102c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1658009e50 tx=0x7f1658009840 comp rx=0 tx=0).stop 2024-04-24T23:04:39.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.304+0000 7f166560e700 1 -- 172.21.15.31:0/795323503 shutdown_connections 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.304+0000 7f166560e700 1 --2- 172.21.15.31:0/795323503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660102c80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.304+0000 7f166560e700 1 -- 172.21.15.31:0/795323503 >> 172.21.15.31:0/795323503 conn(0x7f16600fdf90 msgr2=0x7f1660100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.304+0000 7f166560e700 1 -- 172.21.15.31:0/795323503 shutdown_connections 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.304+0000 7f166560e700 1 -- 172.21.15.31:0/795323503 wait complete. 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f166560e700 1 Processor -- start 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f166560e700 1 -- start start 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f166560e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f166560e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1660078050 con 0x7f1660102870 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f165effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f165effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55198/0 (socket says 172.21.15.31:55198) 2024-04-24T23:04:39.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f165effd700 1 -- 172.21.15.31:0/1955774114 learned_addr learned my addr 172.21.15.31:0/1955774114 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f165effd700 1 -- 172.21.15.31:0/1955774114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1658009ae0 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f165effd700 1 --2- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660077b10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f16580127d0 tx=0x7f1658009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.305+0000 7f16577fe700 1 -- 172.21.15.31:0/1955774114 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f16580092e0 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.306+0000 7f16577fe700 1 -- 172.21.15.31:0/1955774114 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1658007d30 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.306+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16600760c0 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.306+0000 7f16577fe700 1 -- 172.21.15.31:0/1955774114 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1658025960 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.306+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16600765b0 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.306+0000 7f16577fe700 1 -- 172.21.15.31:0/1955774114 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f1658019050 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.306+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166004bed0 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.307+0000 7f16577fe700 1 --2- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f164c036850 0x7f164c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.307+0000 7f16577fe700 1 -- 172.21.15.31:0/1955774114 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f165804a240 con 0x7f1660102870 2024-04-24T23:04:39.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.307+0000 7f165e7fc700 1 --2- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f164c036850 0x7f164c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.307+0000 7f165e7fc700 1 --2- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f164c036850 0x7f164c038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1648009990 tx=0x7f1648006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.311+0000 7f16577fe700 1 -- 172.21.15.31:0/1955774114 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f165801bd90 con 0x7f1660102870 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:38.563+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f1660076d00 con 0x7f1660102870 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.258+0000 7f16577fe700 1 -- 172.21.15.31:0/1955774114 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f165801c500 con 0x7f1660102870 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.259+0000 7f16577fe700 1 -- 172.21.15.31:0/1955774114 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f165804a5e0 con 0x7f1660102870 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f164c036850 msgr2=0x7f164c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 --2- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f164c036850 0x7f164c038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1648009990 tx=0x7f1648006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 msgr2=0x7f1660077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 --2- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660077b10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f16580127d0 tx=0x7f1658009840 comp rx=0 tx=0).stop 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 shutdown_connections 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 --2- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f164c036850 0x7f164c038d00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 --2- 172.21.15.31:0/1955774114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660102870 0x7f1660077b10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 >> 172.21.15.31:0/1955774114 conn(0x7f16600fdf90 msgr2=0x7f16600feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:39.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 shutdown_connections 2024-04-24T23:04:39.839 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:39.263+0000 7f166560e700 1 -- 172.21.15.31:0/1955774114 wait complete. 2024-04-24T23:04:40.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:40 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1955774114' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-24T23:04:40.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:40 smithi031 ceph-mon[26411]: mgrmap e9: smithi031.rrnmwr(active, since 23s) 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "smithi031.rrnmwr", 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.311+0000 7fbdde82a700 1 Processor -- start 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.311+0000 7fbdde82a700 1 -- start start 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.311+0000 7fbdde82a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.200 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.311+0000 7fbdde82a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd81031c0 con 0x7fbdd8102870 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.311+0000 7fbdd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.311+0000 7fbdd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40858/0 (socket says 172.21.15.31:40858) 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.311+0000 7fbdd7fff700 1 -- 172.21.15.31:0/3477735765 learned_addr learned my addr 172.21.15.31:0/3477735765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.311+0000 7fbdd7fff700 1 -- 172.21.15.31:0/3477735765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd8103300 con 0x7fbdd8102870 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.312+0000 7fbdd7fff700 1 --2- 172.21.15.31:0/3477735765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8102c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80098e0 tx=0x7fbdc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5045790b410738c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.312+0000 7fbdd6ffd700 1 -- 172.21.15.31:0/3477735765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc800b860 con 0x7fbdd8102870 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.312+0000 7fbdd6ffd700 1 -- 172.21.15.31:0/3477735765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdc800be90 con 0x7fbdd8102870 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.312+0000 7fbdd6ffd700 1 -- 172.21.15.31:0/3477735765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc80103c0 con 0x7fbdd8102870 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.313+0000 7fbdde82a700 1 -- 172.21.15.31:0/3477735765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 msgr2=0x7fbdd8102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.313+0000 7fbdde82a700 1 --2- 172.21.15.31:0/3477735765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8102c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80098e0 tx=0x7fbdc8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.313+0000 7fbdde82a700 1 -- 172.21.15.31:0/3477735765 shutdown_connections 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.313+0000 7fbdde82a700 1 --2- 172.21.15.31:0/3477735765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8102c80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.201 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.313+0000 7fbdde82a700 1 -- 172.21.15.31:0/3477735765 >> 172.21.15.31:0/3477735765 conn(0x7fbdd80fdf90 msgr2=0x7fbdd8100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.313+0000 7fbdde82a700 1 -- 172.21.15.31:0/3477735765 shutdown_connections 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.313+0000 7fbdde82a700 1 -- 172.21.15.31:0/3477735765 wait complete. 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.313+0000 7fbdde82a700 1 Processor -- start 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.314+0000 7fbdde82a700 1 -- start start 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.314+0000 7fbdde82a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.314+0000 7fbdde82a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8078050 con 0x7fbdd8102870 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.314+0000 7fbdd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.314+0000 7fbdd7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40868/0 (socket says 172.21.15.31:40868) 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.314+0000 7fbdd7fff700 1 -- 172.21.15.31:0/4150609388 learned_addr learned my addr 172.21.15.31:0/4150609388 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:41.202 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.314+0000 7fbdd7fff700 1 -- 172.21.15.31:0/4150609388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80094f0 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.315+0000 7fbdd7fff700 1 --2- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8077b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011680 tx=0x7fbdc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.315+0000 7fbdd4ff9700 1 -- 172.21.15.31:0/4150609388 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc8010890 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.315+0000 7fbdd4ff9700 1 -- 172.21.15.31:0/4150609388 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdc8010ec0 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.315+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd80760c0 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.315+0000 7fbdd4ff9700 1 -- 172.21.15.31:0/4150609388 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbdc80243f0 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.315+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd80765b0 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.316+0000 7fbdd4ff9700 1 -- 172.21.15.31:0/4150609388 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fbdc802d3f0 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.316+0000 7fbdd4ff9700 1 --2- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbdc00368f0 0x7fbdc0038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.316+0000 7fbdd4ff9700 1 -- 172.21.15.31:0/4150609388 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fbdc804b320 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.316+0000 7fbdd77fe700 1 -- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbdc00368f0 msgr2=0x7fbdc0038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2092175909 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.316+0000 7fbdd77fe700 1 --2- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbdc00368f0 0x7fbdc0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.316+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd804bed0 con 0x7fbdd8102870 2024-04-24T23:04:41.203 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.321+0000 7fbdd4ff9700 1 -- 172.21.15.31:0/4150609388 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbdc802b8e0 con 0x7fbdd8102870 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.517+0000 7fbdd77fe700 1 -- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbdc00368f0 msgr2=0x7fbdc0038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2092175909 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.517+0000 7fbdd77fe700 1 --2- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbdc00368f0 0x7fbdc0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.617+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fbdd8076a30 con 0x7fbdd8102870 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.618+0000 7fbdd4ff9700 1 -- 172.21.15.31:0/4150609388 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fbdc802c350 con 0x7fbdd8102870 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.620+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbdc00368f0 msgr2=0x7fbdc0038da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.620+0000 7fbdde82a700 1 --2- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbdc00368f0 0x7fbdc0038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.620+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 msgr2=0x7fbdd8077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.620+0000 7fbdde82a700 1 --2- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8077b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011680 tx=0x7fbdc8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.620+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 shutdown_connections 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.620+0000 7fbdde82a700 1 --2- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7fbdc00368f0 0x7fbdc0038da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.620+0000 7fbdde82a700 1 --2- 172.21.15.31:0/4150609388 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdd8102870 0x7fbdd8077b10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.620+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 >> 172.21.15.31:0/4150609388 conn(0x7fbdd80fdf90 msgr2=0x7fbdd80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:41.204 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.621+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 shutdown_connections 2024-04-24T23:04:41.205 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:40.621+0000 7fbdde82a700 1 -- 172.21.15.31:0/4150609388 wait complete. 2024-04-24T23:04:41.205 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-04-24T23:04:41.205 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-04-24T23:04:41.483 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:41 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/4150609388' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: Active manager daemon smithi031.rrnmwr restarted 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: osdmap e3: 0 total, 0 up, 0 in 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: mgrmap e10: smithi031.rrnmwr(active, starting, since 0.00313371s) 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:04:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:04:46.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:46.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:04:46.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:04:46.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:46 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:04:47.744 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.680+0000 7f8d4d6c4700 1 Processor -- start 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.680+0000 7f8d4d6c4700 1 -- start start 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.680+0000 7f8d4d6c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d480771f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.680+0000 7f8d4d6c4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480790f0 con 0x7f8d48078ce0 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.680+0000 7f8d46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d480771f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.680+0000 7f8d46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d480771f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40880/0 (socket says 172.21.15.31:40880) 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.680+0000 7f8d46ffd700 1 -- 172.21.15.31:0/3109327461 learned_addr learned my addr 172.21.15.31:0/3109327461 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.680+0000 7f8d46ffd700 1 -- 172.21.15.31:0/3109327461 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48077730 con 0x7f8d48078ce0 2024-04-24T23:04:47.745 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d46ffd700 1 --2- 172.21.15.31:0/3109327461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d480771f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d380098e0 tx=0x7f8d38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31899899badda0a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d45ffb700 1 -- 172.21.15.31:0/3109327461 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d3800b860 con 0x7f8d48078ce0 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d45ffb700 1 -- 172.21.15.31:0/3109327461 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d3800be90 con 0x7f8d48078ce0 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d45ffb700 1 -- 172.21.15.31:0/3109327461 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d380103c0 con 0x7f8d48078ce0 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/3109327461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 msgr2=0x7f8d480771f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d4d6c4700 1 --2- 172.21.15.31:0/3109327461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d480771f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d380098e0 tx=0x7f8d38009250 comp rx=0 tx=0).stop 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/3109327461 shutdown_connections 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d4d6c4700 1 --2- 172.21.15.31:0/3109327461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d480771f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.681+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/3109327461 >> 172.21.15.31:0/3109327461 conn(0x7f8d480fdfd0 msgr2=0x7f8d481003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.682+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/3109327461 shutdown_connections 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.682+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/3109327461 wait complete. 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.682+0000 7f8d4d6c4700 1 Processor -- start 2024-04-24T23:04:47.746 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.682+0000 7f8d4d6c4700 1 -- start start 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.682+0000 7f8d4d6c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d481138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d4d6c4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48113df0 con 0x7f8d48078ce0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d481138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d46ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d481138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40888/0 (socket says 172.21.15.31:40888) 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d46ffd700 1 -- 172.21.15.31:0/1384460767 learned_addr learned my addr 172.21.15.31:0/1384460767 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d46ffd700 1 -- 172.21.15.31:0/1384460767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380094f0 con 0x7f8d48078ce0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d46ffd700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d481138b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d380094c0 tx=0x7f8d38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d380118f0 con 0x7f8d48078ce0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d38010d60 con 0x7f8d48078ce0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d48113ff0 con 0x7f8d48078ce0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d3801a430 con 0x7f8d48078ce0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.683+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d48114b60 con 0x7f8d48078ce0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.684+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f8d3801a590 con 0x7f8d48078ce0 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.685+0000 7f8d2f7fe700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 0x7f8d30038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.747 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.685+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 --> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d300394b0 con 0x7f8d300368f0 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.685+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f8d3804a710 con 0x7f8d48078ce0 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.685+0000 7f8d467fc700 1 -- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 msgr2=0x7f8d30038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2092175909 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.685+0000 7f8d467fc700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 0x7f8d30038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.886+0000 7f8d467fc700 1 -- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 msgr2=0x7f8d30038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2092175909 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:41.886+0000 7f8d467fc700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 0x7f8d30038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:42.287+0000 7f8d467fc700 1 -- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 msgr2=0x7f8d30038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2092175909 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:42.287+0000 7f8d467fc700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 0x7f8d30038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:43.088+0000 7f8d467fc700 1 -- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 msgr2=0x7f8d30038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2092175909 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:43.088+0000 7f8d467fc700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 0x7f8d30038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:44.689+0000 7f8d467fc700 1 -- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 msgr2=0x7f8d30038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/2092175909 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:44.689+0000 7f8d467fc700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 0x7f8d30038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:46.068+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f8d3802ad60 con 0x7f8d48078ce0 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:46.068+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 msgr2=0x7f8d30038da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:04:47.748 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:46.068+0000 7f8d2f7fe700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 0x7f8d30038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.070+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f8d3802c970 con 0x7f8d48078ce0 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.070+0000 7f8d2f7fe700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8d3003a3a0 0x7f8d3003c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.070+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 --> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d300394b0 con 0x7f8d3003a3a0 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.070+0000 7f8d467fc700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8d3003a3a0 0x7f8d3003c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.071+0000 7f8d467fc700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8d3003a3a0 0x7f8d3003c780 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0039d0 tx=0x7f8d3c009350 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.072+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mgr.14162 v2:172.21.15.31:6800/1555170645 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f8d300394b0 con 0x7f8d3003a3a0 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 --> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f8d48114480 con 0x7f8d3003a3a0 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d2f7fe700 1 -- 172.21.15.31:0/1384460767 <== mgr.14162 v2:172.21.15.31:6800/1555170645 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f8d48114480 con 0x7f8d3003a3a0 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8d3003a3a0 msgr2=0x7f8d3003c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8d3003a3a0 0x7f8d3003c780 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0039d0 tx=0x7f8d3c009350 comp rx=0 tx=0).stop 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 msgr2=0x7f8d481138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d481138b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8d380094c0 tx=0x7f8d38009250 comp rx=0 tx=0).stop 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 shutdown_connections 2024-04-24T23:04:47.749 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8d3003a3a0 0x7f8d3003c780 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.750 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d48078ce0 0x7f8d481138b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.750 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 --2- 172.21.15.31:0/1384460767 >> [v2:172.21.15.31:6800/2092175909,v1:172.21.15.31:6801/2092175909] conn(0x7f8d300368f0 0x7f8d30038da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:47.750 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 >> 172.21.15.31:0/1384460767 conn(0x7f8d480fdfd0 msgr2=0x7f8d480feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:47.750 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.078+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 shutdown_connections 2024-04-24T23:04:47.750 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:47.079+0000 7f8d4d6c4700 1 -- 172.21.15.31:0/1384460767 wait complete. 2024-04-24T23:04:47.750 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-04-24T23:04:47.750 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2024-04-24T23:04:47.962 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:47 smithi031 ceph-mon[26411]: [24/Apr/2024:23:04:46] ENGINE Bus STARTING 2024-04-24T23:04:47.963 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:47 smithi031 ceph-mon[26411]: [24/Apr/2024:23:04:46] ENGINE Serving on https://172.21.15.31:7150 2024-04-24T23:04:47.963 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:47 smithi031 ceph-mon[26411]: [24/Apr/2024:23:04:46] ENGINE Bus STARTED 2024-04-24T23:04:47.963 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:47 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:47.963 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:47 smithi031 ceph-mon[26411]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-24T23:04:47.963 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:47 smithi031 ceph-mon[26411]: mgrmap e11: smithi031.rrnmwr(active, since 1.006s) 2024-04-24T23:04:47.963 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:47 smithi031 ceph-mon[26411]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-24T23:04:49.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-24T23:04:49.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.181+0000 7fce0c84c700 1 Processor -- start 2024-04-24T23:04:49.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.181+0000 7fce0c84c700 1 -- start start 2024-04-24T23:04:49.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.181+0000 7fce0c84c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:49.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.181+0000 7fce0c84c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce08103280 con 0x7fce08071290 2024-04-24T23:04:49.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.181+0000 7fce0659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:49.007 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.181+0000 7fce0659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40966/0 (socket says 172.21.15.31:40966) 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.181+0000 7fce0659c700 1 -- 172.21.15.31:0/3902532741 learned_addr learned my addr 172.21.15.31:0/3902532741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.182+0000 7fce0659c700 1 -- 172.21.15.31:0/3902532741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce08107dd0 con 0x7fce08071290 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.182+0000 7fce0659c700 1 --2- 172.21.15.31:0/3902532741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08107890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80098e0 tx=0x7fcdf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4032ad9cfd7e8d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.182+0000 7fce0559a700 1 -- 172.21.15.31:0/3902532741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdf800b860 con 0x7fce08071290 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.182+0000 7fce0559a700 1 -- 172.21.15.31:0/3902532741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdf800be90 con 0x7fce08071290 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.182+0000 7fce0559a700 1 -- 172.21.15.31:0/3902532741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdf80103c0 con 0x7fce08071290 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 -- 172.21.15.31:0/3902532741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 msgr2=0x7fce08107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 --2- 172.21.15.31:0/3902532741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08107890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80098e0 tx=0x7fcdf8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 -- 172.21.15.31:0/3902532741 shutdown_connections 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 --2- 172.21.15.31:0/3902532741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08107890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 -- 172.21.15.31:0/3902532741 >> 172.21.15.31:0/3902532741 conn(0x7fce080fcec0 msgr2=0x7fce080ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 -- 172.21.15.31:0/3902532741 shutdown_connections 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 -- 172.21.15.31:0/3902532741 wait complete. 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 Processor -- start 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.183+0000 7fce0c84c700 1 -- start start 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0c84c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0c84c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce08113d40 con 0x7fce08071290 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40968/0 (socket says 172.21.15.31:40968) 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0659c700 1 -- 172.21.15.31:0/3353288520 learned_addr learned my addr 172.21.15.31:0/3353288520 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0659c700 1 -- 172.21.15.31:0/3353288520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80094f0 con 0x7fce08071290 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0659c700 1 --2- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08113800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8006d40 tx=0x7fcdf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fcdf6ffd700 1 -- 172.21.15.31:0/3353288520 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdf8010840 con 0x7fce08071290 2024-04-24T23:04:49.009 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fcdf6ffd700 1 -- 172.21.15.31:0/3353288520 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdf8010e70 con 0x7fce08071290 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce08113f40 con 0x7fce08071290 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fcdf6ffd700 1 -- 172.21.15.31:0/3353288520 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcdf8024ea0 con 0x7fce08071290 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.184+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce08114ab0 con 0x7fce08071290 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.185+0000 7fcdf6ffd700 1 -- 172.21.15.31:0/3353288520 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7fcdf801a4c0 con 0x7fce08071290 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.185+0000 7fcdf6ffd700 1 --2- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fcdf0036770 0x7fcdf0038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.186+0000 7fcdf6ffd700 1 -- 172.21.15.31:0/3353288520 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcdf804a6e0 con 0x7fce08071290 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.185+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0805f5c0 con 0x7fce08071290 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.186+0000 7fce05d9b700 1 --2- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fcdf0036770 0x7fcdf0038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.186+0000 7fce05d9b700 1 --2- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fcdf0036770 0x7fcdf0038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009990 tx=0x7fcdfc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.191+0000 7fcdf6ffd700 1 -- 172.21.15.31:0/3353288520 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcdf8018940 con 0x7fce08071290 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.398+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 --> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fce080fe120 con 0x7fcdf0036770 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.441+0000 7fcdf6ffd700 1 -- 172.21.15.31:0/3353288520 <== mgr.14162 v2:172.21.15.31:6800/1555170645 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fce080fe120 con 0x7fcdf0036770 2024-04-24T23:04:49.010 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fcdf0036770 msgr2=0x7fcdf0038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 --2- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fcdf0036770 0x7fcdf0038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009990 tx=0x7fcdfc006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 msgr2=0x7fce08113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 --2- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08113800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8006d40 tx=0x7fcdf8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 shutdown_connections 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 --2- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fcdf0036770 0x7fcdf0038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 --2- 172.21.15.31:0/3353288520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce08071290 0x7fce08113800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 >> 172.21.15.31:0/3353288520 conn(0x7fce080fcec0 msgr2=0x7fce080fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 shutdown_connections 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:48.444+0000 7fce0c84c700 1 -- 172.21.15.31:0/3353288520 wait complete. 2024-04-24T23:04:49.011 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-04-24T23:04:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:49 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:49 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:49 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:50 smithi031 ceph-mon[26411]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:50 smithi031 ceph-mon[26411]: mgrmap e12: smithi031.rrnmwr(active, since 2s) 2024-04-24T23:04:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:50 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:50 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:50.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$1xlpkVXVwR.NG/C8VV9HXuXV4BclGgi0L0eM9lp4a/Rqx0lVPLso2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713999889, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.472+0000 7f49ae8d4700 1 Processor -- start 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.472+0000 7f49ae8d4700 1 -- start start 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.472+0000 7f49ae8d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.472+0000 7f49ae8d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a8102080 con 0x7f49a8101730 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.472+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.472+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47652/0 (socket says 172.21.15.31:47652) 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.472+0000 7f49a7fff700 1 -- 172.21.15.31:0/1031870639 learned_addr learned my addr 172.21.15.31:0/1031870639 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49a7fff700 1 -- 172.21.15.31:0/1031870639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a81021c0 con 0x7f49a8101730 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49a7fff700 1 --2- 172.21.15.31:0/1031870639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8101b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49980098e0 tx=0x7f4998009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84461da01da4557 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49a6ffd700 1 -- 172.21.15.31:0/1031870639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f499800b860 con 0x7f49a8101730 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49a6ffd700 1 -- 172.21.15.31:0/1031870639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f499800be90 con 0x7f49a8101730 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49a6ffd700 1 -- 172.21.15.31:0/1031870639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49980103c0 con 0x7f49a8101730 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49ae8d4700 1 -- 172.21.15.31:0/1031870639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 msgr2=0x7f49a8101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.645 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49ae8d4700 1 --2- 172.21.15.31:0/1031870639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8101b40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49980098e0 tx=0x7f4998009250 comp rx=0 tx=0).stop 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49ae8d4700 1 -- 172.21.15.31:0/1031870639 shutdown_connections 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.473+0000 7f49ae8d4700 1 --2- 172.21.15.31:0/1031870639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8101b40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.474+0000 7f49ae8d4700 1 -- 172.21.15.31:0/1031870639 >> 172.21.15.31:0/1031870639 conn(0x7f49a80fcef0 msgr2=0x7f49a80ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.474+0000 7f49ae8d4700 1 -- 172.21.15.31:0/1031870639 shutdown_connections 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.474+0000 7f49ae8d4700 1 -- 172.21.15.31:0/1031870639 wait complete. 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.474+0000 7f49ae8d4700 1 Processor -- start 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.474+0000 7f49ae8d4700 1 -- start start 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49ae8d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49ae8d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a81142b0 con 0x7f49a8101730 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8076b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47658/0 (socket says 172.21.15.31:47658) 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49a7fff700 1 -- 172.21.15.31:0/814083968 learned_addr learned my addr 172.21.15.31:0/814083968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49a7fff700 1 -- 172.21.15.31:0/814083968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980094f0 con 0x7f49a8101730 2024-04-24T23:04:50.646 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49a7fff700 1 --2- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8076b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4998011680 tx=0x7f4998009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49a4ff9700 1 -- 172.21.15.31:0/814083968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4998010890 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49a4ff9700 1 -- 172.21.15.31:0/814083968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4998010ec0 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a81143f0 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.475+0000 7f49a4ff9700 1 -- 172.21.15.31:0/814083968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49980243f0 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.476+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a81148e0 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.476+0000 7f49a4ff9700 1 -- 172.21.15.31:0/814083968 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f499802d3f0 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.477+0000 7f49a4ff9700 1 --2- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f49900368f0 0x7f4990038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.477+0000 7f49a4ff9700 1 -- 172.21.15.31:0/814083968 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f499804a690 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.477+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a804bed0 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.477+0000 7f49a77fe700 1 --2- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f49900368f0 0x7f4990038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.477+0000 7f49a77fe700 1 --2- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f49900368f0 0x7f4990038da0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f499c009990 tx=0x7f499c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.482+0000 7f49a4ff9700 1 -- 172.21.15.31:0/814083968 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f499802b8e0 con 0x7f49a8101730 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.704+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 --> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f49a805f3b0 con 0x7f49900368f0 2024-04-24T23:04:50.647 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.951+0000 7f49a4ff9700 1 -- 172.21.15.31:0/814083968 <== mgr.14162 v2:172.21.15.31:6800/1555170645 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f49a805f3b0 con 0x7f49900368f0 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f49900368f0 msgr2=0x7f4990038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 --2- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f49900368f0 0x7f4990038da0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f499c009990 tx=0x7f499c006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 msgr2=0x7f49a8076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 --2- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8076b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4998011680 tx=0x7f4998009250 comp rx=0 tx=0).stop 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 shutdown_connections 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 --2- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f49900368f0 0x7f4990038da0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 --2- 172.21.15.31:0/814083968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a8101730 0x7f49a8076b60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 >> 172.21.15.31:0/814083968 conn(0x7f49a80fcef0 msgr2=0x7f49a80fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 shutdown_connections 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:49.953+0000 7f49ae8d4700 1 -- 172.21.15.31:0/814083968 wait complete. 2024-04-24T23:04:50.648 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-04-24T23:04:51.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:51 smithi031 ceph-mon[26411]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:04:52.161 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:52 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3568153657' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-24T23:04:52.370 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-04-24T23:04:52.370 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.582+0000 7f8cef959700 1 Processor -- start 2024-04-24T23:04:52.370 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.582+0000 7f8cef959700 1 -- start start 2024-04-24T23:04:52.370 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.582+0000 7f8cef959700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.370 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.582+0000 7f8cef959700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce8103650 con 0x7f8ce8102c70 2024-04-24T23:04:52.370 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.583+0000 7f8ced6f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.370 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.583+0000 7f8ced6f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47674/0 (socket says 172.21.15.31:47674) 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.583+0000 7f8ced6f5700 1 -- 172.21.15.31:0/2500243739 learned_addr learned my addr 172.21.15.31:0/2500243739 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.583+0000 7f8ced6f5700 1 -- 172.21.15.31:0/2500243739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8103790 con 0x7f8ce8102c70 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.583+0000 7f8ced6f5700 1 --2- 172.21.15.31:0/2500243739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8103080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80098e0 tx=0x7f8cd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52037fbf7f97d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.583+0000 7f8ce7fff700 1 -- 172.21.15.31:0/2500243739 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cd800b860 con 0x7f8ce8102c70 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.583+0000 7f8ce7fff700 1 -- 172.21.15.31:0/2500243739 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cd800be90 con 0x7f8ce8102c70 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.583+0000 7f8ce7fff700 1 -- 172.21.15.31:0/2500243739 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cd80103c0 con 0x7f8ce8102c70 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.584+0000 7f8cef959700 1 -- 172.21.15.31:0/2500243739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 msgr2=0x7f8ce8103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.584+0000 7f8cef959700 1 --2- 172.21.15.31:0/2500243739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8103080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80098e0 tx=0x7f8cd8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.584+0000 7f8cef959700 1 -- 172.21.15.31:0/2500243739 shutdown_connections 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.584+0000 7f8cef959700 1 --2- 172.21.15.31:0/2500243739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8103080 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.584+0000 7f8cef959700 1 -- 172.21.15.31:0/2500243739 >> 172.21.15.31:0/2500243739 conn(0x7f8ce80fc6f0 msgr2=0x7f8ce80feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.584+0000 7f8cef959700 1 -- 172.21.15.31:0/2500243739 shutdown_connections 2024-04-24T23:04:52.371 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.584+0000 7f8cef959700 1 -- 172.21.15.31:0/2500243739 wait complete. 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8cef959700 1 Processor -- start 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8cef959700 1 -- start start 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8cef959700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8cef959700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce8113da0 con 0x7f8ce8102c70 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8ced6f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8ced6f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8113860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47684/0 (socket says 172.21.15.31:47684) 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8ced6f5700 1 -- 172.21.15.31:0/3568153657 learned_addr learned my addr 172.21.15.31:0/3568153657 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8ced6f5700 1 -- 172.21.15.31:0/3568153657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80094f0 con 0x7f8ce8102c70 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8ced6f5700 1 --2- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8113860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80090b0 tx=0x7f8cd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8ce5ffb700 1 -- 172.21.15.31:0/3568153657 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cd8010840 con 0x7f8ce8102c70 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8ce5ffb700 1 -- 172.21.15.31:0/3568153657 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cd8010e70 con 0x7f8ce8102c70 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.585+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce8113fa0 con 0x7f8ce8102c70 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.586+0000 7f8ce5ffb700 1 -- 172.21.15.31:0/3568153657 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8cd8024e40 con 0x7f8ce8102c70 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.586+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce8114ac0 con 0x7f8ce8102c70 2024-04-24T23:04:52.372 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.586+0000 7f8ce5ffb700 1 -- 172.21.15.31:0/3568153657 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f8cd80109a0 con 0x7f8ce8102c70 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.587+0000 7f8ce5ffb700 1 --2- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8cd00368a0 0x7f8cd0038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.587+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce805f5c0 con 0x7f8ce8102c70 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.587+0000 7f8ce5ffb700 1 -- 172.21.15.31:0/3568153657 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8cd804a850 con 0x7f8ce8102c70 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.587+0000 7f8cecef4700 1 --2- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8cd00368a0 0x7f8cd0038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.587+0000 7f8cecef4700 1 --2- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8cd00368a0 0x7f8cd0038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009990 tx=0x7f8cdc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.592+0000 7f8ce5ffb700 1 -- 172.21.15.31:0/3568153657 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8cd802e440 con 0x7f8ce8102c70 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.785+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f8ce804bed0 con 0x7f8ce8102c70 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.786+0000 7f8ce5ffb700 1 -- 172.21.15.31:0/3568153657 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8cd802c340 con 0x7f8ce8102c70 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8cd00368a0 msgr2=0x7f8cd0038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 --2- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8cd00368a0 0x7f8cd0038d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009990 tx=0x7f8cdc006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 msgr2=0x7f8ce8113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 --2- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8113860 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80090b0 tx=0x7f8cd8009250 comp rx=0 tx=0).stop 2024-04-24T23:04:52.373 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 shutdown_connections 2024-04-24T23:04:52.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 --2- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f8cd00368a0 0x7f8cd0038d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 --2- 172.21.15.31:0/3568153657 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce8102c70 0x7f8ce8113860 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:52.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 >> 172.21.15.31:0/3568153657 conn(0x7f8ce80fc6f0 msgr2=0x7f8ce80fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:52.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 shutdown_connections 2024-04-24T23:04:52.374 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:51.788+0000 7f8cef959700 1 -- 172.21.15.31:0/3568153657 wait complete. 2024-04-24T23:04:52.378 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-24T23:04:52.378 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-04-24T23:04:52.386 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-24T23:04:52.386 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-04-24T23:04:52.386 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-04-24T23:04:52.386 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-24T23:04:52.388 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-04-24T23:04:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031:8443/ 2024-04-24T23:04:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-04-24T23:04:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: Password: bzg5vq4dft 2024-04-24T23:04:52.388 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:52.388 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config directory 2024-04-24T23:04:52.389 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling autotune for osd_memory_target 2024-04-24T23:04:53.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.785+0000 7f43c844c700 1 Processor -- start 2024-04-24T23:04:53.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.785+0000 7f43c844c700 1 -- start start 2024-04-24T23:04:53.559 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c844c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c0101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c844c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0102080 con 0x7f43c0101730 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c61e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c0101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c61e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c0101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47700/0 (socket says 172.21.15.31:47700) 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c61e8700 1 -- 172.21.15.31:0/1503892242 learned_addr learned my addr 172.21.15.31:0/1503892242 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c61e8700 1 -- 172.21.15.31:0/1503892242 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c01021c0 con 0x7f43c0101730 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c61e8700 1 --2- 172.21.15.31:0/1503892242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c0101b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f43b400bb80 tx=0x7f43b400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d8d121c1b35291 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c51e6700 1 -- 172.21.15.31:0/1503892242 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b400dd00 con 0x7f43c0101730 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c51e6700 1 -- 172.21.15.31:0/1503892242 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43b400de60 con 0x7f43c0101730 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.786+0000 7f43c51e6700 1 -- 172.21.15.31:0/1503892242 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b4018920 con 0x7f43c0101730 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.787+0000 7f43c844c700 1 -- 172.21.15.31:0/1503892242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 msgr2=0x7f43c0101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.787+0000 7f43c844c700 1 --2- 172.21.15.31:0/1503892242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c0101b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f43b400bb80 tx=0x7f43b400b4f0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.787+0000 7f43c844c700 1 -- 172.21.15.31:0/1503892242 shutdown_connections 2024-04-24T23:04:53.560 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.787+0000 7f43c844c700 1 --2- 172.21.15.31:0/1503892242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c0101b40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.787+0000 7f43c844c700 1 -- 172.21.15.31:0/1503892242 >> 172.21.15.31:0/1503892242 conn(0x7f43c00fcef0 msgr2=0x7f43c00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.787+0000 7f43c844c700 1 -- 172.21.15.31:0/1503892242 shutdown_connections 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.787+0000 7f43c844c700 1 -- 172.21.15.31:0/1503892242 wait complete. 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.788+0000 7f43c844c700 1 Processor -- start 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.788+0000 7f43c844c700 1 -- start start 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.788+0000 7f43c844c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c010f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.788+0000 7f43c844c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c010f910 con 0x7f43c0101730 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.788+0000 7f43c61e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c010f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.788+0000 7f43c61e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c010f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47704/0 (socket says 172.21.15.31:47704) 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.788+0000 7f43c61e8700 1 -- 172.21.15.31:0/612020549 learned_addr learned my addr 172.21.15.31:0/612020549 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.788+0000 7f43c61e8700 1 -- 172.21.15.31:0/612020549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b400b790 con 0x7f43c0101730 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.789+0000 7f43c61e8700 1 --2- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c010f3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43b4006d40 tx=0x7f43b400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.789+0000 7f43b2ffd700 1 -- 172.21.15.31:0/612020549 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b400dc70 con 0x7f43c0101730 2024-04-24T23:04:53.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.789+0000 7f43b2ffd700 1 -- 172.21.15.31:0/612020549 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43b401d050 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.789+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c010fb10 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.789+0000 7f43b2ffd700 1 -- 172.21.15.31:0/612020549 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b40223d0 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.789+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c0110680 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.790+0000 7f43b2ffd700 1 -- 172.21.15.31:0/612020549 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f43b4022600 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.790+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c005f5c0 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.790+0000 7f43b2ffd700 1 --2- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f43ac0368f0 0x7f43ac038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.790+0000 7f43b2ffd700 1 -- 172.21.15.31:0/612020549 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f43b4014070 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.790+0000 7f43c59e7700 1 --2- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f43ac0368f0 0x7f43ac038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.790+0000 7f43c59e7700 1 --2- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f43ac0368f0 0x7f43ac038da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009990 tx=0x7f43bc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.795+0000 7f43b2ffd700 1 -- 172.21.15.31:0/612020549 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f43b400ebe0 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.994+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f43c004bed0 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.994+0000 7f43b2ffd700 1 -- 172.21.15.31:0/612020549 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f43b400fed0 con 0x7f43c0101730 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f43ac0368f0 msgr2=0x7f43ac038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 --2- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f43ac0368f0 0x7f43ac038da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f43bc009990 tx=0x7f43bc006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 msgr2=0x7f43c010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 --2- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c010f3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f43b4006d40 tx=0x7f43b400b4f0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 shutdown_connections 2024-04-24T23:04:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 --2- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f43ac0368f0 0x7f43ac038da0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 --2- 172.21.15.31:0/612020549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43c0101730 0x7f43c010f3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 >> 172.21.15.31:0/612020549 conn(0x7f43c00fcef0 msgr2=0x7f43c0105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 shutdown_connections 2024-04-24T23:04:53.563 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:52.996+0000 7f43c844c700 1 -- 172.21.15.31:0/612020549 wait complete. 2024-04-24T23:04:54.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:54 smithi031 ceph-mon[26411]: mgrmap e13: smithi031.rrnmwr(active, since 6s) 2024-04-24T23:04:54.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.944+0000 7f2060af2700 1 Processor -- start 2024-04-24T23:04:54.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.944+0000 7f2060af2700 1 -- start start 2024-04-24T23:04:54.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.944+0000 7f2060af2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.944+0000 7f2060af2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c079070 con 0x7f205c078c60 2024-04-24T23:04:54.834 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.944+0000 7f205a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.944+0000 7f205a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47708/0 (socket says 172.21.15.31:47708) 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.944+0000 7f205a59c700 1 -- 172.21.15.31:0/2428014048 learned_addr learned my addr 172.21.15.31:0/2428014048 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.945+0000 7f205a59c700 1 -- 172.21.15.31:0/2428014048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c077750 con 0x7f205c078c60 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.945+0000 7f205a59c700 1 --2- 172.21.15.31:0/2428014048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c077210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f204c0098e0 tx=0x7f204c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3e8a4ca8dad9f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.945+0000 7f205959a700 1 -- 172.21.15.31:0/2428014048 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f204c00b860 con 0x7f205c078c60 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.945+0000 7f205959a700 1 -- 172.21.15.31:0/2428014048 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f204c00be90 con 0x7f205c078c60 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.945+0000 7f205959a700 1 -- 172.21.15.31:0/2428014048 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f204c0103c0 con 0x7f205c078c60 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.946+0000 7f2060af2700 1 -- 172.21.15.31:0/2428014048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 msgr2=0x7f205c077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.946+0000 7f2060af2700 1 --2- 172.21.15.31:0/2428014048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c077210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f204c0098e0 tx=0x7f204c009250 comp rx=0 tx=0).stop 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.946+0000 7f2060af2700 1 -- 172.21.15.31:0/2428014048 shutdown_connections 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.946+0000 7f2060af2700 1 --2- 172.21.15.31:0/2428014048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c077210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.946+0000 7f2060af2700 1 -- 172.21.15.31:0/2428014048 >> 172.21.15.31:0/2428014048 conn(0x7f205c0fe060 msgr2=0x7f205c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.946+0000 7f2060af2700 1 -- 172.21.15.31:0/2428014048 shutdown_connections 2024-04-24T23:04:54.835 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.946+0000 7f2060af2700 1 -- 172.21.15.31:0/2428014048 wait complete. 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f2060af2700 1 Processor -- start 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f2060af2700 1 -- start start 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f2060af2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f2060af2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c113d80 con 0x7f205c078c60 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f205a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f205a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47716/0 (socket says 172.21.15.31:47716) 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f205a59c700 1 -- 172.21.15.31:0/1645597189 learned_addr learned my addr 172.21.15.31:0/1645597189 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f205a59c700 1 -- 172.21.15.31:0/1645597189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0094f0 con 0x7f205c078c60 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.947+0000 7f205a59c700 1 --2- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c113840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f204c0094c0 tx=0x7f204c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.948+0000 7f204affd700 1 -- 172.21.15.31:0/1645597189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f204c010890 con 0x7f205c078c60 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.948+0000 7f204affd700 1 -- 172.21.15.31:0/1645597189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f204c010ec0 con 0x7f205c078c60 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.948+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c113f80 con 0x7f205c078c60 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.948+0000 7f204affd700 1 -- 172.21.15.31:0/1645597189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f204c0243f0 con 0x7f205c078c60 2024-04-24T23:04:54.836 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.948+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c114af0 con 0x7f205c078c60 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.948+0000 7f204affd700 1 -- 172.21.15.31:0/1645597189 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f204c02d3f0 con 0x7f205c078c60 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.949+0000 7f204affd700 1 --2- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f2044036990 0x7f2044038e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.949+0000 7f204affd700 1 -- 172.21.15.31:0/1645597189 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f204c01a7e0 con 0x7f205c078c60 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.949+0000 7f2059d9b700 1 --2- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f2044036990 0x7f2044038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.949+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c05f5c0 con 0x7f205c078c60 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.949+0000 7f2059d9b700 1 --2- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f2044036990 0x7f2044038e40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2050009990 tx=0x7f2050006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:53.954+0000 7f204affd700 1 -- 172.21.15.31:0/1645597189 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f204c02b8e0 con 0x7f205c078c60 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.257+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f205c04bed0 con 0x7f205c078c60 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.262+0000 7f204affd700 1 -- 172.21.15.31:0/1645597189 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f204c02b2f0 con 0x7f205c078c60 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f2044036990 msgr2=0x7f2044038e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.837 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 --2- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f2044036990 0x7f2044038e40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2050009990 tx=0x7f2050006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 msgr2=0x7f205c113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 --2- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c113840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f204c0094c0 tx=0x7f204c009250 comp rx=0 tx=0).stop 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 shutdown_connections 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 --2- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f2044036990 0x7f2044038e40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 --2- 172.21.15.31:0/1645597189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205c078c60 0x7f205c113840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 >> 172.21.15.31:0/1645597189 conn(0x7f205c0fe060 msgr2=0x7f205c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 shutdown_connections 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr 2024-04-24T23:04:54.264+0000 7f2060af2700 1 -- 172.21.15.31:0/1645597189 wait complete. 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:54.838 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:04:54.839 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-04-24T23:04:54.873 INFO:tasks.cephadm:Fetching config... 2024-04-24T23:04:54.873 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:04:54.873 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-24T23:04:54.931 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-24T23:04:54.931 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:04:54.931 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-24T23:04:54.987 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-24T23:04:54.987 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:04:54.987 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/keyring of=/dev/stdout 2024-04-24T23:04:55.052 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-24T23:04:55.052 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:04:55.052 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-24T23:04:55.107 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-24T23:04:55.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYAxz0iE9pYj1EToztyOSmF0ydR1oBKqBGWN0/729C31TCxcZ1DsvEYLInoDGYnzTwuH9u6b7U5Z36k0UZL57H6fSxGDdha2uBECQmeWdeQYgFyVdEuZ4JbwhIwzuYpYoX7xdCIwb/eKsQTmQ/sohw5MwwjE+D1MUCo6bvM+crKjXCMKIYxttbXVuNC0f1CVuR6KSVKrExpkEApSIbhys5uqQ6ruyfApcvv1msIVJQoQPe2VPeMWaTWRR+jXiNqqa5uLUHO7+7Iiqo+cwKZEN40oEsCTpfS+nAta8dOBr3YeQqlU+IQCO2K+0T/Bytko7hnYGW5WyO/EkvN7nYiL5S63Px6TuncEJVAaiw7X91mdWMrO7Nn7vMUplI9Nq9B4+dO2TMXLJUHgv+W7PFb0u9L9tH51p6NmLMSp7W51huoGiYHGQPdatSf3BA2GXYQwFZb2/R6FeWFoMu5pdJyqZ3I05C4SDxhk+rpyOaEkPkshjIzZsQfiXMoHSmV5dsKqU= ceph-ca7b26e4-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:04:55.182 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYAxz0iE9pYj1EToztyOSmF0ydR1oBKqBGWN0/729C31TCxcZ1DsvEYLInoDGYnzTwuH9u6b7U5Z36k0UZL57H6fSxGDdha2uBECQmeWdeQYgFyVdEuZ4JbwhIwzuYpYoX7xdCIwb/eKsQTmQ/sohw5MwwjE+D1MUCo6bvM+crKjXCMKIYxttbXVuNC0f1CVuR6KSVKrExpkEApSIbhys5uqQ6ruyfApcvv1msIVJQoQPe2VPeMWaTWRR+jXiNqqa5uLUHO7+7Iiqo+cwKZEN40oEsCTpfS+nAta8dOBr3YeQqlU+IQCO2K+0T/Bytko7hnYGW5WyO/EkvN7nYiL5S63Px6TuncEJVAaiw7X91mdWMrO7Nn7vMUplI9Nq9B4+dO2TMXLJUHgv+W7PFb0u9L9tH51p6NmLMSp7W51huoGiYHGQPdatSf3BA2GXYQwFZb2/R6FeWFoMu5pdJyqZ3I05C4SDxhk+rpyOaEkPkshjIzZsQfiXMoHSmV5dsKqU= ceph-ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:55.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYAxz0iE9pYj1EToztyOSmF0ydR1oBKqBGWN0/729C31TCxcZ1DsvEYLInoDGYnzTwuH9u6b7U5Z36k0UZL57H6fSxGDdha2uBECQmeWdeQYgFyVdEuZ4JbwhIwzuYpYoX7xdCIwb/eKsQTmQ/sohw5MwwjE+D1MUCo6bvM+crKjXCMKIYxttbXVuNC0f1CVuR6KSVKrExpkEApSIbhys5uqQ6ruyfApcvv1msIVJQoQPe2VPeMWaTWRR+jXiNqqa5uLUHO7+7Iiqo+cwKZEN40oEsCTpfS+nAta8dOBr3YeQqlU+IQCO2K+0T/Bytko7hnYGW5WyO/EkvN7nYiL5S63Px6TuncEJVAaiw7X91mdWMrO7Nn7vMUplI9Nq9B4+dO2TMXLJUHgv+W7PFb0u9L9tH51p6NmLMSp7W51huoGiYHGQPdatSf3BA2GXYQwFZb2/R6FeWFoMu5pdJyqZ3I05C4SDxhk+rpyOaEkPkshjIzZsQfiXMoHSmV5dsKqU= ceph-ca7b26e4-028e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-24T23:04:55.234 INFO:teuthology.orchestra.run.smithi040.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYAxz0iE9pYj1EToztyOSmF0ydR1oBKqBGWN0/729C31TCxcZ1DsvEYLInoDGYnzTwuH9u6b7U5Z36k0UZL57H6fSxGDdha2uBECQmeWdeQYgFyVdEuZ4JbwhIwzuYpYoX7xdCIwb/eKsQTmQ/sohw5MwwjE+D1MUCo6bvM+crKjXCMKIYxttbXVuNC0f1CVuR6KSVKrExpkEApSIbhys5uqQ6ruyfApcvv1msIVJQoQPe2VPeMWaTWRR+jXiNqqa5uLUHO7+7Iiqo+cwKZEN40oEsCTpfS+nAta8dOBr3YeQqlU+IQCO2K+0T/Bytko7hnYGW5WyO/EkvN7nYiL5S63Px6TuncEJVAaiw7X91mdWMrO7Nn7vMUplI9Nq9B4+dO2TMXLJUHgv+W7PFb0u9L9tH51p6NmLMSp7W51huoGiYHGQPdatSf3BA2GXYQwFZb2/R6FeWFoMu5pdJyqZ3I05C4SDxhk+rpyOaEkPkshjIzZsQfiXMoHSmV5dsKqU= ceph-ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:04:55.246 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-24T23:04:55.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:55 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1645597189' entity='client.admin' 2024-04-24T23:04:55.517 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.093+0000 7f20fbb26700 1 -- 172.21.15.31:0/3713777530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 msgr2=0x7f20f40fe4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.093+0000 7f20fbb26700 1 --2- 172.21.15.31:0/3713777530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 0x7f20f40fe4a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009c60 tx=0x7f20e4004d10 comp rx=0 tx=0).stop 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.093+0000 7f20fbb26700 1 -- 172.21.15.31:0/3713777530 shutdown_connections 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.093+0000 7f20fbb26700 1 --2- 172.21.15.31:0/3713777530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 0x7f20f40fe4a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.093+0000 7f20fbb26700 1 -- 172.21.15.31:0/3713777530 >> 172.21.15.31:0/3713777530 conn(0x7f20f40f6850 msgr2=0x7f20f40f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.093+0000 7f20fbb26700 1 -- 172.21.15.31:0/3713777530 shutdown_connections 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.093+0000 7f20fbb26700 1 -- 172.21.15.31:0/3713777530 wait complete. 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.094+0000 7f20fbb26700 1 Processor -- start 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.094+0000 7f20fbb26700 1 -- start start 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.094+0000 7f20fbb26700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 0x7f20f40fd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:58.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.094+0000 7f20fbb26700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f40fdce0 con 0x7f20f40fe090 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.094+0000 7f20f98c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 0x7f20f40fd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.094+0000 7f20f98c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 0x7f20f40fd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47730/0 (socket says 172.21.15.31:47730) 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.094+0000 7f20f98c2700 1 -- 172.21.15.31:0/57207950 learned_addr learned my addr 172.21.15.31:0/57207950 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.094+0000 7f20f98c2700 1 -- 172.21.15.31:0/57207950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4009940 con 0x7f20f40fe090 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.095+0000 7f20f98c2700 1 --2- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 0x7f20f40fd7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20e4004e60 tx=0x7f20e4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.095+0000 7f20f27fc700 1 -- 172.21.15.31:0/57207950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20e4012410 con 0x7f20f40fe090 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.095+0000 7f20f27fc700 1 -- 172.21.15.31:0/57207950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20e4012a40 con 0x7f20f40fe090 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.095+0000 7f20f27fc700 1 -- 172.21.15.31:0/57207950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20e401ba40 con 0x7f20f40fe090 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.095+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f40fbdb0 con 0x7f20f40fe090 2024-04-24T23:04:58.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.095+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f40fc2a0 con 0x7f20f40fe090 2024-04-24T23:04:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.096+0000 7f20f27fc700 1 -- 172.21.15.31:0/57207950 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f20e4019040 con 0x7f20f40fe090 2024-04-24T23:04:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.096+0000 7f20f27fc700 1 --2- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f20dc0368d0 0x7f20dc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:04:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.096+0000 7f20f27fc700 1 -- 172.21.15.31:0/57207950 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f20e4049ca0 con 0x7f20f40fe090 2024-04-24T23:04:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.096+0000 7f20f90c1700 1 --2- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f20dc0368d0 0x7f20dc038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:04:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.096+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f405f5c0 con 0x7f20f40fe090 2024-04-24T23:04:58.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.097+0000 7f20f90c1700 1 --2- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f20dc0368d0 0x7f20dc038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009990 tx=0x7f20e8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:04:58.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.102+0000 7f20f27fc700 1 -- 172.21.15.31:0/57207950 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f20e4017080 con 0x7f20f40fe090 2024-04-24T23:04:58.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.300+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f20f404bed0 con 0x7f20f40fe090 2024-04-24T23:04:58.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.303+0000 7f20f27fc700 1 -- 172.21.15.31:0/57207950 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f20e401f020 con 0x7f20f40fe090 2024-04-24T23:04:58.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.305+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f20dc0368d0 msgr2=0x7f20dc038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.305+0000 7f20fbb26700 1 --2- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f20dc0368d0 0x7f20dc038d80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009990 tx=0x7f20e8006e40 comp rx=0 tx=0).stop 2024-04-24T23:04:58.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.305+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 msgr2=0x7f20f40fd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:04:58.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.306+0000 7f20fbb26700 1 --2- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 0x7f20f40fd7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f20e4004e60 tx=0x7f20e4004d10 comp rx=0 tx=0).stop 2024-04-24T23:04:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.306+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 shutdown_connections 2024-04-24T23:04:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.306+0000 7f20fbb26700 1 --2- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f20dc0368d0 0x7f20dc038d80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.306+0000 7f20fbb26700 1 --2- 172.21.15.31:0/57207950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20f40fe090 0x7f20f40fd7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:04:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.306+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 >> 172.21.15.31:0/57207950 conn(0x7f20f40f6850 msgr2=0x7f20f4103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:04:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.306+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 shutdown_connections 2024-04-24T23:04:58.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:04:58.306+0000 7f20fbb26700 1 -- 172.21.15.31:0/57207950 wait complete. 2024-04-24T23:04:59.454 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-24T23:04:59.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-24T23:04:59.646 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:59 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/57207950' entity='client.admin' 2024-04-24T23:04:59.647 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:59 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:59.647 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:59 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:59.647 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:59 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:04:59.647 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:59 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:04:59.647 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:59 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:04:59.647 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:59 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T23:04:59.647 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:04:59 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:00.471 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:05:00.599 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:00 smithi031 ceph-mon[26411]: Deploying daemon crash.smithi031 on smithi031 2024-04-24T23:05:02.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.331+0000 7f6278c2d700 1 -- 172.21.15.31:0/3097584293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 msgr2=0x7f62740fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:02.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.331+0000 7f6278c2d700 1 --2- 172.21.15.31:0/3097584293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 0x7f62740fbd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f62640048c0 tx=0x7f6264009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:02.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.331+0000 7f6278c2d700 1 -- 172.21.15.31:0/3097584293 shutdown_connections 2024-04-24T23:05:02.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.331+0000 7f6278c2d700 1 --2- 172.21.15.31:0/3097584293 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 0x7f62740fbd50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.331+0000 7f6278c2d700 1 -- 172.21.15.31:0/3097584293 >> 172.21.15.31:0/3097584293 conn(0x7f62740f70a0 msgr2=0x7f62740f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:02.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.331+0000 7f6278c2d700 1 -- 172.21.15.31:0/3097584293 shutdown_connections 2024-04-24T23:05:02.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.332+0000 7f6278c2d700 1 -- 172.21.15.31:0/3097584293 wait complete. 2024-04-24T23:05:02.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.332+0000 7f6278c2d700 1 Processor -- start 2024-04-24T23:05:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.332+0000 7f6278c2d700 1 -- start start 2024-04-24T23:05:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.332+0000 7f6278c2d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 0x7f627410bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.332+0000 7f6278c2d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627410c4d0 con 0x7f62740fb940 2024-04-24T23:05:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.332+0000 7f627259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 0x7f627410bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.332+0000 7f627259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 0x7f627410bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55026/0 (socket says 172.21.15.31:55026) 2024-04-24T23:05:02.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.332+0000 7f627259c700 1 -- 172.21.15.31:0/2373538452 learned_addr learned my addr 172.21.15.31:0/2373538452 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:05:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.333+0000 7f627259c700 1 -- 172.21.15.31:0/2373538452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6264005040 con 0x7f62740fb940 2024-04-24T23:05:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.333+0000 7f627259c700 1 --2- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 0x7f627410bf90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62640096b0 tx=0x7f6264009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.333+0000 7f626affd700 1 -- 172.21.15.31:0/2373538452 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f626400bed0 con 0x7f62740fb940 2024-04-24T23:05:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.333+0000 7f626affd700 1 -- 172.21.15.31:0/2373538452 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f626400b5c0 con 0x7f62740fb940 2024-04-24T23:05:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.333+0000 7f626affd700 1 -- 172.21.15.31:0/2373538452 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f626401c5b0 con 0x7f62740fb940 2024-04-24T23:05:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.333+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627410c6d0 con 0x7f62740fb940 2024-04-24T23:05:02.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.333+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627410d240 con 0x7f62740fb940 2024-04-24T23:05:02.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.334+0000 7f626affd700 1 -- 172.21.15.31:0/2373538452 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f626401c710 con 0x7f62740fb940 2024-04-24T23:05:02.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.334+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6274101e50 con 0x7f62740fb940 2024-04-24T23:05:02.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.334+0000 7f626affd700 1 --2- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f62600368d0 0x7f6260038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:02.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.334+0000 7f626affd700 1 -- 172.21.15.31:0/2373538452 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6264049b00 con 0x7f62740fb940 2024-04-24T23:05:02.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.334+0000 7f6271d9b700 1 --2- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f62600368d0 0x7f6260038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:02.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.335+0000 7f6271d9b700 1 --2- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f62600368d0 0x7f6260038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f625c009990 tx=0x7f625c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:02.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.339+0000 7f626affd700 1 -- 172.21.15.31:0/2373538452 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6264025400 con 0x7f62740fb940 2024-04-24T23:05:02.552 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.551+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 --> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f627410d480 con 0x7f62600368d0 2024-04-24T23:05:02.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.554+0000 7f626affd700 1 -- 172.21.15.31:0/2373538452 <== mgr.14162 v2:172.21.15.31:6800/1555170645 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f627410d480 con 0x7f62600368d0 2024-04-24T23:05:02.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f62600368d0 msgr2=0x7f6260038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 --2- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f62600368d0 0x7f6260038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f625c009990 tx=0x7f625c006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 msgr2=0x7f627410bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 --2- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 0x7f627410bf90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f62640096b0 tx=0x7f6264009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 shutdown_connections 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 --2- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f62600368d0 0x7f6260038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 --2- 172.21.15.31:0/2373538452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62740fb940 0x7f627410bf90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 >> 172.21.15.31:0/2373538452 conn(0x7f62740f70a0 msgr2=0x7f6274102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 shutdown_connections 2024-04-24T23:05:02.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:02.557+0000 7f6278c2d700 1 -- 172.21.15.31:0/2373538452 wait complete. 2024-04-24T23:05:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:03 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:03 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:03 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:03 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:03 smithi031 ceph-mon[26411]: Deploying daemon node-exporter.smithi031 on smithi031 2024-04-24T23:05:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:03 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:03.308 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi040 2024-04-24T23:05:03.308 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:05:03.308 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/etc/ceph/ceph.conf 2024-04-24T23:05:03.328 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:05:03.328 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:05:03.387 INFO:tasks.cephadm:Adding host smithi040 to orchestrator... 2024-04-24T23:05:03.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch host add smithi040 2024-04-24T23:05:03.649 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:05:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:04 smithi031 ceph-mon[26411]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 -- 172.21.15.31:0/4162294632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 msgr2=0x7f52e40ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 --2- 172.21.15.31:0/4162294632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 0x7f52e40ffa00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f52d40048c0 tx=0x7f52d4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 -- 172.21.15.31:0/4162294632 shutdown_connections 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 --2- 172.21.15.31:0/4162294632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 0x7f52e40ffa00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 -- 172.21.15.31:0/4162294632 >> 172.21.15.31:0/4162294632 conn(0x7f52e40f70a0 msgr2=0x7f52e40f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 -- 172.21.15.31:0/4162294632 shutdown_connections 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 -- 172.21.15.31:0/4162294632 wait complete. 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 Processor -- start 2024-04-24T23:05:06.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 -- start start 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.962+0000 7f52ea84e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 0x7f52e410d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52ea84e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e410d5f0 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 0x7f52e410d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52e3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 0x7f52e410d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55044/0 (socket says 172.21.15.31:55044) 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52e3fff700 1 -- 172.21.15.31:0/3136605970 learned_addr learned my addr 172.21.15.31:0/3136605970 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52e3fff700 1 -- 172.21.15.31:0/3136605970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4005040 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52e3fff700 1 --2- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 0x7f52e410d0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52d400b900 tx=0x7f52d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52e0ff9700 1 -- 172.21.15.31:0/3136605970 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52d400b5c0 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52e0ff9700 1 -- 172.21.15.31:0/3136605970 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52d400be70 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e410d7f0 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52e0ff9700 1 -- 172.21.15.31:0/3136605970 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52d401c590 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.963+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e410e360 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.964+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e4102f30 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.965+0000 7f52e0ff9700 1 -- 172.21.15.31:0/3136605970 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f52d4004d30 con 0x7f52e40fd620 2024-04-24T23:05:06.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.965+0000 7f52e0ff9700 1 --2- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f52cc036920 0x7f52cc038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.965+0000 7f52e0ff9700 1 -- 172.21.15.31:0/3136605970 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f52d4049910 con 0x7f52e40fd620 2024-04-24T23:05:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.965+0000 7f52e37fe700 1 --2- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f52cc036920 0x7f52cc038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.965+0000 7f52e37fe700 1 --2- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f52cc036920 0x7f52cc038dd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009990 tx=0x7f52d8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:06.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:05.969+0000 7f52e0ff9700 1 -- 172.21.15.31:0/3136605970 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f52d4011370 con 0x7f52e40fd620 2024-04-24T23:05:06.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:06.172+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 --> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi040", "target": ["mon-mgr", ""]}) v1 -- 0x7f52e405f660 con 0x7f52cc036920 2024-04-24T23:05:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:07 smithi031 ceph-mon[26411]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:07 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:07 smithi031 ceph-mon[26411]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi040", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:07 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:07 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:07 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:07 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:08.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.212+0000 7f52e0ff9700 1 -- 172.21.15.31:0/3136605970 <== mgr.14162 v2:172.21.15.31:6800/1555170645 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f52e405f660 con 0x7f52cc036920 2024-04-24T23:05:08.214 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi040' with addr '172.21.15.40' 2024-04-24T23:05:08.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.214+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f52cc036920 msgr2=0x7f52cc038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.214+0000 7f52ea84e700 1 --2- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f52cc036920 0x7f52cc038dd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f52d8009990 tx=0x7f52d8006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.214+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 msgr2=0x7f52e410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.214+0000 7f52ea84e700 1 --2- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 0x7f52e410d0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f52d400b900 tx=0x7f52d4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.215+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 shutdown_connections 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.215+0000 7f52ea84e700 1 --2- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f52cc036920 0x7f52cc038dd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.215+0000 7f52ea84e700 1 --2- 172.21.15.31:0/3136605970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52e40fd620 0x7f52e410d0b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.215+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 >> 172.21.15.31:0/3136605970 conn(0x7f52e40f70a0 msgr2=0x7f52e4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.215+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 shutdown_connections 2024-04-24T23:05:08.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:08.215+0000 7f52ea84e700 1 -- 172.21.15.31:0/3136605970 wait complete. 2024-04-24T23:05:08.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:08 smithi031 ceph-mon[26411]: Deploying cephadm binary to smithi040 2024-04-24T23:05:08.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:08 smithi031 ceph-mon[26411]: Deploying daemon alertmanager.smithi031 on smithi031 2024-04-24T23:05:09.337 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-24T23:05:09.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:09 smithi031 ceph-mon[26411]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:09.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:09 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:09.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:09 smithi031 ceph-mon[26411]: Added host smithi040 2024-04-24T23:05:10.650 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:05:11.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:11 smithi031 ceph-mon[26411]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:11.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:11 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:13.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:13 smithi031 ceph-mon[26411]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.659+0000 7f38a50d5700 1 -- 172.21.15.31:0/841549325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 msgr2=0x7f38a00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.659+0000 7f38a50d5700 1 --2- 172.21.15.31:0/841549325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 0x7f38a00fce40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38900048c0 tx=0x7f3890009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.660+0000 7f38a50d5700 1 -- 172.21.15.31:0/841549325 shutdown_connections 2024-04-24T23:05:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.660+0000 7f38a50d5700 1 --2- 172.21.15.31:0/841549325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 0x7f38a00fce40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.660+0000 7f38a50d5700 1 -- 172.21.15.31:0/841549325 >> 172.21.15.31:0/841549325 conn(0x7f38a00f8170 msgr2=0x7f38a00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.660+0000 7f38a50d5700 1 -- 172.21.15.31:0/841549325 shutdown_connections 2024-04-24T23:05:13.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.660+0000 7f38a50d5700 1 -- 172.21.15.31:0/841549325 wait complete. 2024-04-24T23:05:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.660+0000 7f38a50d5700 1 Processor -- start 2024-04-24T23:05:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.660+0000 7f38a50d5700 1 -- start start 2024-04-24T23:05:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f38a50d5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 0x7f38a010f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f38a50d5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a010f790 con 0x7f38a00fca30 2024-04-24T23:05:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f389ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 0x7f38a010f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f389ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 0x7f38a010f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39896/0 (socket says 172.21.15.31:39896) 2024-04-24T23:05:13.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f389ed9d700 1 -- 172.21.15.31:0/78959844 learned_addr learned my addr 172.21.15.31:0/78959844 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:05:13.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f389ed9d700 1 -- 172.21.15.31:0/78959844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3890005040 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f389ed9d700 1 --2- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 0x7f38a010f250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3890009680 tx=0x7f3890009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f38877fe700 1 -- 172.21.15.31:0/78959844 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f389000b5c0 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f38877fe700 1 -- 172.21.15.31:0/78959844 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f389000be70 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f38877fe700 1 -- 172.21.15.31:0/78959844 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f38900135a0 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a010f990 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.661+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a0110500 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.662+0000 7f38877fe700 1 -- 172.21.15.31:0/78959844 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f3890013810 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.662+0000 7f38877fe700 1 --2- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f38880368d0 0x7f3888038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.662+0000 7f38877fe700 1 -- 172.21.15.31:0/78959844 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3890049350 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.662+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a0102f40 con 0x7f38a00fca30 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.663+0000 7f389e59c700 1 --2- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f38880368d0 0x7f3888038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:13.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.663+0000 7f389e59c700 1 --2- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f38880368d0 0x7f3888038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3894009990 tx=0x7f3894006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:13.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.667+0000 7f38877fe700 1 -- 172.21.15.31:0/78959844 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3890021020 con 0x7f38a00fca30 2024-04-24T23:05:13.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.870+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 --> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38a01028e0 con 0x7f38880368d0 2024-04-24T23:05:13.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.872+0000 7f38877fe700 1 -- 172.21.15.31:0/78959844 <== mgr.14162 v2:172.21.15.31:6800/1555170645 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f38a01028e0 con 0x7f38880368d0 2024-04-24T23:05:13.873 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:05:13.873 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.40", "hostname": "smithi040", "labels": [], "status": ""}] 2024-04-24T23:05:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f38880368d0 msgr2=0x7f3888038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 --2- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f38880368d0 0x7f3888038d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3894009990 tx=0x7f3894006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 msgr2=0x7f38a010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 --2- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 0x7f38a010f250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3890009680 tx=0x7f3890009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 shutdown_connections 2024-04-24T23:05:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 --2- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f38880368d0 0x7f3888038d80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 --2- 172.21.15.31:0/78959844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38a00fca30 0x7f38a010f250 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 >> 172.21.15.31:0/78959844 conn(0x7f38a00f8170 msgr2=0x7f38a00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 shutdown_connections 2024-04-24T23:05:13.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:13.874+0000 7f38a50d5700 1 -- 172.21.15.31:0/78959844 wait complete. 2024-04-24T23:05:15.365 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:15 smithi031 ceph-mon[26411]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:05:15.365 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:15 smithi031 ceph-mon[26411]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:15.390 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-24T23:05:15.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-24T23:05:16.199 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T23:05:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:17.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:17 smithi031 ceph-mon[26411]: Deploying daemon grafana.smithi031 on smithi031 2024-04-24T23:05:17.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.598+0000 7feedbf14700 1 -- 172.21.15.31:0/1933864775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 msgr2=0x7feed40fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.598+0000 7feedbf14700 1 --2- 172.21.15.31:0/1933864775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 0x7feed40fbd50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7feed00048c0 tx=0x7feed0009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.598+0000 7feedbf14700 1 -- 172.21.15.31:0/1933864775 shutdown_connections 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.598+0000 7feedbf14700 1 --2- 172.21.15.31:0/1933864775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 0x7feed40fbd50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.598+0000 7feedbf14700 1 -- 172.21.15.31:0/1933864775 >> 172.21.15.31:0/1933864775 conn(0x7feed40f70a0 msgr2=0x7feed40f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feedbf14700 1 -- 172.21.15.31:0/1933864775 shutdown_connections 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feedbf14700 1 -- 172.21.15.31:0/1933864775 wait complete. 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feedbf14700 1 Processor -- start 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feedbf14700 1 -- start start 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feedbf14700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 0x7feed4066860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feedbf14700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed410fdc0 con 0x7feed40fb940 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feed9cb0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 0x7feed4066860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feed9cb0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 0x7feed4066860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39916/0 (socket says 172.21.15.31:39916) 2024-04-24T23:05:17.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.599+0000 7feed9cb0700 1 -- 172.21.15.31:0/3521705129 learned_addr learned my addr 172.21.15.31:0/3521705129 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.600+0000 7feed9cb0700 1 -- 172.21.15.31:0/3521705129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed0005040 con 0x7feed40fb940 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.600+0000 7feed9cb0700 1 --2- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 0x7feed4066860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feed0009680 tx=0x7feed0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.600+0000 7feeca7fc700 1 -- 172.21.15.31:0/3521705129 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feed000b5c0 con 0x7feed40fb940 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.600+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed410ff00 con 0x7feed40fb940 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.600+0000 7feeca7fc700 1 -- 172.21.15.31:0/3521705129 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feed000be70 con 0x7feed40fb940 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.600+0000 7feeca7fc700 1 -- 172.21.15.31:0/3521705129 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7feed00135a0 con 0x7feed40fb940 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.600+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed41103f0 con 0x7feed40fb940 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.601+0000 7feeca7fc700 1 -- 172.21.15.31:0/3521705129 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7feed0004d30 con 0x7feed40fb940 2024-04-24T23:05:17.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.601+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed4105900 con 0x7feed40fb940 2024-04-24T23:05:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.601+0000 7feeca7fc700 1 --2- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7feec0036880 0x7feec0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.601+0000 7feeca7fc700 1 -- 172.21.15.31:0/3521705129 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7feed00493a0 con 0x7feed40fb940 2024-04-24T23:05:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.601+0000 7feed94af700 1 --2- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7feec0036880 0x7feec0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:17.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.601+0000 7feed94af700 1 --2- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7feec0036880 0x7feec0038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feec4009990 tx=0x7feec4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:17.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.606+0000 7feeca7fc700 1 -- 172.21.15.31:0/3521705129 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feed001ca90 con 0x7feed40fb940 2024-04-24T23:05:17.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:17.810+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7feed4101e50 con 0x7feed40fb940 2024-04-24T23:05:18.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.157+0000 7feeca7fc700 1 -- 172.21.15.31:0/3521705129 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7feed0017910 con 0x7feed40fb940 2024-04-24T23:05:18.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7feec0036880 msgr2=0x7feec0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 --2- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7feec0036880 0x7feec0038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feec4009990 tx=0x7feec4006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:18.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 msgr2=0x7feed4066860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:18.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 --2- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 0x7feed4066860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feed0009680 tx=0x7feed0009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:18.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 shutdown_connections 2024-04-24T23:05:18.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 --2- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7feec0036880 0x7feec0038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 --2- 172.21.15.31:0/3521705129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feed40fb940 0x7feed4066860 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:18.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 >> 172.21.15.31:0/3521705129 conn(0x7feed40f70a0 msgr2=0x7feed4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:18.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 shutdown_connections 2024-04-24T23:05:18.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:05:18.160+0000 7feedbf14700 1 -- 172.21.15.31:0/3521705129 wait complete. 2024-04-24T23:05:18.163 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-04-24T23:05:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:18 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3521705129' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:05:19.156 INFO:tasks.cephadm:Adding mon.smithi031 on smithi031 2024-04-24T23:05:19.156 INFO:tasks.cephadm:Adding mon.smithi040 on smithi040 2024-04-24T23:05:19.157 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi031:172.21.15.31=smithi031;smithi040:172.21.15.40=smithi040' 2024-04-24T23:05:19.417 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:19.481 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:19.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:19 smithi031 ceph-mon[26411]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:19 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3521705129' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:05:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:19 smithi031 ceph-mon[26411]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:05:20.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.928+0000 7f890aaaa700 1 -- 172.21.15.40:0/1895352721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 msgr2=0x7f89040fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:20.932 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.928+0000 7f890aaaa700 1 --2- 172.21.15.40:0/1895352721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 0x7f89040fbd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88f80048c0 tx=0x7f88f8009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.928+0000 7f890aaaa700 1 -- 172.21.15.40:0/1895352721 shutdown_connections 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.928+0000 7f890aaaa700 1 --2- 172.21.15.40:0/1895352721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 0x7f89040fbd50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.928+0000 7f890aaaa700 1 -- 172.21.15.40:0/1895352721 >> 172.21.15.40:0/1895352721 conn(0x7f89040f70a0 msgr2=0x7f89040f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.928+0000 7f890aaaa700 1 -- 172.21.15.40:0/1895352721 shutdown_connections 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.928+0000 7f890aaaa700 1 -- 172.21.15.40:0/1895352721 wait complete. 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.929+0000 7f890aaaa700 1 Processor -- start 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.929+0000 7f890aaaa700 1 -- start start 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.929+0000 7f890aaaa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 0x7f89040f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.929+0000 7f890aaaa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040f9570 con 0x7f89040fb940 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.929+0000 7f8908846700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 0x7f89040f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f8908846700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 0x7f89040f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:34178/0 (socket says 172.21.15.40:34178) 2024-04-24T23:05:20.933 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f8908846700 1 -- 172.21.15.40:0/1961140643 learned_addr learned my addr 172.21.15.40:0/1961140643 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f8908846700 1 -- 172.21.15.40:0/1961140643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f8005040 con 0x7f89040fb940 2024-04-24T23:05:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f8908846700 1 --2- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 0x7f89040f9030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f88f80096b0 tx=0x7f88f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f89017fa700 1 -- 172.21.15.40:0/1961140643 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88f800bed0 con 0x7f89040fb940 2024-04-24T23:05:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f89017fa700 1 -- 172.21.15.40:0/1961140643 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88f800b5c0 con 0x7f89040fb940 2024-04-24T23:05:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89040f7640 con 0x7f89040fb940 2024-04-24T23:05:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f89017fa700 1 -- 172.21.15.40:0/1961140643 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88f801c5b0 con 0x7f89040fb940 2024-04-24T23:05:20.934 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.930+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89040f7b30 con 0x7f89040fb940 2024-04-24T23:05:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.931+0000 7f89017fa700 1 -- 172.21.15.40:0/1961140643 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f88f8013420 con 0x7f89040fb940 2024-04-24T23:05:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.931+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890405f5c0 con 0x7f89040fb940 2024-04-24T23:05:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.931+0000 7f89017fa700 1 --2- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f88f4036920 0x7f88f4038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:20.935 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.931+0000 7f89017fa700 1 -- 172.21.15.40:0/1961140643 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f88f8049ac0 con 0x7f89040fb940 2024-04-24T23:05:20.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.932+0000 7f8903fff700 1 --2- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f88f4036920 0x7f88f4038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:20.936 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.932+0000 7f8903fff700 1 --2- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f88f4036920 0x7f88f4038dd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009990 tx=0x7f88f0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:20.941 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:20.939+0000 7f89017fa700 1 -- 172.21.15.40:0/1961140643 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f88f8004d30 con 0x7f89040fb940 2024-04-24T23:05:21.145 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.143+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 --> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi031:172.21.15.31=smithi031;smithi040:172.21.15.40=smithi040", "target": ["mon-mgr", ""]}) v1 -- 0x7f89040f7fa0 con 0x7f88f4036920 2024-04-24T23:05:21.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.151+0000 7f89017fa700 1 -- 172.21.15.40:0/1961140643 <== mgr.14162 v2:172.21.15.31:6800/1555170645 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f89040f7fa0 con 0x7f88f4036920 2024-04-24T23:05:21.152 INFO:teuthology.orchestra.run.smithi040.stdout:Scheduled mon update... 2024-04-24T23:05:21.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f88f4036920 msgr2=0x7f88f4038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:21.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 --2- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f88f4036920 0x7f88f4038dd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f88f0009990 tx=0x7f88f0006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:21.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 msgr2=0x7f89040f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:21.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 --2- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 0x7f89040f9030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f88f80096b0 tx=0x7f88f8009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:21.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 shutdown_connections 2024-04-24T23:05:21.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 --2- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f88f4036920 0x7f88f4038dd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.154 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 --2- 172.21.15.40:0/1961140643 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89040fb940 0x7f89040f9030 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:21.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 >> 172.21.15.40:0/1961140643 conn(0x7f89040f70a0 msgr2=0x7f8904065050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:21.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 shutdown_connections 2024-04-24T23:05:21.155 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:21.152+0000 7f890aaaa700 1 -- 172.21.15.40:0/1961140643 wait complete. 2024-04-24T23:05:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:21 smithi031 ceph-mon[26411]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:21 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:21 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:21.733 DEBUG:teuthology.orchestra.run.smithi040:mon.smithi040> sudo journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi040.service 2024-04-24T23:05:21.736 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:21.736 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:22.028 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:22.075 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:22.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:22 smithi031 ceph-mon[26411]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi031:172.21.15.31=smithi031;smithi040:172.21.15.40=smithi040", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:05:22.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:22 smithi031 ceph-mon[26411]: Saving service mon spec with placement smithi031:172.21.15.31=smithi031;smithi040:172.21.15.40=smithi040;count:2 2024-04-24T23:05:23.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:23 smithi031 ceph-mon[26411]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 -- 172.21.15.40:0/2780350199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 msgr2=0x7f61180fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 --2- 172.21.15.40:0/2780350199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 0x7f61180fce40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f61080048c0 tx=0x7f6108009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 -- 172.21.15.40:0/2780350199 shutdown_connections 2024-04-24T23:05:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 --2- 172.21.15.40:0/2780350199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 0x7f61180fce40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 -- 172.21.15.40:0/2780350199 >> 172.21.15.40:0/2780350199 conn(0x7f61180f8170 msgr2=0x7f61180fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 -- 172.21.15.40:0/2780350199 shutdown_connections 2024-04-24T23:05:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 -- 172.21.15.40:0/2780350199 wait complete. 2024-04-24T23:05:23.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 Processor -- start 2024-04-24T23:05:23.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.506+0000 7f611d6c4700 1 -- start start 2024-04-24T23:05:23.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f611d6c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 0x7f6118071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:23.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f611d6c4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6118072240 con 0x7f61180fca30 2024-04-24T23:05:23.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f6116ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 0x7f6118071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:23.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f6116ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 0x7f6118071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:34350/0 (socket says 172.21.15.40:34350) 2024-04-24T23:05:23.511 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f6116ffd700 1 -- 172.21.15.40:0/4001096806 learned_addr learned my addr 172.21.15.40:0/4001096806 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f6116ffd700 1 -- 172.21.15.40:0/4001096806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6108005040 con 0x7f61180fca30 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f6116ffd700 1 --2- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 0x7f6118071d00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f61080096b0 tx=0x7f6108009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f60ff7fe700 1 -- 172.21.15.40:0/4001096806 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f610800bed0 con 0x7f61180fca30 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f60ff7fe700 1 -- 172.21.15.40:0/4001096806 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f610800b5c0 con 0x7f61180fca30 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6118070310 con 0x7f61180fca30 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f60ff7fe700 1 -- 172.21.15.40:0/4001096806 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f610801c5b0 con 0x7f61180fca30 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.507+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6118070800 con 0x7f61180fca30 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.508+0000 7f60ff7fe700 1 -- 172.21.15.40:0/4001096806 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f610801c710 con 0x7f61180fca30 2024-04-24T23:05:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.508+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611805f5c0 con 0x7f61180fca30 2024-04-24T23:05:23.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.509+0000 7f60ff7fe700 1 --2- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f61000368d0 0x7f6100038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:23.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.509+0000 7f60ff7fe700 1 -- 172.21.15.40:0/4001096806 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6108049b80 con 0x7f61180fca30 2024-04-24T23:05:23.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.509+0000 7f61167fc700 1 --2- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f61000368d0 0x7f6100038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:23.513 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.509+0000 7f61167fc700 1 --2- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f61000368d0 0x7f6100038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f610c009990 tx=0x7f610c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:23.516 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.514+0000 7f60ff7fe700 1 -- 172.21.15.40:0/4001096806 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6108025400 con 0x7f61180fca30 2024-04-24T23:05:23.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.789+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6118106410 con 0x7f61180fca30 2024-04-24T23:05:23.792 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.790+0000 7f60ff7fe700 1 -- 172.21.15.40:0/4001096806 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f6108004d30 con 0x7f61180fca30 2024-04-24T23:05:23.792 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:23.792 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:23.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f61000368d0 msgr2=0x7f6100038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 --2- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f61000368d0 0x7f6100038d80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f610c009990 tx=0x7f610c006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 msgr2=0x7f6118071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 --2- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 0x7f6118071d00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f61080096b0 tx=0x7f6108009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 shutdown_connections 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 --2- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f61000368d0 0x7f6100038d80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 --2- 172.21.15.40:0/4001096806 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61180fca30 0x7f6118071d00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 >> 172.21.15.40:0/4001096806 conn(0x7f61180f8170 msgr2=0x7f6118103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 shutdown_connections 2024-04-24T23:05:23.794 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:23.792+0000 7f611d6c4700 1 -- 172.21.15.40:0/4001096806 wait complete. 2024-04-24T23:05:23.795 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:24.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:24 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/4001096806' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:25.364 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:25.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:25.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:25 smithi031 ceph-mon[26411]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:25.626 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:25.682 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.138+0000 7ff60be9d700 1 -- 172.21.15.40:0/1101631597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 msgr2=0x7ff6040fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.138+0000 7ff60be9d700 1 --2- 172.21.15.40:0/1101631597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 0x7ff6040fce30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff6000048c0 tx=0x7ff600009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.138+0000 7ff60be9d700 1 -- 172.21.15.40:0/1101631597 shutdown_connections 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.138+0000 7ff60be9d700 1 --2- 172.21.15.40:0/1101631597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 0x7ff6040fce30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.138+0000 7ff60be9d700 1 -- 172.21.15.40:0/1101631597 >> 172.21.15.40:0/1101631597 conn(0x7ff6040f81a0 msgr2=0x7ff6040fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.138+0000 7ff60be9d700 1 -- 172.21.15.40:0/1101631597 shutdown_connections 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.138+0000 7ff60be9d700 1 -- 172.21.15.40:0/1101631597 wait complete. 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.139+0000 7ff60be9d700 1 Processor -- start 2024-04-24T23:05:27.142 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.139+0000 7ff60be9d700 1 -- start start 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.139+0000 7ff60be9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 0x7ff60410f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.139+0000 7ff60be9d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60410f7f0 con 0x7ff6040fca20 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.139+0000 7ff609c39700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 0x7ff60410f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.139+0000 7ff609c39700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 0x7ff60410f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:34370/0 (socket says 172.21.15.40:34370) 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.139+0000 7ff609c39700 1 -- 172.21.15.40:0/28749199 learned_addr learned my addr 172.21.15.40:0/28749199 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.139+0000 7ff609c39700 1 -- 172.21.15.40:0/28749199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600005040 con 0x7ff6040fca20 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.140+0000 7ff609c39700 1 --2- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 0x7ff60410f2b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff6000096b0 tx=0x7ff600009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.140+0000 7ff5fa7fc700 1 -- 172.21.15.40:0/28749199 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff60000bed0 con 0x7ff6040fca20 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.140+0000 7ff5fa7fc700 1 -- 172.21.15.40:0/28749199 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff60000b5c0 con 0x7ff6040fca20 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.140+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60410f9f0 con 0x7ff6040fca20 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.140+0000 7ff5fa7fc700 1 -- 172.21.15.40:0/28749199 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff60001c5b0 con 0x7ff6040fca20 2024-04-24T23:05:27.143 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.140+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff604110560 con 0x7ff6040fca20 2024-04-24T23:05:27.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.141+0000 7ff5fa7fc700 1 -- 172.21.15.40:0/28749199 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7ff600013420 con 0x7ff6040fca20 2024-04-24T23:05:27.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.141+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff604102f30 con 0x7ff6040fca20 2024-04-24T23:05:27.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.141+0000 7ff5fa7fc700 1 --2- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7ff5f0036880 0x7ff5f0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:27.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.141+0000 7ff5fa7fc700 1 -- 172.21.15.40:0/28749199 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff600049b10 con 0x7ff6040fca20 2024-04-24T23:05:27.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.141+0000 7ff609438700 1 --2- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7ff5f0036880 0x7ff5f0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:27.144 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.141+0000 7ff609438700 1 --2- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7ff5f0036880 0x7ff5f0038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009990 tx=0x7ff5f4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:27.152 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.150+0000 7ff5fa7fc700 1 -- 172.21.15.40:0/28749199 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff600025400 con 0x7ff6040fca20 2024-04-24T23:05:27.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.424+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff6041107a0 con 0x7ff6040fca20 2024-04-24T23:05:27.426 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.425+0000 7ff5fa7fc700 1 -- 172.21.15.40:0/28749199 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff600004d30 con 0x7ff6040fca20 2024-04-24T23:05:27.427 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:27.427 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:27.428 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.426+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7ff5f0036880 msgr2=0x7ff5f0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.426+0000 7ff60be9d700 1 --2- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7ff5f0036880 0x7ff5f0038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009990 tx=0x7ff5f4006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.426+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 msgr2=0x7ff60410f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.426+0000 7ff60be9d700 1 --2- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 0x7ff60410f2b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff6000096b0 tx=0x7ff600009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.427+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 shutdown_connections 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.427+0000 7ff60be9d700 1 --2- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7ff5f0036880 0x7ff5f0038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.427+0000 7ff60be9d700 1 --2- 172.21.15.40:0/28749199 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6040fca20 0x7ff60410f2b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.427+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 >> 172.21.15.40:0/28749199 conn(0x7ff6040f81a0 msgr2=0x7ff6040fa580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.427+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 shutdown_connections 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:27.427+0000 7ff60be9d700 1 -- 172.21.15.40:0/28749199 wait complete. 2024-04-24T23:05:27.429 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:27.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:27 smithi031 ceph-mon[26411]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:28 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/28749199' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:29.000 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:29.000 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:29.271 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:29.319 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:29.422 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:29 smithi031 ceph-mon[26411]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 -- 172.21.15.40:0/3863604552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 msgr2=0x7f3b000fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 --2- 172.21.15.40:0/3863604552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 0x7f3b000fce40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3af40048c0 tx=0x7f3af4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 -- 172.21.15.40:0/3863604552 shutdown_connections 2024-04-24T23:05:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 --2- 172.21.15.40:0/3863604552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 0x7f3b000fce40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:30.722 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 -- 172.21.15.40:0/3863604552 >> 172.21.15.40:0/3863604552 conn(0x7f3b000f8170 msgr2=0x7f3b000fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 -- 172.21.15.40:0/3863604552 shutdown_connections 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 -- 172.21.15.40:0/3863604552 wait complete. 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 Processor -- start 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.719+0000 7f3b06d62700 1 -- start start 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.720+0000 7f3b06d62700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 0x7f3b0010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.720+0000 7f3b06d62700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0010d5f0 con 0x7f3b000fca30 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.720+0000 7f3b04afe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 0x7f3b0010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.720+0000 7f3b04afe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 0x7f3b0010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:34380/0 (socket says 172.21.15.40:34380) 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.720+0000 7f3b04afe700 1 -- 172.21.15.40:0/78379857 learned_addr learned my addr 172.21.15.40:0/78379857 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.720+0000 7f3b04afe700 1 -- 172.21.15.40:0/78379857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af4005040 con 0x7f3b000fca30 2024-04-24T23:05:30.723 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.720+0000 7f3b04afe700 1 --2- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 0x7f3b0010d0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009680 tx=0x7f3af4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.721+0000 7f3afd7fa700 1 -- 172.21.15.40:0/78379857 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3af400b5c0 con 0x7f3b000fca30 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.721+0000 7f3afd7fa700 1 -- 172.21.15.40:0/78379857 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3af400be70 con 0x7f3b000fca30 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.721+0000 7f3afd7fa700 1 -- 172.21.15.40:0/78379857 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3af40135a0 con 0x7f3b000fca30 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.721+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0010d7f0 con 0x7f3b000fca30 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.721+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0010e360 con 0x7f3b000fca30 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.721+0000 7f3afd7fa700 1 -- 172.21.15.40:0/78379857 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f3af401c430 con 0x7f3b000fca30 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.722+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b00102f40 con 0x7f3b000fca30 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.722+0000 7f3afd7fa700 1 --2- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f3af0036880 0x7f3af0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.722+0000 7f3afd7fa700 1 -- 172.21.15.40:0/78379857 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3af4049430 con 0x7f3b000fca30 2024-04-24T23:05:30.725 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.722+0000 7f3afffff700 1 --2- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f3af0036880 0x7f3af0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:30.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.722+0000 7f3afffff700 1 --2- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f3af0036880 0x7f3af0038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009990 tx=0x7f3aec006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:30.728 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:30.727+0000 7f3afd7fa700 1 -- 172.21.15.40:0/78379857 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3af40117e0 con 0x7f3b000fca30 2024-04-24T23:05:31.006 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.005+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3b0010ed70 con 0x7f3b000fca30 2024-04-24T23:05:31.013 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.011+0000 7f3afd7fa700 1 -- 172.21.15.40:0/78379857 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3af4021070 con 0x7f3b000fca30 2024-04-24T23:05:31.013 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:31.013 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f3af0036880 msgr2=0x7f3af0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 --2- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f3af0036880 0x7f3af0038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009990 tx=0x7f3aec006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 msgr2=0x7f3b0010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 --2- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 0x7f3b0010d0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3af4009680 tx=0x7f3af4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 shutdown_connections 2024-04-24T23:05:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 --2- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f3af0036880 0x7f3af0038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 --2- 172.21.15.40:0/78379857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b000fca30 0x7f3b0010d0b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:31.015 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 >> 172.21.15.40:0/78379857 conn(0x7f3b000f8170 msgr2=0x7f3b00103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 shutdown_connections 2024-04-24T23:05:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:31.013+0000 7f3b06d62700 1 -- 172.21.15.40:0/78379857 wait complete. 2024-04-24T23:05:31.016 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:31.241 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:31 smithi031 ceph-mon[26411]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:31.241 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:31 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/78379857' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:32.330 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:32 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:32.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:32 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:32.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:32 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:32.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:32 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:32.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:32 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:32.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:32 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:32.331 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:32 smithi031 ceph-mon[26411]: Deploying daemon prometheus.smithi031 on smithi031 2024-04-24T23:05:32.644 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:32.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:32.905 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:32.952 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:33.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:33 smithi031 ceph-mon[26411]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:34.342 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.338+0000 7facc246c700 1 -- 172.21.15.40:0/3633316637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 msgr2=0x7facbc0fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.338+0000 7facc246c700 1 --2- 172.21.15.40:0/3633316637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 0x7facbc0fce40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7facac0048c0 tx=0x7facac009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.338+0000 7facc246c700 1 -- 172.21.15.40:0/3633316637 shutdown_connections 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.338+0000 7facc246c700 1 --2- 172.21.15.40:0/3633316637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 0x7facbc0fce40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.338+0000 7facc246c700 1 -- 172.21.15.40:0/3633316637 >> 172.21.15.40:0/3633316637 conn(0x7facbc0f8170 msgr2=0x7facbc0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.339+0000 7facc246c700 1 -- 172.21.15.40:0/3633316637 shutdown_connections 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.339+0000 7facc246c700 1 -- 172.21.15.40:0/3633316637 wait complete. 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.339+0000 7facc246c700 1 Processor -- start 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.339+0000 7facc246c700 1 -- start start 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.339+0000 7facc246c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 0x7facbc10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.339+0000 7facc246c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facbc10d5f0 con 0x7facbc0fca30 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 0x7facbc10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.343 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facbbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 0x7facbc10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:51140/0 (socket says 172.21.15.40:51140) 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facbbfff700 1 -- 172.21.15.40:0/149670426 learned_addr learned my addr 172.21.15.40:0/149670426 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facbbfff700 1 -- 172.21.15.40:0/149670426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac005040 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facbbfff700 1 --2- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 0x7facbc10d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7facac0096b0 tx=0x7facac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facb8ff9700 1 -- 172.21.15.40:0/149670426 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facac00bed0 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facbc10d7f0 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facb8ff9700 1 -- 172.21.15.40:0/149670426 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7facac00b5c0 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facb8ff9700 1 -- 172.21.15.40:0/149670426 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7facac01c5b0 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.340+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facbc10e360 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.341+0000 7facb8ff9700 1 -- 172.21.15.40:0/149670426 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7facac013420 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.341+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facbc102f40 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.341+0000 7facb8ff9700 1 --2- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7faca4036880 0x7faca4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.341+0000 7facb8ff9700 1 -- 172.21.15.40:0/149670426 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7facac049b10 con 0x7facbc0fca30 2024-04-24T23:05:34.344 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.342+0000 7facbb7fe700 1 --2- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7faca4036880 0x7faca4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:34.345 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.342+0000 7facbb7fe700 1 --2- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7faca4036880 0x7faca4038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7facb0009990 tx=0x7facb0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:34.347 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.347+0000 7facb8ff9700 1 -- 172.21.15.40:0/149670426 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7facac025400 con 0x7facbc0fca30 2024-04-24T23:05:34.621 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.620+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7facbc10ed70 con 0x7facbc0fca30 2024-04-24T23:05:34.622 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.621+0000 7facb8ff9700 1 -- 172.21.15.40:0/149670426 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7facac004d30 con 0x7facbc0fca30 2024-04-24T23:05:34.622 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:34.622 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7faca4036880 msgr2=0x7faca4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 --2- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7faca4036880 0x7faca4038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7facb0009990 tx=0x7facb0006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 msgr2=0x7facbc10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 --2- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 0x7facbc10d0b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7facac0096b0 tx=0x7facac009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 shutdown_connections 2024-04-24T23:05:34.624 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 --2- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7faca4036880 0x7faca4038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 --2- 172.21.15.40:0/149670426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facbc0fca30 0x7facbc10d0b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:34.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 >> 172.21.15.40:0/149670426 conn(0x7facbc0f8170 msgr2=0x7facbc103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:34.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 shutdown_connections 2024-04-24T23:05:34.625 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:34.623+0000 7facc246c700 1 -- 172.21.15.40:0/149670426 wait complete. 2024-04-24T23:05:34.625 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:35 smithi031 ceph-mon[26411]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:35 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/149670426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:36.204 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:36.205 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:36.465 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:36.513 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:37.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:37 smithi031 ceph-mon[26411]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:37.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:37 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 -- 172.21.15.40:0/2341652966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 msgr2=0x7f60f40fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 --2- 172.21.15.40:0/2341652966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 0x7f60f40fbd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60e40048c0 tx=0x7f60e4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 -- 172.21.15.40:0/2341652966 shutdown_connections 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 --2- 172.21.15.40:0/2341652966 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 0x7f60f40fbd50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 -- 172.21.15.40:0/2341652966 >> 172.21.15.40:0/2341652966 conn(0x7f60f40f70a0 msgr2=0x7f60f40f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 -- 172.21.15.40:0/2341652966 shutdown_connections 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 -- 172.21.15.40:0/2341652966 wait complete. 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 Processor -- start 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.906+0000 7f60f8cef700 1 -- start start 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.907+0000 7f60f8cef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 0x7f60f410bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.907+0000 7f60f8cef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f410c500 con 0x7f60f40fb940 2024-04-24T23:05:37.908 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.907+0000 7f60f259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 0x7f60f410bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.907+0000 7f60f259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 0x7f60f410bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:51158/0 (socket says 172.21.15.40:51158) 2024-04-24T23:05:37.909 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.907+0000 7f60f259c700 1 -- 172.21.15.40:0/1024841048 learned_addr learned my addr 172.21.15.40:0/1024841048 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.907+0000 7f60f259c700 1 -- 172.21.15.40:0/1024841048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4005040 con 0x7f60f40fb940 2024-04-24T23:05:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.907+0000 7f60f259c700 1 --2- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 0x7f60f410bfc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f60e40096b0 tx=0x7f60e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.908+0000 7f60e2ffd700 1 -- 172.21.15.40:0/1024841048 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60e400bed0 con 0x7f60f40fb940 2024-04-24T23:05:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.908+0000 7f60e2ffd700 1 -- 172.21.15.40:0/1024841048 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60e400b5c0 con 0x7f60f40fb940 2024-04-24T23:05:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.908+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f410c700 con 0x7f60f40fb940 2024-04-24T23:05:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.908+0000 7f60e2ffd700 1 -- 172.21.15.40:0/1024841048 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f60e401c5b0 con 0x7f60f40fb940 2024-04-24T23:05:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.908+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f410d270 con 0x7f60f40fb940 2024-04-24T23:05:37.910 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.908+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f4101e50 con 0x7f60f40fb940 2024-04-24T23:05:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.909+0000 7f60e2ffd700 1 -- 172.21.15.40:0/1024841048 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f60e4004d30 con 0x7f60f40fb940 2024-04-24T23:05:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.909+0000 7f60e2ffd700 1 --2- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f60dc0368d0 0x7f60dc038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.909+0000 7f60e2ffd700 1 -- 172.21.15.40:0/1024841048 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f60e4049a80 con 0x7f60f40fb940 2024-04-24T23:05:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.909+0000 7f60f1d9b700 1 --2- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f60dc0368d0 0x7f60dc038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:37.911 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.910+0000 7f60f1d9b700 1 --2- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f60dc0368d0 0x7f60dc038d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009990 tx=0x7f60e8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:37.915 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:37.914+0000 7f60e2ffd700 1 -- 172.21.15.40:0/1024841048 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f60e4011940 con 0x7f60f40fb940 2024-04-24T23:05:38.188 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.187+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f60f410dc80 con 0x7f60f40fb940 2024-04-24T23:05:38.189 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.188+0000 7f60e2ffd700 1 -- 172.21.15.40:0/1024841048 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f60e4011370 con 0x7f60f40fb940 2024-04-24T23:05:38.189 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:38.189 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f60dc0368d0 msgr2=0x7f60dc038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 --2- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f60dc0368d0 0x7f60dc038d80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009990 tx=0x7f60e8006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 msgr2=0x7f60f410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 --2- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 0x7f60f410bfc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f60e40096b0 tx=0x7f60e4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 shutdown_connections 2024-04-24T23:05:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 --2- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f60dc0368d0 0x7f60dc038d80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.191 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 --2- 172.21.15.40:0/1024841048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f40fb940 0x7f60f410bfc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 >> 172.21.15.40:0/1024841048 conn(0x7f60f40f70a0 msgr2=0x7f60f4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 shutdown_connections 2024-04-24T23:05:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:38.190+0000 7f60f8cef700 1 -- 172.21.15.40:0/1024841048 wait complete. 2024-04-24T23:05:38.192 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:39.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:39 smithi031 ceph-mon[26411]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:39.421 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:39 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/1024841048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:39.771 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:39.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:40.029 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:40.076 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:41.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:41 smithi031 ceph-mon[26411]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:05:41.464 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.462+0000 7fb60929b700 1 -- 172.21.15.40:0/3808446917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 msgr2=0x7fb6040fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.462+0000 7fb60929b700 1 --2- 172.21.15.40:0/3808446917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 0x7fb6040fce30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40048c0 tx=0x7fb5f4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.462+0000 7fb60929b700 1 -- 172.21.15.40:0/3808446917 shutdown_connections 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.462+0000 7fb60929b700 1 --2- 172.21.15.40:0/3808446917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 0x7fb6040fce30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.462+0000 7fb60929b700 1 -- 172.21.15.40:0/3808446917 >> 172.21.15.40:0/3808446917 conn(0x7fb6040f81a0 msgr2=0x7fb6040fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.463+0000 7fb60929b700 1 -- 172.21.15.40:0/3808446917 shutdown_connections 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.463+0000 7fb60929b700 1 -- 172.21.15.40:0/3808446917 wait complete. 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.463+0000 7fb60929b700 1 Processor -- start 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.463+0000 7fb60929b700 1 -- start start 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.463+0000 7fb60929b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 0x7fb60410d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.463+0000 7fb60929b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60410d5c0 con 0x7fb6040fca20 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb602ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 0x7fb60410d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.465 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb602ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 0x7fb60410d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:51174/0 (socket says 172.21.15.40:51174) 2024-04-24T23:05:41.466 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb602ffd700 1 -- 172.21.15.40:0/962901900 learned_addr learned my addr 172.21.15.40:0/962901900 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb602ffd700 1 -- 172.21.15.40:0/962901900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4005040 con 0x7fb6040fca20 2024-04-24T23:05:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb602ffd700 1 --2- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 0x7fb60410d080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40096b0 tx=0x7fb5f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb5eb7fe700 1 -- 172.21.15.40:0/962901900 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5f400bed0 con 0x7fb6040fca20 2024-04-24T23:05:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb5eb7fe700 1 -- 172.21.15.40:0/962901900 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f400b5c0 con 0x7fb6040fca20 2024-04-24T23:05:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60410d7c0 con 0x7fb6040fca20 2024-04-24T23:05:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb5eb7fe700 1 -- 172.21.15.40:0/962901900 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5f401c5b0 con 0x7fb6040fca20 2024-04-24T23:05:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.464+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60410e330 con 0x7fb6040fca20 2024-04-24T23:05:41.467 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.465+0000 7fb5eb7fe700 1 -- 172.21.15.40:0/962901900 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fb5f4013420 con 0x7fb6040fca20 2024-04-24T23:05:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.465+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb604102f30 con 0x7fb6040fca20 2024-04-24T23:05:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.465+0000 7fb5eb7fe700 1 --2- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fb5ec036880 0x7fb5ec038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.466+0000 7fb5eb7fe700 1 -- 172.21.15.40:0/962901900 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb5f4049ac0 con 0x7fb6040fca20 2024-04-24T23:05:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.466+0000 7fb6027fc700 1 --2- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fb5ec036880 0x7fb5ec038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:41.468 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.466+0000 7fb6027fc700 1 --2- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fb5ec036880 0x7fb5ec038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009990 tx=0x7fb5f8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:41.471 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.471+0000 7fb5eb7fe700 1 -- 172.21.15.40:0/962901900 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb5f4013a80 con 0x7fb6040fca20 2024-04-24T23:05:41.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.745+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb60410ed40 con 0x7fb6040fca20 2024-04-24T23:05:41.747 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.746+0000 7fb5eb7fe700 1 -- 172.21.15.40:0/962901900 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fb5f4011370 con 0x7fb6040fca20 2024-04-24T23:05:41.747 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:41.747 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fb5ec036880 msgr2=0x7fb5ec038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 --2- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fb5ec036880 0x7fb5ec038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009990 tx=0x7fb5f8006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 msgr2=0x7fb60410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 --2- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 0x7fb60410d080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40096b0 tx=0x7fb5f4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 shutdown_connections 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 --2- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7fb5ec036880 0x7fb5ec038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 --2- 172.21.15.40:0/962901900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6040fca20 0x7fb60410d080 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 >> 172.21.15.40:0/962901900 conn(0x7fb6040f81a0 msgr2=0x7fb604103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 shutdown_connections 2024-04-24T23:05:41.749 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:41.748+0000 7fb60929b700 1 -- 172.21.15.40:0/962901900 wait complete. 2024-04-24T23:05:41.750 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:42.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:42 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:42.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:42 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:42.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:42 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:42.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:42 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-24T23:05:42.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:42 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/962901900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:43.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:43 smithi031 ceph-mon[26411]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-24T23:05:43.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:43 smithi031 ceph-mon[26411]: mgrmap e14: smithi031.rrnmwr(active, since 56s) 2024-04-24T23:05:43.260 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:43.260 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:43.519 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:43.567 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.022+0000 7f24b07c2700 1 -- 172.21.15.40:0/3462759842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 msgr2=0x7f24a80fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.022+0000 7f24b07c2700 1 --2- 172.21.15.40:0/3462759842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 0x7f24a80fce40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f24a40048c0 tx=0x7f24a4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.022+0000 7f24b07c2700 1 -- 172.21.15.40:0/3462759842 shutdown_connections 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.022+0000 7f24b07c2700 1 --2- 172.21.15.40:0/3462759842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 0x7f24a80fce40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.022+0000 7f24b07c2700 1 -- 172.21.15.40:0/3462759842 >> 172.21.15.40:0/3462759842 conn(0x7f24a80f8170 msgr2=0x7f24a80fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.022+0000 7f24b07c2700 1 -- 172.21.15.40:0/3462759842 shutdown_connections 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.022+0000 7f24b07c2700 1 -- 172.21.15.40:0/3462759842 wait complete. 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.023+0000 7f24b07c2700 1 Processor -- start 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.023+0000 7f24b07c2700 1 -- start start 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.023+0000 7f24b07c2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 0x7f24a810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:45.025 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.023+0000 7f24b07c2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a810d5f0 con 0x7f24a80fca30 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.023+0000 7f24ae55e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 0x7f24a810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.023+0000 7f24ae55e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 0x7f24a810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:44644/0 (socket says 172.21.15.40:44644) 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.023+0000 7f24ae55e700 1 -- 172.21.15.40:0/1116600960 learned_addr learned my addr 172.21.15.40:0/1116600960 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.023+0000 7f24ae55e700 1 -- 172.21.15.40:0/1116600960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4005040 con 0x7f24a80fca30 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.024+0000 7f24ae55e700 1 --2- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 0x7f24a810d0b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f24a40096b0 tx=0x7f24a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.024+0000 7f249affd700 1 -- 172.21.15.40:0/1116600960 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24a400bed0 con 0x7f24a80fca30 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.024+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a810d7f0 con 0x7f24a80fca30 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.024+0000 7f249affd700 1 -- 172.21.15.40:0/1116600960 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24a400b5c0 con 0x7f24a80fca30 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.024+0000 7f249affd700 1 -- 172.21.15.40:0/1116600960 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f24a401c5b0 con 0x7f24a80fca30 2024-04-24T23:05:45.026 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.024+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a810e360 con 0x7f24a80fca30 2024-04-24T23:05:45.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.025+0000 7f249affd700 1 -- 172.21.15.40:0/1116600960 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7f24a4013420 con 0x7f24a80fca30 2024-04-24T23:05:45.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.025+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a8102f40 con 0x7f24a80fca30 2024-04-24T23:05:45.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.025+0000 7f249affd700 1 --2- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f24940368d0 0x7f2494038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:45.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.025+0000 7f249affd700 1 -- 172.21.15.40:0/1116600960 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f24a4049b70 con 0x7f24a80fca30 2024-04-24T23:05:45.027 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.025+0000 7f24add5d700 1 -- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f24940368d0 msgr2=0x7f2494038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1555170645 2024-04-24T23:05:45.028 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.025+0000 7f24add5d700 1 --2- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f24940368d0 0x7f2494038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:05:45.030 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.030+0000 7f249affd700 1 -- 172.21.15.40:0/1116600960 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f24a4025400 con 0x7f24a80fca30 2024-04-24T23:05:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.226+0000 7f24add5d700 1 -- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f24940368d0 msgr2=0x7f2494038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1555170645 2024-04-24T23:05:45.227 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.226+0000 7f24add5d700 1 --2- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f24940368d0 0x7f2494038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:05:45.309 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.308+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f24a810ed70 con 0x7f24a80fca30 2024-04-24T23:05:45.310 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.309+0000 7f249affd700 1 -- 172.21.15.40:0/1116600960 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f24a4004d30 con 0x7f24a80fca30 2024-04-24T23:05:45.310 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:45.310 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f24940368d0 msgr2=0x7f2494038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 --2- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f24940368d0 0x7f2494038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 msgr2=0x7f24a810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 --2- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 0x7f24a810d0b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f24a40096b0 tx=0x7f24a4009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 shutdown_connections 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 --2- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f24940368d0 0x7f2494038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 --2- 172.21.15.40:0/1116600960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24a80fca30 0x7f24a810d0b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 >> 172.21.15.40:0/1116600960 conn(0x7f24a80f8170 msgr2=0x7f24a8103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:45.312 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 shutdown_connections 2024-04-24T23:05:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:45.311+0000 7f24b07c2700 1 -- 172.21.15.40:0/1116600960 wait complete. 2024-04-24T23:05:45.313 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:45.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:45 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/1116600960' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:46.919 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:46.919 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:47.180 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:47.237 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.630+0000 7f7af3ae7700 1 -- 172.21.15.40:0/3762003188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 msgr2=0x7f7aec0fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.630+0000 7f7af3ae7700 1 --2- 172.21.15.40:0/3762003188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 0x7f7aec0fce40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80048c0 tx=0x7f7ae8009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.630+0000 7f7af3ae7700 1 -- 172.21.15.40:0/3762003188 shutdown_connections 2024-04-24T23:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.630+0000 7f7af3ae7700 1 --2- 172.21.15.40:0/3762003188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 0x7f7aec0fce40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.630+0000 7f7af3ae7700 1 -- 172.21.15.40:0/3762003188 >> 172.21.15.40:0/3762003188 conn(0x7f7aec0f8170 msgr2=0x7f7aec0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:48.633 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.630+0000 7f7af3ae7700 1 -- 172.21.15.40:0/3762003188 shutdown_connections 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.630+0000 7f7af3ae7700 1 -- 172.21.15.40:0/3762003188 wait complete. 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.631+0000 7f7af3ae7700 1 Processor -- start 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.631+0000 7f7af3ae7700 1 -- start start 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.631+0000 7f7af3ae7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 0x7f7aec10f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.631+0000 7f7af3ae7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec10f800 con 0x7f7aec0fca30 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.631+0000 7f7af1883700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 0x7f7aec10f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.631+0000 7f7af1883700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 0x7f7aec10f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:44652/0 (socket says 172.21.15.40:44652) 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.631+0000 7f7af1883700 1 -- 172.21.15.40:0/1360221139 learned_addr learned my addr 172.21.15.40:0/1360221139 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.632+0000 7f7af1883700 1 -- 172.21.15.40:0/1360221139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8005040 con 0x7f7aec0fca30 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.632+0000 7f7af1883700 1 --2- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 0x7f7aec10f2c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80096b0 tx=0x7f7ae8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.632+0000 7f7ae27fc700 1 -- 172.21.15.40:0/1360221139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae800bed0 con 0x7f7aec0fca30 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.632+0000 7f7ae27fc700 1 -- 172.21.15.40:0/1360221139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ae800b5c0 con 0x7f7aec0fca30 2024-04-24T23:05:48.634 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.632+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec10fa00 con 0x7f7aec0fca30 2024-04-24T23:05:48.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.632+0000 7f7ae27fc700 1 -- 172.21.15.40:0/1360221139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae801c5b0 con 0x7f7aec0fca30 2024-04-24T23:05:48.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.632+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec110570 con 0x7f7aec0fca30 2024-04-24T23:05:48.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.633+0000 7f7ae27fc700 1 -- 172.21.15.40:0/1360221139 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7f7ae8013420 con 0x7f7aec0fca30 2024-04-24T23:05:48.635 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.633+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec102f40 con 0x7f7aec0fca30 2024-04-24T23:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.633+0000 7f7ae27fc700 1 --2- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f7ad8036920 0x7f7ad8038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.633+0000 7f7ae27fc700 1 -- 172.21.15.40:0/1360221139 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7ae8049a80 con 0x7f7aec0fca30 2024-04-24T23:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.633+0000 7f7af1082700 1 -- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f7ad8036920 msgr2=0x7f7ad8038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1555170645 2024-04-24T23:05:48.636 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.633+0000 7f7af1082700 1 --2- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f7ad8036920 0x7f7ad8038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:05:48.639 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.638+0000 7f7ae27fc700 1 -- 172.21.15.40:0/1360221139 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7ae8025400 con 0x7f7aec0fca30 2024-04-24T23:05:48.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.834+0000 7f7af1082700 1 -- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f7ad8036920 msgr2=0x7f7ad8038dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:6800/1555170645 2024-04-24T23:05:48.835 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.834+0000 7f7af1082700 1 --2- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f7ad8036920 0x7f7ad8038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-24T23:05:48.916 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.915+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7aec110870 con 0x7f7aec0fca30 2024-04-24T23:05:48.917 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.916+0000 7f7ae27fc700 1 -- 172.21.15.40:0/1360221139 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f7ae8004d30 con 0x7f7aec0fca30 2024-04-24T23:05:48.917 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:48.917 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f7ad8036920 msgr2=0x7f7ad8038dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 --2- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f7ad8036920 0x7f7ad8038dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 msgr2=0x7f7aec10f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 --2- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 0x7f7aec10f2c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80096b0 tx=0x7f7ae8009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 shutdown_connections 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 --2- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:6800/1555170645,v1:172.21.15.31:6801/1555170645] conn(0x7f7ad8036920 0x7f7ad8038dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 --2- 172.21.15.40:0/1360221139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec0fca30 0x7f7aec10f2c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 >> 172.21.15.40:0/1360221139 conn(0x7f7aec0f8170 msgr2=0x7f7aec0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 shutdown_connections 2024-04-24T23:05:48.919 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:48.918+0000 7f7af3ae7700 1 -- 172.21.15.40:0/1360221139 wait complete. 2024-04-24T23:05:48.920 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:49.010 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/1360221139' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: Active manager daemon smithi031.rrnmwr restarted 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: mgrmap e15: smithi031.rrnmwr(active, starting, since 0.00316946s) 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:05:49.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:50.424 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:50.425 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:50.740 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:50.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:50 smithi031 ceph-mon[26411]: mgrmap e16: smithi031.rrnmwr(active, since 1.01679s) 2024-04-24T23:05:50.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:50.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.099+0000 7fbf952c0700 1 -- 172.21.15.40:0/3453015244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 msgr2=0x7fbf900ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.099+0000 7fbf952c0700 1 --2- 172.21.15.40:0/3453015244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 0x7fbf900ff630 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009c60 tx=0x7fbf80004d10 comp rx=0 tx=0).stop 2024-04-24T23:05:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.099+0000 7fbf952c0700 1 -- 172.21.15.40:0/3453015244 shutdown_connections 2024-04-24T23:05:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.099+0000 7fbf952c0700 1 --2- 172.21.15.40:0/3453015244 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 0x7fbf900ff630 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.102 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.099+0000 7fbf952c0700 1 -- 172.21.15.40:0/3453015244 >> 172.21.15.40:0/3453015244 conn(0x7fbf900f70a0 msgr2=0x7fbf900f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.099+0000 7fbf952c0700 1 -- 172.21.15.40:0/3453015244 shutdown_connections 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.099+0000 7fbf952c0700 1 -- 172.21.15.40:0/3453015244 wait complete. 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.100+0000 7fbf952c0700 1 Processor -- start 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.100+0000 7fbf952c0700 1 -- start start 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.100+0000 7fbf952c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 0x7fbf900fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.100+0000 7fbf952c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900fd4c0 con 0x7fbf90065d30 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.100+0000 7fbf8effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 0x7fbf900fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.100+0000 7fbf8effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 0x7fbf900fcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:32800/0 (socket says 172.21.15.40:32800) 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.100+0000 7fbf8effd700 1 -- 172.21.15.40:0/1481088803 learned_addr learned my addr 172.21.15.40:0/1481088803 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.100+0000 7fbf8effd700 1 -- 172.21.15.40:0/1481088803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf80009940 con 0x7fbf90065d30 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.101+0000 7fbf8effd700 1 --2- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 0x7fbf900fcf80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbf800097b0 tx=0x7fbf80004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.101+0000 7fbf877fe700 1 -- 172.21.15.40:0/1481088803 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf80012410 con 0x7fbf90065d30 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.101+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf900fb590 con 0x7fbf90065d30 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.101+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf900fba80 con 0x7fbf90065d30 2024-04-24T23:05:52.103 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.101+0000 7fbf877fe700 1 -- 172.21.15.40:0/1481088803 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf80012a40 con 0x7fbf90065d30 2024-04-24T23:05:52.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.101+0000 7fbf877fe700 1 -- 172.21.15.40:0/1481088803 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbf8001b880 con 0x7fbf90065d30 2024-04-24T23:05:52.104 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.102+0000 7fbf877fe700 1 -- 172.21.15.40:0/1481088803 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fbf80019040 con 0x7fbf90065d30 2024-04-24T23:05:52.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.102+0000 7fbf877fe700 1 --2- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbf7c036970 0x7fbf7c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:52.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.102+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf700052e0 con 0x7fbf90065d30 2024-04-24T23:05:52.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.102+0000 7fbf877fe700 1 -- 172.21.15.40:0/1481088803 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fbf80049f80 con 0x7fbf90065d30 2024-04-24T23:05:52.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.103+0000 7fbf8e7fc700 1 --2- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbf7c036970 0x7fbf7c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:52.105 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.103+0000 7fbf8e7fc700 1 --2- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbf7c036970 0x7fbf7c038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009990 tx=0x7fbf78006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:52.108 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.107+0000 7fbf877fe700 1 -- 172.21.15.40:0/1481088803 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbf8002b8f0 con 0x7fbf90065d30 2024-04-24T23:05:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:51 smithi031 ceph-mon[26411]: [24/Apr/2024:23:05:50] ENGINE Bus STARTING 2024-04-24T23:05:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:51 smithi031 ceph-mon[26411]: [24/Apr/2024:23:05:50] ENGINE Serving on https://172.21.15.31:7150 2024-04-24T23:05:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:51 smithi031 ceph-mon[26411]: [24/Apr/2024:23:05:50] ENGINE Bus STARTED 2024-04-24T23:05:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:51 smithi031 ceph-mon[26411]: mgrmap e17: smithi031.rrnmwr(active, since 2s) 2024-04-24T23:05:52.390 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.389+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbf70005590 con 0x7fbf90065d30 2024-04-24T23:05:52.391 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.390+0000 7fbf877fe700 1 -- 172.21.15.40:0/1481088803 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fbf80017070 con 0x7fbf90065d30 2024-04-24T23:05:52.391 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:52.391 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbf7c036970 msgr2=0x7fbf7c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 --2- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbf7c036970 0x7fbf7c038e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009990 tx=0x7fbf78006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 msgr2=0x7fbf900fcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 --2- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 0x7fbf900fcf80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbf800097b0 tx=0x7fbf80004d10 comp rx=0 tx=0).stop 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 shutdown_connections 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 --2- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbf7c036970 0x7fbf7c038e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 --2- 172.21.15.40:0/1481088803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf90065d30 0x7fbf900fcf80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 >> 172.21.15.40:0/1481088803 conn(0x7fbf900f70a0 msgr2=0x7fbf90103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 shutdown_connections 2024-04-24T23:05:52.393 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:52.392+0000 7fbf952c0700 1 -- 172.21.15.40:0/1481088803 wait complete. 2024-04-24T23:05:52.394 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:53.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:52 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/1481088803' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:54.037 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:54.038 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:54.333 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:54.388 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-24T23:05:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.452+0000 7f37a371f700 1 -- 172.21.15.40:0/797139751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 msgr2=0x7f379c0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.452+0000 7f37a371f700 1 --2- 172.21.15.40:0/797139751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 0x7f379c0fbd50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37980048c0 tx=0x7f3798009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.453+0000 7f37a371f700 1 -- 172.21.15.40:0/797139751 shutdown_connections 2024-04-24T23:05:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.453+0000 7f37a371f700 1 --2- 172.21.15.40:0/797139751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 0x7f379c0fbd50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.453+0000 7f37a371f700 1 -- 172.21.15.40:0/797139751 >> 172.21.15.40:0/797139751 conn(0x7f379c0f70a0 msgr2=0x7f379c0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.453+0000 7f37a371f700 1 -- 172.21.15.40:0/797139751 shutdown_connections 2024-04-24T23:05:56.454 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.453+0000 7f37a371f700 1 -- 172.21.15.40:0/797139751 wait complete. 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.453+0000 7f37a371f700 1 Processor -- start 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.453+0000 7f37a371f700 1 -- start start 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f37a371f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 0x7f379c10bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f37a371f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c10c4d0 con 0x7f379c0fb940 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f37a14bb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 0x7f379c10bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f37a14bb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 0x7f379c10bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:32814/0 (socket says 172.21.15.40:32814) 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f37a14bb700 1 -- 172.21.15.40:0/2265044909 learned_addr learned my addr 172.21.15.40:0/2265044909 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f37a14bb700 1 -- 172.21.15.40:0/2265044909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798005040 con 0x7f379c0fb940 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f37a14bb700 1 --2- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 0x7f379c10bf90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f37980096b0 tx=0x7f3798009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f3791ffb700 1 -- 172.21.15.40:0/2265044909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f379800bed0 con 0x7f379c0fb940 2024-04-24T23:05:56.456 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.454+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c10c6d0 con 0x7f379c0fb940 2024-04-24T23:05:56.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.455+0000 7f3791ffb700 1 -- 172.21.15.40:0/2265044909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f379800b5c0 con 0x7f379c0fb940 2024-04-24T23:05:56.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.455+0000 7f3791ffb700 1 -- 172.21.15.40:0/2265044909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f379801c5b0 con 0x7f379c0fb940 2024-04-24T23:05:56.457 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.455+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c10d240 con 0x7f379c0fb940 2024-04-24T23:05:56.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.455+0000 7f3791ffb700 1 -- 172.21.15.40:0/2265044909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f379801c710 con 0x7f379c0fb940 2024-04-24T23:05:56.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.456+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c101e50 con 0x7f379c0fb940 2024-04-24T23:05:56.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.456+0000 7f3791ffb700 1 --2- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f3788038bd0 0x7f378803b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:05:56.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.456+0000 7f3791ffb700 1 -- 172.21.15.40:0/2265044909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f3798049c90 con 0x7f379c0fb940 2024-04-24T23:05:56.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.456+0000 7f37a0cba700 1 --2- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f3788038bd0 0x7f378803b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:05:56.458 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.456+0000 7f37a0cba700 1 --2- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f3788038bd0 0x7f378803b080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f378c009990 tx=0x7f378c006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:05:56.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.461+0000 7f3791ffb700 1 -- 172.21.15.40:0/2265044909 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3798025400 con 0x7f379c0fb940 2024-04-24T23:05:56.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.742+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f379c05f5c0 con 0x7f379c0fb940 2024-04-24T23:05:56.743 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.742+0000 7f3791ffb700 1 -- 172.21.15.40:0/2265044909 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3798011370 con 0x7f379c0fb940 2024-04-24T23:05:56.743 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:05:56.744 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:05:56.745 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.744+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f3788038bd0 msgr2=0x7f378803b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.744+0000 7f37a371f700 1 --2- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f3788038bd0 0x7f378803b080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f378c009990 tx=0x7f378c006e40 comp rx=0 tx=0).stop 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.744+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 msgr2=0x7f379c10bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.744+0000 7f37a371f700 1 --2- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 0x7f379c10bf90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f37980096b0 tx=0x7f3798009e30 comp rx=0 tx=0).stop 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.744+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 shutdown_connections 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.744+0000 7f37a371f700 1 --2- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f3788038bd0 0x7f378803b080 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.744+0000 7f37a371f700 1 --2- 172.21.15.40:0/2265044909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0fb940 0x7f379c10bf90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.744+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 >> 172.21.15.40:0/2265044909 conn(0x7f379c0f70a0 msgr2=0x7f379c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.745+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 shutdown_connections 2024-04-24T23:05:56.746 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:05:56.745+0000 7f37a371f700 1 -- 172.21.15.40:0/2265044909 wait complete. 2024-04-24T23:05:56.747 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:05:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:05:57.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:05:57.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/2265044909' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:05:57.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:05:57.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:05:57.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:05:57.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:57 smithi031 ceph-mon[26411]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:05:58.468 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:05:58.469 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:05:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:05:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:05:58 smithi031 ceph-mon[26411]: Deploying daemon crash.smithi040 on smithi040 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.430+0000 7f74bace8700 1 -- 172.21.15.40:0/2594886640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 msgr2=0x7f74b40fde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.430+0000 7f74bace8700 1 --2- 172.21.15.40:0/2594886640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 0x7f74b40fde30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f74a80048c0 tx=0x7f74a8009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.430+0000 7f74bace8700 1 -- 172.21.15.40:0/2594886640 shutdown_connections 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.430+0000 7f74bace8700 1 --2- 172.21.15.40:0/2594886640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 0x7f74b40fde30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.430+0000 7f74bace8700 1 -- 172.21.15.40:0/2594886640 >> 172.21.15.40:0/2594886640 conn(0x7f74b40f74d0 msgr2=0x7f74b40f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.430+0000 7f74bace8700 1 -- 172.21.15.40:0/2594886640 shutdown_connections 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.430+0000 7f74bace8700 1 -- 172.21.15.40:0/2594886640 wait complete. 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.431+0000 7f74bace8700 1 Processor -- start 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.431+0000 7f74bace8700 1 -- start start 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.432+0000 7f74bace8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 0x7f74b41120a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.432+0000 7f74bace8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b41125e0 con 0x7f74b40fda60 2024-04-24T23:06:00.433 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.432+0000 7f74b8a84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 0x7f74b41120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:00.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.432+0000 7f74b8a84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 0x7f74b41120a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:32836/0 (socket says 172.21.15.40:32836) 2024-04-24T23:06:00.434 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.432+0000 7f74b8a84700 1 -- 172.21.15.40:0/211124897 learned_addr learned my addr 172.21.15.40:0/211124897 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:06:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.432+0000 7f74b8a84700 1 -- 172.21.15.40:0/211124897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8005040 con 0x7f74b40fda60 2024-04-24T23:06:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.432+0000 7f74b8a84700 1 --2- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 0x7f74b41120a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f74a800b820 tx=0x7f74a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.433+0000 7f74b17fa700 1 -- 172.21.15.40:0/211124897 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74a80093c0 con 0x7f74b40fda60 2024-04-24T23:06:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.433+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b4112780 con 0x7f74b40fda60 2024-04-24T23:06:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.433+0000 7f74b17fa700 1 -- 172.21.15.40:0/211124897 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74a8013420 con 0x7f74b40fda60 2024-04-24T23:06:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.433+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b4112bf0 con 0x7f74b40fda60 2024-04-24T23:06:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.433+0000 7f74b17fa700 1 -- 172.21.15.40:0/211124897 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f74a801c5b0 con 0x7f74b40fda60 2024-04-24T23:06:00.435 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.434+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b4066540 con 0x7f74b40fda60 2024-04-24T23:06:00.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.434+0000 7f74b17fa700 1 -- 172.21.15.40:0/211124897 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f74a801c7a0 con 0x7f74b40fda60 2024-04-24T23:06:00.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.434+0000 7f74b17fa700 1 --2- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f74a40369c0 0x7f74a4038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:00.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.434+0000 7f74b17fa700 1 -- 172.21.15.40:0/211124897 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f74a8049af0 con 0x7f74b40fda60 2024-04-24T23:06:00.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.434+0000 7f74b3fff700 1 --2- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f74a40369c0 0x7f74a4038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:00.436 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.435+0000 7f74b3fff700 1 --2- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f74a40369c0 0x7f74a4038e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009990 tx=0x7f74a0006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:00.439 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.439+0000 7f74b17fa700 1 -- 172.21.15.40:0/211124897 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f74a8023300 con 0x7f74b40fda60 2024-04-24T23:06:00.720 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.719+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f74b4106de0 con 0x7f74b40fda60 2024-04-24T23:06:00.724 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.723+0000 7f74b17fa700 1 -- 172.21.15.40:0/211124897 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f74a8011b60 con 0x7f74b40fda60 2024-04-24T23:06:00.724 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:06:00.724 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:06:00.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f74a40369c0 msgr2=0x7f74a4038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:00.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 --2- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f74a40369c0 0x7f74a4038e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009990 tx=0x7f74a0006e40 comp rx=0 tx=0).stop 2024-04-24T23:06:00.726 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 msgr2=0x7f74b41120a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 --2- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 0x7f74b41120a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f74a800b820 tx=0x7f74a8009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 shutdown_connections 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 --2- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f74a40369c0 0x7f74a4038e70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 --2- 172.21.15.40:0/211124897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40fda60 0x7f74b41120a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 >> 172.21.15.40:0/211124897 conn(0x7f74b40f74d0 msgr2=0x7f74b41036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 shutdown_connections 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:00.725+0000 7f74bace8700 1 -- 172.21.15.40:0/211124897 wait complete. 2024-04-24T23:06:00.727 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:06:01.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:00 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/211124897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:02.488 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:02.488 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:02.488 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:02.488 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:02.488 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:02 smithi031 ceph-mon[26411]: Deploying daemon node-exporter.smithi040 on smithi040 2024-04-24T23:06:02.570 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:06:02.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:06:02.832 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:05.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:05.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:05.787 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 -- 172.21.15.40:0/2417516933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 msgr2=0x7f13900fde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 --2- 172.21.15.40:0/2417516933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 0x7f13900fde40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f138c0048c0 tx=0x7f138c009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 -- 172.21.15.40:0/2417516933 shutdown_connections 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 --2- 172.21.15.40:0/2417516933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 0x7f13900fde40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 -- 172.21.15.40:0/2417516933 >> 172.21.15.40:0/2417516933 conn(0x7f1390071800 msgr2=0x7f1390071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 -- 172.21.15.40:0/2417516933 shutdown_connections 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 -- 172.21.15.40:0/2417516933 wait complete. 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 Processor -- start 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.785+0000 7f1397fe3700 1 -- start start 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.786+0000 7f1397fe3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 0x7f1390113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:05.788 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.786+0000 7f1397fe3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13901136d0 con 0x7f13900fda70 2024-04-24T23:06:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.786+0000 7f1395d7f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 0x7f1390113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.786+0000 7f1395d7f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 0x7f1390113190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:41686/0 (socket says 172.21.15.40:41686) 2024-04-24T23:06:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.786+0000 7f1395d7f700 1 -- 172.21.15.40:0/3535578304 learned_addr learned my addr 172.21.15.40:0/3535578304 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:06:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.786+0000 7f1395d7f700 1 -- 172.21.15.40:0/3535578304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c005040 con 0x7f13900fda70 2024-04-24T23:06:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.787+0000 7f1395d7f700 1 --2- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 0x7f1390113190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f138c00b7f0 tx=0x7f138c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.787+0000 7f13867fc700 1 -- 172.21.15.40:0/3535578304 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f138c009370 con 0x7f13900fda70 2024-04-24T23:06:05.790 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.787+0000 7f13867fc700 1 -- 172.21.15.40:0/3535578304 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f138c013420 con 0x7f13900fda70 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.787+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13901138d0 con 0x7f13900fda70 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.787+0000 7f13867fc700 1 -- 172.21.15.40:0/3535578304 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f138c01c590 con 0x7f13900fda70 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.787+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1390113d40 con 0x7f13900fda70 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.788+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13900658b0 con 0x7f13900fda70 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.788+0000 7f13867fc700 1 -- 172.21.15.40:0/3535578304 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f138c013a70 con 0x7f13900fda70 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.788+0000 7f13867fc700 1 --2- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f137c036970 0x7f137c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.788+0000 7f13867fc700 1 -- 172.21.15.40:0/3535578304 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f138c049400 con 0x7f13900fda70 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.788+0000 7f139557e700 1 --2- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f137c036970 0x7f137c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:05.791 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.789+0000 7f139557e700 1 --2- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f137c036970 0x7f137c038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1380009990 tx=0x7f1380006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:05.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:05.793+0000 7f13867fc700 1 -- 172.21.15.40:0/3535578304 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f138c024d30 con 0x7f13900fda70 2024-04-24T23:06:06.080 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.078+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f139005f5c0 con 0x7f13900fda70 2024-04-24T23:06:06.081 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.079+0000 7f13867fc700 1 -- 172.21.15.40:0/3535578304 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f138c021070 con 0x7f13900fda70 2024-04-24T23:06:06.081 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:06:06.081 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f137c036970 msgr2=0x7f137c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 --2- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f137c036970 0x7f137c038e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1380009990 tx=0x7f1380006e40 comp rx=0 tx=0).stop 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 msgr2=0x7f1390113190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 --2- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 0x7f1390113190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f138c00b7f0 tx=0x7f138c009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 shutdown_connections 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 --2- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f137c036970 0x7f137c038e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 --2- 172.21.15.40:0/3535578304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13900fda70 0x7f1390113190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 >> 172.21.15.40:0/3535578304 conn(0x7f1390071800 msgr2=0x7f1390103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:06.083 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 shutdown_connections 2024-04-24T23:06:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:06.081+0000 7f1397fe3700 1 -- 172.21.15.40:0/3535578304 wait complete. 2024-04-24T23:06:06.084 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:06:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:06:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:06.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: Deploying daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:06:06.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:06 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/3535578304' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:08.130 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:06:08.131 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:06:08.790 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:10.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: Deploying daemon mon.smithi040 on smithi040 2024-04-24T23:06:10.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:10.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.415+0000 7f04275db700 1 -- 172.21.15.40:0/947119286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 msgr2=0x7f0420101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.415+0000 7f04275db700 1 --2- 172.21.15.40:0/947119286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 0x7f0420101da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f041c0048c0 tx=0x7f041c009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:10.418 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.415+0000 7f04275db700 1 -- 172.21.15.40:0/947119286 shutdown_connections 2024-04-24T23:06:10.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.415+0000 7f04275db700 1 --2- 172.21.15.40:0/947119286 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 0x7f0420101da0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:10.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.415+0000 7f04275db700 1 -- 172.21.15.40:0/947119286 >> 172.21.15.40:0/947119286 conn(0x7f0420071800 msgr2=0x7f0420071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:10.421 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.416+0000 7f04275db700 1 -- 172.21.15.40:0/947119286 shutdown_connections 2024-04-24T23:06:10.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.417+0000 7f04275db700 1 -- 172.21.15.40:0/947119286 wait complete. 2024-04-24T23:06:10.423 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.417+0000 7f04275db700 1 Processor -- start 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.417+0000 7f04275db700 1 -- start start 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.417+0000 7f04275db700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 0x7f042010e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.417+0000 7f04275db700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042010e610 con 0x7f04201019d0 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.417+0000 7f0425377700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 0x7f042010e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.418+0000 7f0425377700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 0x7f042010e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.40:41714/0 (socket says 172.21.15.40:41714) 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.418+0000 7f0425377700 1 -- 172.21.15.40:0/3648803644 learned_addr learned my addr 172.21.15.40:0/3648803644 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.418+0000 7f0425377700 1 -- 172.21.15.40:0/3648803644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c005040 con 0x7f04201019d0 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.418+0000 7f0425377700 1 --2- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 0x7f042010e0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b820 tx=0x7f041c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.418+0000 7f0415ffb700 1 -- 172.21.15.40:0/3648803644 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f041c0093c0 con 0x7f04201019d0 2024-04-24T23:06:10.424 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.418+0000 7f0415ffb700 1 -- 172.21.15.40:0/3648803644 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f041c013420 con 0x7f04201019d0 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.418+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042010e810 con 0x7f04201019d0 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.418+0000 7f0415ffb700 1 -- 172.21.15.40:0/3648803644 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f041c01c5b0 con 0x7f04201019d0 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.419+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042010ed00 con 0x7f04201019d0 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.420+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042010ef40 con 0x7f04201019d0 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.420+0000 7f0415ffb700 1 -- 172.21.15.40:0/3648803644 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7f041c013a90 con 0x7f04201019d0 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.420+0000 7f0415ffb700 1 --2- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f040c036970 0x7f040c038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.420+0000 7f0415ffb700 1 -- 172.21.15.40:0/3648803644 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f041c049aa0 con 0x7f04201019d0 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.420+0000 7f0424b76700 1 --2- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f040c036970 0x7f040c038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:10.425 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.421+0000 7f0424b76700 1 --2- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f040c036970 0x7f040c038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f0410006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:10.427 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.425+0000 7f0415ffb700 1 -- 172.21.15.40:0/3648803644 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f042010ef40 con 0x7f04201019d0 2024-04-24T23:06:10.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.709+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f042010ef40 con 0x7f04201019d0 2024-04-24T23:06:10.711 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.710+0000 7f0415ffb700 1 -- 172.21.15.40:0/3648803644 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f042010ef40 con 0x7f04201019d0 2024-04-24T23:06:10.712 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:06:10.712 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":1,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:03:38.159717Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f040c036970 msgr2=0x7f040c038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 --2- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f040c036970 0x7f040c038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f0410006e40 comp rx=0 tx=0).stop 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 msgr2=0x7f042010e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 --2- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 0x7f042010e0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f041c00b820 tx=0x7f041c009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 shutdown_connections 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 --2- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f040c036970 0x7f040c038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 --2- 172.21.15.40:0/3648803644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04201019d0 0x7f042010e0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 >> 172.21.15.40:0/3648803644 conn(0x7f0420071800 msgr2=0x7f0420103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 shutdown_connections 2024-04-24T23:06:10.714 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:10.712+0000 7f04275db700 1 -- 172.21.15.40:0/3648803644 wait complete. 2024-04-24T23:06:10.715 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 1 2024-04-24T23:06:11.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:10 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/3648803644' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:11 smithi031 ceph-mon[26411]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:12.687 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-24T23:06:12.688 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-24T23:06:13.403 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi040/config 2024-04-24T23:06:14.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:14 smithi031 ceph-mon[26411]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.197 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 podman[33100]: 2024-04-24 23:06:15.176031829 +0000 UTC m=+0.590423631 container start bf68fa6cba2920982fabee711fc9966b26475039df38b819188d015d0210bf48 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T23:06:15.475 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable), process ceph-mon, pid 2 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pidfile_write: ignore empty --pid-file 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: load: jerasure load: lrc 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: RocksDB version: 6.15.5 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Compile date Apr 24 2024 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: DB SUMMARY 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: DB Session ID: 5RO4B4YT3AUJKY7C7WHH 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi040/store.db dir, Total Num: 0, files: 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi040/store.db: 000004.log size: 511 ; 2024-04-24T23:06:15.476 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.env: 0x562426ea7b00 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.fs: Posix File System 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.info_log: 0x562428e17620 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.statistics: (nil) 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.use_fsync: 0 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:06:15.477 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:06:15.478 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:06:15.478 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:06:15.478 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:06:15.478 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:06:15.478 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.db_log_dir: 2024-04-24T23:06:15.478 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi040/store.db 2024-04-24T23:06:15.478 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:06:15.478 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:06:15.479 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:06:15.479 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:06:15.479 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:06:15.479 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:06:15.479 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:06:15.479 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.write_buffer_manager: 0x562429b0aa20 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.unordered_write: 0 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:06:15.480 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.row_cache: None 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.wal_filter: None 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.preserve_deletes: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:06:15.481 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:06:15.482 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_open_files: -1 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Compression algorithms supported: 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: kZSTD supported: 0 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:06:15.483 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi040/store.db/MANIFEST-000003 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.merge_operator: 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_filter: None 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:06:15.484 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562428deaf58) 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_type: 0 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_block_index_type: 0 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_shortening: 1 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: hash_index_allow_collision: 1 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: checksum: 1 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: no_block_cache: 0 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache: 0x562428e1d090 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:06:15.485 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_options: 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: capacity : 536870912 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: num_shard_bits : 4 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: strict_capacity_limit : 0 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_compressed: (nil) 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: persistent_cache: (nil) 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_size: 4096 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_size_deviation: 10 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_restart_interval: 16 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_block_restart_interval: 1 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: metadata_block_size: 4096 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: partition_filters: 0 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: use_delta_encoding: 1 2024-04-24T23:06:15.486 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: whole_key_filtering: 1 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: verify_compression: 0 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: format_version: 4 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: enable_index_compression: 1 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_align: 0 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compression: NoCompression 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.num_levels: 7 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:06:15.487 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:06:15.488 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:06:15.489 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.arena_block_size: 4194304 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:06:15.490 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.ttl: 2592000 2024-04-24T23:06:15.491 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi040/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999975202624, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-04-24T23:06:15.492 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999975202935, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1713999975, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "d42dca37-49fb-4208-82fb-014dcee596d3", "db_session_id": "5RO4B4YT3AUJKY7C7WHH"}} 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713999975203259, "job": 1, "event": "recovery_finished"} 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi040/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x562428db1180 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: DB pointer 0x562428e86000 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** DB Stats ** 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:06:15.494 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-04-24T23:06:15.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:06:15.496 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.9 0.00 0.00 1 0.000 0 0 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:06:15.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040 does not exist in monmap, will attempt to join an existing cluster 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: using public_addr v2:172.21.15.40:0/0 -> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: starting mon.smithi040 rank -1 at public addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] at bind addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi040 fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(???) e0 preinit fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).mds e1 new map 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).mds e1 print_map 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: e1 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: legacy client fscid: -1 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:06:15.498 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: No filesystems configured 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.499 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi040", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Deploying cephadm binary to smithi040 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Deploying daemon alertmanager.smithi031 on smithi031 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Added host smithi040 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.500 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Deploying daemon grafana.smithi031 on smithi031 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3521705129' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3521705129' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: osdmap e4: 0 total, 0 up, 0 in 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.501 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.14194 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi031:172.21.15.31=smithi031;smithi040:172.21.15.40=smithi040", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Saving service mon spec with placement smithi031:172.21.15.31=smithi031;smithi040:172.21.15.40=smithi040;count:2 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/4001096806' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/28749199' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/78379857' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.502 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Deploying daemon prometheus.smithi031 on smithi031 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/149670426' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/1024841048' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.503 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/962901900' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14162 172.21.15.31:0/3793171661' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mgrmap e14: smithi031.rrnmwr(active, since 56s) 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/1116600960' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/1360221139' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Active manager daemon smithi031.rrnmwr restarted 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mgrmap e15: smithi031.rrnmwr(active, starting, since 0.00316946s) 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:06:15.504 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mgrmap e16: smithi031.rrnmwr(active, since 1.01679s) 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: [24/Apr/2024:23:05:50] ENGINE Bus STARTING 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: [24/Apr/2024:23:05:50] ENGINE Serving on https://172.21.15.31:7150 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: [24/Apr/2024:23:05:50] ENGINE Bus STARTED 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mgrmap e17: smithi031.rrnmwr(active, since 2s) 2024-04-24T23:06:15.505 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/1481088803' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd/host:smithi040", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:06:15.506 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/2265044909' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:15.507 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Deploying daemon crash.smithi040 on smithi040 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/211124897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Deploying daemon node-exporter.smithi040 on smithi040 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-24T23:06:15.508 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Deploying daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/3535578304' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: Deploying daemon mon.smithi040 on smithi040 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/3648803644' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:15.509 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.510 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:15.510 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 ceph-mon[33128]: mon.smithi040@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-04-24T23:06:15.510 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 bash[33100]: bf68fa6cba2920982fabee711fc9966b26475039df38b819188d015d0210bf48 2024-04-24T23:06:15.510 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:15 smithi040 systemd[1]: Started Ceph mon.smithi040 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:06:20.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: mon.smithi031 calling monitor election 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/282806972' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: mon.smithi040 calling monitor election 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: mon.smithi031 is new leader, mons smithi031,smithi040 in quorum (ranks 0,1) 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: monmap e2: 2 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: fsmap 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: mgrmap e17: smithi031.rrnmwr(active, since 31s) 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: overall HEALTH_OK 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/282806972' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/282806972' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/282806972' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:06:20.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:20 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:20.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: mon.smithi031 calling monitor election 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/282806972' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: mon.smithi040 calling monitor election 2024-04-24T23:06:20.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: mon.smithi031 is new leader, mons smithi031,smithi040 in quorum (ranks 0,1) 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: monmap e2: 2 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],smithi040=[v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: fsmap 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: osdmap e5: 0 total, 0 up, 0 in 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: mgrmap e17: smithi031.rrnmwr(active, since 31s) 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: overall HEALTH_OK 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:06:20.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/282806972' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:06:20.752 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:20.752 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/282806972' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:06:20.752 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/282806972' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:06:20.752 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:20 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:21.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.579+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3571988213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89fc0061f0 msgr2=0x7f89fc008650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.579+0000 7f8a17f3b700 1 --2- 172.21.15.40:0/3571988213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89fc0061f0 0x7f89fc008650 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8a10065750 tx=0x7f8a0c002040 comp rx=0 tx=0).stop 2024-04-24T23:06:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.579+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3571988213 shutdown_connections 2024-04-24T23:06:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.579+0000 7f8a17f3b700 1 --2- 172.21.15.40:0/3571988213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89fc0061f0 0x7f89fc008650 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.579+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3571988213 >> 172.21.15.40:0/3571988213 conn(0x7f8a10071970 msgr2=0x7f8a10071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.579+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3571988213 shutdown_connections 2024-04-24T23:06:21.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.579+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3571988213 wait complete. 2024-04-24T23:06:21.582 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:21 smithi040 ceph-mon[33128]: mgrmap e18: smithi031.rrnmwr(active, since 31s), standbys: smithi040.qtdwuq 2024-04-24T23:06:21.582 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:21.582 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi040.qtdwuq", "id": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:06:21.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:21.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:21.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:21.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:21 smithi040 ceph-mon[33128]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:21.583 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a17f3b700 1 Processor -- start 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a17f3b700 1 -- start start 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a17f3b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 0x7f8a10113340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a17f3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a10113880 0x7f8a1010d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a17f3b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10113e90 con 0x7f8a10113880 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a17f3b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a10114000 con 0x7f89fc0061f0 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a15cd7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 0x7f8a10113340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a15cd7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 0x7f8a10113340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57540/0 (socket says 172.21.15.40:57540) 2024-04-24T23:06:21.585 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a15cd7700 1 -- 172.21.15.40:0/3159036119 learned_addr learned my addr 172.21.15.40:0/3159036119 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a154d6700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a10113880 0x7f8a1010d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a15cd7700 1 -- 172.21.15.40:0/3159036119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 msgr2=0x7f8a10113340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a15cd7700 1 -- 172.21.15.40:0/3159036119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 msgr2=0x7f8a10113340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a15cd7700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 0x7f8a10113340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.580+0000 7f8a15cd7700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 0x7f8a10113340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a154d6700 1 -- 172.21.15.40:0/3159036119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 msgr2=0x7f8a10113340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a154d6700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 0x7f8a10113340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a154d6700 1 -- 172.21.15.40:0/3159036119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c005040 con 0x7f8a10113880 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a154d6700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a10113880 0x7f8a1010d2e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000b790 tx=0x7f8a000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a067fc700 1 -- 172.21.15.40:0/3159036119 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a00011810 con 0x7f8a10113880 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a067fc700 1 -- 172.21.15.40:0/3159036119 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a00011e40 con 0x7f8a10113880 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1010d940 con 0x7f8a10113880 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a067fc700 1 -- 172.21.15.40:0/3159036119 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a0000f2e0 con 0x7f8a10113880 2024-04-24T23:06:21.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.581+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1010de80 con 0x7f8a10113880 2024-04-24T23:06:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.582+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1010e150 con 0x7f8a10113880 2024-04-24T23:06:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.583+0000 7f8a067fc700 1 -- 172.21.15.40:0/3159036119 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f8a00010380 con 0x7f8a10113880 2024-04-24T23:06:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.583+0000 7f8a067fc700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f89f4069320 0x7f89f406b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.583+0000 7f8a067fc700 1 -- 172.21.15.40:0/3159036119 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f8a00086400 con 0x7f8a10113880 2024-04-24T23:06:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.583+0000 7f8a15cd7700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f89f4069320 0x7f89f406b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:21.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.584+0000 7f8a15cd7700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f89f4069320 0x7f89f406b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c004f50 tx=0x7f8a0c002040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:21.590 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.588+0000 7f8a067fc700 1 -- 172.21.15.40:0/3159036119 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8a1010e150 con 0x7f8a10113880 2024-04-24T23:06:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:21 smithi031 ceph-mon[26411]: mgrmap e18: smithi031.rrnmwr(active, since 31s), standbys: smithi040.qtdwuq 2024-04-24T23:06:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi040.qtdwuq", "id": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:06:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:21 smithi031 ceph-mon[26411]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:06:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.871+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8a1010e150 con 0x7f8a10113880 2024-04-24T23:06:21.873 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.871+0000 7f8a067fc700 1 -- 172.21.15.40:0/3159036119 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1013 (secure 0 0 0) 0x7f8a1010e150 con 0x7f8a10113880 2024-04-24T23:06:21.873 INFO:teuthology.orchestra.run.smithi040.stdout: 2024-04-24T23:06:21.874 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","modified":"2024-04-24T23:06:15.229793Z","created":"2024-04-24T23:03:38.159717Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi040","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:3300","nonce":0},{"type":"v1","addr":"172.21.15.40:6789","nonce":0}]},"addr":"172.21.15.40:6789/0","public_addr":"172.21.15.40:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-24T23:06:21.875 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f89f4069320 msgr2=0x7f89f406b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f89f4069320 0x7f89f406b7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c004f50 tx=0x7f8a0c002040 comp rx=0 tx=0).stop 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a10113880 msgr2=0x7f8a1010d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a10113880 0x7f8a1010d2e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000b790 tx=0x7f8a000091a0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 shutdown_connections 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89fc0061f0 0x7f8a10113340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f89f4069320 0x7f89f406b7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 --2- 172.21.15.40:0/3159036119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a10113880 0x7f8a1010d2e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 >> 172.21.15.40:0/3159036119 conn(0x7f8a10071970 msgr2=0x7f8a101022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.873+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 shutdown_connections 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:06:21.874+0000 7f8a17f3b700 1 -- 172.21.15.40:0/3159036119 wait complete. 2024-04-24T23:06:21.876 INFO:teuthology.orchestra.run.smithi040.stderr:dumped monmap epoch 2 2024-04-24T23:06:22.386 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:22 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/3159036119' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:22.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:22 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/3159036119' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-24T23:06:22.755 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-24T23:06:22.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-24T23:06:23.034 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: Reconfiguring mon.smithi031 (unknown last config time)... 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: Reconfiguring daemon mon.smithi031 on smithi031 2024-04-24T23:06:23.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:23 smithi031 ceph-mon[26411]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:23.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: Reconfiguring mon.smithi031 (unknown last config time)... 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: Reconfiguring daemon mon.smithi031 on smithi031 2024-04-24T23:06:23.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:23 smithi040 ceph-mon[33128]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:25.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:25 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:25.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:25 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:25.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 -- 172.21.15.31:0/3818828348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc0fbd40 msgr2=0x7f68dc0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:25.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 --2- 172.21.15.31:0/3818828348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc0fbd40 0x7f68dc0fc1b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68c8004940 tx=0x7f68c8009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 -- 172.21.15.31:0/3818828348 shutdown_connections 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 --2- 172.21.15.31:0/3818828348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc0fbd40 0x7f68dc0fc1b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 --2- 172.21.15.31:0/3818828348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68dc101d90 0x7f68dc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 -- 172.21.15.31:0/3818828348 >> 172.21.15.31:0/3818828348 conn(0x7f68dc071d00 msgr2=0x7f68dc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 -- 172.21.15.31:0/3818828348 shutdown_connections 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 -- 172.21.15.31:0/3818828348 wait complete. 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 Processor -- start 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.510+0000 7f68e30de700 1 -- start start 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68e30de700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68dc0fbd40 0x7f68dc10eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68e30de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc101d90 0x7f68dc10f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68e30de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc10f6c0 con 0x7f68dc101d90 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68e30de700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68dc10f800 con 0x7f68dc0fbd40 2024-04-24T23:06:25.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc101d90 0x7f68dc10f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc101d90 0x7f68dc10f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46854/0 (socket says 172.21.15.31:46854) 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68dbfff700 1 -- 172.21.15.31:0/570109706 learned_addr learned my addr 172.21.15.31:0/570109706 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68e0e7a700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68dc0fbd40 0x7f68dc10eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68dbfff700 1 -- 172.21.15.31:0/570109706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68dc0fbd40 msgr2=0x7f68dc10eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68dbfff700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68dc0fbd40 0x7f68dc10eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68dbfff700 1 -- 172.21.15.31:0/570109706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c8005040 con 0x7f68dc101d90 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68dbfff700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc101d90 0x7f68dc10f070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b3b0 tx=0x7f68c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68d97fa700 1 -- 172.21.15.31:0/570109706 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68c800e040 con 0x7f68dc101d90 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68d97fa700 1 -- 172.21.15.31:0/570109706 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68c8013420 con 0x7f68dc101d90 2024-04-24T23:06:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc117ba0 con 0x7f68dc101d90 2024-04-24T23:06:25.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.511+0000 7f68d97fa700 1 -- 172.21.15.31:0/570109706 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68c801c630 con 0x7f68dc101d90 2024-04-24T23:06:25.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.512+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68dc1180b0 con 0x7f68dc101d90 2024-04-24T23:06:25.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.513+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68dc0fd430 con 0x7f68dc101d90 2024-04-24T23:06:25.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.513+0000 7f68d97fa700 1 -- 172.21.15.31:0/570109706 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f68c8013590 con 0x7f68dc101d90 2024-04-24T23:06:25.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.513+0000 7f68d97fa700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f68cc069210 0x7f68cc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:25.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.514+0000 7f68d97fa700 1 -- 172.21.15.31:0/570109706 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f68c8087200 con 0x7f68dc101d90 2024-04-24T23:06:25.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.514+0000 7f68e0e7a700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f68cc069210 0x7f68cc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:25.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.514+0000 7f68e0e7a700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f68cc069210 0x7f68cc06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68d0004f40 tx=0x7f68d000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:25.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.518+0000 7f68d97fa700 1 -- 172.21.15.31:0/570109706 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f68c80546b0 con 0x7f68dc101d90 2024-04-24T23:06:25.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.719+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f68dc04bed0 con 0x7f68dc101d90 2024-04-24T23:06:25.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.719+0000 7f68d97fa700 1 -- 172.21.15.31:0/570109706 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7f68c8057cd0 con 0x7f68dc101d90 2024-04-24T23:06:25.721 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:25.721 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-04-24T23:06:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:06:25.721 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] 2024-04-24T23:06:25.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f68cc069210 msgr2=0x7f68cc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:25.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f68cc069210 0x7f68cc06b6c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68d0004f40 tx=0x7f68d000b040 comp rx=0 tx=0).stop 2024-04-24T23:06:25.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc101d90 msgr2=0x7f68dc10f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc101d90 0x7f68dc10f070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68c800b3b0 tx=0x7f68c8009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 shutdown_connections 2024-04-24T23:06:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68dc0fbd40 0x7f68dc10eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f68cc069210 0x7f68cc06b6c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 --2- 172.21.15.31:0/570109706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68dc101d90 0x7f68dc10f070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 >> 172.21.15.31:0/570109706 conn(0x7f68dc071d00 msgr2=0x7f68dc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 shutdown_connections 2024-04-24T23:06:25.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:25.722+0000 7f68e30de700 1 -- 172.21.15.31:0/570109706 wait complete. 2024-04-24T23:06:26.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:26.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:26.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:26.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: Reconfiguring mgr.smithi031.rrnmwr (unknown last config time)... 2024-04-24T23:06:26.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.rrnmwr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:26.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:26.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:26.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: Reconfiguring daemon mgr.smithi031.rrnmwr on smithi031 2024-04-24T23:06:26.166 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:26 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/570109706' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:26.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:26.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:26.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:26.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: Reconfiguring mgr.smithi031.rrnmwr (unknown last config time)... 2024-04-24T23:06:26.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.rrnmwr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:26.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:26.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:26.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: Reconfiguring daemon mgr.smithi031.rrnmwr on smithi031 2024-04-24T23:06:26.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:26 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/570109706' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:27.134 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-24T23:06:27.134 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:06:27.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:06:27.162 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:06:27.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:27.228 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:06:27.228 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.conf 2024-04-24T23:06:27.262 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:06:27.262 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:06:27.330 INFO:tasks.cephadm:Deploying OSDs... 2024-04-24T23:06:27.331 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:06:27.331 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:06:27.346 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:06:27.346 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-24 23:05:40.537971390 +0000 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-24 23:00:21.836855795 +0000 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-24 23:00:21.836855795 +0000 2024-04-24T23:06:27.402 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-24 23:00:21.836855795 +0000 2024-04-24T23:06:27.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:06:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-24T23:06:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-24T23:06:27.469 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000207958 s, 2.5 MB/s 2024-04-24T23:06:27.470 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:06:27.526 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-04-24T23:06:27.581 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:06:27.582 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:27.582 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-24T23:06:27.582 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:27.582 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:27.582 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-24 23:05:40.537971390 +0000 2024-04-24T23:06:27.582 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-24 23:00:22.198849908 +0000 2024-04-24T23:06:27.582 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-24 23:00:22.198849908 +0000 2024-04-24T23:06:27.582 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-24 23:00:22.198849908 +0000 2024-04-24T23:06:27.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:06:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-24T23:06:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-24T23:06:27.646 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000226162 s, 2.3 MB/s 2024-04-24T23:06:27.647 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:06:27.703 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-24 23:05:40.538971374 +0000 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-24 23:00:22.542844313 +0000 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-24 23:00:22.542844313 +0000 2024-04-24T23:06:27.759 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-24 23:00:22.542844313 +0000 2024-04-24T23:06:27.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:06:27.822 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-24T23:06:27.822 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-24T23:06:27.822 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000192739 s, 2.7 MB/s 2024-04-24T23:06:27.823 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:06:27.879 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-04-24T23:06:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:06:27.934 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:27.934 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-24T23:06:27.934 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:27.934 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:27.934 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-04-24 23:05:40.538971374 +0000 2024-04-24T23:06:27.934 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-24 23:00:22.892838621 +0000 2024-04-24T23:06:27.935 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-24 23:00:22.892838621 +0000 2024-04-24T23:06:27.935 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-04-24 23:00:22.892838621 +0000 2024-04-24T23:06:27.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:06:27.995 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:27 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:27.995 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:27 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:27.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:27 smithi031 ceph-mon[26411]: Reconfiguring crash.smithi031 (monmap changed)... 2024-04-24T23:06:27.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:27 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:06:27.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:27 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:27.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:27 smithi031 ceph-mon[26411]: Reconfiguring daemon crash.smithi031 on smithi031 2024-04-24T23:06:27.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:27 smithi031 ceph-mon[26411]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:27.998 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-04-24T23:06:27.998 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-04-24T23:06:27.998 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000272445 s, 1.9 MB/s 2024-04-24T23:06:27.999 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:06:28.056 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:06:28.056 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T23:06:28.075 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T23:06:28.075 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2024-04-24T23:06:28.134 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T23:06:28.134 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:28.134 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-24T23:06:28.134 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:28.135 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:28.135 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-24 23:06:13.650867741 +0000 2024-04-24T23:06:28.135 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-24 23:00:21.830603699 +0000 2024-04-24T23:06:28.135 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-24 23:00:21.830603699 +0000 2024-04-24T23:06:28.135 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-24 23:00:21.830603699 +0000 2024-04-24T23:06:28.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T23:06:28.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:27 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:28.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:27 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:28.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:27 smithi040 ceph-mon[33128]: Reconfiguring crash.smithi031 (monmap changed)... 2024-04-24T23:06:28.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:27 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:06:28.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:27 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:28.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:27 smithi040 ceph-mon[33128]: Reconfiguring daemon crash.smithi031 on smithi031 2024-04-24T23:06:28.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:27 smithi040 ceph-mon[33128]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:28.190 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-24T23:06:28.190 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-24T23:06:28.190 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000285608 s, 1.8 MB/s 2024-04-24T23:06:28.191 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T23:06:28.252 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-24T23:06:28.310 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-24 23:06:13.650867741 +0000 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-24 23:00:22.176599391 +0000 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-24 23:00:22.176599391 +0000 2024-04-24T23:06:28.311 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-24 23:00:22.176599391 +0000 2024-04-24T23:06:28.311 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T23:06:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-24T23:06:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-24T23:06:28.379 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000220946 s, 2.3 MB/s 2024-04-24T23:06:28.380 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T23:06:28.440 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2024-04-24T23:06:28.498 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T23:06:28.499 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:28.499 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-24T23:06:28.499 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:28.499 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:28.499 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-24 23:06:13.650867741 +0000 2024-04-24T23:06:28.499 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-24 23:00:22.537594897 +0000 2024-04-24T23:06:28.499 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-24 23:00:22.537594897 +0000 2024-04-24T23:06:28.499 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-24 23:00:22.537594897 +0000 2024-04-24T23:06:28.499 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T23:06:28.565 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-24T23:06:28.566 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-24T23:06:28.566 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000252617 s, 2.0 MB/s 2024-04-24T23:06:28.570 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T23:06:28.628 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2024-04-24T23:06:28.685 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T23:06:28.685 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T23:06:28.685 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-24T23:06:28.685 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:06:28.685 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T23:06:28.686 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2024-04-24 23:06:13.651867729 +0000 2024-04-24T23:06:28.686 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-24 23:00:22.898590402 +0000 2024-04-24T23:06:28.686 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-24 23:00:22.898590402 +0000 2024-04-24T23:06:28.686 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: 2024-04-24 23:00:22.898590402 +0000 2024-04-24T23:06:28.686 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2024-04-24T23:06:28.751 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000209921 s, 2.4 MB/s 2024-04-24T23:06:28.752 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T23:06:28.810 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-04-24T23:06:28.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:06:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:29 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:29 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:29 smithi040 ceph-mon[33128]: Reconfiguring alertmanager.smithi031 (dependencies changed)... 2024-04-24T23:06:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:29 smithi040 ceph-mon[33128]: Reconfiguring daemon alertmanager.smithi031 on smithi031 2024-04-24T23:06:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:29 smithi040 ceph-mon[33128]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:29.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:29 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:29.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:29 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:29.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:29 smithi031 ceph-mon[26411]: Reconfiguring alertmanager.smithi031 (dependencies changed)... 2024-04-24T23:06:29.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:29 smithi031 ceph-mon[26411]: Reconfiguring daemon alertmanager.smithi031 on smithi031 2024-04-24T23:06:29.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:29 smithi031 ceph-mon[26411]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:30.015 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:06:32.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:32 smithi040 ceph-mon[33128]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:32.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:32 smithi031 ceph-mon[26411]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:33.998 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:06:33.999 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:06:33.999 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-24T23:06:33.999 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-24T23:06:33.999 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130659 s, 803 MB/s 2024-04-24T23:06:33.999 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-24T23:06:34.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:34 smithi031 ceph-mon[26411]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:34.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:34 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:34.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:34 smithi040 ceph-mon[33128]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:34.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:34 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:35.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-04-24T23:06:35.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:35.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:35.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:35 smithi031 ceph-mon[26411]: Reconfiguring grafana.smithi031 (dependencies changed)... 2024-04-24T23:06:35.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:35 smithi031 ceph-mon[26411]: Reconfiguring daemon grafana.smithi031 on smithi031 2024-04-24T23:06:35.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:35 smithi031 ceph-mon[26411]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:35 smithi040 ceph-mon[33128]: Reconfiguring grafana.smithi031 (dependencies changed)... 2024-04-24T23:06:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:35 smithi040 ceph-mon[33128]: Reconfiguring daemon grafana.smithi031 on smithi031 2024-04-24T23:06:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:35 smithi040 ceph-mon[33128]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:36.166 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:06:38.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:38 smithi031 ceph-mon[26411]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:38 smithi040 ceph-mon[33128]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.220+0000 7f53b909f700 1 -- 172.21.15.31:0/3206435539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fcf60 msgr2=0x7f53b40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.220+0000 7f53b909f700 1 --2- 172.21.15.31:0/3206435539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fcf60 0x7f53b40fd3d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f53a4004940 tx=0x7f53a4009e30 comp rx=0 tx=0).stop 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.220+0000 7f53b909f700 1 -- 172.21.15.31:0/3206435539 shutdown_connections 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.220+0000 7f53b909f700 1 --2- 172.21.15.31:0/3206435539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fcf60 0x7f53b40fd3d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.220+0000 7f53b909f700 1 --2- 172.21.15.31:0/3206435539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b40fbcb0 0x7f53b40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.220+0000 7f53b909f700 1 -- 172.21.15.31:0/3206435539 >> 172.21.15.31:0/3206435539 conn(0x7f53b40f7430 msgr2=0x7f53b40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.220+0000 7f53b909f700 1 -- 172.21.15.31:0/3206435539 shutdown_connections 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.220+0000 7f53b909f700 1 -- 172.21.15.31:0/3206435539 wait complete. 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b909f700 1 Processor -- start 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b909f700 1 -- start start 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b909f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fbcb0 0x7f53b4110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b909f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b40fcf60 0x7f53b4112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b909f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4066330 con 0x7f53b40fbcb0 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b909f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b4066470 con 0x7f53b40fcf60 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fbcb0 0x7f53b4110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b2d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fbcb0 0x7f53b4110750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45804/0 (socket says 172.21.15.31:45804) 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b2d9d700 1 -- 172.21.15.31:0/2645788951 learned_addr learned my addr 172.21.15.31:0/2645788951 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.221+0000 7f53b259c700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b40fcf60 0x7f53b4112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53b2d9d700 1 -- 172.21.15.31:0/2645788951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b40fcf60 msgr2=0x7f53b4112ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53b2d9d700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b40fcf60 0x7f53b4112ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53b2d9d700 1 -- 172.21.15.31:0/2645788951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a4005040 con 0x7f53b40fbcb0 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53b2d9d700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fbcb0 0x7f53b4110750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f539c00c830 tx=0x7f539c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53ab7fe700 1 -- 172.21.15.31:0/2645788951 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f539c0106f0 con 0x7f53b40fbcb0 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53ab7fe700 1 -- 172.21.15.31:0/2645788951 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f539c010d20 con 0x7f53b40fbcb0 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b4113240 con 0x7f53b40fbcb0 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53ab7fe700 1 -- 172.21.15.31:0/2645788951 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f539c0113a0 con 0x7f53b40fbcb0 2024-04-24T23:06:39.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.222+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b4113800 con 0x7f53b40fbcb0 2024-04-24T23:06:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.223+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b405f5c0 con 0x7f53b40fbcb0 2024-04-24T23:06:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.223+0000 7f53ab7fe700 1 -- 172.21.15.31:0/2645788951 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f539c012400 con 0x7f53b40fbcb0 2024-04-24T23:06:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.224+0000 7f53ab7fe700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f53a0069210 0x7f53a006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:06:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.224+0000 7f53ab7fe700 1 -- 172.21.15.31:0/2645788951 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f539c085f50 con 0x7f53b40fbcb0 2024-04-24T23:06:39.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.224+0000 7f53b259c700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f53a0069210 0x7f53a006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:06:39.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.224+0000 7f53b259c700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f53a0069210 0x7f53a006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53a400b970 tx=0x7f53a4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:06:39.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.228+0000 7f53ab7fe700 1 -- 172.21.15.31:0/2645788951 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f539c053400 con 0x7f53b40fbcb0 2024-04-24T23:06:39.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:06:39.476+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f53b41012c0 con 0x7f53a0069210 2024-04-24T23:06:40.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:40 smithi031 ceph-mon[26411]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:40.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:40.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:40.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:40.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:40 smithi040 ceph-mon[33128]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:40.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:06:40.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:06:40.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:41.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:41 smithi031 ceph-mon[26411]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:41.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:41 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:41.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:41 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:41.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:41 smithi031 ceph-mon[26411]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-24T23:06:41.157 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:41 smithi031 ceph-mon[26411]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-24T23:06:41.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:41 smithi040 ceph-mon[33128]: from='client.14242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:06:41.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:41 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:41.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:41 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:41.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:41 smithi040 ceph-mon[33128]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-24T23:06:41.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:41 smithi040 ceph-mon[33128]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-24T23:06:42.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:42 smithi031 ceph-mon[26411]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:42.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:42 smithi040 ceph-mon[33128]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:44.298 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:44 smithi031 ceph-mon[26411]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:44.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:44 smithi040 ceph-mon[33128]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:46.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:46 smithi040 ceph-mon[33128]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:46.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:46 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:46.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:46 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:46.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:46 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:06:46.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:46 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:46.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:46 smithi031 ceph-mon[26411]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:46.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:46 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:46.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:46 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:46.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:46 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:06:46.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:46 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:47.360 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: Reconfiguring crash.smithi040 (monmap changed)... 2024-04-24T23:06:47.361 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: Reconfiguring daemon crash.smithi040 on smithi040 2024-04-24T23:06:47.361 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2204518501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1ff679-6b2e-4059-a299-2d2356bbe91c"}]: dispatch 2024-04-24T23:06:47.361 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2204518501' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f1ff679-6b2e-4059-a299-2d2356bbe91c"}]': finished 2024-04-24T23:06:47.361 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:06:47.361 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:06:47.361 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:47.362 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:47.362 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:47.362 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:47.362 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: Reconfiguring crash.smithi040 (monmap changed)... 2024-04-24T23:06:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: Reconfiguring daemon crash.smithi040 on smithi040 2024-04-24T23:06:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2204518501' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f1ff679-6b2e-4059-a299-2d2356bbe91c"}]: dispatch 2024-04-24T23:06:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2204518501' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f1ff679-6b2e-4059-a299-2d2356bbe91c"}]': finished 2024-04-24T23:06:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: osdmap e6: 1 total, 0 up, 1 in 2024-04-24T23:06:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:06:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:47.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:06:47.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:06:47.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:48 smithi040 ceph-mon[33128]: Reconfiguring mgr.smithi040.qtdwuq (monmap changed)... 2024-04-24T23:06:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:48 smithi040 ceph-mon[33128]: Reconfiguring daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:06:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:48 smithi040 ceph-mon[33128]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:48 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/820205273' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:48.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:48 smithi031 ceph-mon[26411]: Reconfiguring mgr.smithi040.qtdwuq (monmap changed)... 2024-04-24T23:06:48.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:48 smithi031 ceph-mon[26411]: Reconfiguring daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:06:48.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:48 smithi031 ceph-mon[26411]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:48.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:48 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/820205273' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:06:49.348 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:49.348 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:49.348 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:49 smithi040 ceph-mon[33128]: Reconfiguring mon.smithi040 (monmap changed)... 2024-04-24T23:06:49.348 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:49.348 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:49.348 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:49.348 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:49 smithi031 ceph-mon[26411]: Reconfiguring mon.smithi040 (monmap changed)... 2024-04-24T23:06:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:06:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:06:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:06:50.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: Reconfiguring daemon mon.smithi040 on smithi040 2024-04-24T23:06:50.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:50.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.402 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.403 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: Reconfiguring daemon mon.smithi040 on smithi040 2024-04-24T23:06:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:50.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:51.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:51 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:06:51.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:51 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-24T23:06:51.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:51 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:06:51.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:51 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-24T23:06:51.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:51 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:06:51.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:51 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-24T23:06:51.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:51 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:06:51.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:51 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-24T23:06:51.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:51 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:06:51.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:51 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-24T23:06:51.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:51 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:06:51.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:51 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-24T23:06:52.653 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:52 smithi031 ceph-mon[26411]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:52.653 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:52 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:52.653 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:52 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:52.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:52 smithi040 ceph-mon[33128]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:52.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:52 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:52.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:52 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:54.413 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:54 smithi031 ceph-mon[26411]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:54 smithi040 ceph-mon[33128]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:55.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:55 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:06:55.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:55 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:55.563 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:55 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:06:55.564 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:55 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:56.516 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:56 smithi031 ceph-mon[26411]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:56.516 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:56 smithi031 ceph-mon[26411]: Deploying daemon osd.0 on smithi031 2024-04-24T23:06:56.516 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:56.516 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:56.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:56 smithi040 ceph-mon[33128]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:56.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:56 smithi040 ceph-mon[33128]: Deploying daemon osd.0 on smithi031 2024-04-24T23:06:56.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:56.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:57.549 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:57 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:06:57.549 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:57 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:06:57.549 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:57 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:57.549 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:57 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:06:58.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:58 smithi040 ceph-mon[33128]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:58 smithi031 ceph-mon[26411]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:59.602 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:59.602 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:59.602 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:06:59 smithi031 ceph-mon[26411]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:06:59.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:59.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:06:59.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:06:59 smithi040 ceph-mon[33128]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:02.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:02 smithi031 ceph-mon[26411]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:02.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:02 smithi040 ceph-mon[33128]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:04.382 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:07:04.383 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:04.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:07:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:05 smithi031 ceph-mon[26411]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:05 smithi031 ceph-mon[26411]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T23:07:05.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:05 smithi040 ceph-mon[33128]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:05.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:05 smithi040 ceph-mon[33128]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T23:07:06.361 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-04-24T23:07:06.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.359+0000 7f53ab7fe700 1 -- 172.21.15.31:0/2645788951 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f53b41012c0 con 0x7f53a0069210 2024-04-24T23:07:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f53a0069210 msgr2=0x7f53a006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f53a0069210 0x7f53a006b6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53a400b970 tx=0x7f53a4009e30 comp rx=0 tx=0).stop 2024-04-24T23:07:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fbcb0 msgr2=0x7f53b4110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:06.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fbcb0 0x7f53b4110750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f539c00c830 tx=0x7f539c004470 comp rx=0 tx=0).stop 2024-04-24T23:07:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 shutdown_connections 2024-04-24T23:07:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f53a0069210 0x7f53a006b6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53b40fbcb0 0x7f53b4110750 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 --2- 172.21.15.31:0/2645788951 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f53b40fcf60 0x7f53b4112ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 >> 172.21.15.31:0/2645788951 conn(0x7f53b40f7430 msgr2=0x7f53b40f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 shutdown_connections 2024-04-24T23:07:06.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:06.362+0000 7f53b909f700 1 -- 172.21.15.31:0/2645788951 wait complete. 2024-04-24T23:07:06.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:06 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:06.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:06 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:06.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:06 smithi040 ceph-mon[33128]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:06.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:06 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:06.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:06 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:06.667 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:06.668 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:06.668 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:06 smithi031 ceph-mon[26411]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:06.668 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:06.668 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:06 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:06.954 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.0.service 2024-04-24T23:07:06.956 INFO:tasks.cephadm:Deploying osd.1 on smithi031 with /dev/vg_nvme/lv_3... 2024-04-24T23:07:06.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:07:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:07 smithi031 ceph-mon[26411]: from='osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:07:07.749 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:07:07 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[46025]: 2024-04-24T23:07:07.248+0000 7f461afee540 -1 osd.0 0 log_to_monitors true 2024-04-24T23:07:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:07 smithi040 ceph-mon[33128]: from='osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:07:08.290 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:07:08.678 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:08 smithi031 ceph-mon[26411]: from='osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:07:08.678 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:08 smithi031 ceph-mon[26411]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:07:08.679 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:08 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:08.679 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:08 smithi031 ceph-mon[26411]: from='osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:07:08.679 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:08 smithi031 ceph-mon[26411]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:08.681 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:07:08 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[46025]: 2024-04-24T23:07:08.479+0000 7f46117c0700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T23:07:08.682 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:07:08 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[46025]: 2024-04-24T23:07:08.487+0000 7f460a5b7700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:08.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:08 smithi040 ceph-mon[33128]: from='osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:07:08.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:08 smithi040 ceph-mon[33128]: osdmap e7: 1 total, 0 up, 1 in 2024-04-24T23:07:08.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:08 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:08.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:08 smithi040 ceph-mon[33128]: from='osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:07:08.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:08 smithi040 ceph-mon[33128]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:09 smithi031 ceph-mon[26411]: from='osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-24T23:07:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:09 smithi031 ceph-mon[26411]: osdmap e8: 1 total, 0 up, 1 in 2024-04-24T23:07:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:09.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:09.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:09 smithi040 ceph-mon[33128]: from='osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-24T23:07:09.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:09 smithi040 ceph-mon[33128]: osdmap e8: 1 total, 0 up, 1 in 2024-04-24T23:07:09.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:09 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:09.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:09 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:10 smithi031 ceph-mon[26411]: purged_snaps scrub starts 2024-04-24T23:07:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:10 smithi031 ceph-mon[26411]: purged_snaps scrub ok 2024-04-24T23:07:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:10 smithi031 ceph-mon[26411]: osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] boot 2024-04-24T23:07:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:10 smithi031 ceph-mon[26411]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:07:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:10 smithi031 ceph-mon[26411]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:10.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:10 smithi040 ceph-mon[33128]: purged_snaps scrub starts 2024-04-24T23:07:10.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:10 smithi040 ceph-mon[33128]: purged_snaps scrub ok 2024-04-24T23:07:10.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:10.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:10 smithi040 ceph-mon[33128]: osd.0 [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] boot 2024-04-24T23:07:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:10 smithi040 ceph-mon[33128]: osdmap e9: 1 total, 1 up, 1 in 2024-04-24T23:07:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:07:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:10 smithi040 ceph-mon[33128]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-24T23:07:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:07:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:07:11.329 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-24T23:07:11.329 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-24T23:07:11.329 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127485 s, 823 MB/s 2024-04-24T23:07:11.330 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-24T23:07:11.734 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:11 smithi031 ceph-mon[26411]: osdmap e10: 1 total, 1 up, 1 in 2024-04-24T23:07:11.735 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:11.735 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:11.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:11 smithi040 ceph-mon[33128]: osdmap e10: 1 total, 1 up, 1 in 2024-04-24T23:07:11.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:11.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:11.969 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-04-24T23:07:12.694 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:07:12.806 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:12 smithi031 ceph-mon[26411]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:12.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:12 smithi040 ceph-mon[33128]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:14.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:14 smithi031 ceph-mon[26411]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:14.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:14 smithi040 ceph-mon[33128]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:15.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.182+0000 7f6292846700 1 -- 172.21.15.31:0/919787695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 msgr2=0x7f628c0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:15.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.182+0000 7f6292846700 1 --2- 172.21.15.31:0/919787695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 0x7f628c0ffa30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f627c004940 tx=0x7f627c009e30 comp rx=0 tx=0).stop 2024-04-24T23:07:15.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.182+0000 7f6292846700 1 -- 172.21.15.31:0/919787695 shutdown_connections 2024-04-24T23:07:15.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.182+0000 7f6292846700 1 --2- 172.21.15.31:0/919787695 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c0fff70 0x7f628c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:15.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.182+0000 7f6292846700 1 --2- 172.21.15.31:0/919787695 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 0x7f628c0ffa30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:15.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.182+0000 7f6292846700 1 -- 172.21.15.31:0/919787695 >> 172.21.15.31:0/919787695 conn(0x7f628c0f74c0 msgr2=0x7f628c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:15.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.182+0000 7f6292846700 1 -- 172.21.15.31:0/919787695 shutdown_connections 2024-04-24T23:07:15.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.182+0000 7f6292846700 1 -- 172.21.15.31:0/919787695 wait complete. 2024-04-24T23:07:15.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f6292846700 1 Processor -- start 2024-04-24T23:07:15.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f6292846700 1 -- start start 2024-04-24T23:07:15.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f6292846700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 0x7f628c10fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:15.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f6292846700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c0fff70 0x7f628c110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:15.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f6292846700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c1107f0 con 0x7f628c066b50 2024-04-24T23:07:15.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f6292846700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c110fe0 con 0x7f628c0fff70 2024-04-24T23:07:15.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f628bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 0x7f628c10fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:15.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f628bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 0x7f628c10fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:37438/0 (socket says 172.21.15.31:37438) 2024-04-24T23:07:15.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f628bfff700 1 -- 172.21.15.31:0/258323630 learned_addr learned my addr 172.21.15.31:0/258323630 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:07:15.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f628b7fe700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c0fff70 0x7f628c110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:15.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.183+0000 7f628bfff700 1 -- 172.21.15.31:0/258323630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c0fff70 msgr2=0x7f628c110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:15.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.184+0000 7f628bfff700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c0fff70 0x7f628c110160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:15.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.184+0000 7f628bfff700 1 -- 172.21.15.31:0/258323630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c005040 con 0x7f628c066b50 2024-04-24T23:07:15.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.184+0000 7f628bfff700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 0x7f628c10fc20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f627c00b3b0 tx=0x7f627c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:15.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.184+0000 7f6288ff9700 1 -- 172.21.15.31:0/258323630 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627c00e040 con 0x7f628c066b50 2024-04-24T23:07:15.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.184+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c111260 con 0x7f628c066b50 2024-04-24T23:07:15.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.184+0000 7f6288ff9700 1 -- 172.21.15.31:0/258323630 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f627c00bbf0 con 0x7f628c066b50 2024-04-24T23:07:15.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.184+0000 7f6288ff9700 1 -- 172.21.15.31:0/258323630 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627c01c630 con 0x7f628c066b50 2024-04-24T23:07:15.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.184+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c111820 con 0x7f628c066b50 2024-04-24T23:07:15.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.185+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c05f5c0 con 0x7f628c066b50 2024-04-24T23:07:15.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.186+0000 7f6288ff9700 1 -- 172.21.15.31:0/258323630 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f627c00bd60 con 0x7f628c066b50 2024-04-24T23:07:15.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.186+0000 7f6288ff9700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6278069210 0x7f627806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:15.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.187+0000 7f6288ff9700 1 -- 172.21.15.31:0/258323630 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f627c087680 con 0x7f628c066b50 2024-04-24T23:07:15.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.187+0000 7f628b7fe700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6278069210 0x7f627806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:15.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.187+0000 7f628b7fe700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6278069210 0x7f627806b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6274009bf0 tx=0x7f62740091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:15.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.190+0000 7f6288ff9700 1 -- 172.21.15.31:0/258323630 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f627c058360 con 0x7f628c066b50 2024-04-24T23:07:15.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:15.420+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f628c111b70 con 0x7f6278069210 2024-04-24T23:07:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: Detected new or changed devices on smithi031 2024-04-24T23:07:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: Adjusting osd_memory_target on smithi031 to 12869M 2024-04-24T23:07:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:16.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:16.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:16.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:16.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:16.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:16.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:16.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:15 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: Detected new or changed devices on smithi031 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: Adjusting osd_memory_target on smithi031 to 12869M 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:16.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:16.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:15 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:17.033 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:16 smithi031 ceph-mon[26411]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:17.034 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:16 smithi031 ceph-mon[26411]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T23:07:17.034 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:16 smithi031 ceph-mon[26411]: Cluster is now healthy 2024-04-24T23:07:17.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:16 smithi040 ceph-mon[33128]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:17.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:16 smithi040 ceph-mon[33128]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T23:07:17.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:16 smithi040 ceph-mon[33128]: Cluster is now healthy 2024-04-24T23:07:17.904 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:17 smithi031 ceph-mon[26411]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:18.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:17 smithi040 ceph-mon[33128]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:19.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:19 smithi031 ceph-mon[26411]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:19.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:19 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2140505088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6885f0d2-70e8-4c54-a381-0b3683a0bc49"}]: dispatch 2024-04-24T23:07:20.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:19 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2140505088' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6885f0d2-70e8-4c54-a381-0b3683a0bc49"}]': finished 2024-04-24T23:07:20.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:19 smithi031 ceph-mon[26411]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:07:20.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:20.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:07:20.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:19 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1603629249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:20.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:19 smithi040 ceph-mon[33128]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:20.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:19 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2140505088' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6885f0d2-70e8-4c54-a381-0b3683a0bc49"}]: dispatch 2024-04-24T23:07:20.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:19 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2140505088' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6885f0d2-70e8-4c54-a381-0b3683a0bc49"}]': finished 2024-04-24T23:07:20.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:19 smithi040 ceph-mon[33128]: osdmap e11: 2 total, 1 up, 2 in 2024-04-24T23:07:20.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:20.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:07:20.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:19 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1603629249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:21.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:21.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:21.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:21 smithi040 ceph-mon[33128]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:21 smithi031 ceph-mon[26411]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:23 smithi031 ceph-mon[26411]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:24.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:23 smithi040 ceph-mon[33128]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:25.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:25 smithi031 ceph-mon[26411]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:26.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:25 smithi040 ceph-mon[33128]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:26.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:26 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:07:26.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:26 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:26.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:26 smithi031 ceph-mon[26411]: Deploying daemon osd.1 on smithi031 2024-04-24T23:07:27.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:26 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:07:27.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:26 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:27.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:26 smithi040 ceph-mon[33128]: Deploying daemon osd.1 on smithi031 2024-04-24T23:07:27.977 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:27 smithi031 ceph-mon[26411]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:28.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:27 smithi040 ceph-mon[33128]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:30.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:29 smithi040 ceph-mon[33128]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:30.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:29 smithi031 ceph-mon[26411]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:32.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:31 smithi031 ceph-mon[26411]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:32.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:31 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:32.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:31 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:32.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:31 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:31 smithi040 ceph-mon[33128]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:31 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:31 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:31 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:34.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:33 smithi040 ceph-mon[33128]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:34.187 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:33 smithi031 ceph-mon[26411]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:34.990 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:34 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:07:35.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.045+0000 7f6288ff9700 1 -- 172.21.15.31:0/258323630 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f628c111b70 con 0x7f6278069210 2024-04-24T23:07:35.047 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 1 on host 'smithi031' 2024-04-24T23:07:35.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6278069210 msgr2=0x7f627806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6278069210 0x7f627806b6c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6274009bf0 tx=0x7f62740091a0 comp rx=0 tx=0).stop 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 msgr2=0x7f628c10fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 0x7f628c10fc20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f627c00b3b0 tx=0x7f627c009e30 comp rx=0 tx=0).stop 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 shutdown_connections 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6278069210 0x7f627806b6c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c066b50 0x7f628c10fc20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 --2- 172.21.15.31:0/258323630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c0fff70 0x7f628c110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 >> 172.21.15.31:0/258323630 conn(0x7f628c0f74c0 msgr2=0x7f628c0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 shutdown_connections 2024-04-24T23:07:35.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:35.047+0000 7f6292846700 1 -- 172.21.15.31:0/258323630 wait complete. 2024-04-24T23:07:35.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:34 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:07:35.989 DEBUG:teuthology.orchestra.run.smithi031:osd.1> sudo journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.1.service 2024-04-24T23:07:35.991 INFO:tasks.cephadm:Deploying osd.2 on smithi031 with /dev/vg_nvme/lv_2... 2024-04-24T23:07:35.991 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:07:36.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:36 smithi031 ceph-mon[26411]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:36.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:36 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:36.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:36 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:36.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:36 smithi031 ceph-mon[26411]: from='osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:07:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:36 smithi040 ceph-mon[33128]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:36 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:36 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:36 smithi040 ceph-mon[33128]: from='osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:07:36.735 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:07:37.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:37 smithi031 ceph-mon[26411]: from='osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:07:37.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:37 smithi031 ceph-mon[26411]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:07:37.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:37 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:37.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:37 smithi031 ceph-mon[26411]: from='osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:07:37.249 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:07:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[53346]: 2024-04-24T23:07:37.033+0000 7f51e2544700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T23:07:37.249 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:07:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[53346]: 2024-04-24T23:07:37.037+0000 7f51de341700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:07:37.285 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:37 smithi040 ceph-mon[33128]: from='osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:07:37.285 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:37 smithi040 ceph-mon[33128]: osdmap e12: 2 total, 1 up, 2 in 2024-04-24T23:07:37.285 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:37 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:37.285 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:37 smithi040 ceph-mon[33128]: from='osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:07:38.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:38 smithi031 ceph-mon[26411]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:38.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:38 smithi031 ceph-mon[26411]: from='osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-24T23:07:38.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:38 smithi031 ceph-mon[26411]: osdmap e13: 2 total, 1 up, 2 in 2024-04-24T23:07:38.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:38 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:38.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:38 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:38 smithi040 ceph-mon[33128]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:38 smithi040 ceph-mon[33128]: from='osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-24T23:07:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:38 smithi040 ceph-mon[33128]: osdmap e13: 2 total, 1 up, 2 in 2024-04-24T23:07:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:38 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:38.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:38 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:39.294 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:39 smithi031 ceph-mon[26411]: purged_snaps scrub starts 2024-04-24T23:07:39.294 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:39 smithi031 ceph-mon[26411]: purged_snaps scrub ok 2024-04-24T23:07:39.294 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:39 smithi031 ceph-mon[26411]: osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] boot 2024-04-24T23:07:39.294 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:39 smithi031 ceph-mon[26411]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:07:39.294 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:39 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:39.295 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:39 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:39.295 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:39 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:39.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:39 smithi040 ceph-mon[33128]: purged_snaps scrub starts 2024-04-24T23:07:39.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:39 smithi040 ceph-mon[33128]: purged_snaps scrub ok 2024-04-24T23:07:39.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:39 smithi040 ceph-mon[33128]: osd.1 [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] boot 2024-04-24T23:07:39.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:39 smithi040 ceph-mon[33128]: osdmap e14: 2 total, 2 up, 2 in 2024-04-24T23:07:39.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:39 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:07:39.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:39 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:39.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:39 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:07:39.766 INFO:teuthology.orchestra.run.smithi031.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:07:39.767 INFO:teuthology.orchestra.run.smithi031.stderr: stderr: 10+0 records in 2024-04-24T23:07:39.767 INFO:teuthology.orchestra.run.smithi031.stderr:10+0 records out 2024-04-24T23:07:39.767 INFO:teuthology.orchestra.run.smithi031.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122242 s, 858 MB/s 2024-04-24T23:07:39.767 INFO:teuthology.orchestra.run.smithi031.stderr:--> Zapping successful for: 2024-04-24T23:07:40.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:40 smithi031 ceph-mon[26411]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:40.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:40 smithi031 ceph-mon[26411]: osdmap e15: 2 total, 2 up, 2 in 2024-04-24T23:07:40.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:40 smithi040 ceph-mon[33128]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-24T23:07:40.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:40 smithi040 ceph-mon[33128]: osdmap e15: 2 total, 2 up, 2 in 2024-04-24T23:07:41.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-04-24T23:07:42.191 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:42 smithi031 ceph-mon[26411]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:42.373 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:07:42.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:42 smithi040 ceph-mon[33128]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:43.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: Detected new or changed devices on smithi031 2024-04-24T23:07:43.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:43.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:43.569 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: Adjusting osd_memory_target on smithi031 to 6434M 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.570 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:43 smithi031 ceph-mon[26411]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: Detected new or changed devices on smithi031 2024-04-24T23:07:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: Adjusting osd_memory_target on smithi031 to 6434M 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:43 smithi040 ceph-mon[33128]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.795+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/1670247895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 msgr2=0x7f1d740fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.795+0000 7f1d7b0e7700 1 --2- 172.21.15.31:0/1670247895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 0x7f1d740fd270 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d68004940 tx=0x7f1d68009e30 comp rx=0 tx=0).stop 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.795+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/1670247895 shutdown_connections 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.795+0000 7f1d7b0e7700 1 --2- 172.21.15.31:0/1670247895 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d740fdfd0 0x7f1d740fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.795+0000 7f1d7b0e7700 1 --2- 172.21.15.31:0/1670247895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 0x7f1d740fd270 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.795+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/1670247895 >> 172.21.15.31:0/1670247895 conn(0x7f1d740f8540 msgr2=0x7f1d740fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.795+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/1670247895 shutdown_connections 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.795+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/1670247895 wait complete. 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d7b0e7700 1 Processor -- start 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d7b0e7700 1 -- start start 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d7b0e7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 0x7f1d7410d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d7b0e7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d740fdfd0 0x7f1d7410de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d7b0e7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7410e500 con 0x7f1d740fce60 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d7b0e7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d74112ea0 con 0x7f1d740fdfd0 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d78e83700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 0x7f1d7410d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d78e83700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 0x7f1d7410d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42732/0 (socket says 172.21.15.31:42732) 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d78e83700 1 -- 172.21.15.31:0/759993538 learned_addr learned my addr 172.21.15.31:0/759993538 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d73fff700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d740fdfd0 0x7f1d7410de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d78e83700 1 -- 172.21.15.31:0/759993538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d740fdfd0 msgr2=0x7f1d7410de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d78e83700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d740fdfd0 0x7f1d7410de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.796+0000 7f1d78e83700 1 -- 172.21.15.31:0/759993538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68005040 con 0x7f1d740fce60 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.797+0000 7f1d78e83700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 0x7f1d7410d930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b3b0 tx=0x7f1d68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.797+0000 7f1d717fa700 1 -- 172.21.15.31:0/759993538 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d6800e040 con 0x7f1d740fce60 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.797+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d741130a0 con 0x7f1d740fce60 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.797+0000 7f1d717fa700 1 -- 172.21.15.31:0/759993538 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d6800bcc0 con 0x7f1d740fce60 2024-04-24T23:07:43.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.797+0000 7f1d717fa700 1 -- 172.21.15.31:0/759993538 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d6801c630 con 0x7f1d740fce60 2024-04-24T23:07:43.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.797+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d74113610 con 0x7f1d740fce60 2024-04-24T23:07:43.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.798+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7404c710 con 0x7f1d740fce60 2024-04-24T23:07:43.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.798+0000 7f1d717fa700 1 -- 172.21.15.31:0/759993538 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f1d68013980 con 0x7f1d740fce60 2024-04-24T23:07:43.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.799+0000 7f1d717fa700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f1d64069210 0x7f1d6406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.799+0000 7f1d717fa700 1 -- 172.21.15.31:0/759993538 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f1d680872b0 con 0x7f1d740fce60 2024-04-24T23:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.799+0000 7f1d73fff700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f1d64069210 0x7f1d6406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.799+0000 7f1d73fff700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f1d64069210 0x7f1d6406b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1d60006fd0 tx=0x7f1d60008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:07:43.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:43.803+0000 7f1d717fa700 1 -- 172.21.15.31:0/759993538 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1d68057cc0 con 0x7f1d740fce60 2024-04-24T23:07:44.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:07:44.005+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d741022b0 con 0x7f1d64069210 2024-04-24T23:07:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:44 smithi031 ceph-mon[26411]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:44 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:44 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:44 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:44 smithi040 ceph-mon[33128]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:07:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:44 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:07:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:44 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:07:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:44 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:45.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:45 smithi031 ceph-mon[26411]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:45.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:45 smithi040 ceph-mon[33128]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:46 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3271254290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b57c743-5826-4bf9-ac00-0cb539b30224"}]: dispatch 2024-04-24T23:07:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:46 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3271254290' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b57c743-5826-4bf9-ac00-0cb539b30224"}]': finished 2024-04-24T23:07:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:46 smithi040 ceph-mon[33128]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:07:46.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:46 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:46.971 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:46 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3271254290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b57c743-5826-4bf9-ac00-0cb539b30224"}]: dispatch 2024-04-24T23:07:46.971 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:46 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3271254290' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b57c743-5826-4bf9-ac00-0cb539b30224"}]': finished 2024-04-24T23:07:46.971 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:46 smithi031 ceph-mon[26411]: osdmap e16: 3 total, 2 up, 3 in 2024-04-24T23:07:46.971 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:46 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:07:47.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:47 smithi031 ceph-mon[26411]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:47.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:47 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/4078323376' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:47 smithi040 ceph-mon[33128]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:47 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/4078323376' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:07:50.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:49 smithi040 ceph-mon[33128]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:50.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:07:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:49 smithi031 ceph-mon[26411]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:07:52.114 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:51 smithi031 ceph-mon[26411]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:52.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:51 smithi040 ceph-mon[33128]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:53.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:53 smithi031 ceph-mon[26411]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:53.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:53 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:07:53.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:53 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:54.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:53 smithi040 ceph-mon[33128]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:54.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:53 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:07:54.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:53 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:07:55.141 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:54 smithi031 ceph-mon[26411]: Deploying daemon osd.2 on smithi031 2024-04-24T23:07:55.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:54 smithi040 ceph-mon[33128]: Deploying daemon osd.2 on smithi031 2024-04-24T23:07:55.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:55 smithi031 ceph-mon[26411]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:56.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:55 smithi040 ceph-mon[33128]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:57.983 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:57 smithi031 ceph-mon[26411]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:57 smithi040 ceph-mon[33128]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:07:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:59 smithi031 ceph-mon[26411]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:07:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:07:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:00.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:00.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:00.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:59 smithi040 ceph-mon[33128]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:08:00.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:07:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:02.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.009+0000 7f1d717fa700 1 -- 172.21.15.31:0/759993538 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1d741022b0 con 0x7f1d64069210 2024-04-24T23:08:02.011 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 2 on host 'smithi031' 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f1d64069210 msgr2=0x7f1d6406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f1d64069210 0x7f1d6406b6c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1d60006fd0 tx=0x7f1d60008040 comp rx=0 tx=0).stop 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 msgr2=0x7f1d7410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 0x7f1d7410d930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b3b0 tx=0x7f1d68009e30 comp rx=0 tx=0).stop 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 shutdown_connections 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f1d64069210 0x7f1d6406b6c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d740fce60 0x7f1d7410d930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 --2- 172.21.15.31:0/759993538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1d740fdfd0 0x7f1d7410de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:02.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 >> 172.21.15.31:0/759993538 conn(0x7f1d740f8540 msgr2=0x7f1d740f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:02.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 shutdown_connections 2024-04-24T23:08:02.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:08:02.012+0000 7f1d7b0e7700 1 -- 172.21.15.31:0/759993538 wait complete. 2024-04-24T23:08:02.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:01 smithi040 ceph-mon[33128]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:08:02.234 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:01 smithi031 ceph-mon[26411]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:08:02.963 DEBUG:teuthology.orchestra.run.smithi031:osd.2> sudo journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.2.service 2024-04-24T23:08:02.966 INFO:tasks.cephadm:Deploying osd.3 on smithi040 with /dev/vg_nvme/lv_4... 2024-04-24T23:08:02.966 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-24T23:08:02.993 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:02 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:02.993 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:02 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:02.993 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:02 smithi040 ceph-mon[33128]: from='osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:08:03.188 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:03.188 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:03.188 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:02 smithi031 ceph-mon[26411]: from='osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:08:03.235 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi040/config 2024-04-24T23:08:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:03 smithi031 ceph-mon[26411]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:08:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:03 smithi031 ceph-mon[26411]: from='osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:08:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:03 smithi031 ceph-mon[26411]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:08:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:03 smithi031 ceph-mon[26411]: from='osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:08:04.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:03 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[60435]: 2024-04-24T23:08:03.985+0000 7f98742f1700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T23:08:04.250 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:03 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[60435]: 2024-04-24T23:08:03.992+0000 7f986e8eb700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:04.359 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:03 smithi040 ceph-mon[33128]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:08:04.359 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:03 smithi040 ceph-mon[33128]: from='osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:08:04.359 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:03 smithi040 ceph-mon[33128]: osdmap e17: 3 total, 2 up, 3 in 2024-04-24T23:08:04.359 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:04.359 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:03 smithi040 ceph-mon[33128]: from='osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:08:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-24T23:08:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-24T23:08:05.106 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-24T23:08:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-24T23:08:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112044 s, 936 MB/s 2024-04-24T23:08:05.106 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-24T23:08:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:04 smithi031 ceph-mon[26411]: from='osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-24T23:08:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:04 smithi031 ceph-mon[26411]: osdmap e18: 3 total, 2 up, 3 in 2024-04-24T23:08:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:08:05.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:04 smithi040 ceph-mon[33128]: from='osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-04-24T23:08:05.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:04 smithi040 ceph-mon[33128]: osdmap e18: 3 total, 2 up, 3 in 2024-04-24T23:08:05.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:05.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:05.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:08:05.706 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_4 2024-04-24T23:08:06.013 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi040/config 2024-04-24T23:08:06.247 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: purged_snaps scrub starts 2024-04-24T23:08:06.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: purged_snaps scrub ok 2024-04-24T23:08:06.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:08:06.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:06.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] boot 2024-04-24T23:08:06.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:08:06.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:06.248 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-24T23:08:06.249 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:06.249 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:05 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:06.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: purged_snaps scrub starts 2024-04-24T23:08:06.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: purged_snaps scrub ok 2024-04-24T23:08:06.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: pgmap v75: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-24T23:08:06.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:06.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: osd.2 [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] boot 2024-04-24T23:08:06.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: osdmap e19: 3 total, 3 up, 3 in 2024-04-24T23:08:06.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:08:06.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-24T23:08:06.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:06.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:07.108 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:07 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-24T23:08:07.108 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:07 smithi040 ceph-mon[33128]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:08:07.108 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:07 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:08:07.209 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:07 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-24T23:08:07.210 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:07 smithi031 ceph-mon[26411]: osdmap e20: 3 total, 3 up, 3 in 2024-04-24T23:08:07.210 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:07 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-24T23:08:07.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.582+0000 7f9413a10700 1 -- 172.21.15.40:0/303197450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fbcb0 msgr2=0x7f940c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:07.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.582+0000 7f9413a10700 1 --2- 172.21.15.40:0/303197450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fbcb0 0x7f940c0fc0c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9408004940 tx=0x7f9408009e30 comp rx=0 tx=0).stop 2024-04-24T23:08:07.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.582+0000 7f9413a10700 1 -- 172.21.15.40:0/303197450 shutdown_connections 2024-04-24T23:08:07.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.582+0000 7f9413a10700 1 --2- 172.21.15.40:0/303197450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f940c0fcf60 0x7f940c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:07.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.582+0000 7f9413a10700 1 --2- 172.21.15.40:0/303197450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fbcb0 0x7f940c0fc0c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.582+0000 7f9413a10700 1 -- 172.21.15.40:0/303197450 >> 172.21.15.40:0/303197450 conn(0x7f940c0f7430 msgr2=0x7f940c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9413a10700 1 -- 172.21.15.40:0/303197450 shutdown_connections 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9413a10700 1 -- 172.21.15.40:0/303197450 wait complete. 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9413a10700 1 Processor -- start 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9413a10700 1 -- start start 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9413a10700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f940c0fbcb0 0x7f940c0ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9413a10700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fcf60 0x7f940c0fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9413a10700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0fe7e0 con 0x7f940c0fbcb0 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9413a10700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940c0fe920 con 0x7f940c0fcf60 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9410fab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fcf60 0x7f940c0fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9410fab700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fcf60 0x7f940c0fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:55810/0 (socket says 172.21.15.40:55810) 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f94117ac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f940c0fbcb0 0x7f940c0ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.583+0000 7f9410fab700 1 -- 172.21.15.40:0/2973731932 learned_addr learned my addr 172.21.15.40:0/2973731932 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:08:07.587 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9410fab700 1 -- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f940c0fbcb0 msgr2=0x7f940c0ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9410fab700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f940c0fbcb0 0x7f940c0ffd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9410fab700 1 -- 172.21.15.40:0/2973731932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9408005040 con 0x7f940c0fcf60 2024-04-24T23:08:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9410fab700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fcf60 0x7f940c0fe2a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00eb40 tx=0x7f93fc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9401ffb700 1 -- 172.21.15.40:0/2973731932 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93fc007cc0 con 0x7f940c0fcf60 2024-04-24T23:08:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9401ffb700 1 -- 172.21.15.40:0/2973731932 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93fc00f040 con 0x7f940c0fcf60 2024-04-24T23:08:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f940c0feba0 con 0x7f940c0fcf60 2024-04-24T23:08:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9401ffb700 1 -- 172.21.15.40:0/2973731932 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93fc00ca10 con 0x7f940c0fcf60 2024-04-24T23:08:07.588 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.584+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f940c0ff0e0 con 0x7f940c0fcf60 2024-04-24T23:08:07.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.585+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f940c101af0 con 0x7f940c0fcf60 2024-04-24T23:08:07.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.586+0000 7f9401ffb700 1 -- 172.21.15.40:0/2973731932 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f93fc00cb70 con 0x7f940c0fcf60 2024-04-24T23:08:07.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.586+0000 7f9401ffb700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f93f8069210 0x7f93f806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:07.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.586+0000 7f9401ffb700 1 -- 172.21.15.40:0/2973731932 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7f93fc0862a0 con 0x7f940c0fcf60 2024-04-24T23:08:07.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.586+0000 7f94117ac700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f93f8069210 0x7f93f806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:07.589 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.587+0000 7f94117ac700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f93f8069210 0x7f93f806b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9408004890 tx=0x7f9408009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:07.592 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.591+0000 7f9401ffb700 1 -- 172.21.15.40:0/2973731932 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f93fc04e250 con 0x7f940c0fcf60 2024-04-24T23:08:07.790 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:08:07 smithi031 sudo[62396]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:08:07.790 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:08:07 smithi031 sudo[62396]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:07.790 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:08:07 smithi031 sudo[62396]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:07.790 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:08:07 smithi031 sudo[62396]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:07.793 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:07.791+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f940c05e3a0 con 0x7f93f8069210 2024-04-24T23:08:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:08 smithi040 ceph-mon[33128]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:08 smithi040 ceph-mon[33128]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:08:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:08 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:08:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:08 smithi040 ceph-mon[33128]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:08:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:08 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:08 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:08 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:08 smithi031 ceph-mon[26411]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:08 smithi031 ceph-mon[26411]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-24T23:08:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:08 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-24T23:08:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:08 smithi031 ceph-mon[26411]: osdmap e21: 3 total, 3 up, 3 in 2024-04-24T23:08:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:08 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:08 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:08 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:08.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[62425]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:08:08.250 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[62425]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:08.250 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[62425]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:08.250 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[62425]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:08.675 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[64815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:08:08.675 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[64815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:08.675 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[64815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:08.675 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[64815]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:09.061 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:09.059+0000 7f9401ffb700 1 -- 172.21.15.40:0/2973731932 <== mon.1 v2:172.21.15.40:3300/0 7 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f93fc0533f0 con 0x7f940c0fcf60 2024-04-24T23:08:09.118 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:09 smithi040 ceph-mon[33128]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:09.118 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:09 smithi040 ceph-mon[33128]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T23:08:09.118 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:09 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:09 smithi031 ceph-mon[26411]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:09 smithi031 ceph-mon[26411]: osdmap e22: 3 total, 3 up, 3 in 2024-04-24T23:08:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:09.249 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[64821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:08:09.249 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[64821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:09.249 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[64821]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:09.249 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:08:08 smithi031 sudo[64821]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:09.749 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:09 smithi031 sudo[64888]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:08:09.749 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:09 smithi031 sudo[64888]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:09.749 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:09 smithi031 sudo[64888]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:09.749 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:09 smithi031 sudo[64888]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:10.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:09 smithi031 sudo[64891]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:08:10.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:09 smithi031 sudo[64891]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:10.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:09 smithi031 sudo[64891]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:10.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:08:09 smithi031 sudo[64891]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 sudo[64894]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:08:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 sudo[64894]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 sudo[64894]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 sudo[64894]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:11.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 sudo[64897]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:08:11.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 sudo[64897]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:11.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 sudo[64897]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:11.085 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 sudo[64897]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: Detected new or changed devices on smithi031 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: Adjusting osd_memory_target on smithi031 to 4289M 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:08:11.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: Cluster is now healthy 2024-04-24T23:08:11.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: mgrmap e19: smithi031.rrnmwr(active, since 2m), standbys: smithi040.qtdwuq 2024-04-24T23:08:11.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:11.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:11.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:11.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:10 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: Detected new or changed devices on smithi031 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: Adjusting osd_memory_target on smithi031 to 4289M 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: Cluster is now healthy 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: mgrmap e19: smithi031.rrnmwr(active, since 2m), standbys: smithi040.qtdwuq 2024-04-24T23:08:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:11.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:11.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:11.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:10 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 sudo[36579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-24T23:08:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 sudo[36579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 sudo[36579]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 sudo[36579]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:12.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 sudo[36582]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-24T23:08:12.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 sudo[36582]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 sudo[36582]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 sudo[36582]: pam_unix(sudo:session): session closed for user root 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/356723720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dc47b54-65da-4579-bcac-2258be1f4cf8"}]: dispatch 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dc47b54-65da-4579-bcac-2258be1f4cf8"}]: dispatch 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1dc47b54-65da-4579-bcac-2258be1f4cf8"}]': finished 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:08:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:08:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:08:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:08:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:08:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/356723720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dc47b54-65da-4579-bcac-2258be1f4cf8"}]: dispatch 2024-04-24T23:08:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:08:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1dc47b54-65da-4579-bcac-2258be1f4cf8"}]: dispatch 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1dc47b54-65da-4579-bcac-2258be1f4cf8"}]': finished 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: osdmap e23: 4 total, 3 up, 4 in 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-24T23:08:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:13.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:12 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/2317653605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:13.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:12 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/2317653605' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:14.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:13 smithi040 ceph-mon[33128]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:14.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:13 smithi031 ceph-mon[26411]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:15.885 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:15 smithi040 ceph-mon[33128]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-04-24T23:08:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:15 smithi031 ceph-mon[26411]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-04-24T23:08:18.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:17 smithi040 ceph-mon[33128]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-24T23:08:18.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:17 smithi031 ceph-mon[26411]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-24T23:08:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:19 smithi040 ceph-mon[33128]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-24T23:08:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:08:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:19 smithi040 ceph-mon[33128]: Deploying daemon osd.3 on smithi040 2024-04-24T23:08:19.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:08:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:19 smithi031 ceph-mon[26411]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-24T23:08:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:08:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:19 smithi031 ceph-mon[26411]: Deploying daemon osd.3 on smithi040 2024-04-24T23:08:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:08:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:21 smithi040 ceph-mon[33128]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T23:08:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:21 smithi031 ceph-mon[26411]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-24T23:08:23.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:23 smithi040 ceph-mon[33128]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T23:08:24.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:23 smithi031 ceph-mon[26411]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T23:08:25.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:25 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:25.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:25 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:25.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:25 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:25.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:25 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:25.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:25 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:25.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:25 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:26.442 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:26 smithi040 ceph-mon[33128]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T23:08:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:26 smithi031 ceph-mon[26411]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-24T23:08:27.162 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 3 on host 'smithi040' 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.158+0000 7f9401ffb700 1 -- 172.21.15.40:0/2973731932 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f940c05e3a0 con 0x7f93f8069210 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f93f8069210 msgr2=0x7f93f806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f93f8069210 0x7f93f806b6c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9408004890 tx=0x7f9408009e30 comp rx=0 tx=0).stop 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fcf60 msgr2=0x7f940c0fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fcf60 0x7f940c0fe2a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00eb40 tx=0x7f93fc00b040 comp rx=0 tx=0).stop 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 shutdown_connections 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f93f8069210 0x7f93f806b6c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f940c0fbcb0 0x7f940c0ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 --2- 172.21.15.40:0/2973731932 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f940c0fcf60 0x7f940c0fe2a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 >> 172.21.15.40:0/2973731932 conn(0x7f940c0f7430 msgr2=0x7f940c065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:27.163 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 shutdown_connections 2024-04-24T23:08:27.164 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:27.160+0000 7f9413a10700 1 -- 172.21.15.40:0/2973731932 wait complete. 2024-04-24T23:08:28.197 DEBUG:teuthology.orchestra.run.smithi040:osd.3> sudo journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.3.service 2024-04-24T23:08:28.199 INFO:tasks.cephadm:Deploying osd.4 on smithi040 with /dev/vg_nvme/lv_3... 2024-04-24T23:08:28.199 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-24T23:08:28.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:28 smithi040 ceph-mon[33128]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:28.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:28 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:28.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:28 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:28.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:28 smithi040 ceph-mon[33128]: from='osd.3 [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:08:28.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:28 smithi040 ceph-mon[33128]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:08:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:28 smithi031 ceph-mon[26411]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:28 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:28 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:28 smithi031 ceph-mon[26411]: from='osd.3 [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:08:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:28 smithi031 ceph-mon[26411]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:08:28.818 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi040/config 2024-04-24T23:08:29.196 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:29 smithi040 ceph-mon[33128]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:08:29.196 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:29 smithi040 ceph-mon[33128]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:08:29.196 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:29 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:29.196 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:29 smithi040 ceph-mon[33128]: from='osd.3 [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:08:29.196 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:29 smithi040 ceph-mon[33128]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:08:29.196 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:08:29 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[38653]: 2024-04-24T23:08:29.189+0000 7f0a21c53700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T23:08:29.196 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:08:29 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[38653]: 2024-04-24T23:08:29.194+0000 7f0a1d24f700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:29 smithi031 ceph-mon[26411]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:08:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:29 smithi031 ceph-mon[26411]: osdmap e24: 4 total, 3 up, 4 in 2024-04-24T23:08:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:29 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:29 smithi031 ceph-mon[26411]: from='osd.3 [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:08:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:29 smithi031 ceph-mon[26411]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:08:30.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:30 smithi031 ceph-mon[26411]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:30.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:30 smithi031 ceph-mon[26411]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-24T23:08:30.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:30 smithi031 ceph-mon[26411]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T23:08:30.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:30 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:30.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:30 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:30.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:30 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:30.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:30 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:30.584 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:30 smithi040 ceph-mon[33128]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:30.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:30 smithi040 ceph-mon[33128]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-24T23:08:30.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:30 smithi040 ceph-mon[33128]: osdmap e25: 4 total, 3 up, 4 in 2024-04-24T23:08:30.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:30 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:30.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:30 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:30.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:30 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:30.585 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:30 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-24T23:08:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-24T23:08:30.773 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-24T23:08:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-24T23:08:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120967 s, 867 MB/s 2024-04-24T23:08:30.773 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-24T23:08:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:31 smithi031 ceph-mon[26411]: purged_snaps scrub starts 2024-04-24T23:08:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:31 smithi031 ceph-mon[26411]: purged_snaps scrub ok 2024-04-24T23:08:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:31 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:31 smithi031 ceph-mon[26411]: osd.3 [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] boot 2024-04-24T23:08:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:31 smithi031 ceph-mon[26411]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:08:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:31 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:31 smithi031 ceph-mon[26411]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:08:31.632 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:31 smithi040 ceph-mon[33128]: purged_snaps scrub starts 2024-04-24T23:08:31.632 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:31 smithi040 ceph-mon[33128]: purged_snaps scrub ok 2024-04-24T23:08:31.632 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:31 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:31.632 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:31 smithi040 ceph-mon[33128]: osd.3 [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] boot 2024-04-24T23:08:31.632 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:31 smithi040 ceph-mon[33128]: osdmap e26: 4 total, 4 up, 4 in 2024-04-24T23:08:31.632 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:31 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:08:31.632 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:31 smithi040 ceph-mon[33128]: osdmap e27: 4 total, 4 up, 4 in 2024-04-24T23:08:32.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:32 smithi040 ceph-mon[33128]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:32.497 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:32 smithi040 ceph-mon[33128]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:08:32.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:32 smithi031 ceph-mon[26411]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-24T23:08:32.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:32 smithi031 ceph-mon[26411]: osdmap e28: 4 total, 4 up, 4 in 2024-04-24T23:08:32.526 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_3 2024-04-24T23:08:32.874 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi040/config 2024-04-24T23:08:34.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:34 smithi040 ceph-mon[33128]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:34.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:34 smithi040 ceph-mon[33128]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:08:34.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:34 smithi031 ceph-mon[26411]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:34.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:34 smithi031 ceph-mon[26411]: osdmap e29: 4 total, 4 up, 4 in 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.249+0000 7f5992e19700 1 -- 172.21.15.40:0/186823304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fbcc0 msgr2=0x7f598c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.249+0000 7f5992e19700 1 --2- 172.21.15.40:0/186823304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fbcc0 0x7f598c0fc0d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5980004940 tx=0x7f5980009e30 comp rx=0 tx=0).stop 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.249+0000 7f5992e19700 1 -- 172.21.15.40:0/186823304 shutdown_connections 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.249+0000 7f5992e19700 1 --2- 172.21.15.40:0/186823304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f598c0fcf70 0x7f598c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.249+0000 7f5992e19700 1 --2- 172.21.15.40:0/186823304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fbcc0 0x7f598c0fc0d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.249+0000 7f5992e19700 1 -- 172.21.15.40:0/186823304 >> 172.21.15.40:0/186823304 conn(0x7f598c0f7440 msgr2=0x7f598c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.249+0000 7f5992e19700 1 -- 172.21.15.40:0/186823304 shutdown_connections 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.249+0000 7f5992e19700 1 -- 172.21.15.40:0/186823304 wait complete. 2024-04-24T23:08:35.253 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f5992e19700 1 Processor -- start 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f5992e19700 1 -- start start 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f5992e19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f598c0fbcc0 0x7f598c066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f5992e19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fcf70 0x7f598c112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f5992e19700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c112d00 con 0x7f598c0fbcc0 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f5992e19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c112e40 con 0x7f598c0fcf70 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f598bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fcf70 0x7f598c112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f598bfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fcf70 0x7f598c112730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44770/0 (socket says 172.21.15.40:44770) 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.250+0000 7f598bfff700 1 -- 172.21.15.40:0/646929448 learned_addr learned my addr 172.21.15.40:0/646929448 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f5990bb5700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f598c0fbcc0 0x7f598c066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f598bfff700 1 -- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f598c0fbcc0 msgr2=0x7f598c066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f598bfff700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f598c0fbcc0 0x7f598c066d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f598bfff700 1 -- 172.21.15.40:0/646929448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5980005040 con 0x7f598c0fcf70 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f598bfff700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fcf70 0x7f598c112730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f597800b790 tx=0x7f5978009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:35.254 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f59897fa700 1 -- 172.21.15.40:0/646929448 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f597800f6f0 con 0x7f598c0fcf70 2024-04-24T23:08:35.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f59897fa700 1 -- 172.21.15.40:0/646929448 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f597800fd20 con 0x7f598c0fcf70 2024-04-24T23:08:35.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c113120 con 0x7f598c0fcf70 2024-04-24T23:08:35.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f59897fa700 1 -- 172.21.15.40:0/646929448 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59780103a0 con 0x7f598c0fcf70 2024-04-24T23:08:35.255 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.251+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c1136e0 con 0x7f598c0fcf70 2024-04-24T23:08:35.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.252+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c04c710 con 0x7f598c0fcf70 2024-04-24T23:08:35.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.253+0000 7f59897fa700 1 -- 172.21.15.40:0/646929448 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5978011400 con 0x7f598c0fcf70 2024-04-24T23:08:35.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.253+0000 7f59897fa700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f597c0692a0 0x7f597c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:08:35.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.253+0000 7f59897fa700 1 -- 172.21.15.40:0/646929448 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f5978086800 con 0x7f598c0fcf70 2024-04-24T23:08:35.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.253+0000 7f5990bb5700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f597c0692a0 0x7f597c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:08:35.256 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.254+0000 7f5990bb5700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f597c0692a0 0x7f597c06b750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f598000b3b0 tx=0x7f5980009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:08:35.258 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.257+0000 7f59897fa700 1 -- 172.21.15.40:0/646929448 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5978056d00 con 0x7f598c0fcf70 2024-04-24T23:08:35.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:08:35.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:35.459+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f598c101290 con 0x7f597c0692a0 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: osdmap e30: 4 total, 4 up, 4 in 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:35.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:35.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:08:35.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:36 smithi040 ceph-mon[33128]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:08:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:36 smithi040 ceph-mon[33128]: Detected new or changed devices on smithi040 2024-04-24T23:08:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:36 smithi040 ceph-mon[33128]: Adjusting osd_memory_target on smithi040 to 15941M 2024-04-24T23:08:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:36 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:36 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:36 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:36 smithi031 ceph-mon[26411]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-24T23:08:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:36 smithi031 ceph-mon[26411]: Detected new or changed devices on smithi040 2024-04-24T23:08:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:36 smithi031 ceph-mon[26411]: Adjusting osd_memory_target on smithi040 to 15941M 2024-04-24T23:08:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:36 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:08:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:36 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:08:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:36 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:37.147 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:37 smithi040 ceph-mon[33128]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:37.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:37 smithi031 ceph-mon[26411]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:08:38.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:38 smithi031 ceph-mon[26411]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T23:08:38.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:38 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/3241910107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4"}]: dispatch 2024-04-24T23:08:38.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:38 smithi031 ceph-mon[26411]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4"}]: dispatch 2024-04-24T23:08:38.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:38 smithi031 ceph-mon[26411]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4"}]': finished 2024-04-24T23:08:38.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:38 smithi031 ceph-mon[26411]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:08:38.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:38 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:38 smithi040 ceph-mon[33128]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-24T23:08:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:38 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/3241910107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4"}]: dispatch 2024-04-24T23:08:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:38 smithi040 ceph-mon[33128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4"}]: dispatch 2024-04-24T23:08:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:38 smithi040 ceph-mon[33128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4"}]': finished 2024-04-24T23:08:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:38 smithi040 ceph-mon[33128]: osdmap e31: 5 total, 4 up, 5 in 2024-04-24T23:08:38.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:38 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:39.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:39 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/67764887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:39.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:39 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/67764887' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:08:40.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:40 smithi040 ceph-mon[33128]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:08:40.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:40 smithi031 ceph-mon[26411]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-24T23:08:42.277 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:42 smithi040 ceph-mon[33128]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T23:08:42.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:42 smithi031 ceph-mon[26411]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-24T23:08:44.211 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:44 smithi040 ceph-mon[33128]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-24T23:08:44.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:44 smithi031 ceph-mon[26411]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-04-24T23:08:46.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:46 smithi040 ceph-mon[33128]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:46.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:46 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:08:46.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:46 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:46 smithi031 ceph-mon[26411]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:46 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:08:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:46 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:08:47.375 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:47 smithi040 ceph-mon[33128]: Deploying daemon osd.4 on smithi040 2024-04-24T23:08:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:47 smithi031 ceph-mon[26411]: Deploying daemon osd.4 on smithi040 2024-04-24T23:08:48.236 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:48 smithi040 ceph-mon[33128]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:48.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:48 smithi031 ceph-mon[26411]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:49.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:08:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:08:50.202 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:50 smithi040 ceph-mon[33128]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:50 smithi031 ceph-mon[26411]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:52.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:51 smithi040 ceph-mon[33128]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:52.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:52.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:52.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:51 smithi031 ceph-mon[26411]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:08:53.806 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:53 smithi040 ceph-mon[33128]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:53.877 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.875+0000 7f59897fa700 1 -- 172.21.15.40:0/646929448 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f598c101290 con 0x7f597c0692a0 2024-04-24T23:08:53.878 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 4 on host 'smithi040' 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f597c0692a0 msgr2=0x7f597c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f597c0692a0 0x7f597c06b750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f598000b3b0 tx=0x7f5980009e30 comp rx=0 tx=0).stop 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fcf70 msgr2=0x7f598c112730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fcf70 0x7f598c112730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f597800b790 tx=0x7f5978009250 comp rx=0 tx=0).stop 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 shutdown_connections 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f597c0692a0 0x7f597c06b750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f598c0fbcc0 0x7f598c066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 --2- 172.21.15.40:0/646929448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f598c0fcf70 0x7f598c112730 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 >> 172.21.15.40:0/646929448 conn(0x7f598c0f7440 msgr2=0x7f598c0f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:08:53.880 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 shutdown_connections 2024-04-24T23:08:53.881 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:08:53.878+0000 7f5992e19700 1 -- 172.21.15.40:0/646929448 wait complete. 2024-04-24T23:08:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:53 smithi031 ceph-mon[26411]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:54.493 DEBUG:teuthology.orchestra.run.smithi040:osd.4> sudo journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.4.service 2024-04-24T23:08:54.495 INFO:tasks.cephadm:Deploying osd.5 on smithi040 with /dev/vg_nvme/lv_2... 2024-04-24T23:08:54.495 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-24T23:08:54.788 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:08:54 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[45250]: 2024-04-24T23:08:54.668+0000 7efc855eb540 -1 osd.4 0 log_to_monitors true 2024-04-24T23:08:55.046 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:55.046 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:55.047 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:54 smithi040 ceph-mon[33128]: from='osd.4 [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:55.047 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:54 smithi040 ceph-mon[33128]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:55.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:55.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:55.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:54 smithi031 ceph-mon[26411]: from='osd.4 [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:55.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:54 smithi031 ceph-mon[26411]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:08:55.285 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi040/config 2024-04-24T23:08:56.082 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:08:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[45250]: 2024-04-24T23:08:55.862+0000 7efc7d5c0700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T23:08:56.082 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:08:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[45250]: 2024-04-24T23:08:55.869+0000 7efc74bb4700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:08:56.082 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:55 smithi040 ceph-mon[33128]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:56.082 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:55 smithi040 ceph-mon[33128]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:08:56.083 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:55 smithi040 ceph-mon[33128]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:08:56.083 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:55 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:56.083 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:55 smithi040 ceph-mon[33128]: from='osd.4 [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:08:56.083 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:55 smithi040 ceph-mon[33128]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:08:56.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:55 smithi031 ceph-mon[26411]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:56.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:55 smithi031 ceph-mon[26411]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:08:56.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:55 smithi031 ceph-mon[26411]: osdmap e32: 5 total, 4 up, 5 in 2024-04-24T23:08:56.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:55 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:56.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:55 smithi031 ceph-mon[26411]: from='osd.4 [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:08:56.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:55 smithi031 ceph-mon[26411]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:08:57.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-24T23:08:57.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T23:08:57.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:57.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:57.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:57.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: osd.4 [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] boot 2024-04-24T23:08:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:08:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-24T23:08:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: osdmap e33: 5 total, 4 up, 5 in 2024-04-24T23:08:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:08:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: osd.4 [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] boot 2024-04-24T23:08:57.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: osdmap e34: 5 total, 5 up, 5 in 2024-04-24T23:08:57.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:08:57.285 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-24T23:08:57.285 INFO:teuthology.orchestra.run.smithi040.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-24T23:08:57.285 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10+0 records in 2024-04-24T23:08:57.285 INFO:teuthology.orchestra.run.smithi040.stderr:10+0 records out 2024-04-24T23:08:57.285 INFO:teuthology.orchestra.run.smithi040.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121784 s, 861 MB/s 2024-04-24T23:08:57.285 INFO:teuthology.orchestra.run.smithi040.stderr:--> Zapping successful for: 2024-04-24T23:08:57.929 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi040:vg_nvme/lv_2 2024-04-24T23:08:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:57 smithi040 ceph-mon[33128]: purged_snaps scrub starts 2024-04-24T23:08:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:57 smithi040 ceph-mon[33128]: purged_snaps scrub ok 2024-04-24T23:08:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:57 smithi040 ceph-mon[33128]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:58.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:57 smithi031 ceph-mon[26411]: purged_snaps scrub starts 2024-04-24T23:08:58.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:57 smithi031 ceph-mon[26411]: purged_snaps scrub ok 2024-04-24T23:08:58.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:57 smithi031 ceph-mon[26411]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-24T23:08:59.066 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:58 smithi040 ceph-mon[33128]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T23:08:59.097 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi040/config 2024-04-24T23:08:59.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:58 smithi031 ceph-mon[26411]: osdmap e35: 5 total, 5 up, 5 in 2024-04-24T23:09:00.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:08:59 smithi040 ceph-mon[33128]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:00.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:08:59 smithi031 ceph-mon[26411]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.576+0000 7fab3f057700 1 -- 172.21.15.40:0/2645949632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fdfd0 msgr2=0x7fab380fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.576+0000 7fab3f057700 1 --2- 172.21.15.40:0/2645949632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fdfd0 0x7fab380fe440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fab24004940 tx=0x7fab24009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.576+0000 7fab3f057700 1 -- 172.21.15.40:0/2645949632 shutdown_connections 2024-04-24T23:09:01.580 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.576+0000 7fab3f057700 1 --2- 172.21.15.40:0/2645949632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fdfd0 0x7fab380fe440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.576+0000 7fab3f057700 1 --2- 172.21.15.40:0/2645949632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab380fce60 0x7fab380fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.576+0000 7fab3f057700 1 -- 172.21.15.40:0/2645949632 >> 172.21.15.40:0/2645949632 conn(0x7fab380f8540 msgr2=0x7fab380fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.576+0000 7fab3f057700 1 -- 172.21.15.40:0/2645949632 shutdown_connections 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.576+0000 7fab3f057700 1 -- 172.21.15.40:0/2645949632 wait complete. 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3f057700 1 Processor -- start 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3f057700 1 -- start start 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3f057700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fce60 0x7fab38071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3f057700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab380fdfd0 0x7fab380702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3f057700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38072200 con 0x7fab380fdfd0 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3f057700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab380707f0 con 0x7fab380fce60 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3cdf3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fce60 0x7fab38071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3cdf3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fce60 0x7fab38071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50288/0 (socket says 172.21.15.40:50288) 2024-04-24T23:09:01.581 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.577+0000 7fab3cdf3700 1 -- 172.21.15.40:0/1332975505 learned_addr learned my addr 172.21.15.40:0/1332975505 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab37fff700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab380fdfd0 0x7fab380702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab3cdf3700 1 -- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab380fdfd0 msgr2=0x7fab380702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab3cdf3700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab380fdfd0 0x7fab380702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab3cdf3700 1 -- 172.21.15.40:0/1332975505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24005040 con 0x7fab380fce60 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab3cdf3700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fce60 0x7fab38071c00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00c8c0 tx=0x7fab2c0044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab357fa700 1 -- 172.21.15.40:0/1332975505 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2c00a980 con 0x7fab380fce60 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab357fa700 1 -- 172.21.15.40:0/1332975505 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab2c00ce60 con 0x7fab380fce60 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab38070aa0 con 0x7fab380fce60 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab357fa700 1 -- 172.21.15.40:0/1332975505 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2c0115e0 con 0x7fab380fce60 2024-04-24T23:09:01.582 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.578+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab38071060 con 0x7fab380fce60 2024-04-24T23:09:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.579+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3804c710 con 0x7fab380fce60 2024-04-24T23:09:01.583 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.579+0000 7fab357fa700 1 -- 172.21.15.40:0/1332975505 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fab2c012400 con 0x7fab380fce60 2024-04-24T23:09:01.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.580+0000 7fab357fa700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fab280692a0 0x7fab2806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:01.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.580+0000 7fab357fa700 1 -- 172.21.15.40:0/1332975505 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7fab2c086db0 con 0x7fab380fce60 2024-04-24T23:09:01.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.580+0000 7fab37fff700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fab280692a0 0x7fab2806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:01.584 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.581+0000 7fab37fff700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fab280692a0 0x7fab2806b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fab24004f90 tx=0x7fab24009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:01.586 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.584+0000 7fab357fa700 1 -- 172.21.15.40:0/1332975505 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fab2c057060 con 0x7fab380fce60 2024-04-24T23:09:01.795 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:01.793+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fab38102360 con 0x7fab280692a0 2024-04-24T23:09:01.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: Detected new or changed devices on smithi040 2024-04-24T23:09:01.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:01.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:01.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: Adjusting osd_memory_target on smithi040 to 7970M 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:01.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:01 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: Detected new or changed devices on smithi040 2024-04-24T23:09:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: Adjusting osd_memory_target on smithi040 to 7970M 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: pgmap v120: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:01 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:02 smithi040 ceph-mon[33128]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:02 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:02 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:02 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:02 smithi031 ceph-mon[26411]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi040:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:09:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-24T23:09:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-24T23:09:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:02 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:03 smithi040 ceph-mon[33128]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:04.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:03 smithi031 ceph-mon[26411]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:09:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:04 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/831231999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "027b7b7a-afb6-4507-8cb2-274ebadbb2ca"}]: dispatch 2024-04-24T23:09:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:04 smithi040 ceph-mon[33128]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "027b7b7a-afb6-4507-8cb2-274ebadbb2ca"}]: dispatch 2024-04-24T23:09:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:04 smithi040 ceph-mon[33128]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "027b7b7a-afb6-4507-8cb2-274ebadbb2ca"}]': finished 2024-04-24T23:09:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:04 smithi040 ceph-mon[33128]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:09:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:04.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:09:04.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:04 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/831231999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "027b7b7a-afb6-4507-8cb2-274ebadbb2ca"}]: dispatch 2024-04-24T23:09:04.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:04 smithi031 ceph-mon[26411]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "027b7b7a-afb6-4507-8cb2-274ebadbb2ca"}]: dispatch 2024-04-24T23:09:04.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:04 smithi031 ceph-mon[26411]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "027b7b7a-afb6-4507-8cb2-274ebadbb2ca"}]': finished 2024-04-24T23:09:04.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:04 smithi031 ceph-mon[26411]: osdmap e36: 6 total, 5 up, 6 in 2024-04-24T23:09:04.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:05 smithi040 ceph-mon[33128]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:05 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/1881585303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:05.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:05 smithi031 ceph-mon[26411]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:05.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:05 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/1881585303' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-24T23:09:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:07 smithi040 ceph-mon[33128]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:07 smithi031 ceph-mon[26411]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:10.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:09 smithi040 ceph-mon[33128]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:09 smithi031 ceph-mon[26411]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:12.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:11 smithi040 ceph-mon[33128]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:12.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:09:12.056 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:11 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:11 smithi031 ceph-mon[26411]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:09:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:11 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:12.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:12 smithi031 ceph-mon[26411]: Deploying daemon osd.5 on smithi040 2024-04-24T23:09:13.027 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:12 smithi040 ceph-mon[33128]: Deploying daemon osd.5 on smithi040 2024-04-24T23:09:14.132 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:13 smithi040 ceph-mon[33128]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:14.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:13 smithi031 ceph-mon[26411]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:16.083 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:15 smithi040 ceph-mon[33128]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:15 smithi031 ceph-mon[26411]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:17 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:17 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:17 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:17 smithi040 ceph-mon[33128]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:17 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:17 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:17 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:17 smithi031 ceph-mon[26411]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:19.458 INFO:teuthology.orchestra.run.smithi040.stdout:Created osd(s) 5 on host 'smithi040' 2024-04-24T23:09:19.459 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.456+0000 7fab357fa700 1 -- 172.21.15.40:0/1332975505 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fab38102360 con 0x7fab280692a0 2024-04-24T23:09:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fab280692a0 msgr2=0x7fab2806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:19.460 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fab280692a0 0x7fab2806b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fab24004f90 tx=0x7fab24009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fce60 msgr2=0x7fab38071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fce60 0x7fab38071c00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00c8c0 tx=0x7fab2c0044c0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 shutdown_connections 2024-04-24T23:09:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fab380fce60 0x7fab38071c00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fab280692a0 0x7fab2806b750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 --2- 172.21.15.40:0/1332975505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab380fdfd0 0x7fab380702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 >> 172.21.15.40:0/1332975505 conn(0x7fab380f8540 msgr2=0x7fab380f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:19.461 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 shutdown_connections 2024-04-24T23:09:19.462 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:19.458+0000 7fab3f057700 1 -- 172.21.15.40:0/1332975505 wait complete. 2024-04-24T23:09:20.064 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:19 smithi040 ceph-mon[33128]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:20.064 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:09:20.064 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:20.064 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:20.090 DEBUG:teuthology.orchestra.run.smithi040:osd.5> sudo journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.5.service 2024-04-24T23:09:20.093 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-24T23:09:20.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:09:20.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:19 smithi031 ceph-mon[26411]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:20.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:09:20.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:20.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:20.370 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:20.844 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:09:20 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[52137]: 2024-04-24T23:09:20.491+0000 7fd4c04c5540 -1 osd.5 0 log_to_monitors true 2024-04-24T23:09:21.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:20 smithi040 ceph-mon[33128]: from='osd.5 [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:21.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:20 smithi040 ceph-mon[33128]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:21.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:20 smithi031 ceph-mon[26411]: from='osd.5 [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:21.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:20 smithi031 ceph-mon[26411]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:09:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.872+0000 7f37cd373700 1 -- 172.21.15.31:0/2725926225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 msgr2=0x7f37c80ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.872+0000 7f37cd373700 1 --2- 172.21.15.31:0/2725926225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 0x7f37c80ffa00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009ce0 tx=0x7f37b8004d10 comp rx=0 tx=0).stop 2024-04-24T23:09:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.872+0000 7f37cd373700 1 -- 172.21.15.31:0/2725926225 shutdown_connections 2024-04-24T23:09:21.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.872+0000 7f37cd373700 1 --2- 172.21.15.31:0/2725926225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c80fff40 0x7f37c8102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.872+0000 7f37cd373700 1 --2- 172.21.15.31:0/2725926225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 0x7f37c80ffa00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.872+0000 7f37cd373700 1 -- 172.21.15.31:0/2725926225 >> 172.21.15.31:0/2725926225 conn(0x7f37c80f7430 msgr2=0x7f37c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.872+0000 7f37cd373700 1 -- 172.21.15.31:0/2725926225 shutdown_connections 2024-04-24T23:09:21.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.872+0000 7f37cd373700 1 -- 172.21.15.31:0/2725926225 wait complete. 2024-04-24T23:09:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37cd373700 1 Processor -- start 2024-04-24T23:09:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37cd373700 1 -- start start 2024-04-24T23:09:21.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37cd373700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 0x7f37c80fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37cd373700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c80fff40 0x7f37c80fb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37cd373700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80fbeb0 con 0x7f37c8066260 2024-04-24T23:09:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37cd373700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c80fc020 con 0x7f37c80fff40 2024-04-24T23:09:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 0x7f37c80fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37c6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 0x7f37c80fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32804/0 (socket says 172.21.15.31:32804) 2024-04-24T23:09:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37c6ffd700 1 -- 172.21.15.31:0/2597074595 learned_addr learned my addr 172.21.15.31:0/2597074595 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.873+0000 7f37c67fc700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c80fff40 0x7f37c80fb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:21.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37c6ffd700 1 -- 172.21.15.31:0/2597074595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c80fff40 msgr2=0x7f37c80fb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37c6ffd700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c80fff40 0x7f37c80fb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37c6ffd700 1 -- 172.21.15.31:0/2597074595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b8009940 con 0x7f37c8066260 2024-04-24T23:09:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37c6ffd700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 0x7f37c80fd430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009380 tx=0x7f37b8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37bf7fe700 1 -- 172.21.15.31:0/2597074595 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37b800e040 con 0x7f37c8066260 2024-04-24T23:09:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c80fc2a0 con 0x7f37c8066260 2024-04-24T23:09:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37bf7fe700 1 -- 172.21.15.31:0/2597074595 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37b801cd00 con 0x7f37c8066260 2024-04-24T23:09:21.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37bf7fe700 1 -- 172.21.15.31:0/2597074595 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37b8012b90 con 0x7f37c8066260 2024-04-24T23:09:21.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.874+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c8115a80 con 0x7f37c8066260 2024-04-24T23:09:21.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.875+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c805f5c0 con 0x7f37c8066260 2024-04-24T23:09:21.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.876+0000 7f37bf7fe700 1 -- 172.21.15.31:0/2597074595 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f37b801c830 con 0x7f37c8066260 2024-04-24T23:09:21.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.876+0000 7f37bf7fe700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37b40692a0 0x7f37b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:21.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.876+0000 7f37bf7fe700 1 -- 172.21.15.31:0/2597074595 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4354+0+0 (secure 0 0 0) 0x7f37b80890a0 con 0x7f37c8066260 2024-04-24T23:09:21.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.876+0000 7f37c67fc700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37b40692a0 0x7f37b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:21.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.877+0000 7f37c67fc700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37b40692a0 0x7f37b406b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009b80 tx=0x7f37b000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:21.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:21.881+0000 7f37bf7fe700 1 -- 172.21.15.31:0/2597074595 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f37b80557a0 con 0x7f37c8066260 2024-04-24T23:09:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:21 smithi031 ceph-mon[26411]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:21 smithi031 ceph-mon[26411]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:09:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:21 smithi031 ceph-mon[26411]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:09:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:21 smithi031 ceph-mon[26411]: from='osd.5 [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:09:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:21 smithi031 ceph-mon[26411]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:09:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:21 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.075+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f37c804bed0 con 0x7f37c8066260 2024-04-24T23:09:22.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.075+0000 7f37bf7fe700 1 -- 172.21.15.31:0/2597074595 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7f37b8058dc0 con 0x7f37c8066260 2024-04-24T23:09:22.077 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:09:22.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37b40692a0 msgr2=0x7f37b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:22.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37b40692a0 0x7f37b406b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009b80 tx=0x7f37b000b040 comp rx=0 tx=0).stop 2024-04-24T23:09:22.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 msgr2=0x7f37c80fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:22.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 0x7f37c80fd430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f37b8009380 tx=0x7f37b8004d10 comp rx=0 tx=0).stop 2024-04-24T23:09:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 shutdown_connections 2024-04-24T23:09:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37b40692a0 0x7f37b406b750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c8066260 0x7f37c80fd430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 --2- 172.21.15.31:0/2597074595 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37c80fff40 0x7f37c80fb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 >> 172.21.15.31:0/2597074595 conn(0x7f37c80f7430 msgr2=0x7f37c80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 shutdown_connections 2024-04-24T23:09:22.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:22.078+0000 7f37cd373700 1 -- 172.21.15.31:0/2597074595 wait complete. 2024-04-24T23:09:22.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[52137]: 2024-04-24T23:09:21.854+0000 7fd4b6c97700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T23:09:22.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[52137]: 2024-04-24T23:09:21.866+0000 7fd4afa8e700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:09:22.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-mon[33128]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:22.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-mon[33128]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:09:22.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-mon[33128]: osdmap e37: 6 total, 5 up, 6 in 2024-04-24T23:09:22.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:22.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-mon[33128]: from='osd.5 [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:09:22.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-mon[33128]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:09:22.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:22.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:21 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:22.660 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1714000136,"num_in_osds":6,"osd_in_since":1714000144,"num_remapped_pgs":0} 2024-04-24T23:09:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:22 smithi031 ceph-mon[26411]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-24T23:09:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:22 smithi031 ceph-mon[26411]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:09:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:22 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:22 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:22 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2597074595' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:09:23.003 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:22 smithi040 ceph-mon[33128]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]': finished 2024-04-24T23:09:23.004 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:22 smithi040 ceph-mon[33128]: osdmap e38: 6 total, 5 up, 6 in 2024-04-24T23:09:23.004 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:22 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:23.004 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:22 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:23.004 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:22 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2597074595' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:09:23.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-24T23:09:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:23 smithi031 ceph-mon[26411]: purged_snaps scrub starts 2024-04-24T23:09:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:23 smithi031 ceph-mon[26411]: purged_snaps scrub ok 2024-04-24T23:09:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:23 smithi031 ceph-mon[26411]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:23 smithi031 ceph-mon[26411]: osd.5 [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] boot 2024-04-24T23:09:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:23 smithi031 ceph-mon[26411]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:09:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:24.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:23 smithi040 ceph-mon[33128]: purged_snaps scrub starts 2024-04-24T23:09:24.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:23 smithi040 ceph-mon[33128]: purged_snaps scrub ok 2024-04-24T23:09:24.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:23 smithi040 ceph-mon[33128]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-24T23:09:24.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:23 smithi040 ceph-mon[33128]: osd.5 [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] boot 2024-04-24T23:09:24.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:23 smithi040 ceph-mon[33128]: osdmap e39: 6 total, 6 up, 6 in 2024-04-24T23:09:24.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:09:25.141 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:25.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:24 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: osdmap e40: 6 total, 6 up, 6 in 2024-04-24T23:09:25.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:09:25.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:24 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:09:25.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 -- 172.21.15.31:0/2776728875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 msgr2=0x7f40f40ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 --2- 172.21.15.31:0/2776728875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 0x7f40f40ffda0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009ce0 tx=0x7f40ec004d10 comp rx=0 tx=0).stop 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 -- 172.21.15.31:0/2776728875 shutdown_connections 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 --2- 172.21.15.31:0/2776728875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f41002e0 0x7f40f41026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 --2- 172.21.15.31:0/2776728875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 0x7f40f40ffda0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 -- 172.21.15.31:0/2776728875 >> 172.21.15.31:0/2776728875 conn(0x7f40f40f7440 msgr2=0x7f40f40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 -- 172.21.15.31:0/2776728875 shutdown_connections 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 -- 172.21.15.31:0/2776728875 wait complete. 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.619+0000 7f40f8d27700 1 Processor -- start 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f8d27700 1 -- start start 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f8d27700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 0x7f40f40fd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f8d27700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f41002e0 0x7f40f40fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f8d27700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40fbe80 con 0x7f40f40fd9c0 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f8d27700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f40fbff0 con 0x7f40f41002e0 2024-04-24T23:09:25.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 0x7f40f40fd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 0x7f40f40fd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32814/0 (socket says 172.21.15.31:32814) 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f259c700 1 -- 172.21.15.31:0/382356984 learned_addr learned my addr 172.21.15.31:0/382356984 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f1d9b700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f41002e0 0x7f40f40fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f259c700 1 -- 172.21.15.31:0/382356984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f41002e0 msgr2=0x7f40f40fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f259c700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f41002e0 0x7f40f40fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f259c700 1 -- 172.21.15.31:0/382356984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec009940 con 0x7f40f40fd9c0 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40f259c700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 0x7f40f40fd400 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009910 tx=0x7f40ec004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.620+0000 7f40eaffd700 1 -- 172.21.15.31:0/382356984 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40ec00e040 con 0x7f40f40fd9c0 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.621+0000 7f40eaffd700 1 -- 172.21.15.31:0/382356984 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40ec008da0 con 0x7f40f40fd9c0 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.621+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f40fc270 con 0x7f40f40fd9c0 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.621+0000 7f40eaffd700 1 -- 172.21.15.31:0/382356984 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40ec01c620 con 0x7f40f40fd9c0 2024-04-24T23:09:25.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.621+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f4115c40 con 0x7f40f40fd9c0 2024-04-24T23:09:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.621+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f405f5c0 con 0x7f40f40fd9c0 2024-04-24T23:09:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.622+0000 7f40eaffd700 1 -- 172.21.15.31:0/382356984 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f40ec012a50 con 0x7f40f40fd9c0 2024-04-24T23:09:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.622+0000 7f40eaffd700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f40e00692b0 0x7f40e006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.622+0000 7f40eaffd700 1 -- 172.21.15.31:0/382356984 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f40ec0889c0 con 0x7f40f40fd9c0 2024-04-24T23:09:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.622+0000 7f40f1d9b700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f40e00692b0 0x7f40e006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:25.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.623+0000 7f40f1d9b700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f40e00692b0 0x7f40e006b760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0102e0 tx=0x7f40dc00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:25.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.627+0000 7f40eaffd700 1 -- 172.21.15.31:0/382356984 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f40ec054fb0 con 0x7f40f40fd9c0 2024-04-24T23:09:25.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.824+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f40f4105970 con 0x7f40f40fd9c0 2024-04-24T23:09:25.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.824+0000 7f40eaffd700 1 -- 172.21.15.31:0/382356984 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f40ec004e10 con 0x7f40f40fd9c0 2024-04-24T23:09:25.826 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:09:25.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f40e00692b0 msgr2=0x7f40e006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:25.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f40e00692b0 0x7f40e006b760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0102e0 tx=0x7f40dc00b040 comp rx=0 tx=0).stop 2024-04-24T23:09:25.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 msgr2=0x7f40f40fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:25.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 0x7f40f40fd400 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009910 tx=0x7f40ec004d10 comp rx=0 tx=0).stop 2024-04-24T23:09:25.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 shutdown_connections 2024-04-24T23:09:25.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f40e00692b0 0x7f40e006b760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:25.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40f40fd9c0 0x7f40f40fd400 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:25.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 --2- 172.21.15.31:0/382356984 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f41002e0 0x7f40f40fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:25.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.826+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 >> 172.21.15.31:0/382356984 conn(0x7f40f40f7440 msgr2=0x7f40f40f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:25.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.827+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 shutdown_connections 2024-04-24T23:09:25.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:25.827+0000 7f40f8d27700 1 -- 172.21.15.31:0/382356984 wait complete. 2024-04-24T23:09:26.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:25 smithi031 ceph-mon[26411]: Detected new or changed devices on smithi040 2024-04-24T23:09:26.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:25 smithi031 ceph-mon[26411]: Adjusting osd_memory_target on smithi040 to 5313M 2024-04-24T23:09:26.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:25 smithi031 ceph-mon[26411]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:26.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:25 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/382356984' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:09:26.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:25 smithi040 ceph-mon[33128]: Detected new or changed devices on smithi040 2024-04-24T23:09:26.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:25 smithi040 ceph-mon[33128]: Adjusting osd_memory_target on smithi040 to 5313M 2024-04-24T23:09:26.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:25 smithi040 ceph-mon[33128]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:26.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:25 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/382356984' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-24T23:09:26.475 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1714000162,"num_in_osds":6,"osd_in_since":1714000144,"num_remapped_pgs":0} 2024-04-24T23:09:26.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:09:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:28.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:27 smithi040 ceph-mon[33128]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:28.163 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:27 smithi031 ceph-mon[26411]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 -- 172.21.15.31:0/1878890011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 msgr2=0x7f8008065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 --2- 172.21.15.31:0/1878890011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 0x7f8008065ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8004940 tx=0x7f7ff8009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 -- 172.21.15.31:0/1878890011 shutdown_connections 2024-04-24T23:09:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 --2- 172.21.15.31:0/1878890011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 0x7f8008065ab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 --2- 172.21.15.31:0/1878890011 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008102e50 0x7f8008103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 -- 172.21.15.31:0/1878890011 >> 172.21.15.31:0/1878890011 conn(0x7f8008071d00 msgr2=0x7f8008072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 -- 172.21.15.31:0/1878890011 shutdown_connections 2024-04-24T23:09:28.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 -- 172.21.15.31:0/1878890011 wait complete. 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.392+0000 7f800fa36700 1 Processor -- start 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800fa36700 1 -- start start 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800fa36700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 0x7f8008115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800fa36700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008102e50 0x7f80081162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800fa36700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80081169b0 con 0x7f8008065640 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800fa36700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800810fcf0 con 0x7f8008102e50 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800d7d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 0x7f8008115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800d7d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 0x7f8008115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:32834/0 (socket says 172.21.15.31:32834) 2024-04-24T23:09:28.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800d7d2700 1 -- 172.21.15.31:0/1282160691 learned_addr learned my addr 172.21.15.31:0/1282160691 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800cfd1700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008102e50 0x7f80081162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800d7d2700 1 -- 172.21.15.31:0/1282160691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008102e50 msgr2=0x7f80081162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800d7d2700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008102e50 0x7f80081162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800d7d2700 1 -- 172.21.15.31:0/1282160691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff8005040 con 0x7f8008065640 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.393+0000 7f800d7d2700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 0x7f8008115d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f800400b790 tx=0x7f8004009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.394+0000 7f7ffdffb700 1 -- 172.21.15.31:0/1282160691 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f800400f770 con 0x7f8008065640 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.394+0000 7f7ffdffb700 1 -- 172.21.15.31:0/1282160691 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f800400fda0 con 0x7f8008065640 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.394+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800810ff70 con 0x7f8008065640 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.394+0000 7f7ffdffb700 1 -- 172.21.15.31:0/1282160691 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f800400d310 con 0x7f8008065640 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.394+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8008110530 con 0x7f8008065640 2024-04-24T23:09:28.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.395+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80080fe540 con 0x7f8008065640 2024-04-24T23:09:28.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.395+0000 7f7ffdffb700 1 -- 172.21.15.31:0/1282160691 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8004010420 con 0x7f8008065640 2024-04-24T23:09:28.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.395+0000 7f7ffdffb700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f7ff40692a0 0x7f7ff406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:28.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.396+0000 7f7ffdffb700 1 -- 172.21.15.31:0/1282160691 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8004086d70 con 0x7f8008065640 2024-04-24T23:09:28.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.396+0000 7f800cfd1700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f7ff40692a0 0x7f7ff406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:28.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.396+0000 7f800cfd1700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f7ff40692a0 0x7f7ff406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8004f90 tx=0x7f7ff8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:28.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.400+0000 7f7ffdffb700 1 -- 172.21.15.31:0/1282160691 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8004056f10 con 0x7f8008065640 2024-04-24T23:09:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.598+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f800804bf00 con 0x7f8008065640 2024-04-24T23:09:28.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.599+0000 7f7ffdffb700 1 -- 172.21.15.31:0/1282160691 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11033 (secure 0 0 0) 0x7f8004056aa0 con 0x7f8008065640 2024-04-24T23:09:28.600 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:09:28.601 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:03:41.065781+0000","modified":"2024-04-24T23:09:23.859599+0000","last_up_change":"2024-04-24T23:09:22.852760+0000","last_in_change":"2024-04-24T23:09:04.280370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:08:05.473853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0f1ff679-6b2e-4059-a299-2d2356bbe91c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6803","nonce":1397311567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6805","nonce":1397311567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6809","nonce":1397311567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6807","nonce":1397311567}]},"public_addr":"172.21.15.31:6803/1397311567","cluster_addr":"172.21.15.31:6805/1397311567","heartbeat_back_addr":"172.21.15.31:6809/1397311567","heartbeat_front_addr":"172.21.15.31:6807/1397311567","state":["exists","up"]},{"osd":1,"uuid":"6885f0d2-70e8-4c54-a381-0b3683a0bc49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6811","nonce":1768854803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6813","nonce":1768854803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6817","nonce":1768854803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6815","nonce":1768854803}]},"public_addr":"172.21.15.31:6811/1768854803","cluster_addr":"172.21.15.31:6813/1768854803","heartbeat_back_addr":"172.21.15.31:6817/1768854803","heartbeat_front_addr":"172.21.15.31:6815/1768854803","state":["exists","up"]},{"osd":2,"uuid":"9b57c743-5826-4bf9-ac00-0cb539b30224","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6819","nonce":563563256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6821","nonce":563563256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6825","nonce":563563256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6823","nonce":563563256}]},"public_addr":"172.21.15.31:6819/563563256","cluster_addr":"172.21.15.31:6821/563563256","heartbeat_back_addr":"172.21.15.31:6825/563563256","heartbeat_front_addr":"172.21.15.31:6823/563563256","state":["exists","up"]},{"osd":3,"uuid":"1dc47b54-65da-4579-bcac-2258be1f4cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6801","nonce":824499182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6803","nonce":824499182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6807","nonce":824499182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6805","nonce":824499182}]},"public_addr":"172.21.15.40:6801/824499182","cluster_addr":"172.21.15.40:6803/824499182","heartbeat_back_addr":"172.21.15.40:6807/824499182","heartbeat_front_addr":"172.21.15.40:6805/824499182","state":["exists","up"]},{"osd":4,"uuid":"e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6809","nonce":1464558178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6811","nonce":1464558178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6815","nonce":1464558178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6813","nonce":1464558178}]},"public_addr":"172.21.15.40:6809/1464558178","cluster_addr":"172.21.15.40:6811/1464558178","heartbeat_back_addr":"172.21.15.40:6815/1464558178","heartbeat_front_addr":"172.21.15.40:6813/1464558178","state":["exists","up"]},{"osd":5,"uuid":"027b7b7a-afb6-4507-8cb2-274ebadbb2ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6817","nonce":2359746556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6819","nonce":2359746556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6823","nonce":2359746556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6821","nonce":2359746556}]},"public_addr":"172.21.15.40:6817/2359746556","cluster_addr":"172.21.15.40:6819/2359746556","heartbeat_back_addr":"172.21.15.40:6823/2359746556","heartbeat_front_addr":"172.21.15.40:6821/2359746556","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:08.221856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:36.407326+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:03.612047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:29.069233+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:55.660200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:21.507297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1805904019":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/39610504":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6801/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6800/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/2457576751":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6801/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6800/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/2308793407":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6801/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/382933206":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/1340983696":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/3645410271":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/611668999":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/1654686211":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6800/4106752127":"2024-04-25T23:04:15.327046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:09:28.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f7ff40692a0 msgr2=0x7f7ff406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f7ff40692a0 0x7f7ff406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8004f90 tx=0x7f7ff8009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 msgr2=0x7f8008115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 0x7f8008115d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f800400b790 tx=0x7f8004009250 comp rx=0 tx=0).stop 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 shutdown_connections 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f7ff40692a0 0x7f7ff406b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8008065640 0x7f8008115d90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 --2- 172.21.15.31:0/1282160691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8008102e50 0x7f80081162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 >> 172.21.15.31:0/1282160691 conn(0x7f8008071d00 msgr2=0x7f80080fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 shutdown_connections 2024-04-24T23:09:28.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:28.601+0000 7f800fa36700 1 -- 172.21.15.31:0/1282160691 wait complete. 2024-04-24T23:09:29.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:28 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1282160691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:29.237 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:28 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1282160691' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:29.267 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:08:05.473853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-24T23:09:29.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-24T23:09:29.533 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:29.991 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:29 smithi031 ceph-mon[26411]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:30.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:29 smithi040 ceph-mon[33128]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:31.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.117+0000 7f31ec117700 1 -- 172.21.15.31:0/3612786025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 msgr2=0x7f31e40fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:31.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.117+0000 7f31ec117700 1 --2- 172.21.15.31:0/3612786025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 0x7f31e40fe1b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f31e0004940 tx=0x7f31e0009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:31.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.117+0000 7f31ec117700 1 -- 172.21.15.31:0/3612786025 shutdown_connections 2024-04-24T23:09:31.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.117+0000 7f31ec117700 1 --2- 172.21.15.31:0/3612786025 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40fe780 0x7f31e4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:31.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.117+0000 7f31ec117700 1 --2- 172.21.15.31:0/3612786025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 0x7f31e40fe1b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.117+0000 7f31ec117700 1 -- 172.21.15.31:0/3612786025 >> 172.21.15.31:0/3612786025 conn(0x7f31e40f7850 msgr2=0x7f31e40f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.117+0000 7f31ec117700 1 -- 172.21.15.31:0/3612786025 shutdown_connections 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.117+0000 7f31ec117700 1 -- 172.21.15.31:0/3612786025 wait complete. 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31ec117700 1 Processor -- start 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31ec117700 1 -- start start 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31ec117700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 0x7f31e4113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31ec117700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40fe780 0x7f31e4114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31ec117700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e41146e0 con 0x7f31e40fdde0 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31ec117700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e410dad0 con 0x7f31e40fe780 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31e9eb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 0x7f31e4113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31e9eb3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 0x7f31e4113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59096/0 (socket says 172.21.15.31:59096) 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31e9eb3700 1 -- 172.21.15.31:0/3000025618 learned_addr learned my addr 172.21.15.31:0/3000025618 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31e96b2700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40fe780 0x7f31e4114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:31.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31e9eb3700 1 -- 172.21.15.31:0/3000025618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40fe780 msgr2=0x7f31e4114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31e9eb3700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40fe780 0x7f31e4114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.118+0000 7f31e9eb3700 1 -- 172.21.15.31:0/3000025618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e0005040 con 0x7f31e40fdde0 2024-04-24T23:09:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.119+0000 7f31e9eb3700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 0x7f31e4113b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31e000b3b0 tx=0x7f31e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.119+0000 7f31da7fc700 1 -- 172.21.15.31:0/3000025618 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31e000e040 con 0x7f31e40fdde0 2024-04-24T23:09:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.119+0000 7f31da7fc700 1 -- 172.21.15.31:0/3000025618 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31e0013420 con 0x7f31e40fdde0 2024-04-24T23:09:31.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.119+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e410dd50 con 0x7f31e40fdde0 2024-04-24T23:09:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.119+0000 7f31da7fc700 1 -- 172.21.15.31:0/3000025618 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31e001c630 con 0x7f31e40fdde0 2024-04-24T23:09:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.119+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e410e290 con 0x7f31e40fdde0 2024-04-24T23:09:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.120+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e4066750 con 0x7f31e40fdde0 2024-04-24T23:09:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.120+0000 7f31da7fc700 1 -- 172.21.15.31:0/3000025618 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f31e0013590 con 0x7f31e40fdde0 2024-04-24T23:09:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.121+0000 7f31da7fc700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f31d00692b0 0x7f31d006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.121+0000 7f31da7fc700 1 -- 172.21.15.31:0/3000025618 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f31e00880d0 con 0x7f31e40fdde0 2024-04-24T23:09:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.121+0000 7f31e96b2700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f31d00692b0 0x7f31d006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:31.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.121+0000 7f31e96b2700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f31d00692b0 0x7f31d006b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f31e410ec90 tx=0x7f31d400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:31.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.125+0000 7f31da7fc700 1 -- 172.21.15.31:0/3000025618 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f31e00581d0 con 0x7f31e40fdde0 2024-04-24T23:09:31.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.321+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31e4109e70 con 0x7f31e40fdde0 2024-04-24T23:09:31.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.321+0000 7f31da7fc700 1 -- 172.21.15.31:0/3000025618 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31e0057d60 con 0x7f31e40fdde0 2024-04-24T23:09:31.323 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f31d00692b0 msgr2=0x7f31d006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f31d00692b0 0x7f31d006b760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f31e410ec90 tx=0x7f31d400b040 comp rx=0 tx=0).stop 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 msgr2=0x7f31e4113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 0x7f31e4113b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31e000b3b0 tx=0x7f31e0009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 shutdown_connections 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f31d00692b0 0x7f31d006b760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31e40fdde0 0x7f31e4113b50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 --2- 172.21.15.31:0/3000025618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f31e40fe780 0x7f31e4114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 >> 172.21.15.31:0/3000025618 conn(0x7f31e40f7850 msgr2=0x7f31e40f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 shutdown_connections 2024-04-24T23:09:31.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:31.323+0000 7f31ec117700 1 -- 172.21.15.31:0/3000025618 wait complete. 2024-04-24T23:09:32.002 INFO:tasks.cephadm:Setting up client nodes... 2024-04-24T23:09:32.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T23:09:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:31 smithi040 ceph-mon[33128]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:31 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3000025618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:09:32.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:31 smithi031 ceph-mon[26411]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:32.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:31 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3000025618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:09:32.269 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.840+0000 7f7938f19700 1 -- 172.21.15.31:0/2282093696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 msgr2=0x7f79340fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.840+0000 7f7938f19700 1 --2- 172.21.15.31:0/2282093696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 0x7f79340fd270 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7924004940 tx=0x7f7924009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.840+0000 7f7938f19700 1 -- 172.21.15.31:0/2282093696 shutdown_connections 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.840+0000 7f7938f19700 1 --2- 172.21.15.31:0/2282093696 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79340fdfd0 0x7f79340fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.840+0000 7f7938f19700 1 --2- 172.21.15.31:0/2282093696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 0x7f79340fd270 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.840+0000 7f7938f19700 1 -- 172.21.15.31:0/2282093696 >> 172.21.15.31:0/2282093696 conn(0x7f79340f8540 msgr2=0x7f79340fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.840+0000 7f7938f19700 1 -- 172.21.15.31:0/2282093696 shutdown_connections 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f7938f19700 1 -- 172.21.15.31:0/2282093696 wait complete. 2024-04-24T23:09:33.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f7938f19700 1 Processor -- start 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f7938f19700 1 -- start start 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f7938f19700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 0x7f7934071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f7938f19700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79340fdfd0 0x7f79340702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f7938f19700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7934072200 con 0x7f79340fce60 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f7938f19700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79340707f0 con 0x7f79340fdfd0 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f793259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 0x7f7934071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f793259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 0x7f7934071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59104/0 (socket says 172.21.15.31:59104) 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f793259c700 1 -- 172.21.15.31:0/1957865746 learned_addr learned my addr 172.21.15.31:0/1957865746 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.841+0000 7f7931d9b700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79340fdfd0 0x7f79340702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f793259c700 1 -- 172.21.15.31:0/1957865746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79340fdfd0 msgr2=0x7f79340702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f793259c700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79340fdfd0 0x7f79340702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f793259c700 1 -- 172.21.15.31:0/1957865746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924005040 con 0x7f79340fce60 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f793259c700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 0x7f7934071c00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7924009150 tx=0x7f7924009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f7922ffd700 1 -- 172.21.15.31:0/1957865746 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f792400e040 con 0x7f79340fce60 2024-04-24T23:09:33.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f7922ffd700 1 -- 172.21.15.31:0/1957865746 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f792400bcc0 con 0x7f79340fce60 2024-04-24T23:09:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7934070a40 con 0x7f79340fce60 2024-04-24T23:09:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f7922ffd700 1 -- 172.21.15.31:0/1957865746 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f792401c630 con 0x7f79340fce60 2024-04-24T23:09:33.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.842+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7934071000 con 0x7f79340fce60 2024-04-24T23:09:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.843+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793404c710 con 0x7f79340fce60 2024-04-24T23:09:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.844+0000 7f7922ffd700 1 -- 172.21.15.31:0/1957865746 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7924013420 con 0x7f79340fce60 2024-04-24T23:09:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.844+0000 7f7922ffd700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f791c0692a0 0x7f791c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.844+0000 7f7922ffd700 1 -- 172.21.15.31:0/1957865746 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7924087b40 con 0x7f79340fce60 2024-04-24T23:09:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.844+0000 7f7931d9b700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f791c0692a0 0x7f791c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:33.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.845+0000 7f7931d9b700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f791c0692a0 0x7f791c06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7928007fd0 tx=0x7f7928009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:33.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:33.848+0000 7f7922ffd700 1 -- 172.21.15.31:0/1957865746 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7924057ce0 con 0x7f79340fce60 2024-04-24T23:09:34.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.143+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f79341058c0 con 0x7f79340fce60 2024-04-24T23:09:34.145 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:33 smithi031 ceph-mon[26411]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:34.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.147+0000 7f7922ffd700 1 -- 172.21.15.31:0/1957865746 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f7924057870 con 0x7f79340fce60 2024-04-24T23:09:34.148 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-04-24T23:09:34.148 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAukSlm/zCcCBAAmoeRXtca9CdZKy9GGnnVDg== 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f791c0692a0 msgr2=0x7f791c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f791c0692a0 0x7f791c06b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7928007fd0 tx=0x7f7928009040 comp rx=0 tx=0).stop 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 msgr2=0x7f7934071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 0x7f7934071c00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7924009150 tx=0x7f7924009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 shutdown_connections 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f791c0692a0 0x7f791c06b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79340fce60 0x7f7934071c00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 --2- 172.21.15.31:0/1957865746 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f79340fdfd0 0x7f79340702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 >> 172.21.15.31:0/1957865746 conn(0x7f79340f8540 msgr2=0x7f79340f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:34.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 shutdown_connections 2024-04-24T23:09:34.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:34.149+0000 7f7938f19700 1 -- 172.21.15.31:0/1957865746 wait complete. 2024-04-24T23:09:34.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:33 smithi040 ceph-mon[33128]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:34.809 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:09:34.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-24T23:09:34.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T23:09:34.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-24T23:09:35.121 INFO:teuthology.orchestra.run.smithi040.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi040/config 2024-04-24T23:09:35.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:34 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:09:35.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:34 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1957865746' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:09:35.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:34 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1957865746' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:09:35.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:34 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:09:35.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:34 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1957865746' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:09:35.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:34 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1957865746' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:09:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:36 smithi040 ceph-mon[33128]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:36 smithi031 ceph-mon[26411]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:36.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 -- 172.21.15.40:0/987822258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fdfd0 msgr2=0x7f82b80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:36.706 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 --2- 172.21.15.40:0/987822258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fdfd0 0x7f82b80fe440 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f82ac004940 tx=0x7f82ac009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 -- 172.21.15.40:0/987822258 shutdown_connections 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 --2- 172.21.15.40:0/987822258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fdfd0 0x7f82b80fe440 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 --2- 172.21.15.40:0/987822258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b80fce60 0x7f82b80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 -- 172.21.15.40:0/987822258 >> 172.21.15.40:0/987822258 conn(0x7f82b80f8540 msgr2=0x7f82b80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 -- 172.21.15.40:0/987822258 shutdown_connections 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 -- 172.21.15.40:0/987822258 wait complete. 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 Processor -- start 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.704+0000 7f82bcf4a700 1 -- start start 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82bcf4a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fce60 0x7f82b8071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82bcf4a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b80fdfd0 0x7f82b80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82bcf4a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b8072200 con 0x7f82b80fdfd0 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82bcf4a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80707f0 con 0x7f82b80fce60 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82b659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fce60 0x7f82b8071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:36.707 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82b659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fce60 0x7f82b8071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:60394/0 (socket says 172.21.15.40:60394) 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82b659c700 1 -- 172.21.15.40:0/463548724 learned_addr learned my addr 172.21.15.40:0/463548724 (peer_addr_for_me v2:172.21.15.40:0/0) 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82b659c700 1 -- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b80fdfd0 msgr2=0x7f82b80702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82b659c700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b80fdfd0 0x7f82b80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82b659c700 1 -- 172.21.15.40:0/463548724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac005040 con 0x7f82b80fce60 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82b659c700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fce60 0x7f82b8071c00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82a800c8c0 tx=0x7f82a80044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82a6ffd700 1 -- 172.21.15.40:0/463548724 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82a800a980 con 0x7f82b80fce60 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82a6ffd700 1 -- 172.21.15.40:0/463548724 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82a800ce60 con 0x7f82b80fce60 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b8070aa0 con 0x7f82b80fce60 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82a6ffd700 1 -- 172.21.15.40:0/463548724 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82a80115e0 con 0x7f82b80fce60 2024-04-24T23:09:36.708 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.705+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b8071060 con 0x7f82b80fce60 2024-04-24T23:09:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.706+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b804c710 con 0x7f82b80fce60 2024-04-24T23:09:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.707+0000 7f82a6ffd700 1 -- 172.21.15.40:0/463548724 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f82a8012400 con 0x7f82b80fce60 2024-04-24T23:09:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.707+0000 7f82a6ffd700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f82a00692a0 0x7f82a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.707+0000 7f82a6ffd700 1 -- 172.21.15.40:0/463548724 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f82a8086f60 con 0x7f82b80fce60 2024-04-24T23:09:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.708+0000 7f82b5d9b700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f82a00692a0 0x7f82a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:36.709 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.708+0000 7f82b5d9b700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f82a00692a0 0x7f82a006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f82ac004f90 tx=0x7f82ac009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:36.713 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.712+0000 7f82a6ffd700 1 -- 172.21.15.40:0/463548724 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f82a8057060 con 0x7f82b80fce60 2024-04-24T23:09:36.988 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.987+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f82b81058c0 con 0x7f82b80fce60 2024-04-24T23:09:36.993 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.993+0000 7f82a6ffd700 1 -- 172.21.15.40:0/463548724 <== mon.1 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7f82a80126b0 con 0x7f82b80fce60 2024-04-24T23:09:36.994 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2024-04-24T23:09:36.994 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQAwkSlmUQ/2OhAAaEm7d8UJSrr/hCxAJe8ksA== 2024-04-24T23:09:36.995 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.994+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f82a00692a0 msgr2=0x7f82a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f82a00692a0 0x7f82a006b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f82ac004f90 tx=0x7f82ac009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fce60 msgr2=0x7f82b8071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fce60 0x7f82b8071c00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82a800c8c0 tx=0x7f82a80044c0 comp rx=0 tx=0).stop 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 shutdown_connections 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82b80fce60 0x7f82b8071c00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f82a00692a0 0x7f82a006b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 --2- 172.21.15.40:0/463548724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82b80fdfd0 0x7f82b80702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 >> 172.21.15.40:0/463548724 conn(0x7f82b80f8540 msgr2=0x7f82b80f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 shutdown_connections 2024-04-24T23:09:36.996 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:09:36.995+0000 7f82bcf4a700 1 -- 172.21.15.40:0/463548724 wait complete. 2024-04-24T23:09:37.302 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:37 smithi040 ceph-mon[33128]: from='client.? 172.21.15.40:0/463548724' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:09:37.302 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:37 smithi040 ceph-mon[33128]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:09:37.302 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:37 smithi040 ceph-mon[33128]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:09:37.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:37 smithi031 ceph-mon[26411]: from='client.? 172.21.15.40:0/463548724' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:09:37.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:37 smithi031 ceph-mon[26411]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-24T23:09:37.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:37 smithi031 ceph-mon[26411]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-24T23:09:37.588 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:09:37.588 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-24T23:09:37.588 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T23:09:37.626 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T23:09:37.626 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-24T23:09:37.627 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-24T23:09:37.901 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:38.288 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:38 smithi031 ceph-mon[26411]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:38.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:38 smithi040 ceph-mon[33128]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.395+0000 7fd8b5471700 1 -- 172.21.15.31:0/1626193866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fe7b0 msgr2=0x7fd8b0106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:39.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.395+0000 7fd8b5471700 1 --2- 172.21.15.31:0/1626193866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fe7b0 0x7fd8b0106df0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0004940 tx=0x7fd8a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.395+0000 7fd8b5471700 1 -- 172.21.15.31:0/1626193866 shutdown_connections 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.395+0000 7fd8b5471700 1 --2- 172.21.15.31:0/1626193866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fe7b0 0x7fd8b0106df0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.395+0000 7fd8b5471700 1 --2- 172.21.15.31:0/1626193866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b00fde10 0x7fd8b00fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.395+0000 7fd8b5471700 1 -- 172.21.15.31:0/1626193866 >> 172.21.15.31:0/1626193866 conn(0x7fd8b00f7820 msgr2=0x7fd8b00f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.395+0000 7fd8b5471700 1 -- 172.21.15.31:0/1626193866 shutdown_connections 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.395+0000 7fd8b5471700 1 -- 172.21.15.31:0/1626193866 wait complete. 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8b5471700 1 Processor -- start 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8b5471700 1 -- start start 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8b5471700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fde10 0x7fd8b0113c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8b5471700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b00fe7b0 0x7fd8b01141b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8b5471700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0114800 con 0x7fd8b00fde10 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8b5471700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b010dde0 con 0x7fd8b00fe7b0 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fde10 0x7fd8b0113c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8aeffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fde10 0x7fd8b0113c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52844/0 (socket says 172.21.15.31:52844) 2024-04-24T23:09:39.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8aeffd700 1 -- 172.21.15.31:0/628117663 learned_addr learned my addr 172.21.15.31:0/628117663 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8ae7fc700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b00fe7b0 0x7fd8b01141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8aeffd700 1 -- 172.21.15.31:0/628117663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b00fe7b0 msgr2=0x7fd8b01141b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8aeffd700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b00fe7b0 0x7fd8b01141b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.396+0000 7fd8aeffd700 1 -- 172.21.15.31:0/628117663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a0005040 con 0x7fd8b00fde10 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.397+0000 7fd8aeffd700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fde10 0x7fd8b0113c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c830 tx=0x7fd8a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.397+0000 7fd8977fe700 1 -- 172.21.15.31:0/628117663 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a40106f0 con 0x7fd8b00fde10 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.397+0000 7fd8977fe700 1 -- 172.21.15.31:0/628117663 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8a4010d20 con 0x7fd8b00fde10 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.397+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b010e0c0 con 0x7fd8b00fde10 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.397+0000 7fd8977fe700 1 -- 172.21.15.31:0/628117663 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8a40113a0 con 0x7fd8b00fde10 2024-04-24T23:09:39.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.397+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b010e600 con 0x7fd8b00fde10 2024-04-24T23:09:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.398+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0066b30 con 0x7fd8b00fde10 2024-04-24T23:09:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.398+0000 7fd8977fe700 1 -- 172.21.15.31:0/628117663 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd8a4012400 con 0x7fd8b00fde10 2024-04-24T23:09:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.399+0000 7fd8977fe700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd8980692a0 0x7fd89806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.399+0000 7fd8977fe700 1 -- 172.21.15.31:0/628117663 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd8a4086d30 con 0x7fd8b00fde10 2024-04-24T23:09:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.399+0000 7fd8ae7fc700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd8980692a0 0x7fd89806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:39.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.399+0000 7fd8ae7fc700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd8980692a0 0x7fd89806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd8b010f510 tx=0x7fd8a0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:39.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.403+0000 7fd8977fe700 1 -- 172.21.15.31:0/628117663 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd8a4056ed0 con 0x7fd8b00fde10 2024-04-24T23:09:39.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.653+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd8b004c710 con 0x7fd8b00fde10 2024-04-24T23:09:39.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.656+0000 7fd8977fe700 1 -- 172.21.15.31:0/628117663 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167176 (secure 0 0 0) 0x7fd8a4056a60 con 0x7fd8b00fde10 2024-04-24T23:09:39.661 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:09:39.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd8980692a0 msgr2=0x7fd89806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:39.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd8980692a0 0x7fd89806b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd8b010f510 tx=0x7fd8a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:39.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fde10 msgr2=0x7fd8b0113c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:39.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fde10 0x7fd8b0113c70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c830 tx=0x7fd8a4004470 comp rx=0 tx=0).stop 2024-04-24T23:09:39.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 shutdown_connections 2024-04-24T23:09:39.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd8980692a0 0x7fd89806b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd8b00fde10 0x7fd8b0113c70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 --2- 172.21.15.31:0/628117663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8b00fe7b0 0x7fd8b01141b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:39.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 >> 172.21.15.31:0/628117663 conn(0x7fd8b00f7820 msgr2=0x7fd8b00f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:39.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 shutdown_connections 2024-04-24T23:09:39.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:39.659+0000 7fd8b5471700 1 -- 172.21.15.31:0/628117663 wait complete. 2024-04-24T23:09:40.333 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:40 smithi031 ceph-mon[26411]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:40.334 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:40 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/628117663' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:09:40.361 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"flags":0,"active_gid":14208,"active_name":"smithi031.rrnmwr","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1061697388},{"type":"v1","addr":"172.21.15.31:6801","nonce":1061697388}]},"active_addr":"172.21.15.31:6801/1061697388","active_change":"2024-04-24T23:05:48.958479+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14228,"name":"smithi040.qtdwuq","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/","prometheus":"http://172.21.15.31:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":823375198}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2443214115}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":582471448}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":271213718}]}]} 2024-04-24T23:09:40.365 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-24T23:09:40.365 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-24T23:09:40.365 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:09:40.632 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:40.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:40 smithi040 ceph-mon[33128]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:40.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:40 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/628117663' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-24T23:09:42.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 -- 172.21.15.31:0/3771647638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 msgr2=0x7fb104106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 --2- 172.21.15.31:0/3771647638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 0x7fb104106eb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8004940 tx=0x7fb0f8009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 -- 172.21.15.31:0/3771647638 shutdown_connections 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 --2- 172.21.15.31:0/3771647638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 0x7fb104106eb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 --2- 172.21.15.31:0/3771647638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1040fde40 0x7fb1040fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 -- 172.21.15.31:0/3771647638 >> 172.21.15.31:0/3771647638 conn(0x7fb1040f7820 msgr2=0x7fb1040f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 -- 172.21.15.31:0/3771647638 shutdown_connections 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 -- 172.21.15.31:0/3771647638 wait complete. 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 Processor -- start 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.168+0000 7fb1092a2700 1 -- start start 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1092a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1040fde40 0x7fb104115f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1092a2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 0x7fb104116470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1092a2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104116ac0 con 0x7fb1040fe7e0 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1092a2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041100a0 con 0x7fb1040fde40 2024-04-24T23:09:42.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1027fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 0x7fb104116470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1027fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 0x7fb104116470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52870/0 (socket says 172.21.15.31:52870) 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1027fc700 1 -- 172.21.15.31:0/3591201410 learned_addr learned my addr 172.21.15.31:0/3591201410 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1027fc700 1 -- 172.21.15.31:0/3591201410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1040fde40 msgr2=0x7fb104115f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1027fc700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1040fde40 0x7fb104115f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1027fc700 1 -- 172.21.15.31:0/3591201410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f8005040 con 0x7fb1040fe7e0 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1027fc700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 0x7fb104116470 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80093b0 tx=0x7fb0f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb0eb7fe700 1 -- 172.21.15.31:0/3591201410 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f800e040 con 0x7fb1040fe7e0 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb0eb7fe700 1 -- 172.21.15.31:0/3591201410 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0f8004290 con 0x7fb1040fe7e0 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb104110320 con 0x7fb1040fe7e0 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb0eb7fe700 1 -- 172.21.15.31:0/3591201410 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0f801c3f0 con 0x7fb1040fe7e0 2024-04-24T23:09:42.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.169+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb104110860 con 0x7fb1040fe7e0 2024-04-24T23:09:42.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.170+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb104066a70 con 0x7fb1040fe7e0 2024-04-24T23:09:42.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.171+0000 7fb0eb7fe700 1 -- 172.21.15.31:0/3591201410 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb0f8013890 con 0x7fb1040fe7e0 2024-04-24T23:09:42.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.171+0000 7fb0eb7fe700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0ec0692a0 0x7fb0ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:42.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.171+0000 7fb0eb7fe700 1 -- 172.21.15.31:0/3591201410 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb0f8088040 con 0x7fb1040fe7e0 2024-04-24T23:09:42.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.171+0000 7fb102ffd700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0ec0692a0 0x7fb0ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:42.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.172+0000 7fb102ffd700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0ec0692a0 0x7fb0ec06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4004d00 tx=0x7fb0f4009ce0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:42.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.176+0000 7fb0eb7fe700 1 -- 172.21.15.31:0/3591201410 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb0f80581e0 con 0x7fb1040fe7e0 2024-04-24T23:09:42.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:42 smithi031 ceph-mon[26411]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.371+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb104109ee0 con 0x7fb1040fe7e0 2024-04-24T23:09:42.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.372+0000 7fb0eb7fe700 1 -- 172.21.15.31:0/3591201410 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11033 (secure 0 0 0) 0x7fb0f8057d70 con 0x7fb1040fe7e0 2024-04-24T23:09:42.374 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:09:42.374 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:03:41.065781+0000","modified":"2024-04-24T23:09:23.859599+0000","last_up_change":"2024-04-24T23:09:22.852760+0000","last_in_change":"2024-04-24T23:09:04.280370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:08:05.473853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0f1ff679-6b2e-4059-a299-2d2356bbe91c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6803","nonce":1397311567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6805","nonce":1397311567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6809","nonce":1397311567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6807","nonce":1397311567}]},"public_addr":"172.21.15.31:6803/1397311567","cluster_addr":"172.21.15.31:6805/1397311567","heartbeat_back_addr":"172.21.15.31:6809/1397311567","heartbeat_front_addr":"172.21.15.31:6807/1397311567","state":["exists","up"]},{"osd":1,"uuid":"6885f0d2-70e8-4c54-a381-0b3683a0bc49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6811","nonce":1768854803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6813","nonce":1768854803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6817","nonce":1768854803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6815","nonce":1768854803}]},"public_addr":"172.21.15.31:6811/1768854803","cluster_addr":"172.21.15.31:6813/1768854803","heartbeat_back_addr":"172.21.15.31:6817/1768854803","heartbeat_front_addr":"172.21.15.31:6815/1768854803","state":["exists","up"]},{"osd":2,"uuid":"9b57c743-5826-4bf9-ac00-0cb539b30224","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6819","nonce":563563256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6821","nonce":563563256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6825","nonce":563563256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6823","nonce":563563256}]},"public_addr":"172.21.15.31:6819/563563256","cluster_addr":"172.21.15.31:6821/563563256","heartbeat_back_addr":"172.21.15.31:6825/563563256","heartbeat_front_addr":"172.21.15.31:6823/563563256","state":["exists","up"]},{"osd":3,"uuid":"1dc47b54-65da-4579-bcac-2258be1f4cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6801","nonce":824499182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6803","nonce":824499182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6807","nonce":824499182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6805","nonce":824499182}]},"public_addr":"172.21.15.40:6801/824499182","cluster_addr":"172.21.15.40:6803/824499182","heartbeat_back_addr":"172.21.15.40:6807/824499182","heartbeat_front_addr":"172.21.15.40:6805/824499182","state":["exists","up"]},{"osd":4,"uuid":"e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6809","nonce":1464558178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6811","nonce":1464558178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6815","nonce":1464558178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6813","nonce":1464558178}]},"public_addr":"172.21.15.40:6809/1464558178","cluster_addr":"172.21.15.40:6811/1464558178","heartbeat_back_addr":"172.21.15.40:6815/1464558178","heartbeat_front_addr":"172.21.15.40:6813/1464558178","state":["exists","up"]},{"osd":5,"uuid":"027b7b7a-afb6-4507-8cb2-274ebadbb2ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6817","nonce":2359746556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6819","nonce":2359746556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6823","nonce":2359746556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6821","nonce":2359746556}]},"public_addr":"172.21.15.40:6817/2359746556","cluster_addr":"172.21.15.40:6819/2359746556","heartbeat_back_addr":"172.21.15.40:6823/2359746556","heartbeat_front_addr":"172.21.15.40:6821/2359746556","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:08.221856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:36.407326+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:03.612047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:29.069233+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:55.660200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:21.507297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1805904019":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/39610504":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6801/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6800/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/2457576751":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6801/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6800/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/2308793407":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6801/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/382933206":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/1340983696":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/3645410271":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/611668999":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/1654686211":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6800/4106752127":"2024-04-25T23:04:15.327046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:09:42.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0ec0692a0 msgr2=0x7fb0ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.376 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0ec0692a0 0x7fb0ec06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4004d00 tx=0x7fb0f4009ce0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 msgr2=0x7fb104116470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 0x7fb104116470 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80093b0 tx=0x7fb0f8009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 shutdown_connections 2024-04-24T23:09:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1040fde40 0x7fb104115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0ec0692a0 0x7fb0ec06b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 --2- 172.21.15.31:0/3591201410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1040fe7e0 0x7fb104116470 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:42.377 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 >> 172.21.15.31:0/3591201410 conn(0x7fb1040f7820 msgr2=0x7fb1040fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 shutdown_connections 2024-04-24T23:09:42.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:42.374+0000 7fb1092a2700 1 -- 172.21.15.31:0/3591201410 wait complete. 2024-04-24T23:09:42.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:42 smithi040 ceph-mon[33128]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:43.003 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-24T23:09:43.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-24T23:09:43.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:43 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3591201410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:43.272 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:43 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3591201410' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:44.453 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:44 smithi031 ceph-mon[26411]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:44 smithi040 ceph-mon[33128]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.809+0000 7f2f554d3700 1 -- 172.21.15.31:0/2242546086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f501026a0 msgr2=0x7f2f500fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.809+0000 7f2f554d3700 1 --2- 172.21.15.31:0/2242546086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f501026a0 0x7f2f500fbd10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f44004940 tx=0x7f2f44009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 -- 172.21.15.31:0/2242546086 shutdown_connections 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 --2- 172.21.15.31:0/2242546086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f501026a0 0x7f2f500fbd10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 --2- 172.21.15.31:0/2242546086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f50101d00 0x7f2f501020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 -- 172.21.15.31:0/2242546086 >> 172.21.15.31:0/2242546086 conn(0x7f2f500f7870 msgr2=0x7f2f500f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 -- 172.21.15.31:0/2242546086 shutdown_connections 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 -- 172.21.15.31:0/2242546086 wait complete. 2024-04-24T23:09:44.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 Processor -- start 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 -- start start 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101d00 0x7f2f5010eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f501026a0 0x7f2f5010f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5010f5e0 con 0x7f2f50101d00 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.810+0000 7f2f554d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5010f720 con 0x7f2f501026a0 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f4effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101d00 0x7f2f5010eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f4effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101d00 0x7f2f5010eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52900/0 (socket says 172.21.15.31:52900) 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f4effd700 1 -- 172.21.15.31:0/3301932522 learned_addr learned my addr 172.21.15.31:0/3301932522 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f4e7fc700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f501026a0 0x7f2f5010f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f4effd700 1 -- 172.21.15.31:0/3301932522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f501026a0 msgr2=0x7f2f5010f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f4effd700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f501026a0 0x7f2f5010f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:44.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f4effd700 1 -- 172.21.15.31:0/3301932522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44005040 con 0x7f2f50101d00 2024-04-24T23:09:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f4effd700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101d00 0x7f2f5010eb60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c830 tx=0x7f2f40004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f377fe700 1 -- 172.21.15.31:0/3301932522 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f400106f0 con 0x7f2f50101d00 2024-04-24T23:09:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f50117bd0 con 0x7f2f50101d00 2024-04-24T23:09:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f377fe700 1 -- 172.21.15.31:0/3301932522 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f40010d20 con 0x7f2f50101d00 2024-04-24T23:09:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f377fe700 1 -- 172.21.15.31:0/3301932522 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f400113a0 con 0x7f2f50101d00 2024-04-24T23:09:44.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.811+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f50117f20 con 0x7f2f50101d00 2024-04-24T23:09:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.812+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f500fd450 con 0x7f2f50101d00 2024-04-24T23:09:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.813+0000 7f2f377fe700 1 -- 172.21.15.31:0/3301932522 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2f40012400 con 0x7f2f50101d00 2024-04-24T23:09:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.813+0000 7f2f377fe700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f2f380692a0 0x7f2f3806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.813+0000 7f2f377fe700 1 -- 172.21.15.31:0/3301932522 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2f40086e00 con 0x7f2f50101d00 2024-04-24T23:09:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.813+0000 7f2f4e7fc700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f2f380692a0 0x7f2f3806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:44.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.814+0000 7f2f4e7fc700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f2f380692a0 0x7f2f3806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f44004890 tx=0x7f2f44009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:44.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:44.818+0000 7f2f377fe700 1 -- 172.21.15.31:0/3301932522 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2f40053470 con 0x7f2f50101d00 2024-04-24T23:09:45.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.014+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f500667f0 con 0x7f2f50101d00 2024-04-24T23:09:45.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.015+0000 7f2f377fe700 1 -- 172.21.15.31:0/3301932522 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11033 (secure 0 0 0) 0x7f2f40056a90 con 0x7f2f50101d00 2024-04-24T23:09:45.017 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:09:45.017 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":40,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:03:41.065781+0000","modified":"2024-04-24T23:09:23.859599+0000","last_up_change":"2024-04-24T23:09:22.852760+0000","last_in_change":"2024-04-24T23:09:04.280370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:08:05.473853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"0f1ff679-6b2e-4059-a299-2d2356bbe91c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6803","nonce":1397311567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6805","nonce":1397311567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6809","nonce":1397311567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6807","nonce":1397311567}]},"public_addr":"172.21.15.31:6803/1397311567","cluster_addr":"172.21.15.31:6805/1397311567","heartbeat_back_addr":"172.21.15.31:6809/1397311567","heartbeat_front_addr":"172.21.15.31:6807/1397311567","state":["exists","up"]},{"osd":1,"uuid":"6885f0d2-70e8-4c54-a381-0b3683a0bc49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6811","nonce":1768854803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6813","nonce":1768854803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6817","nonce":1768854803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6815","nonce":1768854803}]},"public_addr":"172.21.15.31:6811/1768854803","cluster_addr":"172.21.15.31:6813/1768854803","heartbeat_back_addr":"172.21.15.31:6817/1768854803","heartbeat_front_addr":"172.21.15.31:6815/1768854803","state":["exists","up"]},{"osd":2,"uuid":"9b57c743-5826-4bf9-ac00-0cb539b30224","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6819","nonce":563563256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6821","nonce":563563256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6825","nonce":563563256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6823","nonce":563563256}]},"public_addr":"172.21.15.31:6819/563563256","cluster_addr":"172.21.15.31:6821/563563256","heartbeat_back_addr":"172.21.15.31:6825/563563256","heartbeat_front_addr":"172.21.15.31:6823/563563256","state":["exists","up"]},{"osd":3,"uuid":"1dc47b54-65da-4579-bcac-2258be1f4cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6801","nonce":824499182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6803","nonce":824499182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6807","nonce":824499182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6805","nonce":824499182}]},"public_addr":"172.21.15.40:6801/824499182","cluster_addr":"172.21.15.40:6803/824499182","heartbeat_back_addr":"172.21.15.40:6807/824499182","heartbeat_front_addr":"172.21.15.40:6805/824499182","state":["exists","up"]},{"osd":4,"uuid":"e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6809","nonce":1464558178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6811","nonce":1464558178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6815","nonce":1464558178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6813","nonce":1464558178}]},"public_addr":"172.21.15.40:6809/1464558178","cluster_addr":"172.21.15.40:6811/1464558178","heartbeat_back_addr":"172.21.15.40:6815/1464558178","heartbeat_front_addr":"172.21.15.40:6813/1464558178","state":["exists","up"]},{"osd":5,"uuid":"027b7b7a-afb6-4507-8cb2-274ebadbb2ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6817","nonce":2359746556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6819","nonce":2359746556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6823","nonce":2359746556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6821","nonce":2359746556}]},"public_addr":"172.21.15.40:6817/2359746556","cluster_addr":"172.21.15.40:6819/2359746556","heartbeat_back_addr":"172.21.15.40:6823/2359746556","heartbeat_front_addr":"172.21.15.40:6821/2359746556","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:08.221856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:36.407326+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:03.612047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:29.069233+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:55.660200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:21.507297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/1805904019":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/39610504":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6801/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6800/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/2457576751":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6801/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6800/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/2308793407":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6801/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/382933206":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/1340983696":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/3645410271":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/611668999":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/1654686211":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6800/4106752127":"2024-04-25T23:04:15.327046+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:09:45.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f2f380692a0 msgr2=0x7f2f3806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:45.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f2f380692a0 0x7f2f3806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2f44004890 tx=0x7f2f44009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101d00 msgr2=0x7f2f5010eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101d00 0x7f2f5010eb60 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c830 tx=0x7f2f40004470 comp rx=0 tx=0).stop 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 shutdown_connections 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f2f380692a0 0x7f2f3806b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f50101d00 0x7f2f5010eb60 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 --2- 172.21.15.31:0/3301932522 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2f501026a0 0x7f2f5010f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 >> 172.21.15.31:0/3301932522 conn(0x7f2f500f7870 msgr2=0x7f2f500f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 shutdown_connections 2024-04-24T23:09:45.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:45.017+0000 7f2f554d3700 1 -- 172.21.15.31:0/3301932522 wait complete. 2024-04-24T23:09:45.419 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:45 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3301932522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:45.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-24T23:09:45.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-24T23:09:45.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-24T23:09:45.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-24T23:09:45.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-24T23:09:45.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-24T23:09:45.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:45 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3301932522' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:09:45.904 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:45.907 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:46.308 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:46 smithi031 ceph-mon[26411]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:46.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:46 smithi040 ceph-mon[33128]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:47.545 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:48.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:48 smithi031 ceph-mon[26411]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:48.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.510+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/3727173451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 msgr2=0x7fa4e0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:48.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.510+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/3727173451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 0x7fa4e0102160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc004940 tx=0x7fa4dc009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:48.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.510+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/3727173451 shutdown_connections 2024-04-24T23:09:48.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.510+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/3727173451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4e00fbd40 0x7fa4e00fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:48.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.510+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/3727173451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 0x7fa4e0102160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:48.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.510+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/3727173451 >> 172.21.15.31:0/3727173451 conn(0x7fa4e0071d00 msgr2=0x7fa4e0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.510+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/3727173451 shutdown_connections 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.510+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/3727173451 wait complete. 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e7b3d700 1 Processor -- start 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e7b3d700 1 -- start start 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e7b3d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4e00fbd40 0x7fa4e0106b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e7b3d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 0x7fa4e01070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e7b3d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e0107630 con 0x7fa4e0101d90 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e7b3d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e0107770 con 0x7fa4e00fbd40 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e50d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 0x7fa4e01070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e50d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 0x7fa4e01070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52910/0 (socket says 172.21.15.31:52910) 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e50d8700 1 -- 172.21.15.31:0/2212749189 learned_addr learned my addr 172.21.15.31:0/2212749189 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e58d9700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4e00fbd40 0x7fa4e0106b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:48.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e50d8700 1 -- 172.21.15.31:0/2212749189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4e00fbd40 msgr2=0x7fa4e0106b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e50d8700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4e00fbd40 0x7fa4e0106b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e50d8700 1 -- 172.21.15.31:0/2212749189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc005040 con 0x7fa4e0101d90 2024-04-24T23:09:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.511+0000 7fa4e50d8700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 0x7fa4e01070a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000eb40 tx=0x7fa4d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.512+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d0007cc0 con 0x7fa4e0101d90 2024-04-24T23:09:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.512+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e01079c0 con 0x7fa4e0101d90 2024-04-24T23:09:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.512+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4d000f040 con 0x7fa4e0101d90 2024-04-24T23:09:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.512+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d0009620 con 0x7fa4e0101d90 2024-04-24T23:09:48.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.512+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e011a150 con 0x7fa4e0101d90 2024-04-24T23:09:48.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.513+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa4e005e660 con 0x7fa4e0101d90 2024-04-24T23:09:48.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.513+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa4d0009780 con 0x7fa4e0101d90 2024-04-24T23:09:48.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4d67fc700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa4cc0692a0 0x7fa4cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:48.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4e58d9700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa4cc0692a0 0x7fa4cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:48.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa4d0086af0 con 0x7fa4e0101d90 2024-04-24T23:09:48.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4d67fc700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] conn(0x7fa4cc06ef70 0x7fa4cc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 --> [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa4cc071a30 con 0x7fa4cc06ef70 2024-04-24T23:09:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4d008b220 con 0x7fa4e0101d90 2024-04-24T23:09:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4e60da700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] conn(0x7fa4cc06ef70 0x7fa4cc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4e58d9700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa4cc0692a0 0x7fa4cc06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00b0d0 tx=0x7fa4dc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.514+0000 7fa4e60da700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] conn(0x7fa4cc06ef70 0x7fa4cc071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:48.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.515+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 <== osd.2 v2:172.21.15.31:6818/563563256 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fa4cc071a30 con 0x7fa4cc06ef70 2024-04-24T23:09:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.530+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 --> [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa4e005eae0 con 0x7fa4cc06ef70 2024-04-24T23:09:48.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.530+0000 7fa4d67fc700 1 -- 172.21.15.31:0/2212749189 <== osd.2 v2:172.21.15.31:6818/563563256 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa4e005eae0 con 0x7fa4cc06ef70 2024-04-24T23:09:48.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.530+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] conn(0x7fa4cc06ef70 msgr2=0x7fa4cc071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.530+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] conn(0x7fa4cc06ef70 0x7fa4cc071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa4cc0692a0 msgr2=0x7fa4cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa4cc0692a0 0x7fa4cc06b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00b0d0 tx=0x7fa4dc009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 msgr2=0x7fa4e01070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 0x7fa4e01070a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000eb40 tx=0x7fa4d000b040 comp rx=0 tx=0).stop 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 shutdown_connections 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6818/563563256,v1:172.21.15.31:6819/563563256] conn(0x7fa4cc06ef70 0x7fa4cc071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4e00fbd40 0x7fa4e0106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa4cc0692a0 0x7fa4cc06b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:48.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 --2- 172.21.15.31:0/2212749189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e0101d90 0x7fa4e01070a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:48.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 >> 172.21.15.31:0/2212749189 conn(0x7fa4e0071d00 msgr2=0x7fa4e00faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:48.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 shutdown_connections 2024-04-24T23:09:48.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:48.531+0000 7fa4e7b3d700 1 -- 172.21.15.31:0/2212749189 wait complete. 2024-04-24T23:09:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:48 smithi040 ceph-mon[33128]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:49.397 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:09:49.425 INFO:teuthology.orchestra.run.smithi031.stdout:81604378648 2024-04-24T23:09:49.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-24T23:09:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.575+0000 7ff97cff0700 1 -- 172.21.15.31:0/3651659022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9780fc190 msgr2=0x7ff978101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.575+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3651659022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9780fc190 0x7ff978101a70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff96c004940 tx=0x7ff96c009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.576+0000 7ff97cff0700 1 -- 172.21.15.31:0/3651659022 shutdown_connections 2024-04-24T23:09:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.576+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3651659022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9780fc190 0x7ff978101a70 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.576+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3651659022 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9780fb7f0 0x7ff9780fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.576+0000 7ff97cff0700 1 -- 172.21.15.31:0/3651659022 >> 172.21.15.31:0/3651659022 conn(0x7ff978071460 msgr2=0x7ff978071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.576+0000 7ff97cff0700 1 -- 172.21.15.31:0/3651659022 shutdown_connections 2024-04-24T23:09:49.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.576+0000 7ff97cff0700 1 -- 172.21.15.31:0/3651659022 wait complete. 2024-04-24T23:09:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97cff0700 1 Processor -- start 2024-04-24T23:09:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97cff0700 1 -- start start 2024-04-24T23:09:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97cff0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9780fb7f0 0x7ff9781139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97cff0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9780fc190 0x7ff978113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:49.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97cff0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff978114530 con 0x7ff9780fc190 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97cff0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97810db50 con 0x7ff9780fb7f0 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff975d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9780fc190 0x7ff978113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9780fb7f0 0x7ff9781139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97659c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9780fb7f0 0x7ff9781139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:54732/0 (socket says 172.21.15.31:54732) 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.577+0000 7ff97659c700 1 -- 172.21.15.31:0/3272468635 learned_addr learned my addr 172.21.15.31:0/3272468635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff97659c700 1 -- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9780fc190 msgr2=0x7ff978113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff97659c700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9780fc190 0x7ff978113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff97659c700 1 -- 172.21.15.31:0/3272468635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c005040 con 0x7ff9780fb7f0 2024-04-24T23:09:49.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff97659c700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9780fb7f0 0x7ff9781139a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c830 tx=0x7ff968004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9680106f0 con 0x7ff9780fb7f0 2024-04-24T23:09:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97810ddd0 con 0x7ff9780fb7f0 2024-04-24T23:09:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff968010d20 con 0x7ff9780fb7f0 2024-04-24T23:09:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9680113a0 con 0x7ff9780fb7f0 2024-04-24T23:09:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.578+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97810e2e0 con 0x7ff9780fb7f0 2024-04-24T23:09:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.579+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff97810e810 con 0x7ff9780fb7f0 2024-04-24T23:09:49.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.580+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff968010850 con 0x7ff9780fb7f0 2024-04-24T23:09:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.581+0000 7ff966ffd700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff9600692a0 0x7ff96006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:49.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.581+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff968086ef0 con 0x7ff9780fb7f0 2024-04-24T23:09:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.581+0000 7ff975d9b700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff9600692a0 0x7ff96006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.581+0000 7ff966ffd700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] conn(0x7ff96006ef70 0x7ff960071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.581+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 --> [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff960071a30 con 0x7ff96006ef70 2024-04-24T23:09:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.581+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff968087260 con 0x7ff9780fb7f0 2024-04-24T23:09:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.581+0000 7ff976d9d700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] conn(0x7ff96006ef70 0x7ff960071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.581+0000 7ff975d9b700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff9600692a0 0x7ff96006b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b3b0 tx=0x7ff96c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:49.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.582+0000 7ff976d9d700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] conn(0x7ff96006ef70 0x7ff960071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:49.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.583+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 <== osd.0 v2:172.21.15.31:6802/1397311567 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff960071a30 con 0x7ff96006ef70 2024-04-24T23:09:49.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.611+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 --> [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff97810e9d0 con 0x7ff96006ef70 2024-04-24T23:09:49.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.611+0000 7ff966ffd700 1 -- 172.21.15.31:0/3272468635 <== osd.0 v2:172.21.15.31:6802/1397311567 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff97810e9d0 con 0x7ff96006ef70 2024-04-24T23:09:49.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.611+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] conn(0x7ff96006ef70 msgr2=0x7ff960071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.611+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] conn(0x7ff96006ef70 0x7ff960071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff9600692a0 msgr2=0x7ff96006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff9600692a0 0x7ff96006b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00b3b0 tx=0x7ff96c009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:49.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9780fb7f0 msgr2=0x7ff9781139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:49.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9780fb7f0 0x7ff9781139a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff96800c830 tx=0x7ff968004470 comp rx=0 tx=0).stop 2024-04-24T23:09:49.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 shutdown_connections 2024-04-24T23:09:49.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6802/1397311567,v1:172.21.15.31:6803/1397311567] conn(0x7ff96006ef70 0x7ff960071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9780fb7f0 0x7ff9781139a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff9600692a0 0x7ff96006b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 --2- 172.21.15.31:0/3272468635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9780fc190 0x7ff978113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:49.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.612+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 >> 172.21.15.31:0/3272468635 conn(0x7ff978071460 msgr2=0x7ff978100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:49.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.613+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 shutdown_connections 2024-04-24T23:09:49.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:49.613+0000 7ff97cff0700 1 -- 172.21.15.31:0/3272468635 wait complete. 2024-04-24T23:09:49.632 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:49.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:09:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:50 smithi031 ceph-mon[26411]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:50 smithi040 ceph-mon[33128]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:51.232 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:52 smithi031 ceph-mon[26411]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:52.303 INFO:teuthology.orchestra.run.smithi031.stdout:38654705699 2024-04-24T23:09:52.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-24T23:09:52.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:52 smithi040 ceph-mon[33128]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.391+0000 7fbb770c6700 1 -- 172.21.15.31:0/4294926973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb70101d10 msgr2=0x7fbb701020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.391+0000 7fbb770c6700 1 --2- 172.21.15.31:0/4294926973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb70101d10 0x7fbb701020e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb64004940 tx=0x7fbb64009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.392+0000 7fbb770c6700 1 -- 172.21.15.31:0/4294926973 shutdown_connections 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.392+0000 7fbb770c6700 1 --2- 172.21.15.31:0/4294926973 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb701026b0 0x7fbb700fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.392+0000 7fbb770c6700 1 --2- 172.21.15.31:0/4294926973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb70101d10 0x7fbb701020e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.392+0000 7fbb770c6700 1 -- 172.21.15.31:0/4294926973 >> 172.21.15.31:0/4294926973 conn(0x7fbb700f7860 msgr2=0x7fbb700f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.392+0000 7fbb770c6700 1 -- 172.21.15.31:0/4294926973 shutdown_connections 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.392+0000 7fbb770c6700 1 -- 172.21.15.31:0/4294926973 wait complete. 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.392+0000 7fbb770c6700 1 Processor -- start 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.392+0000 7fbb770c6700 1 -- start start 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb770c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb70101d10 0x7fbb701128c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb770c6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb701026b0 0x7fbb70112e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:53.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb770c6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb70113450 con 0x7fbb701026b0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb770c6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7010ca50 con 0x7fbb70101d10 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb701026b0 0x7fbb70112e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb701026b0 0x7fbb70112e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51948/0 (socket says 172.21.15.31:51948) 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6ffff700 1 -- 172.21.15.31:0/2037031097 learned_addr learned my addr 172.21.15.31:0/2037031097 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb74e62700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb70101d10 0x7fbb701128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6ffff700 1 -- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb70101d10 msgr2=0x7fbb701128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6ffff700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb70101d10 0x7fbb701128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6ffff700 1 -- 172.21.15.31:0/2037031097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64005040 con 0x7fbb701026b0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6ffff700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb701026b0 0x7fbb70112e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00eaa0 tx=0x7fbb5c004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5c00cb90 con 0x7fbb701026b0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb5c00ccf0 con 0x7fbb701026b0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7010ccd0 con 0x7fbb701026b0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5c010820 con 0x7fbb701026b0 2024-04-24T23:09:53.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.393+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7010d290 con 0x7fbb701026b0 2024-04-24T23:09:53.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.394+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb7010d820 con 0x7fbb701026b0 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.395+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fbb5c010a60 con 0x7fbb701026b0 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.395+0000 7fbb6d7fa700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbb600692a0 0x7fbb6006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.395+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbb5c089680 con 0x7fbb701026b0 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.395+0000 7fbb74e62700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbb600692a0 0x7fbb6006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.395+0000 7fbb6d7fa700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] conn(0x7fbb6006ef70 0x7fbb60071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.395+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 --> [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb60071a30 con 0x7fbb6006ef70 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.395+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb5c08bcd0 con 0x7fbb701026b0 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.396+0000 7fbb74e62700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbb600692a0 0x7fbb6006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400b3b0 tx=0x7fbb64009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.396+0000 7fbb75663700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] conn(0x7fbb6006ef70 0x7fbb60071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.396+0000 7fbb75663700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] conn(0x7fbb6006ef70 0x7fbb60071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:53.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.397+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 <== osd.5 v2:172.21.15.40:6816/2359746556 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbb60071a30 con 0x7fbb6006ef70 2024-04-24T23:09:53.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.412+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 --> [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb700f9210 con 0x7fbb6006ef70 2024-04-24T23:09:53.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.412+0000 7fbb6d7fa700 1 -- 172.21.15.31:0/2037031097 <== osd.5 v2:172.21.15.40:6816/2359746556 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbb700f9210 con 0x7fbb6006ef70 2024-04-24T23:09:53.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] conn(0x7fbb6006ef70 msgr2=0x7fbb60071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:53.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] conn(0x7fbb6006ef70 0x7fbb60071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:53.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbb600692a0 msgr2=0x7fbb6006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:53.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbb600692a0 0x7fbb6006b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400b3b0 tx=0x7fbb64009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb701026b0 msgr2=0x7fbb70112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb701026b0 0x7fbb70112e00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00eaa0 tx=0x7fbb5c004500 comp rx=0 tx=0).stop 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 shutdown_connections 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:6816/2359746556,v1:172.21.15.40:6817/2359746556] conn(0x7fbb6006ef70 0x7fbb60071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb70101d10 0x7fbb701128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fbb600692a0 0x7fbb6006b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 --2- 172.21.15.31:0/2037031097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb701026b0 0x7fbb70112e00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 >> 172.21.15.31:0/2037031097 conn(0x7fbb700f7860 msgr2=0x7fbb700f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 shutdown_connections 2024-04-24T23:09:53.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:53.413+0000 7fbb770c6700 1 -- 172.21.15.31:0/2037031097 wait complete. 2024-04-24T23:09:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:54 smithi031 ceph-mon[26411]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:54 smithi040 ceph-mon[33128]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:54.802 INFO:teuthology.orchestra.run.smithi031.stdout:167503724553 2024-04-24T23:09:54.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.793+0000 7fcd90ab5700 1 -- 172.21.15.31:0/2808339482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c103780 msgr2=0x7fcd8c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.793+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/2808339482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c103780 0x7fcd8c0fcdf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcd74004940 tx=0x7fcd74009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.793+0000 7fcd90ab5700 1 -- 172.21.15.31:0/2808339482 shutdown_connections 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.793+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/2808339482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c103780 0x7fcd8c0fcdf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.793+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/2808339482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd8c102de0 0x7fcd8c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.793+0000 7fcd90ab5700 1 -- 172.21.15.31:0/2808339482 >> 172.21.15.31:0/2808339482 conn(0x7fcd8c0f8950 msgr2=0x7fcd8c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.793+0000 7fcd90ab5700 1 -- 172.21.15.31:0/2808339482 shutdown_connections 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.793+0000 7fcd90ab5700 1 -- 172.21.15.31:0/2808339482 wait complete. 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd90ab5700 1 Processor -- start 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd90ab5700 1 -- start start 2024-04-24T23:09:55.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd90ab5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c102de0 0x7fcd8c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd90ab5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd8c103780 0x7fcd8c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd90ab5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c070d30 con 0x7fcd8c102de0 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd90ab5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c070e70 con 0x7fcd8c103780 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd8a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c102de0 0x7fcd8c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd8a59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c102de0 0x7fcd8c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51980/0 (socket says 172.21.15.31:51980) 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.794+0000 7fcd8a59c700 1 -- 172.21.15.31:0/1756607639 learned_addr learned my addr 172.21.15.31:0/1756607639 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd89d9b700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd8c103780 0x7fcd8c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd8a59c700 1 -- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd8c103780 msgr2=0x7fcd8c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd8a59c700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd8c103780 0x7fcd8c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd8a59c700 1 -- 172.21.15.31:0/1756607639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd74005040 con 0x7fcd8c102de0 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd8a59c700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c102de0 0x7fcd8c070260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c002a50 tx=0x7fcd7c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd7c007cc0 con 0x7fcd8c102de0 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c119e70 con 0x7fcd8c102de0 2024-04-24T23:09:55.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd7c00f040 con 0x7fcd8c102de0 2024-04-24T23:09:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd7c00da30 con 0x7fcd8c102de0 2024-04-24T23:09:55.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.795+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c11a290 con 0x7fcd8c102de0 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.796+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd8c101770 con 0x7fcd8c102de0 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.797+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fcd7c00dbb0 con 0x7fcd8c102de0 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.797+0000 7fcd82ffd700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fcd780692a0 0x7fcd7806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.797+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcd7c087650 con 0x7fcd8c102de0 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.797+0000 7fcd89d9b700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fcd780692a0 0x7fcd7806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.797+0000 7fcd82ffd700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] conn(0x7fcd7806ef70 0x7fcd78071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.797+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 --> [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd78071a30 con 0x7fcd7806ef70 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.797+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd7c087a30 con 0x7fcd8c102de0 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.798+0000 7fcd89d9b700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fcd780692a0 0x7fcd7806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd74004890 tx=0x7fcd74009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:55.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.798+0000 7fcd8ad9d700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] conn(0x7fcd7806ef70 0x7fcd78071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.798+0000 7fcd8ad9d700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] conn(0x7fcd7806ef70 0x7fcd78071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:55.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.799+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 <== osd.3 v2:172.21.15.40:6800/824499182 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fcd78071a30 con 0x7fcd7806ef70 2024-04-24T23:09:55.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.814+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 --> [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcd8c0fe530 con 0x7fcd7806ef70 2024-04-24T23:09:55.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.814+0000 7fcd82ffd700 1 -- 172.21.15.31:0/1756607639 <== osd.3 v2:172.21.15.40:6800/824499182 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcd8c0fe530 con 0x7fcd7806ef70 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] conn(0x7fcd7806ef70 msgr2=0x7fcd78071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] conn(0x7fcd7806ef70 0x7fcd78071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fcd780692a0 msgr2=0x7fcd7806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fcd780692a0 0x7fcd7806b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcd74004890 tx=0x7fcd74009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c102de0 msgr2=0x7fcd8c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c102de0 0x7fcd8c070260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c002a50 tx=0x7fcd7c0046b0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 shutdown_connections 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fcd780692a0 0x7fcd7806b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd8c102de0 0x7fcd8c070260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:6800/824499182,v1:172.21.15.40:6801/824499182] conn(0x7fcd7806ef70 0x7fcd78071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 --2- 172.21.15.31:0/1756607639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd8c103780 0x7fcd8c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 >> 172.21.15.31:0/1756607639 conn(0x7fcd8c0f8950 msgr2=0x7fcd8c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:55.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 shutdown_connections 2024-04-24T23:09:55.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:55.815+0000 7fcd90ab5700 1 -- 172.21.15.31:0/1756607639 wait complete. 2024-04-24T23:09:56.465 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:56 smithi040 ceph-mon[33128]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:56 smithi031 ceph-mon[26411]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:56.871 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.876+0000 7f8ad0923700 1 -- 172.21.15.31:0/1588579624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fddd0 msgr2=0x7f8acc0fe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.876+0000 7f8ad0923700 1 --2- 172.21.15.31:0/1588579624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fddd0 0x7f8acc0fe1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8abc004940 tx=0x7f8abc009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.876+0000 7f8ad0923700 1 -- 172.21.15.31:0/1588579624 shutdown_connections 2024-04-24T23:09:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.876+0000 7f8ad0923700 1 --2- 172.21.15.31:0/1588579624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8acc0fe770 0x7f8acc106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.876+0000 7f8ad0923700 1 --2- 172.21.15.31:0/1588579624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fddd0 0x7f8acc0fe1a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.876+0000 7f8ad0923700 1 -- 172.21.15.31:0/1588579624 >> 172.21.15.31:0/1588579624 conn(0x7f8acc0f7850 msgr2=0x7f8acc0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.876+0000 7f8ad0923700 1 -- 172.21.15.31:0/1588579624 shutdown_connections 2024-04-24T23:09:56.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.877+0000 7f8ad0923700 1 -- 172.21.15.31:0/1588579624 wait complete. 2024-04-24T23:09:56.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.877+0000 7f8ad0923700 1 Processor -- start 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.877+0000 7f8ad0923700 1 -- start start 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ad0923700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8acc0fddd0 0x7f8acc0fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ad0923700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fe770 0x7f8acc0fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ad0923700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc0fc760 con 0x7f8acc0fe770 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ad0923700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8acc0fc8a0 con 0x7f8acc0fddd0 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ac9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fe770 0x7f8acc0fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ac9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fe770 0x7f8acc0fc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51998/0 (socket says 172.21.15.31:51998) 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ac9d9b700 1 -- 172.21.15.31:0/2215265608 learned_addr learned my addr 172.21.15.31:0/2215265608 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8aca59c700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8acc0fddd0 0x7f8acc0fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ac9d9b700 1 -- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8acc0fddd0 msgr2=0x7f8acc0fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ac9d9b700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8acc0fddd0 0x7f8acc0fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ac9d9b700 1 -- 172.21.15.31:0/2215265608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc005040 con 0x7f8acc0fe770 2024-04-24T23:09:56.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8aca59c700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8acc0fddd0 0x7f8acc0fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:09:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.878+0000 7f8ac9d9b700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fe770 0x7f8acc0fc1d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0002a50 tx=0x7f8ac00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.879+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac0007cc0 con 0x7f8acc0fe770 2024-04-24T23:09:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.879+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc119e20 con 0x7f8acc0fe770 2024-04-24T23:09:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.879+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ac000f040 con 0x7f8acc0fe770 2024-04-24T23:09:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.879+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac0009640 con 0x7f8acc0fe770 2024-04-24T23:09:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.879+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8acc11a220 con 0x7f8acc0fe770 2024-04-24T23:09:56.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.880+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8acc0f9830 con 0x7f8acc0fe770 2024-04-24T23:09:56.882 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.881+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8ac00097a0 con 0x7f8acc0fe770 2024-04-24T23:09:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.881+0000 7f8abaffd700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ab40692a0 0x7f8ab406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.881+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8ac0086ba0 con 0x7f8acc0fe770 2024-04-24T23:09:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.881+0000 7f8aca59c700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ab40692a0 0x7f8ab406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.881+0000 7f8abaffd700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] conn(0x7f8ab406ef70 0x7f8ab4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.881+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 --> [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ab4071a30 con 0x7f8ab406ef70 2024-04-24T23:09:56.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.881+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ac008c200 con 0x7f8acc0fe770 2024-04-24T23:09:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.882+0000 7f8acad9d700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] conn(0x7f8ab406ef70 0x7f8ab4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.882+0000 7f8aca59c700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ab40692a0 0x7f8ab406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00b3b0 tx=0x7f8abc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.882+0000 7f8acad9d700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] conn(0x7f8ab406ef70 0x7f8ab4071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:56.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.883+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 <== osd.4 v2:172.21.15.40:6808/1464558178 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8ab4071a30 con 0x7f8ab406ef70 2024-04-24T23:09:56.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.902+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 --> [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8acc0666d0 con 0x7f8ab406ef70 2024-04-24T23:09:56.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.902+0000 7f8abaffd700 1 -- 172.21.15.31:0/2215265608 <== osd.4 v2:172.21.15.40:6808/1464558178 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8acc0666d0 con 0x7f8ab406ef70 2024-04-24T23:09:56.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.902+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] conn(0x7f8ab406ef70 msgr2=0x7f8ab4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.902+0000 7f8ad0923700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] conn(0x7f8ab406ef70 0x7f8ab4071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ab40692a0 msgr2=0x7f8ab406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ab40692a0 0x7f8ab406b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8abc00b3b0 tx=0x7f8abc009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fe770 msgr2=0x7f8acc0fc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fe770 0x7f8acc0fc1d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0002a50 tx=0x7f8ac00046b0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 shutdown_connections 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:6808/1464558178,v1:172.21.15.40:6809/1464558178] conn(0x7f8ab406ef70 0x7f8ab4071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8acc0fddd0 0x7f8acc0fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ab40692a0 0x7f8ab406b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 --2- 172.21.15.31:0/2215265608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc0fe770 0x7f8acc0fc1d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 >> 172.21.15.31:0/2215265608 conn(0x7f8acc0f7850 msgr2=0x7f8acc0f8e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 shutdown_connections 2024-04-24T23:09:56.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:56.903+0000 7f8ad0923700 1 -- 172.21.15.31:0/2215265608 wait complete. 2024-04-24T23:09:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:111669149716 2024-04-24T23:09:57.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-24T23:09:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.914+0000 7f8f64c40700 1 -- 172.21.15.31:0/1891604314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 msgr2=0x7f8f6006ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.914+0000 7f8f64c40700 1 --2- 172.21.15.31:0/1891604314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 0x7f8f6006ef70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8f54004940 tx=0x7f8f54009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.914+0000 7f8f64c40700 1 -- 172.21.15.31:0/1891604314 shutdown_connections 2024-04-24T23:09:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.914+0000 7f8f64c40700 1 --2- 172.21.15.31:0/1891604314 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 0x7f8f6006ef70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.914+0000 7f8f64c40700 1 --2- 172.21.15.31:0/1891604314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f6010a550 0x7f8f6010a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:57.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.914+0000 7f8f64c40700 1 -- 172.21.15.31:0/1891604314 >> 172.21.15.31:0/1891604314 conn(0x7f8f6006a830 msgr2=0x7f8f6006ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.914+0000 7f8f64c40700 1 -- 172.21.15.31:0/1891604314 shutdown_connections 2024-04-24T23:09:57.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.914+0000 7f8f64c40700 1 -- 172.21.15.31:0/1891604314 wait complete. 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f64c40700 1 Processor -- start 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f64c40700 1 -- start start 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f64c40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 0x7f8f601172a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f64c40700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f6010a550 0x7f8f601177e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f64c40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60117d20 con 0x7f8f6006eb00 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f64c40700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60117e60 con 0x7f8f6010a550 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f5e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 0x7f8f601172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f5e59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 0x7f8f601172a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:52018/0 (socket says 172.21.15.31:52018) 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.915+0000 7f8f5e59c700 1 -- 172.21.15.31:0/3889962553 learned_addr learned my addr 172.21.15.31:0/3889962553 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:09:57.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f5dd9b700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f6010a550 0x7f8f601177e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f5e59c700 1 -- 172.21.15.31:0/3889962553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f6010a550 msgr2=0x7f8f601177e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f5e59c700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f6010a550 0x7f8f601177e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f5e59c700 1 -- 172.21.15.31:0/3889962553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54005040 con 0x7f8f6006eb00 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f5e59c700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 0x7f8f601172a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000eb40 tx=0x7f8f5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f50007cc0 con 0x7f8f6006eb00 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f601180e0 con 0x7f8f6006eb00 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f5000f040 con 0x7f8f6006eb00 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f50009620 con 0x7f8f6006eb00 2024-04-24T23:09:57.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.916+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f601205e0 con 0x7f8f6006eb00 2024-04-24T23:09:57.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.917+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f6005e6a0 con 0x7f8f6006eb00 2024-04-24T23:09:57.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.918+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8f50009780 con 0x7f8f6006eb00 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.918+0000 7f8f4effd700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8f480692a0 0x7f8f4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.918+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8f50086af0 con 0x7f8f6006eb00 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.918+0000 7f8f5dd9b700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8f480692a0 0x7f8f4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.918+0000 7f8f4effd700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] conn(0x7f8f4806ef70 0x7f8f48071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.918+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 --> [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f48071a30 con 0x7f8f4806ef70 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.918+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f5008b220 con 0x7f8f6006eb00 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.919+0000 7f8f5ed9d700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] conn(0x7f8f4806ef70 0x7f8f48071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.919+0000 7f8f5dd9b700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8f480692a0 0x7f8f4806b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400b0d0 tx=0x7f8f54009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.919+0000 7f8f5ed9d700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] conn(0x7f8f4806ef70 0x7f8f48071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:09:57.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.920+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 <== osd.1 v2:172.21.15.31:6810/1768854803 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8f48071a30 con 0x7f8f4806ef70 2024-04-24T23:09:57.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.935+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 --> [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f6006fd60 con 0x7f8f4806ef70 2024-04-24T23:09:57.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f4effd700 1 -- 172.21.15.31:0/3889962553 <== osd.1 v2:172.21.15.31:6810/1768854803 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f6006fd60 con 0x7f8f4806ef70 2024-04-24T23:09:57.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] conn(0x7f8f4806ef70 msgr2=0x7f8f48071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:57.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] conn(0x7f8f4806ef70 0x7f8f48071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8f480692a0 msgr2=0x7f8f4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8f480692a0 0x7f8f4806b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400b0d0 tx=0x7f8f54009e30 comp rx=0 tx=0).stop 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 msgr2=0x7f8f601172a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 0x7f8f601172a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000eb40 tx=0x7f8f5000b040 comp rx=0 tx=0).stop 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 shutdown_connections 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6810/1768854803,v1:172.21.15.31:6811/1768854803] conn(0x7f8f4806ef70 0x7f8f48071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8f480692a0 0x7f8f4806b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f6006eb00 0x7f8f601172a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 --2- 172.21.15.31:0/3889962553 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f6010a550 0x7f8f601177e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 >> 172.21.15.31:0/3889962553 conn(0x7f8f6006a830 msgr2=0x7f8f6006d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 shutdown_connections 2024-04-24T23:09:57.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:09:57.936+0000 7f8f64c40700 1 -- 172.21.15.31:0/3889962553 wait complete. 2024-04-24T23:09:58.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:09:58 smithi031 ceph-mon[26411]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:58.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:09:58 smithi040 ceph-mon[33128]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:09:58.987 INFO:teuthology.orchestra.run.smithi031.stdout:146028888079 2024-04-24T23:09:58.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-24T23:09:59.655 INFO:teuthology.orchestra.run.smithi031.stdout:60129542174 2024-04-24T23:09:59.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-24T23:09:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:09:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:00.118 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:00.118 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:00.400 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:00 smithi031 ceph-mon[26411]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:00.400 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:00 smithi031 ceph-mon[26411]: overall HEALTH_OK 2024-04-24T23:10:00.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:00 smithi040 ceph-mon[33128]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:00.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:00 smithi040 ceph-mon[33128]: overall HEALTH_OK 2024-04-24T23:10:00.906 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.714+0000 7f52a028e700 1 -- 172.21.15.31:0/260405428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298102680 msgr2=0x7f52980fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.714+0000 7f52a028e700 1 --2- 172.21.15.31:0/260405428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298102680 0x7f52980fbcf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5288004940 tx=0x7f5288009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.714+0000 7f52a028e700 1 -- 172.21.15.31:0/260405428 shutdown_connections 2024-04-24T23:10:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.714+0000 7f52a028e700 1 --2- 172.21.15.31:0/260405428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298102680 0x7f52980fbcf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.714+0000 7f52a028e700 1 --2- 172.21.15.31:0/260405428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5298101ce0 0x7f52981020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.714+0000 7f52a028e700 1 -- 172.21.15.31:0/260405428 >> 172.21.15.31:0/260405428 conn(0x7f52980f7850 msgr2=0x7f52980f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:01.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.715+0000 7f52a028e700 1 -- 172.21.15.31:0/260405428 shutdown_connections 2024-04-24T23:10:01.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.715+0000 7f52a028e700 1 -- 172.21.15.31:0/260405428 wait complete. 2024-04-24T23:10:01.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.715+0000 7f52a028e700 1 Processor -- start 2024-04-24T23:10:01.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.715+0000 7f52a028e700 1 -- start start 2024-04-24T23:10:01.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f52a028e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298101ce0 0x7f5298113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f52a028e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5298102680 0x7f5298113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f52a028e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5298114630 con 0x7f5298101ce0 2024-04-24T23:10:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f52a028e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529810da20 con 0x7f5298102680 2024-04-24T23:10:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f529e02a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298101ce0 0x7f5298113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f529e02a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298101ce0 0x7f5298113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40104/0 (socket says 172.21.15.31:40104) 2024-04-24T23:10:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f529e02a700 1 -- 172.21.15.31:0/1758482784 learned_addr learned my addr 172.21.15.31:0/1758482784 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f529d829700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5298102680 0x7f5298113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:01.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f529e02a700 1 -- 172.21.15.31:0/1758482784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5298102680 msgr2=0x7f5298113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f529e02a700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5298102680 0x7f5298113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f529e02a700 1 -- 172.21.15.31:0/1758482784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288005040 con 0x7f5298101ce0 2024-04-24T23:10:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f529e02a700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298101ce0 0x7f5298113aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5294002a80 tx=0x7f52940046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.716+0000 7f528effd700 1 -- 172.21.15.31:0/1758482784 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5294007cc0 con 0x7f5298101ce0 2024-04-24T23:10:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.717+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529810dd00 con 0x7f5298101ce0 2024-04-24T23:10:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.717+0000 7f528effd700 1 -- 172.21.15.31:0/1758482784 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f529400f040 con 0x7f5298101ce0 2024-04-24T23:10:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.717+0000 7f528effd700 1 -- 172.21.15.31:0/1758482784 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52940096f0 con 0x7f5298101ce0 2024-04-24T23:10:01.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.717+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529810e2c0 con 0x7f5298101ce0 2024-04-24T23:10:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.718+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52980fd430 con 0x7f5298101ce0 2024-04-24T23:10:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.718+0000 7f528effd700 1 -- 172.21.15.31:0/1758482784 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f529401c400 con 0x7f5298101ce0 2024-04-24T23:10:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.719+0000 7f528effd700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f528406d6b0 0x7f528406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.719+0000 7f528effd700 1 -- 172.21.15.31:0/1758482784 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5294086680 con 0x7f5298101ce0 2024-04-24T23:10:01.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.719+0000 7f529d829700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f528406d6b0 0x7f528406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:01.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.719+0000 7f529d829700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f528406d6b0 0x7f528406fb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f529810ebe0 tx=0x7f5288009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:01.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.723+0000 7f528effd700 1 -- 172.21.15.31:0/1758482784 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f529404d250 con 0x7f5298101ce0 2024-04-24T23:10:01.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.924+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5298108dc0 con 0x7f5298101ce0 2024-04-24T23:10:01.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.924+0000 7f528effd700 1 -- 172.21.15.31:0/1758482784 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5294056770 con 0x7f5298101ce0 2024-04-24T23:10:01.926 INFO:teuthology.orchestra.run.smithi031.stdout:81604378650 2024-04-24T23:10:01.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f528406d6b0 msgr2=0x7f528406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f528406d6b0 0x7f528406fb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f529810ebe0 tx=0x7f5288009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:01.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298101ce0 msgr2=0x7f5298113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:01.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298101ce0 0x7f5298113aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5294002a80 tx=0x7f52940046b0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 shutdown_connections 2024-04-24T23:10:01.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f528406d6b0 0x7f528406fb60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5298101ce0 0x7f5298113aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 --2- 172.21.15.31:0/1758482784 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5298102680 0x7f5298113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:01.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 >> 172.21.15.31:0/1758482784 conn(0x7f52980f7850 msgr2=0x7f5298065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:01.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 shutdown_connections 2024-04-24T23:10:01.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:01.927+0000 7f52a028e700 1 -- 172.21.15.31:0/1758482784 wait complete. 2024-04-24T23:10:02.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:02 smithi031 ceph-mon[26411]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:02.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:02 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1758482784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:10:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:02 smithi040 ceph-mon[33128]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:02 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1758482784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-24T23:10:03.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-04-24T23:10:03.422 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:04 smithi031 ceph-mon[26411]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:10:04.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:04 smithi040 ceph-mon[33128]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:04.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:10:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.170+0000 7fb65d9d4700 1 -- 172.21.15.31:0/3158296153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6581026b0 msgr2=0x7fb6580fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.170+0000 7fb65d9d4700 1 --2- 172.21.15.31:0/3158296153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6581026b0 0x7fb6580fbd20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb640004940 tx=0x7fb640009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.170+0000 7fb65d9d4700 1 -- 172.21.15.31:0/3158296153 shutdown_connections 2024-04-24T23:10:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.170+0000 7fb65d9d4700 1 --2- 172.21.15.31:0/3158296153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6581026b0 0x7fb6580fbd20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.170+0000 7fb65d9d4700 1 --2- 172.21.15.31:0/3158296153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb658101d10 0x7fb6581020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.170+0000 7fb65d9d4700 1 -- 172.21.15.31:0/3158296153 >> 172.21.15.31:0/3158296153 conn(0x7fb6580f7820 msgr2=0x7fb6580f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.170+0000 7fb65d9d4700 1 -- 172.21.15.31:0/3158296153 shutdown_connections 2024-04-24T23:10:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.170+0000 7fb65d9d4700 1 -- 172.21.15.31:0/3158296153 wait complete. 2024-04-24T23:10:05.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.171+0000 7fb65d9d4700 1 Processor -- start 2024-04-24T23:10:05.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.171+0000 7fb65d9d4700 1 -- start start 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.171+0000 7fb65d9d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb658101d10 0x7fb658115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.171+0000 7fb65d9d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6581026b0 0x7fb658116410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.171+0000 7fb65d9d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb658116a60 con 0x7fb658101d10 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.171+0000 7fb65d9d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb658110040 con 0x7fb6581026b0 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb656ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb658101d10 0x7fb658115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb656ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb658101d10 0x7fb658115ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40116/0 (socket says 172.21.15.31:40116) 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb656ffd700 1 -- 172.21.15.31:0/1642803544 learned_addr learned my addr 172.21.15.31:0/1642803544 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb6567fc700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6581026b0 0x7fb658116410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb656ffd700 1 -- 172.21.15.31:0/1642803544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6581026b0 msgr2=0x7fb658116410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb656ffd700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6581026b0 0x7fb658116410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb656ffd700 1 -- 172.21.15.31:0/1642803544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640005040 con 0x7fb658101d10 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb656ffd700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb658101d10 0x7fb658115ed0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb65000cb10 tx=0x7fb650007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb64f7fe700 1 -- 172.21.15.31:0/1642803544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb650004200 con 0x7fb658101d10 2024-04-24T23:10:05.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb658110320 con 0x7fb658101d10 2024-04-24T23:10:05.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb64f7fe700 1 -- 172.21.15.31:0/1642803544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb65000acc0 con 0x7fb658101d10 2024-04-24T23:10:05.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb64f7fe700 1 -- 172.21.15.31:0/1642803544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb650003d90 con 0x7fb658101d10 2024-04-24T23:10:05.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.172+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6581108e0 con 0x7fb658101d10 2024-04-24T23:10:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.173+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6580fd460 con 0x7fb658101d10 2024-04-24T23:10:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.174+0000 7fb64f7fe700 1 -- 172.21.15.31:0/1642803544 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb650003430 con 0x7fb658101d10 2024-04-24T23:10:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.174+0000 7fb64f7fe700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6440692a0 0x7fb64406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.174+0000 7fb64f7fe700 1 -- 172.21.15.31:0/1642803544 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb650087500 con 0x7fb658101d10 2024-04-24T23:10:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.175+0000 7fb6567fc700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6440692a0 0x7fb64406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.175+0000 7fb6567fc700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6440692a0 0x7fb64406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb64000b3b0 tx=0x7fb640009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:05.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.179+0000 7fb64f7fe700 1 -- 172.21.15.31:0/1642803544 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb6500574e0 con 0x7fb658101d10 2024-04-24T23:10:05.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.390+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb65804c710 con 0x7fb658101d10 2024-04-24T23:10:05.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.391+0000 7fb64f7fe700 1 -- 172.21.15.31:0/1642803544 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb65804c710 con 0x7fb658101d10 2024-04-24T23:10:05.393 INFO:teuthology.orchestra.run.smithi031.stdout:111669149718 2024-04-24T23:10:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6440692a0 msgr2=0x7fb64406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6440692a0 0x7fb64406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb64000b3b0 tx=0x7fb640009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb658101d10 msgr2=0x7fb658115ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb658101d10 0x7fb658115ed0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb65000cb10 tx=0x7fb650007f90 comp rx=0 tx=0).stop 2024-04-24T23:10:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 shutdown_connections 2024-04-24T23:10:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6440692a0 0x7fb64406b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb658101d10 0x7fb658115ed0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 --2- 172.21.15.31:0/1642803544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6581026b0 0x7fb658116410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:05.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 >> 172.21.15.31:0/1642803544 conn(0x7fb6580f7820 msgr2=0x7fb658066280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:05.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 shutdown_connections 2024-04-24T23:10:05.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:05.393+0000 7fb65d9d4700 1 -- 172.21.15.31:0/1642803544 wait complete. 2024-04-24T23:10:06.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:06 smithi031 ceph-mon[26411]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:06.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:06 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1642803544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:10:06.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:06 smithi040 ceph-mon[33128]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:06.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:06 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1642803544' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-24T23:10:07.191 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149716 got 111669149718 for osd.3 2024-04-24T23:10:07.191 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.349+0000 7efe0cfce700 1 -- 172.21.15.31:0/2691961632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe080fbd40 msgr2=0x7efe080fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.349+0000 7efe0cfce700 1 --2- 172.21.15.31:0/2691961632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe080fbd40 0x7efe080fc1b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efdf8004940 tx=0x7efdf8009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.349+0000 7efe0cfce700 1 -- 172.21.15.31:0/2691961632 shutdown_connections 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.349+0000 7efe0cfce700 1 --2- 172.21.15.31:0/2691961632 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe080fbd40 0x7efe080fc1b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.349+0000 7efe0cfce700 1 --2- 172.21.15.31:0/2691961632 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe08101d90 0x7efe08102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.349+0000 7efe0cfce700 1 -- 172.21.15.31:0/2691961632 >> 172.21.15.31:0/2691961632 conn(0x7efe08071d00 msgr2=0x7efe08072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.349+0000 7efe0cfce700 1 -- 172.21.15.31:0/2691961632 shutdown_connections 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.349+0000 7efe0cfce700 1 -- 172.21.15.31:0/2691961632 wait complete. 2024-04-24T23:10:07.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe0cfce700 1 Processor -- start 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe0cfce700 1 -- start start 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe0cfce700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe080fbd40 0x7efe08115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe0cfce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe08101d90 0x7efe08116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe0cfce700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe08116a00 con 0x7efe08101d90 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe0cfce700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0810fd80 con 0x7efe080fbd40 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe05d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe08101d90 0x7efe08116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe05d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe08101d90 0x7efe08116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40130/0 (socket says 172.21.15.31:40130) 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe05d9b700 1 -- 172.21.15.31:0/2088146366 learned_addr learned my addr 172.21.15.31:0/2088146366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.350+0000 7efe0659c700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe080fbd40 0x7efe08115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efe05d9b700 1 -- 172.21.15.31:0/2088146366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe080fbd40 msgr2=0x7efe08115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efe05d9b700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe080fbd40 0x7efe08115de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efe05d9b700 1 -- 172.21.15.31:0/2088146366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8005040 con 0x7efe08101d90 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efe0659c700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe080fbd40 0x7efe08115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:07.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efe05d9b700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe08101d90 0x7efe08116320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b8b0 tx=0x7efdf8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efdfeffd700 1 -- 172.21.15.31:0/2088146366 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf800e040 con 0x7efe08101d90 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efdfeffd700 1 -- 172.21.15.31:0/2088146366 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdf8013420 con 0x7efe08101d90 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe08110000 con 0x7efe08101d90 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efdfeffd700 1 -- 172.21.15.31:0/2088146366 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdf801c630 con 0x7efe08101d90 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.351+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe08110540 con 0x7efe08101d90 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.352+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe080fd430 con 0x7efe08101d90 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.352+0000 7efdfeffd700 1 -- 172.21.15.31:0/2088146366 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7efdf8013a00 con 0x7efe08101d90 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.353+0000 7efdfeffd700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7efdf40692a0 0x7efdf406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.353+0000 7efdfeffd700 1 -- 172.21.15.31:0/2088146366 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efdf8088030 con 0x7efe08101d90 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.353+0000 7efe0659c700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7efdf40692a0 0x7efdf406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:07.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.353+0000 7efe0659c700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7efdf40692a0 0x7efdf406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009b10 tx=0x7efdf0008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:07.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.358+0000 7efdfeffd700 1 -- 172.21.15.31:0/2088146366 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efdf8054740 con 0x7efe08101d90 2024-04-24T23:10:07.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.556+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7efe0804bed0 con 0x7efe08101d90 2024-04-24T23:10:07.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.557+0000 7efdfeffd700 1 -- 172.21.15.31:0/2088146366 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efdf8057d60 con 0x7efe08101d90 2024-04-24T23:10:07.559 INFO:teuthology.orchestra.run.smithi031.stdout:146028888080 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7efdf40692a0 msgr2=0x7efdf406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7efdf40692a0 0x7efdf406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009b10 tx=0x7efdf0008040 comp rx=0 tx=0).stop 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe08101d90 msgr2=0x7efe08116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe08101d90 0x7efe08116320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efdf800b8b0 tx=0x7efdf8009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 shutdown_connections 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efe080fbd40 0x7efe08115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7efdf40692a0 0x7efdf406b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 --2- 172.21.15.31:0/2088146366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe08101d90 0x7efe08116320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 >> 172.21.15.31:0/2088146366 conn(0x7efe08071d00 msgr2=0x7efe080fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 shutdown_connections 2024-04-24T23:10:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:07.559+0000 7efe0cfce700 1 -- 172.21.15.31:0/2088146366 wait complete. 2024-04-24T23:10:08.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:08 smithi031 ceph-mon[26411]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:08.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:08 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2088146366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:10:08.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:08 smithi040 ceph-mon[33128]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:08.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:08 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2088146366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-24T23:10:08.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888080 for osd.4 2024-04-24T23:10:08.698 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 -- 172.21.15.31:0/3842792482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070c10 msgr2=0x7fb9441090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 --2- 172.21.15.31:0/3842792482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070c10 0x7fb9441090f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb938004970 tx=0x7fb938009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 -- 172.21.15.31:0/3842792482 shutdown_connections 2024-04-24T23:10:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 --2- 172.21.15.31:0/3842792482 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070c10 0x7fb9441090f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 --2- 172.21.15.31:0/3842792482 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb944070300 0x7fb9440706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 -- 172.21.15.31:0/3842792482 >> 172.21.15.31:0/3842792482 conn(0x7fb9440f8990 msgr2=0x7fb9440fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 -- 172.21.15.31:0/3842792482 shutdown_connections 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 -- 172.21.15.31:0/3842792482 wait complete. 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 Processor -- start 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.124+0000 7fb94c1d3700 1 -- start start 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb94c1d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070300 0x7fb944115b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb94c1d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb944070c10 0x7fb944116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb94c1d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9441166d0 con 0x7fb944070300 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb94c1d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94410fcb0 con 0x7fb944070c10 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb949f6f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070300 0x7fb944115b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb949f6f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070300 0x7fb944115b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46822/0 (socket says 172.21.15.31:46822) 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb949f6f700 1 -- 172.21.15.31:0/281847898 learned_addr learned my addr 172.21.15.31:0/281847898 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb94976e700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb944070c10 0x7fb944116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb949f6f700 1 -- 172.21.15.31:0/281847898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb944070c10 msgr2=0x7fb944116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb949f6f700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb944070c10 0x7fb944116080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb949f6f700 1 -- 172.21.15.31:0/281847898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938005040 con 0x7fb944070300 2024-04-24T23:10:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb949f6f700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070300 0x7fb944115b40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb93400cac0 tx=0x7fb934007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb9427fc700 1 -- 172.21.15.31:0/281847898 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb93400a980 con 0x7fb944070300 2024-04-24T23:10:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94410ff90 con 0x7fb944070300 2024-04-24T23:10:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb9427fc700 1 -- 172.21.15.31:0/281847898 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb934012440 con 0x7fb944070300 2024-04-24T23:10:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb9427fc700 1 -- 172.21.15.31:0/281847898 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9340115e0 con 0x7fb944070300 2024-04-24T23:10:09.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.125+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9441104a0 con 0x7fb944070300 2024-04-24T23:10:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.126+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb944071a40 con 0x7fb944070300 2024-04-24T23:10:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.127+0000 7fb9427fc700 1 -- 172.21.15.31:0/281847898 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb9340125b0 con 0x7fb944070300 2024-04-24T23:10:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.127+0000 7fb9427fc700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb92c0692a0 0x7fb92c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.127+0000 7fb9427fc700 1 -- 172.21.15.31:0/281847898 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb9340875b0 con 0x7fb944070300 2024-04-24T23:10:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.127+0000 7fb94976e700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb92c0692a0 0x7fb92c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.128+0000 7fb94976e700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb92c0692a0 0x7fb92c06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb938004f50 tx=0x7fb938009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.132+0000 7fb9427fc700 1 -- 172.21.15.31:0/281847898 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb934057630 con 0x7fb944070300 2024-04-24T23:10:09.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.329+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb94404c710 con 0x7fb944070300 2024-04-24T23:10:09.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.330+0000 7fb9427fc700 1 -- 172.21.15.31:0/281847898 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb934012860 con 0x7fb944070300 2024-04-24T23:10:09.332 INFO:teuthology.orchestra.run.smithi031.stdout:60129542176 2024-04-24T23:10:09.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb92c0692a0 msgr2=0x7fb92c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb92c0692a0 0x7fb92c06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb938004f50 tx=0x7fb938009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070300 msgr2=0x7fb944115b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070300 0x7fb944115b40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb93400cac0 tx=0x7fb934007ee0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 shutdown_connections 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb92c0692a0 0x7fb92c06b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb944070300 0x7fb944115b40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 --2- 172.21.15.31:0/281847898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb944070c10 0x7fb944116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 >> 172.21.15.31:0/281847898 conn(0x7fb9440f8990 msgr2=0x7fb9440f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 shutdown_connections 2024-04-24T23:10:09.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.332+0000 7fb94c1d3700 1 -- 172.21.15.31:0/281847898 wait complete. 2024-04-24T23:10:09.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.749+0000 7f0fc93f1700 1 -- 172.21.15.31:0/3034952230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc4101d90 msgr2=0x7f0fc4102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.749+0000 7f0fc93f1700 1 --2- 172.21.15.31:0/3034952230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc4101d90 0x7f0fc4102160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc004940 tx=0x7f0fbc009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.749+0000 7f0fc93f1700 1 -- 172.21.15.31:0/3034952230 shutdown_connections 2024-04-24T23:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.749+0000 7f0fc93f1700 1 --2- 172.21.15.31:0/3034952230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc40fbd40 0x7f0fc40fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.749+0000 7f0fc93f1700 1 --2- 172.21.15.31:0/3034952230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc4101d90 0x7f0fc4102160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.749+0000 7f0fc93f1700 1 -- 172.21.15.31:0/3034952230 >> 172.21.15.31:0/3034952230 conn(0x7f0fc4071d00 msgr2=0x7f0fc4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.749+0000 7f0fc93f1700 1 -- 172.21.15.31:0/3034952230 shutdown_connections 2024-04-24T23:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.749+0000 7f0fc93f1700 1 -- 172.21.15.31:0/3034952230 wait complete. 2024-04-24T23:10:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc93f1700 1 Processor -- start 2024-04-24T23:10:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc93f1700 1 -- start start 2024-04-24T23:10:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc93f1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc40fbd40 0x7f0fc4113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc93f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc4101d90 0x7f0fc4113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc93f1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc4114630 con 0x7f0fc40fbd40 2024-04-24T23:10:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc93f1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc410da20 con 0x7f0fc4101d90 2024-04-24T23:10:09.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc40fbd40 0x7f0fc4113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc40fbd40 0x7f0fc4113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46852/0 (socket says 172.21.15.31:46852) 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc2ffd700 1 -- 172.21.15.31:0/1231073561 learned_addr learned my addr 172.21.15.31:0/1231073561 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc27fc700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc4101d90 0x7f0fc4113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc2ffd700 1 -- 172.21.15.31:0/1231073561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc4101d90 msgr2=0x7f0fc4113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc2ffd700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc4101d90 0x7f0fc4113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc2ffd700 1 -- 172.21.15.31:0/1231073561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc005040 con 0x7f0fc40fbd40 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.750+0000 7f0fc2ffd700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc40fbd40 0x7f0fc4113aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b8b0 tx=0x7f0fbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.751+0000 7f0fbb7fe700 1 -- 172.21.15.31:0/1231073561 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fbc00e040 con 0x7f0fc40fbd40 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.751+0000 7f0fbb7fe700 1 -- 172.21.15.31:0/1231073561 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fbc004290 con 0x7f0fc40fbd40 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.751+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc410dca0 con 0x7f0fc40fbd40 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.751+0000 7f0fbb7fe700 1 -- 172.21.15.31:0/1231073561 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fbc01c3f0 con 0x7f0fc40fbd40 2024-04-24T23:10:09.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.751+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc410e260 con 0x7f0fc40fbd40 2024-04-24T23:10:09.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.752+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc40feff0 con 0x7f0fc40fbd40 2024-04-24T23:10:09.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.752+0000 7f0fbb7fe700 1 -- 172.21.15.31:0/1231073561 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0fbc013820 con 0x7f0fc40fbd40 2024-04-24T23:10:09.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.752+0000 7f0fbb7fe700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0fb00692a0 0x7f0fb006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.753+0000 7f0fbb7fe700 1 -- 172.21.15.31:0/1231073561 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0fbc087fe0 con 0x7f0fc40fbd40 2024-04-24T23:10:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.753+0000 7f0fc27fc700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0fb00692a0 0x7f0fb006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:09.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.753+0000 7f0fc27fc700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0fb00692a0 0x7f0fb006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fc410ebe0 tx=0x7f0fac00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:09.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.757+0000 7f0fbb7fe700 1 -- 172.21.15.31:0/1231073561 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0fbc058180 con 0x7f0fc40fbd40 2024-04-24T23:10:09.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.951+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0fc405f620 con 0x7f0fc40fbd40 2024-04-24T23:10:09.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.952+0000 7f0fbb7fe700 1 -- 172.21.15.31:0/1231073561 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0fbc057d10 con 0x7f0fc40fbd40 2024-04-24T23:10:09.953 INFO:teuthology.orchestra.run.smithi031.stdout:38654705703 2024-04-24T23:10:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0fb00692a0 msgr2=0x7f0fb006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0fb00692a0 0x7f0fb006b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fc410ebe0 tx=0x7f0fac00b040 comp rx=0 tx=0).stop 2024-04-24T23:10:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc40fbd40 msgr2=0x7f0fc4113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc40fbd40 0x7f0fc4113aa0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00b8b0 tx=0x7f0fbc009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 shutdown_connections 2024-04-24T23:10:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0fb00692a0 0x7f0fb006b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0fc40fbd40 0x7f0fc4113aa0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 --2- 172.21.15.31:0/1231073561 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0fc4101d90 0x7f0fc4113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 >> 172.21.15.31:0/1231073561 conn(0x7f0fc4071d00 msgr2=0x7f0fc40faef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 shutdown_connections 2024-04-24T23:10:09.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:09.954+0000 7f0fc93f1700 1 -- 172.21.15.31:0/1231073561 wait complete. 2024-04-24T23:10:10.338 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542176 for osd.1 2024-04-24T23:10:10.338 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:10.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:10 smithi031 ceph-mon[26411]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:10.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:10 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/281847898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:10:10.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:10 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1231073561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:10:10.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:10 smithi040 ceph-mon[33128]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:10.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:10 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/281847898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-24T23:10:10.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:10 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1231073561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-24T23:10:10.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705703 for osd.0 2024-04-24T23:10:10.891 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.307+0000 7f9963f89700 1 -- 172.21.15.31:0/1152921719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c103780 msgr2=0x7f995c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.307+0000 7f9963f89700 1 --2- 172.21.15.31:0/1152921719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c103780 0x7f995c0fcdf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f994c004940 tx=0x7f994c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.307+0000 7f9963f89700 1 -- 172.21.15.31:0/1152921719 shutdown_connections 2024-04-24T23:10:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.307+0000 7f9963f89700 1 --2- 172.21.15.31:0/1152921719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c103780 0x7f995c0fcdf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.307+0000 7f9963f89700 1 --2- 172.21.15.31:0/1152921719 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c102de0 0x7f995c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.307+0000 7f9963f89700 1 -- 172.21.15.31:0/1152921719 >> 172.21.15.31:0/1152921719 conn(0x7f995c0f8990 msgr2=0x7f995c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9963f89700 1 -- 172.21.15.31:0/1152921719 shutdown_connections 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9963f89700 1 -- 172.21.15.31:0/1152921719 wait complete. 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9963f89700 1 Processor -- start 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9963f89700 1 -- start start 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9963f89700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c102de0 0x7f995c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9963f89700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c103780 0x7f995c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9963f89700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c0721b0 con 0x7f995c102de0 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9963f89700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995c070ce0 con 0x7f995c103780 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9961d25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c102de0 0x7f995c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9961d25700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c102de0 0x7f995c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46866/0 (socket says 172.21.15.31:46866) 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9961d25700 1 -- 172.21.15.31:0/3567849331 learned_addr learned my addr 172.21.15.31:0/3567849331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.308+0000 7f9961524700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c103780 0x7f995c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f9961d25700 1 -- 172.21.15.31:0/3567849331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c103780 msgr2=0x7f995c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f9961d25700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c103780 0x7f995c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f9961d25700 1 -- 172.21.15.31:0/3567849331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c005040 con 0x7f995c102de0 2024-04-24T23:10:11.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f9961d25700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c102de0 0x7f995c070260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f995800c8c0 tx=0x7f99580044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f99527fc700 1 -- 172.21.15.31:0/3567849331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995800a760 con 0x7f995c102de0 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f99527fc700 1 -- 172.21.15.31:0/3567849331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f995800ad90 con 0x7f995c102de0 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c070fc0 con 0x7f995c102de0 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f99527fc700 1 -- 172.21.15.31:0/3567849331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99580113a0 con 0x7f995c102de0 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.309+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995c11a0a0 con 0x7f995c102de0 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.310+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995c0fe530 con 0x7f995c102de0 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.311+0000 7f99527fc700 1 -- 172.21.15.31:0/3567849331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f9958012400 con 0x7f995c102de0 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.311+0000 7f99527fc700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f99480692a0 0x7f994806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:11.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.311+0000 7f99527fc700 1 -- 172.21.15.31:0/3567849331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f9958086eb0 con 0x7f995c102de0 2024-04-24T23:10:11.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.311+0000 7f9961524700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f99480692a0 0x7f994806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:11.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.312+0000 7f9961524700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f99480692a0 0x7f994806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f994c004890 tx=0x7f994c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:11.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.315+0000 7f99527fc700 1 -- 172.21.15.31:0/3567849331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9958053520 con 0x7f995c102de0 2024-04-24T23:10:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.510+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f995c04c710 con 0x7f995c102de0 2024-04-24T23:10:11.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.511+0000 7f99527fc700 1 -- 172.21.15.31:0/3567849331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f995805c090 con 0x7f995c102de0 2024-04-24T23:10:11.512 INFO:teuthology.orchestra.run.smithi031.stdout:167503724557 2024-04-24T23:10:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.512+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f99480692a0 msgr2=0x7f994806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.512+0000 7f9963f89700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f99480692a0 0x7f994806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f994c004890 tx=0x7f994c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.512+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c102de0 msgr2=0x7f995c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:11.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.512+0000 7f9963f89700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c102de0 0x7f995c070260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f995800c8c0 tx=0x7f99580044c0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.513+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 shutdown_connections 2024-04-24T23:10:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.513+0000 7f9963f89700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f99480692a0 0x7f994806b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.513+0000 7f9963f89700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f995c102de0 0x7f995c070260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.513+0000 7f9963f89700 1 --2- 172.21.15.31:0/3567849331 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f995c103780 0x7f995c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.513+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 >> 172.21.15.31:0/3567849331 conn(0x7f995c0f8990 msgr2=0x7f995c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.513+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 shutdown_connections 2024-04-24T23:10:11.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:11.513+0000 7f9963f89700 1 -- 172.21.15.31:0/3567849331 wait complete. 2024-04-24T23:10:12.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724557 for osd.5 2024-04-24T23:10:12.196 DEBUG:teuthology.parallel:result is None 2024-04-24T23:10:12.197 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-24T23:10:12.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:10:12.490 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:12.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:12 smithi031 ceph-mon[26411]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:12.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:12 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3567849331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:10:12.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:12 smithi040 ceph-mon[33128]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:12.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:12 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3567849331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.061+0000 7fb6be5db700 1 -- 172.21.15.31:0/537064312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 msgr2=0x7fb6b8065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.061+0000 7fb6be5db700 1 --2- 172.21.15.31:0/537064312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 0x7fb6b8065ab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0004940 tx=0x7fb6a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.061+0000 7fb6be5db700 1 -- 172.21.15.31:0/537064312 shutdown_connections 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.061+0000 7fb6be5db700 1 --2- 172.21.15.31:0/537064312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 0x7fb6b8065ab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.061+0000 7fb6be5db700 1 --2- 172.21.15.31:0/537064312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b8102e50 0x7fb6b8103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.061+0000 7fb6be5db700 1 -- 172.21.15.31:0/537064312 >> 172.21.15.31:0/537064312 conn(0x7fb6b8071d00 msgr2=0x7fb6b8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.061+0000 7fb6be5db700 1 -- 172.21.15.31:0/537064312 shutdown_connections 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.061+0000 7fb6be5db700 1 -- 172.21.15.31:0/537064312 wait complete. 2024-04-24T23:10:14.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6be5db700 1 Processor -- start 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6be5db700 1 -- start start 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6be5db700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 0x7fb6b8113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6be5db700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b8102e50 0x7fb6b81140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6be5db700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b81147a0 con 0x7fb6b8065640 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6be5db700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b810db00 con 0x7fb6b8102e50 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 0x7fb6b8113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6b7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 0x7fb6b8113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46882/0 (socket says 172.21.15.31:46882) 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.062+0000 7fb6b7fff700 1 -- 172.21.15.31:0/2194732989 learned_addr learned my addr 172.21.15.31:0/2194732989 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6af7fe700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b8102e50 0x7fb6b81140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6b7fff700 1 -- 172.21.15.31:0/2194732989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b8102e50 msgr2=0x7fb6b81140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6b7fff700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b8102e50 0x7fb6b81140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6b7fff700 1 -- 172.21.15.31:0/2194732989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a0005040 con 0x7fb6b8065640 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6b7fff700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 0x7fb6b8113b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800b790 tx=0x7fb6a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6b57fa700 1 -- 172.21.15.31:0/2194732989 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a800f770 con 0x7fb6b8065640 2024-04-24T23:10:14.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6b57fa700 1 -- 172.21.15.31:0/2194732989 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6a800fda0 con 0x7fb6b8065640 2024-04-24T23:10:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b810dde0 con 0x7fb6b8065640 2024-04-24T23:10:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6b57fa700 1 -- 172.21.15.31:0/2194732989 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6a800d310 con 0x7fb6b8065640 2024-04-24T23:10:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.063+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b810e3a0 con 0x7fb6b8065640 2024-04-24T23:10:14.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.064+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b80fe540 con 0x7fb6b8065640 2024-04-24T23:10:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.065+0000 7fb6b57fa700 1 -- 172.21.15.31:0/2194732989 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb6a8010420 con 0x7fb6b8065640 2024-04-24T23:10:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.065+0000 7fb6b57fa700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6a40692b0 0x7fb6a406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.065+0000 7fb6b57fa700 1 -- 172.21.15.31:0/2194732989 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fb6a8086e80 con 0x7fb6b8065640 2024-04-24T23:10:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.065+0000 7fb6af7fe700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6a40692b0 0x7fb6a406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:14.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.065+0000 7fb6af7fe700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6a40692b0 0x7fb6a406b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000b3b0 tx=0x7fb6a0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:14.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.069+0000 7fb6b57fa700 1 -- 172.21.15.31:0/2194732989 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb6a8056f80 con 0x7fb6b8065640 2024-04-24T23:10:14.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.262+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6b8109fe0 con 0x7fb6a40692b0 2024-04-24T23:10:14.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.263+0000 7fb6b57fa700 1 -- 172.21.15.31:0/2194732989 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28223 (secure 0 0 0) 0x7fb6b8109fe0 con 0x7fb6a40692b0 2024-04-24T23:10:14.265 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6a40692b0 msgr2=0x7fb6a406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6a40692b0 0x7fb6a406b760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6a000b3b0 tx=0x7fb6a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 msgr2=0x7fb6b8113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 0x7fb6b8113b80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800b790 tx=0x7fb6a8009250 comp rx=0 tx=0).stop 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 shutdown_connections 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6a40692b0 0x7fb6a406b760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb6b8065640 0x7fb6b8113b80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 --2- 172.21.15.31:0/2194732989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb6b8102e50 0x7fb6b81140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:14.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 >> 172.21.15.31:0/2194732989 conn(0x7fb6b8071d00 msgr2=0x7fb6b80fa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:14.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 shutdown_connections 2024-04-24T23:10:14.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:14.265+0000 7fb6be5db700 1 -- 172.21.15.31:0/2194732989 wait complete. 2024-04-24T23:10:14.268 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-24T23:10:14.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:14 smithi031 ceph-mon[26411]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:14.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:14 smithi040 ceph-mon[33128]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:15.089 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-04-24T23:10:12.797565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788040,"kb_used_data":3872,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682872,"statfs":{"total":575970213888,"available":574139260928,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002290"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-24T23:09:22.858021+0000","last_change":"2024-04-24T23:08:56.858359+0000","last_active":"2024-04-24T23:09:22.858021+0000","last_peered":"2024-04-24T23:09:22.858021+0000","last_clean":"2024-04-24T23:09:22.858021+0000","last_became_active":"2024-04-24T23:08:56.857849+0000","last_became_peered":"2024-04-24T23:08:56.857849+0000","last_unstale":"2024-04-24T23:09:22.858021+0000","last_undegraded":"2024-04-24T23:09:22.858021+0000","last_fullsized":"2024-04-24T23:09:22.858021+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:05.990869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:05.990869+0000","last_clean_scrub_stamp":"2024-04-24T23:08:05.990869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:16.959929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 23:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.54300000000000004}]},{"osd":1,"last update":"Wed Apr 24 23:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Apr 24 23:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.443},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.49199999999999999}]},{"osd":3,"last update":"Wed Apr 24 23:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 23:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.49199999999999999}]},{"osd":1,"last update":"Wed Apr 24 23:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed Apr 24 23:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.443},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Apr 24 23:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 23:10:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed Apr 24 23:10:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.38900000000000001},"min":{"1min":0.28000000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.36499999999999999}]},{"osd":3,"last update":"Wed Apr 24 23:09:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.67200000000000004}]},{"osd":4,"last update":"Wed Apr 24 23:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 23:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.33400000000000002,"15min":0.32600000000000001},"min":{"1min":0.28599999999999998,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.502,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.35899999999999999}]},{"osd":2,"last update":"Wed Apr 24 23:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.29199999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.32200000000000001}]},{"osd":3,"last update":"Wed Apr 24 23:09:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Apr 24 23:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 23:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.36099999999999999,"15min":0.36499999999999999},"min":{"1min":0.25600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.44,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.26500000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.59599999999999997}]},{"osd":2,"last update":"Wed Apr 24 23:09:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Apr 24 23:09:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.755},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.84299999999999997}]},{"osd":4,"last update":"Wed Apr 24 23:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.85599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:10:15.091 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-24T23:10:15.364 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:15.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:15 smithi031 ceph-mon[26411]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:15 smithi040 ceph-mon[33128]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:16.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:16 smithi031 ceph-mon[26411]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:16.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:16 smithi040 ceph-mon[33128]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.928+0000 7f3762bb5700 1 -- 172.21.15.31:0/860215349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c102de0 msgr2=0x7f375c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.928+0000 7f3762bb5700 1 --2- 172.21.15.31:0/860215349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c102de0 0x7f375c1031b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f374c004940 tx=0x7f374c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.928+0000 7f3762bb5700 1 -- 172.21.15.31:0/860215349 shutdown_connections 2024-04-24T23:10:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.928+0000 7f3762bb5700 1 --2- 172.21.15.31:0/860215349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f375c103780 0x7f375c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.928+0000 7f3762bb5700 1 --2- 172.21.15.31:0/860215349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c102de0 0x7f375c1031b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.928+0000 7f3762bb5700 1 -- 172.21.15.31:0/860215349 >> 172.21.15.31:0/860215349 conn(0x7f375c0f8990 msgr2=0x7f375c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:16.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.928+0000 7f3762bb5700 1 -- 172.21.15.31:0/860215349 shutdown_connections 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.928+0000 7f3762bb5700 1 -- 172.21.15.31:0/860215349 wait complete. 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f3762bb5700 1 Processor -- start 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f3762bb5700 1 -- start start 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f3762bb5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f375c102de0 0x7f375c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f3762bb5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c103780 0x7f375c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f3762bb5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c072160 con 0x7f375c103780 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f3762bb5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c070ce0 con 0x7f375c102de0 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f375bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c103780 0x7f375c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f375bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c103780 0x7f375c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46904/0 (socket says 172.21.15.31:46904) 2024-04-24T23:10:16.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.929+0000 7f375bfff700 1 -- 172.21.15.31:0/2968262073 learned_addr learned my addr 172.21.15.31:0/2968262073 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f3760951700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f375c102de0 0x7f375c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f375bfff700 1 -- 172.21.15.31:0/2968262073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f375c102de0 msgr2=0x7f375c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f375bfff700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f375c102de0 0x7f375c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f375bfff700 1 -- 172.21.15.31:0/2968262073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c005040 con 0x7f375c103780 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f375bfff700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c103780 0x7f375c0707a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3750002a50 tx=0x7f3750004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f37597fa700 1 -- 172.21.15.31:0/2968262073 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3750007cc0 con 0x7f375c103780 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c070fc0 con 0x7f375c103780 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f37597fa700 1 -- 172.21.15.31:0/2968262073 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f375000f040 con 0x7f375c103780 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f37597fa700 1 -- 172.21.15.31:0/2968262073 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f375000d9c0 con 0x7f375c103780 2024-04-24T23:10:16.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.930+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c11a160 con 0x7f375c103780 2024-04-24T23:10:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.931+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c1000f0 con 0x7f375c103780 2024-04-24T23:10:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.932+0000 7f37597fa700 1 -- 172.21.15.31:0/2968262073 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3750009730 con 0x7f375c103780 2024-04-24T23:10:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.932+0000 7f37597fa700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37440692a0 0x7f374406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:16.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.932+0000 7f37597fa700 1 -- 172.21.15.31:0/2968262073 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3750086bc0 con 0x7f375c103780 2024-04-24T23:10:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.932+0000 7f3760951700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37440692a0 0x7f374406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:16.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.932+0000 7f3760951700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37440692a0 0x7f374406b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f374c004f90 tx=0x7f374c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:16.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:16.936+0000 7f37597fa700 1 -- 172.21.15.31:0/2968262073 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f37500531b0 con 0x7f375c103780 2024-04-24T23:10:17.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.131+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f375c0716f0 con 0x7f37440692a0 2024-04-24T23:10:17.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.132+0000 7f37597fa700 1 -- 172.21.15.31:0/2968262073 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28209 (secure 0 0 0) 0x7f375c0716f0 con 0x7f37440692a0 2024-04-24T23:10:17.135 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37440692a0 msgr2=0x7f374406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37440692a0 0x7f374406b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f374c004f90 tx=0x7f374c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c103780 msgr2=0x7f375c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c103780 0x7f375c0707a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3750002a50 tx=0x7f3750004600 comp rx=0 tx=0).stop 2024-04-24T23:10:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 shutdown_connections 2024-04-24T23:10:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f375c102de0 0x7f375c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f37440692a0 0x7f374406b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 --2- 172.21.15.31:0/2968262073 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f375c103780 0x7f375c0707a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:17.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 >> 172.21.15.31:0/2968262073 conn(0x7f375c0f8990 msgr2=0x7f375c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:17.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 shutdown_connections 2024-04-24T23:10:17.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:17.134+0000 7f3762bb5700 1 -- 172.21.15.31:0/2968262073 wait complete. 2024-04-24T23:10:17.137 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-04-24T23:10:17.734 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-04-24T23:10:16.798250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788040,"kb_used_data":3872,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682872,"statfs":{"total":575970213888,"available":574139260928,"internally_reserved":0,"allocated":3964928,"data_stored":2605601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002213"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":190,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-24T23:09:22.858021+0000","last_change":"2024-04-24T23:08:56.858359+0000","last_active":"2024-04-24T23:09:22.858021+0000","last_peered":"2024-04-24T23:09:22.858021+0000","last_clean":"2024-04-24T23:09:22.858021+0000","last_became_active":"2024-04-24T23:08:56.857849+0000","last_became_peered":"2024-04-24T23:08:56.857849+0000","last_unstale":"2024-04-24T23:09:22.858021+0000","last_undegraded":"2024-04-24T23:09:22.858021+0000","last_fullsized":"2024-04-24T23:09:22.858021+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T23:08:05.990869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T23:08:05.990869+0000","last_clean_scrub_stamp":"2024-04-24T23:08:05.990869+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:16.959929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 23:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.54300000000000004}]},{"osd":1,"last update":"Wed Apr 24 23:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Apr 24 23:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.443},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.49199999999999999}]},{"osd":3,"last update":"Wed Apr 24 23:09:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.29199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149721,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 23:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.503}]},{"osd":1,"last update":"Wed Apr 24 23:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.54000000000000004}]},{"osd":2,"last update":"Wed Apr 24 23:09:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52300000000000002}]},{"osd":4,"last update":"Wed Apr 24 23:09:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.32300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378654,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":360448,"data_stored":138532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 23:10:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.35899999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.36899999999999999,"15min":0.371},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.33200000000000002}]},{"osd":1,"last update":"Wed Apr 24 23:10:10 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39000000000000001,"15min":0.38900000000000001},"min":{"1min":0.28000000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Apr 24 23:09:31 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.503}]},{"osd":4,"last update":"Wed Apr 24 23:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 23:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.33400000000000002,"15min":0.32600000000000001},"min":{"1min":0.28599999999999998,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.502,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.35699999999999998,"15min":0.34799999999999998},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.54400000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.30399999999999999}]},{"osd":2,"last update":"Wed Apr 24 23:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.40100000000000002},"min":{"1min":0.29199999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Apr 24 23:09:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed Apr 24 23:09:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.438},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 23:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.36099999999999999,"15min":0.36499999999999999},"min":{"1min":0.25600000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.44,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.377,"15min":0.38100000000000001},"min":{"1min":0.26500000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.52700000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Apr 24 23:10:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.39200000000000002,"15min":0.40000000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.41099999999999998,"15min":0.41599999999999998},"min":{"1min":0.26500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.36899999999999999}]},{"osd":3,"last update":"Wed Apr 24 23:09:32 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.496}]},{"osd":4,"last update":"Wed Apr 24 23:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.46100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T23:10:17.735 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-24T23:10:17.735 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T23:10:17.735 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-24T23:10:17.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-24T23:10:18.000 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:18.427 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:18 smithi031 ceph-mon[26411]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:18.427 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:18 smithi031 ceph-mon[26411]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:18.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:18 smithi040 ceph-mon[33128]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:18.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:18 smithi040 ceph-mon[33128]: from='client.14416 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-24T23:10:19.423 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:10:19.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:10:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 -- 172.21.15.31:0/2270787485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 msgr2=0x7fd498065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 --2- 172.21.15.31:0/2270787485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 0x7fd498065ab0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd480004940 tx=0x7fd480009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:19.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 -- 172.21.15.31:0/2270787485 shutdown_connections 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 --2- 172.21.15.31:0/2270787485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 0x7fd498065ab0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 --2- 172.21.15.31:0/2270787485 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498102e50 0x7fd498103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 -- 172.21.15.31:0/2270787485 >> 172.21.15.31:0/2270787485 conn(0x7fd498071d00 msgr2=0x7fd498072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 -- 172.21.15.31:0/2270787485 shutdown_connections 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 -- 172.21.15.31:0/2270787485 wait complete. 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.670+0000 7fd49deda700 1 Processor -- start 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd49deda700 1 -- start start 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd49deda700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 0x7fd498113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd49deda700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498102e50 0x7fd4981140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd49deda700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd498114700 con 0x7fd498065640 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd49deda700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49810daf0 con 0x7fd498102e50 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd4977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 0x7fd498113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd4977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 0x7fd498113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36668/0 (socket says 172.21.15.31:36668) 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd4977fe700 1 -- 172.21.15.31:0/1945178565 learned_addr learned my addr 172.21.15.31:0/1945178565 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd496ffd700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498102e50 0x7fd4981140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd4977fe700 1 -- 172.21.15.31:0/1945178565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498102e50 msgr2=0x7fd4981140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd4977fe700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498102e50 0x7fd4981140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.671+0000 7fd4977fe700 1 -- 172.21.15.31:0/1945178565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd480005040 con 0x7fd498065640 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.672+0000 7fd4977fe700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 0x7fd498113b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd490002a50 tx=0x7fd4900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.672+0000 7fd48ffff700 1 -- 172.21.15.31:0/1945178565 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd490007cc0 con 0x7fd498065640 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.672+0000 7fd48ffff700 1 -- 172.21.15.31:0/1945178565 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd49000f040 con 0x7fd498065640 2024-04-24T23:10:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.672+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49810ddd0 con 0x7fd498065640 2024-04-24T23:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.672+0000 7fd48ffff700 1 -- 172.21.15.31:0/1945178565 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd49000da30 con 0x7fd498065640 2024-04-24T23:10:19.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.672+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49810e2e0 con 0x7fd498065640 2024-04-24T23:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.673+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd498100100 con 0x7fd498065640 2024-04-24T23:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.674+0000 7fd48ffff700 1 -- 172.21.15.31:0/1945178565 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd49000db90 con 0x7fd498065640 2024-04-24T23:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.674+0000 7fd48ffff700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd48406d6b0 0x7fd48406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.674+0000 7fd48ffff700 1 -- 172.21.15.31:0/1945178565 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd490086c10 con 0x7fd498065640 2024-04-24T23:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.674+0000 7fd496ffd700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd48406d6b0 0x7fd48406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.674+0000 7fd496ffd700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd48406d6b0 0x7fd48406fb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd49810edd0 tx=0x7fd480009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:19.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.678+0000 7fd48ffff700 1 -- 172.21.15.31:0/1945178565 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd490056c90 con 0x7fd498065640 2024-04-24T23:10:19.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.918+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd49805f5c0 con 0x7fd498065640 2024-04-24T23:10:19.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.918+0000 7fd48ffff700 1 -- 172.21.15.31:0/1945178565 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd490056820 con 0x7fd498065640 2024-04-24T23:10:19.920 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:19.920 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd48406d6b0 msgr2=0x7fd48406fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd48406d6b0 0x7fd48406fb60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd49810edd0 tx=0x7fd480009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 msgr2=0x7fd498113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 0x7fd498113b70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd490002a50 tx=0x7fd4900046b0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 shutdown_connections 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd48406d6b0 0x7fd48406fb60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd498065640 0x7fd498113b70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 --2- 172.21.15.31:0/1945178565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd498102e50 0x7fd4981140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:19.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 >> 172.21.15.31:0/1945178565 conn(0x7fd498071d00 msgr2=0x7fd4980fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 shutdown_connections 2024-04-24T23:10:19.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:19.920+0000 7fd49deda700 1 -- 172.21.15.31:0/1945178565 wait complete. 2024-04-24T23:10:20.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:20 smithi031 ceph-mon[26411]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:20.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:20 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1945178565' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:10:20.571 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-24T23:10:20.572 INFO:tasks.cephadm:Setup complete, yielding 2024-04-24T23:10:20.572 INFO:teuthology.run_tasks:Running task print... 2024-04-24T23:10:20.582 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-24T23:10:20.582 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:10:20.590 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:10:20.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-24T23:10:20.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:20 smithi040 ceph-mon[33128]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:20.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:20 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1945178565' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-24T23:10:20.854 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.350+0000 7fc242506700 1 -- 172.21.15.31:0/625962605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 msgr2=0x7fc23c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.350+0000 7fc242506700 1 --2- 172.21.15.31:0/625962605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 0x7fc23c0fd3d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc230004940 tx=0x7fc230009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.350+0000 7fc242506700 1 -- 172.21.15.31:0/625962605 shutdown_connections 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.350+0000 7fc242506700 1 --2- 172.21.15.31:0/625962605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 0x7fc23c0fd3d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.350+0000 7fc242506700 1 --2- 172.21.15.31:0/625962605 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c0fbcb0 0x7fc23c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.350+0000 7fc242506700 1 -- 172.21.15.31:0/625962605 >> 172.21.15.31:0/625962605 conn(0x7fc23c0f7430 msgr2=0x7fc23c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc242506700 1 -- 172.21.15.31:0/625962605 shutdown_connections 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc242506700 1 -- 172.21.15.31:0/625962605 wait complete. 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc242506700 1 Processor -- start 2024-04-24T23:10:22.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc242506700 1 -- start start 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc242506700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c0fbcb0 0x7fc23c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc242506700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 0x7fc23c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc242506700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c10e4d0 con 0x7fc23c0fcf60 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc242506700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c112e70 con 0x7fc23c0fbcb0 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.351+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 0x7fc23c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc23b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 0x7fc23c10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36688/0 (socket says 172.21.15.31:36688) 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc23b7fe700 1 -- 172.21.15.31:0/594883241 learned_addr learned my addr 172.21.15.31:0/594883241 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc23b7fe700 1 -- 172.21.15.31:0/594883241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c0fbcb0 msgr2=0x7fc23c10d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc23bfff700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c0fbcb0 0x7fc23c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc23b7fe700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c0fbcb0 0x7fc23c10d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc23b7fe700 1 -- 172.21.15.31:0/594883241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc230005040 con 0x7fc23c0fcf60 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc23bfff700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c0fbcb0 0x7fc23c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc23b7fe700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 0x7fc23c10de40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b3b0 tx=0x7fc230009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:22.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc238ff9700 1 -- 172.21.15.31:0/594883241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc23000e040 con 0x7fc23c0fcf60 2024-04-24T23:10:22.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc238ff9700 1 -- 172.21.15.31:0/594883241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc23000bcc0 con 0x7fc23c0fcf60 2024-04-24T23:10:22.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c113070 con 0x7fc23c0fcf60 2024-04-24T23:10:22.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc238ff9700 1 -- 172.21.15.31:0/594883241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc23001c630 con 0x7fc23c0fcf60 2024-04-24T23:10:22.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.352+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c1135e0 con 0x7fc23c0fcf60 2024-04-24T23:10:22.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.353+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23c101af0 con 0x7fc23c0fcf60 2024-04-24T23:10:22.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.354+0000 7fc238ff9700 1 -- 172.21.15.31:0/594883241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc230013980 con 0x7fc23c0fcf60 2024-04-24T23:10:22.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.354+0000 7fc238ff9700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc2240692a0 0x7fc22406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:22.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.354+0000 7fc238ff9700 1 -- 172.21.15.31:0/594883241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc230087bc0 con 0x7fc23c0fcf60 2024-04-24T23:10:22.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.354+0000 7fc23bfff700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc2240692a0 0x7fc22406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:22.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.355+0000 7fc23bfff700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc2240692a0 0x7fc22406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006fd0 tx=0x7fc22c008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:22.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.359+0000 7fc238ff9700 1 -- 172.21.15.31:0/594883241 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc230057cf0 con 0x7fc23c0fcf60 2024-04-24T23:10:22.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:22 smithi031 ceph-mon[26411]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:22.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.551+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fc23c04c710 con 0x7fc23c0fcf60 2024-04-24T23:10:22.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.557+0000 7fc238ff9700 1 -- 172.21.15.31:0/594883241 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fc230057880 con 0x7fc23c0fcf60 2024-04-24T23:10:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc2240692a0 msgr2=0x7fc22406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc2240692a0 0x7fc22406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc22c006fd0 tx=0x7fc22c008040 comp rx=0 tx=0).stop 2024-04-24T23:10:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 msgr2=0x7fc23c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 0x7fc23c10de40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc23000b3b0 tx=0x7fc230009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:22.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 shutdown_connections 2024-04-24T23:10:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc23c0fbcb0 0x7fc23c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc2240692a0 0x7fc22406b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 --2- 172.21.15.31:0/594883241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc23c0fcf60 0x7fc23c10de40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 >> 172.21.15.31:0/594883241 conn(0x7fc23c0f7430 msgr2=0x7fc23c066740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 shutdown_connections 2024-04-24T23:10:22.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:22.560+0000 7fc242506700 1 -- 172.21.15.31:0/594883241 wait complete. 2024-04-24T23:10:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:22 smithi040 ceph-mon[33128]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:23.159 INFO:teuthology.run_tasks:Running task print... 2024-04-24T23:10:23.167 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-24T23:10:23.167 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:10:23.184 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:10:23.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-24T23:10:23.455 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:23.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:23 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/594883241' entity='client.admin' 2024-04-24T23:10:23.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:23.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:23 smithi031 ceph-mon[26411]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:23.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:23.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:23.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:23 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:23.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:23 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/594883241' entity='client.admin' 2024-04-24T23:10:23.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:23.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:23 smithi040 ceph-mon[33128]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:23.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:23.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:23.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:23 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.885+0000 7f43211a1700 1 -- 172.21.15.31:0/2568725726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 msgr2=0x7f431c070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.885+0000 7f43211a1700 1 --2- 172.21.15.31:0/2568725726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 0x7f431c070ca0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4304004940 tx=0x7f4304009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.885+0000 7f43211a1700 1 -- 172.21.15.31:0/2568725726 shutdown_connections 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.885+0000 7f43211a1700 1 --2- 172.21.15.31:0/2568725726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 0x7f431c070ca0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.885+0000 7f43211a1700 1 --2- 172.21.15.31:0/2568725726 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c071de0 0x7f431c0702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.885+0000 7f43211a1700 1 -- 172.21.15.31:0/2568725726 >> 172.21.15.31:0/2568725726 conn(0x7f431c0f8340 msgr2=0x7f431c0fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.885+0000 7f43211a1700 1 -- 172.21.15.31:0/2568725726 shutdown_connections 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.885+0000 7f43211a1700 1 -- 172.21.15.31:0/2568725726 wait complete. 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f43211a1700 1 Processor -- start 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f43211a1700 1 -- start start 2024-04-24T23:10:24.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f43211a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 0x7f431c10fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f43211a1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c071de0 0x7f431c10ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f43211a1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c110550 con 0x7f431c070830 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f43211a1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c110da0 con 0x7f431c071de0 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f431bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 0x7f431c10fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f431bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 0x7f431c10fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36704/0 (socket says 172.21.15.31:36704) 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f431bfff700 1 -- 172.21.15.31:0/3862922362 learned_addr learned my addr 172.21.15.31:0/3862922362 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.886+0000 7f431b7fe700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c071de0 0x7f431c10ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f431bfff700 1 -- 172.21.15.31:0/3862922362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c071de0 msgr2=0x7f431c10ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f431bfff700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c071de0 0x7f431c10ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f431bfff700 1 -- 172.21.15.31:0/3862922362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304005040 con 0x7f431c070830 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f431bfff700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 0x7f431c10fa10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f430c002a80 tx=0x7f430c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:24.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f4318ff9700 1 -- 172.21.15.31:0/3862922362 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f430c007cc0 con 0x7f431c070830 2024-04-24T23:10:24.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f4318ff9700 1 -- 172.21.15.31:0/3862922362 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f430c00f040 con 0x7f431c070830 2024-04-24T23:10:24.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c111080 con 0x7f431c070830 2024-04-24T23:10:24.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f4318ff9700 1 -- 172.21.15.31:0/3862922362 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f430c00dad0 con 0x7f431c070830 2024-04-24T23:10:24.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.887+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1115c0 con 0x7f431c070830 2024-04-24T23:10:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.888+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c0fc000 con 0x7f431c070830 2024-04-24T23:10:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.888+0000 7f4318ff9700 1 -- 172.21.15.31:0/3862922362 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f430c00dc30 con 0x7f431c070830 2024-04-24T23:10:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.889+0000 7f4318ff9700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f43080692a0 0x7f430806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.889+0000 7f4318ff9700 1 -- 172.21.15.31:0/3862922362 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f430c086c10 con 0x7f431c070830 2024-04-24T23:10:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.889+0000 7f431b7fe700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f43080692a0 0x7f430806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:24.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.889+0000 7f431b7fe700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f43080692a0 0x7f430806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f430400b3b0 tx=0x7f4304009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:24.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:24.893+0000 7f4318ff9700 1 -- 172.21.15.31:0/3862922362 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f430c056c90 con 0x7f431c070830 2024-04-24T23:10:25.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.104+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f431c111950 con 0x7f43080692a0 2024-04-24T23:10:25.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.105+0000 7f4318ff9700 1 -- 172.21.15.31:0/3862922362 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f431c111950 con 0x7f43080692a0 2024-04-24T23:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-04-24T23:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-04-24T23:10:25.107 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f43080692a0 msgr2=0x7f430806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f43080692a0 0x7f430806b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f430400b3b0 tx=0x7f4304009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 msgr2=0x7f431c10fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 0x7f431c10fa10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f430c002a80 tx=0x7f430c004600 comp rx=0 tx=0).stop 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 shutdown_connections 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f43080692a0 0x7f430806b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f431c070830 0x7f431c10fa10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 --2- 172.21.15.31:0/3862922362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f431c071de0 0x7f431c10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:25.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 >> 172.21.15.31:0/3862922362 conn(0x7f431c0f8340 msgr2=0x7f431c0f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:25.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.107+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 shutdown_connections 2024-04-24T23:10:25.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:25.108+0000 7f43211a1700 1 -- 172.21.15.31:0/3862922362 wait complete. 2024-04-24T23:10:25.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-24T23:10:25.952 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:25.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:25 smithi031 ceph-mon[26411]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:25.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:25 smithi031 ceph-mon[26411]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:26.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:25 smithi040 ceph-mon[33128]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:26.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:25 smithi040 ceph-mon[33128]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:27.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.534+0000 7f44909c0700 1 -- 172.21.15.31:0/903678965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 msgr2=0x7f448c0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.534+0000 7f44909c0700 1 --2- 172.21.15.31:0/903678965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 0x7f448c0fe440 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4474004940 tx=0x7f4474009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.534+0000 7f44909c0700 1 -- 172.21.15.31:0/903678965 shutdown_connections 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.534+0000 7f44909c0700 1 --2- 172.21.15.31:0/903678965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 0x7f448c0fe440 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.534+0000 7f44909c0700 1 --2- 172.21.15.31:0/903678965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f448c0fce60 0x7f448c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.534+0000 7f44909c0700 1 -- 172.21.15.31:0/903678965 >> 172.21.15.31:0/903678965 conn(0x7f448c0f8560 msgr2=0x7f448c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.534+0000 7f44909c0700 1 -- 172.21.15.31:0/903678965 shutdown_connections 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.534+0000 7f44909c0700 1 -- 172.21.15.31:0/903678965 wait complete. 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f44909c0700 1 Processor -- start 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f44909c0700 1 -- start start 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f44909c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f448c0fce60 0x7f448c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f44909c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 0x7f448c10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f44909c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c10e5b0 con 0x7f448c0fdfd0 2024-04-24T23:10:27.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f44909c0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c10eda0 con 0x7f448c0fce60 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f4489d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 0x7f448c10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f4489d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 0x7f448c10df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36726/0 (socket says 172.21.15.31:36726) 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f4489d9b700 1 -- 172.21.15.31:0/4243572051 learned_addr learned my addr 172.21.15.31:0/4243572051 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f448a59c700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f448c0fce60 0x7f448c10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f4489d9b700 1 -- 172.21.15.31:0/4243572051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f448c0fce60 msgr2=0x7f448c10d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f4489d9b700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f448c0fce60 0x7f448c10d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.535+0000 7f4489d9b700 1 -- 172.21.15.31:0/4243572051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474005040 con 0x7f448c0fdfd0 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.536+0000 7f4489d9b700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 0x7f448c10df20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f447400b3b0 tx=0x7f4474009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.536+0000 7f4482ffd700 1 -- 172.21.15.31:0/4243572051 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f447400e040 con 0x7f448c0fdfd0 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.536+0000 7f4482ffd700 1 -- 172.21.15.31:0/4243572051 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f447400bcc0 con 0x7f448c0fdfd0 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.536+0000 7f4482ffd700 1 -- 172.21.15.31:0/4243572051 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f447401c630 con 0x7f448c0fdfd0 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.536+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c10f020 con 0x7f448c0fdfd0 2024-04-24T23:10:27.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.536+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c10f5e0 con 0x7f448c0fdfd0 2024-04-24T23:10:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.537+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c05f5c0 con 0x7f448c0fdfd0 2024-04-24T23:10:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.537+0000 7f4482ffd700 1 -- 172.21.15.31:0/4243572051 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4474013980 con 0x7f448c0fdfd0 2024-04-24T23:10:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.538+0000 7f4482ffd700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44780692a0 0x7f447806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.538+0000 7f4482ffd700 1 -- 172.21.15.31:0/4243572051 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4474087c90 con 0x7f448c0fdfd0 2024-04-24T23:10:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.538+0000 7f448a59c700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44780692a0 0x7f447806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:27.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.538+0000 7f448a59c700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44780692a0 0x7f447806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4484006fd0 tx=0x7f4484008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:27.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.542+0000 7f4482ffd700 1 -- 172.21.15.31:0/4243572051 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4474057d10 con 0x7f448c0fdfd0 2024-04-24T23:10:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.745+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f448c10f8c0 con 0x7f44780692a0 2024-04-24T23:10:27.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.754+0000 7f4482ffd700 1 -- 172.21.15.31:0/4243572051 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f448c10f8c0 con 0x7f44780692a0 2024-04-24T23:10:27.755 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:10:27.755 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (3m) 2m ago 5m 25.5M - 0.25.0 c8568f914cd2 87840bb14366 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (5m) 2m ago 5m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (4m) 66s ago 4m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (3m) 2m ago 4m 78.1M - 9.4.7 954c08fa6188 44ecf082adf8 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:9283 running (6m) 2m ago 6m 469M - 17.2.7-902-gc61472c8 0de407c1b3af 7b9a4ca2c955 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (4m) 66s ago 4m 434M - 17.2.7-902-gc61472c8 0de407c1b3af bb3f989b92a5 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (6m) 2m ago 6m 42.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (4m) 66s ago 4m 38.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 0541b481dfad 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (4m) 66s ago 4m 16.8M - 1.5.0 0da6a335fe13 7bd2678617bc 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 2m ago 3m 34.3M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (2m) 2m ago 2m 34.8M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (2m) 2m ago 2m 12.7M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (2m) 66s ago 2m 42.0M 5313M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:10:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (97s) 66s ago 97s 37.1M 5313M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:10:27.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (71s) 66s ago 71s 11.0M 5313M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:10:27.757 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 2m ago 4m 38.1M - 2.43.0 a07b618ecd1d aac36626882b 2024-04-24T23:10:27.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44780692a0 msgr2=0x7f447806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44780692a0 0x7f447806b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4484006fd0 tx=0x7f4484008040 comp rx=0 tx=0).stop 2024-04-24T23:10:27.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 msgr2=0x7f448c10df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:27.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 0x7f448c10df20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f447400b3b0 tx=0x7f4474009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:27.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 shutdown_connections 2024-04-24T23:10:27.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f448c0fce60 0x7f448c10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44780692a0 0x7f447806b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 --2- 172.21.15.31:0/4243572051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f448c0fdfd0 0x7f448c10df20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:27.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 >> 172.21.15.31:0/4243572051 conn(0x7f448c0f8560 msgr2=0x7f448c0f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:27.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 shutdown_connections 2024-04-24T23:10:27.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:27.756+0000 7f44909c0700 1 -- 172.21.15.31:0/4243572051 wait complete. 2024-04-24T23:10:27.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:27 smithi031 ceph-mon[26411]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:28.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:27 smithi040 ceph-mon[33128]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:28.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-24T23:10:28.723 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:29.107 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:28 smithi031 ceph-mon[26411]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:29.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:28 smithi040 ceph-mon[33128]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:30.130 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:29 smithi031 ceph-mon[26411]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:30.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:29 smithi040 ceph-mon[33128]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 -- 172.21.15.31:0/3592380212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fed80 msgr2=0x7fc6d80faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 --2- 172.21.15.31:0/3592380212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fed80 0x7fc6d80faec0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8004940 tx=0x7fc6c8009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 -- 172.21.15.31:0/3592380212 shutdown_connections 2024-04-24T23:10:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 --2- 172.21.15.31:0/3592380212 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fed80 0x7fc6d80faec0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:30.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 --2- 172.21.15.31:0/3592380212 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d80fe430 0x7fc6d80fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 -- 172.21.15.31:0/3592380212 >> 172.21.15.31:0/3592380212 conn(0x7fc6d80f6bd0 msgr2=0x7fc6d80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 -- 172.21.15.31:0/3592380212 shutdown_connections 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 -- 172.21.15.31:0/3592380212 wait complete. 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 Processor -- start 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.343+0000 7fc6df4e8700 1 -- start start 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6df4e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fe430 0x7fc6d810d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6df4e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d80fed80 0x7fc6d810dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6df4e8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d810e3e0 con 0x7fc6d80fe430 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6df4e8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d810e550 con 0x7fc6d80fed80 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6dd284700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fe430 0x7fc6d810d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6dd284700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fe430 0x7fc6d810d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42780/0 (socket says 172.21.15.31:42780) 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6dd284700 1 -- 172.21.15.31:0/1101445407 learned_addr learned my addr 172.21.15.31:0/1101445407 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6dd284700 1 -- 172.21.15.31:0/1101445407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d80fed80 msgr2=0x7fc6d810dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6dd284700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d80fed80 0x7fc6d810dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:30.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6dd284700 1 -- 172.21.15.31:0/1101445407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c8005040 con 0x7fc6d80fe430 2024-04-24T23:10:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6dd284700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fe430 0x7fc6d810d8a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400eb40 tx=0x7fc6d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6cdffb700 1 -- 172.21.15.31:0/1101445407 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6d4007cc0 con 0x7fc6d80fe430 2024-04-24T23:10:30.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6cdffb700 1 -- 172.21.15.31:0/1101445407 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6d400f040 con 0x7fc6d80fe430 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d8112f50 con 0x7fc6d80fe430 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.344+0000 7fc6cdffb700 1 -- 172.21.15.31:0/1101445407 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6d4009620 con 0x7fc6d80fe430 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.345+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d8113430 con 0x7fc6d80fe430 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.346+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d804c710 con 0x7fc6d80fe430 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.346+0000 7fc6cdffb700 1 -- 172.21.15.31:0/1101445407 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc6d4009780 con 0x7fc6d80fe430 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.346+0000 7fc6cdffb700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc6c40692a0 0x7fc6c406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.346+0000 7fc6cdffb700 1 -- 172.21.15.31:0/1101445407 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc6d4086ad0 con 0x7fc6d80fe430 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.346+0000 7fc6dca83700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc6c40692a0 0x7fc6c406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:30.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.347+0000 7fc6dca83700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc6c40692a0 0x7fc6c406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009b30 tx=0x7fc6c8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:30.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.351+0000 7fc6cdffb700 1 -- 172.21.15.31:0/1101445407 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc6d404e250 con 0x7fc6d80fe430 2024-04-24T23:10:30.556 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.554+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6d8102480 con 0x7fc6c40692a0 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.558+0000 7fc6cdffb700 1 -- 172.21.15.31:0/1101445407 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1161 (secure 0 0 0) 0x7fc6d8102480 con 0x7fc6c40692a0 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:crash 2/2 2m ago 6m * 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 2/2 2m ago 6m count:2 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:mon 2/2 2m ago 5m smithi031:172.21.15.31=smithi031;smithi040:172.21.15.40=smithi040;count:2 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-24T23:10:30.560 INFO:teuthology.orchestra.run.smithi031.stdout:osd 6 2m ago - 2024-04-24T23:10:30.561 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc6c40692a0 msgr2=0x7fc6c406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc6c40692a0 0x7fc6c406b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009b30 tx=0x7fc6c8009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fe430 msgr2=0x7fc6d810d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fe430 0x7fc6d810d8a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400eb40 tx=0x7fc6d400b040 comp rx=0 tx=0).stop 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 shutdown_connections 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc6c40692a0 0x7fc6c406b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d80fe430 0x7fc6d810d8a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 --2- 172.21.15.31:0/1101445407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc6d80fed80 0x7fc6d810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:30.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 >> 172.21.15.31:0/1101445407 conn(0x7fc6d80f6bd0 msgr2=0x7fc6d80f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 shutdown_connections 2024-04-24T23:10:30.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:30.560+0000 7fc6df4e8700 1 -- 172.21.15.31:0/1101445407 wait complete. 2024-04-24T23:10:31.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-24T23:10:31.457 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:31 smithi040 ceph-mon[33128]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:31 smithi040 ceph-mon[33128]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:32.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:31 smithi031 ceph-mon[26411]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:32.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:31 smithi031 ceph-mon[26411]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.046+0000 7f6b97189700 1 -- 172.21.15.31:0/2153554774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 msgr2=0x7f6b900fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.046+0000 7f6b97189700 1 --2- 172.21.15.31:0/2153554774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 0x7f6b900fe440 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c004940 tx=0x7f6b7c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.046+0000 7f6b97189700 1 -- 172.21.15.31:0/2153554774 shutdown_connections 2024-04-24T23:10:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.046+0000 7f6b97189700 1 --2- 172.21.15.31:0/2153554774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 0x7f6b900fe440 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.046+0000 7f6b97189700 1 --2- 172.21.15.31:0/2153554774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b900fce60 0x7f6b900fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.046+0000 7f6b97189700 1 -- 172.21.15.31:0/2153554774 >> 172.21.15.31:0/2153554774 conn(0x7f6b900f8540 msgr2=0x7f6b900fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.046+0000 7f6b97189700 1 -- 172.21.15.31:0/2153554774 shutdown_connections 2024-04-24T23:10:33.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.046+0000 7f6b97189700 1 -- 172.21.15.31:0/2153554774 wait complete. 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.047+0000 7f6b97189700 1 Processor -- start 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.047+0000 7f6b97189700 1 -- start start 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.047+0000 7f6b97189700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b900fce60 0x7f6b90113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.047+0000 7f6b97189700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 0x7f6b90114490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.047+0000 7f6b97189700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90114a90 con 0x7f6b900fdfd0 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.047+0000 7f6b97189700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90114c00 con 0x7f6b900fce60 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 0x7f6b90114490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 0x7f6b90114490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42798/0 (socket says 172.21.15.31:42798) 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8ffff700 1 -- 172.21.15.31:0/1205304029 learned_addr learned my addr 172.21.15.31:0/1205304029 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b94f25700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b900fce60 0x7f6b90113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8ffff700 1 -- 172.21.15.31:0/1205304029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b900fce60 msgr2=0x7f6b90113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8ffff700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b900fce60 0x7f6b90113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8ffff700 1 -- 172.21.15.31:0/1205304029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c005040 con 0x7f6b900fdfd0 2024-04-24T23:10:33.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8ffff700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 0x7f6b90114490 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009150 tx=0x7f6b7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8d7fa700 1 -- 172.21.15.31:0/1205304029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7c00e040 con 0x7f6b900fdfd0 2024-04-24T23:10:33.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8d7fa700 1 -- 172.21.15.31:0/1205304029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b7c004290 con 0x7f6b900fdfd0 2024-04-24T23:10:33.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b901195b0 con 0x7f6b900fdfd0 2024-04-24T23:10:33.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b8d7fa700 1 -- 172.21.15.31:0/1205304029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b7c01c630 con 0x7f6b900fdfd0 2024-04-24T23:10:33.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.048+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b90119b70 con 0x7f6b900fdfd0 2024-04-24T23:10:33.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.049+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b90101960 con 0x7f6b900fdfd0 2024-04-24T23:10:33.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.050+0000 7f6b8d7fa700 1 -- 172.21.15.31:0/1205304029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6b7c00bbf0 con 0x7f6b900fdfd0 2024-04-24T23:10:33.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.050+0000 7f6b8d7fa700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6b800692a0 0x7f6b8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:33.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.050+0000 7f6b8d7fa700 1 -- 172.21.15.31:0/1205304029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6b7c087a40 con 0x7f6b900fdfd0 2024-04-24T23:10:33.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.050+0000 7f6b94f25700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6b800692a0 0x7f6b8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:33.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.051+0000 7f6b94f25700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6b800692a0 0x7f6b8006b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b901153e0 tx=0x7f6b8400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:33.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.055+0000 7f6b8d7fa700 1 -- 172.21.15.31:0/1205304029 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6b7c057be0 con 0x7f6b900fdfd0 2024-04-24T23:10:33.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.255+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b9002b460 con 0x7f6b800692a0 2024-04-24T23:10:33.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.257+0000 7f6b8d7fa700 1 -- 172.21.15.31:0/1205304029 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f6b9002b460 con 0x7f6b800692a0 2024-04-24T23:10:33.258 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-04-24T23:10:33.258 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-04-24T23:10:33.259 INFO:teuthology.orchestra.run.smithi031.stdout:smithi040 172.21.15.40 2024-04-24T23:10:33.259 INFO:teuthology.orchestra.run.smithi031.stdout:2 hosts in cluster 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6b800692a0 msgr2=0x7f6b8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6b800692a0 0x7f6b8006b750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b901153e0 tx=0x7f6b8400b040 comp rx=0 tx=0).stop 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 msgr2=0x7f6b90114490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 0x7f6b90114490 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009150 tx=0x7f6b7c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 shutdown_connections 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6b900fce60 0x7f6b90113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f6b800692a0 0x7f6b8006b750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 --2- 172.21.15.31:0/1205304029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b900fdfd0 0x7f6b90114490 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:33.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 >> 172.21.15.31:0/1205304029 conn(0x7f6b900f8540 msgr2=0x7f6b900f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:33.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 shutdown_connections 2024-04-24T23:10:33.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:33.259+0000 7f6b97189700 1 -- 172.21.15.31:0/1205304029 wait complete. 2024-04-24T23:10:33.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-24T23:10:34.142 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:34.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:33 smithi040 ceph-mon[33128]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:34.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:33 smithi040 ceph-mon[33128]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:34.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:33 smithi031 ceph-mon[26411]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:34.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:33 smithi031 ceph-mon[26411]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:35.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:34 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:10:35.178 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:34 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:10:35.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 -- 172.21.15.31:0/494059357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 msgr2=0x7fa1840fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 --2- 172.21.15.31:0/494059357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 0x7fa1840fc0c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa17c004940 tx=0x7fa17c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 -- 172.21.15.31:0/494059357 shutdown_connections 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 --2- 172.21.15.31:0/494059357 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840fcf60 0x7fa1840fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 --2- 172.21.15.31:0/494059357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 0x7fa1840fc0c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 -- 172.21.15.31:0/494059357 >> 172.21.15.31:0/494059357 conn(0x7fa1840f7430 msgr2=0x7fa1840f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 -- 172.21.15.31:0/494059357 shutdown_connections 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 -- 172.21.15.31:0/494059357 wait complete. 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 Processor -- start 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.669+0000 7fa1890a3700 1 -- start start 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa1890a3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 0x7fa18410c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa1890a3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840fcf60 0x7fa18410ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa1890a3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18410d460 con 0x7fa1840fbcb0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa1890a3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18410dcb0 con 0x7fa1840fcf60 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa182d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 0x7fa18410c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa182d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 0x7fa18410c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42824/0 (socket says 172.21.15.31:42824) 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa182d9d700 1 -- 172.21.15.31:0/618005738 learned_addr learned my addr 172.21.15.31:0/618005738 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa18259c700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840fcf60 0x7fa18410ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa182d9d700 1 -- 172.21.15.31:0/618005738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840fcf60 msgr2=0x7fa18410ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa182d9d700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840fcf60 0x7fa18410ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa182d9d700 1 -- 172.21.15.31:0/618005738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c005040 con 0x7fa1840fbcb0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa18259c700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840fcf60 0x7fa18410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa182d9d700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 0x7fa18410c920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b3b0 tx=0x7fa17c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa17b7fe700 1 -- 172.21.15.31:0/618005738 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa17c00e040 con 0x7fa1840fbcb0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa17b7fe700 1 -- 172.21.15.31:0/618005738 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa17c00bcc0 con 0x7fa1840fbcb0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa18410df30 con 0x7fa1840fbcb0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa17b7fe700 1 -- 172.21.15.31:0/618005738 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa17c01c630 con 0x7fa1840fbcb0 2024-04-24T23:10:35.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.670+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa18410e470 con 0x7fa1840fbcb0 2024-04-24T23:10:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.671+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18405f5c0 con 0x7fa1840fbcb0 2024-04-24T23:10:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.672+0000 7fa17b7fe700 1 -- 172.21.15.31:0/618005738 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa17c013980 con 0x7fa1840fbcb0 2024-04-24T23:10:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.672+0000 7fa17b7fe700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa1700692a0 0x7fa17006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.672+0000 7fa17b7fe700 1 -- 172.21.15.31:0/618005738 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa17c087c80 con 0x7fa1840fbcb0 2024-04-24T23:10:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.673+0000 7fa18259c700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa1700692a0 0x7fa17006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.673+0000 7fa18259c700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa1700692a0 0x7fa17006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009fd0 tx=0x7fa16c009f20 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:35.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.677+0000 7fa17b7fe700 1 -- 172.21.15.31:0/618005738 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa17c057d00 con 0x7fa1840fbcb0 2024-04-24T23:10:35.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.880+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa18410e7a0 con 0x7fa1700692a0 2024-04-24T23:10:35.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.881+0000 7fa17b7fe700 1 -- 172.21.15.31:0/618005738 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fa18410e7a0 con 0x7fa1700692a0 2024-04-24T23:10:35.883 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-24T23:10:35.883 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-24T23:10:35.883 INFO:teuthology.orchestra.run.smithi031.stdout:smithi040 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009J400BGN 372G No 71s ago Has a FileSystem, LVM detected 2024-04-24T23:10:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.883+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa1700692a0 msgr2=0x7fa17006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.883+0000 7fa1890a3700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa1700692a0 0x7fa17006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009fd0 tx=0x7fa16c009f20 comp rx=0 tx=0).stop 2024-04-24T23:10:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.883+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 msgr2=0x7fa18410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.883+0000 7fa1890a3700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 0x7fa18410c920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00b3b0 tx=0x7fa17c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:35.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.884+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 shutdown_connections 2024-04-24T23:10:35.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.884+0000 7fa1890a3700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa1700692a0 0x7fa17006b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.884+0000 7fa1890a3700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1840fbcb0 0x7fa18410c920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.884+0000 7fa1890a3700 1 --2- 172.21.15.31:0/618005738 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa1840fcf60 0x7fa18410ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:35.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.884+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 >> 172.21.15.31:0/618005738 conn(0x7fa1840f7430 msgr2=0x7fa1840f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:35.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.884+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 shutdown_connections 2024-04-24T23:10:35.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:35.884+0000 7fa1890a3700 1 -- 172.21.15.31:0/618005738 wait complete. 2024-04-24T23:10:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:35 smithi040 ceph-mon[33128]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:36.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:35 smithi031 ceph-mon[26411]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:36.513 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:10:36.523 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:10:36.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-24T23:10:36.790 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:37.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:36 smithi040 ceph-mon[33128]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:37.172 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:36 smithi031 ceph-mon[26411]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:38.103 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:37 smithi031 ceph-mon[26411]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:38.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:37 smithi040 ceph-mon[33128]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.207+0000 7f611d569700 1 -- 172.21.15.31:0/2455454059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 msgr2=0x7f61180ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.207+0000 7f611d569700 1 --2- 172.21.15.31:0/2455454059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 0x7f61180ffa30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6108009ce0 tx=0x7f6108004d10 comp rx=0 tx=0).stop 2024-04-24T23:10:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.207+0000 7f611d569700 1 -- 172.21.15.31:0/2455454059 shutdown_connections 2024-04-24T23:10:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.207+0000 7f611d569700 1 --2- 172.21.15.31:0/2455454059 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61180fff70 0x7f6118102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.207+0000 7f611d569700 1 --2- 172.21.15.31:0/2455454059 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 0x7f61180ffa30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.207+0000 7f611d569700 1 -- 172.21.15.31:0/2455454059 >> 172.21.15.31:0/2455454059 conn(0x7f61180f74c0 msgr2=0x7f61180f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.207+0000 7f611d569700 1 -- 172.21.15.31:0/2455454059 shutdown_connections 2024-04-24T23:10:38.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.207+0000 7f611d569700 1 -- 172.21.15.31:0/2455454059 wait complete. 2024-04-24T23:10:38.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f611d569700 1 Processor -- start 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f611d569700 1 -- start start 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f611d569700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 0x7f61180fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f611d569700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61180fff70 0x7f61180fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f611d569700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61180fbf00 con 0x7f6118066b50 2024-04-24T23:10:38.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f611d569700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61180fc040 con 0x7f61180fff70 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f6116ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 0x7f61180fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f6116ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 0x7f61180fd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42846/0 (socket says 172.21.15.31:42846) 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f6116ffd700 1 -- 172.21.15.31:0/1781861478 learned_addr learned my addr 172.21.15.31:0/1781861478 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f61167fc700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61180fff70 0x7f61180fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f6116ffd700 1 -- 172.21.15.31:0/1781861478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61180fff70 msgr2=0x7f61180fb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:38.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f6116ffd700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61180fff70 0x7f61180fb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:38.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.208+0000 7f6116ffd700 1 -- 172.21.15.31:0/1781861478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6108009940 con 0x7f6118066b50 2024-04-24T23:10:38.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.209+0000 7f6116ffd700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 0x7f61180fd3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6108009380 tx=0x7f6108004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.209+0000 7f610f7fe700 1 -- 172.21.15.31:0/1781861478 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610800e040 con 0x7f6118066b50 2024-04-24T23:10:38.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.209+0000 7f610f7fe700 1 -- 172.21.15.31:0/1781861478 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61080128e0 con 0x7f6118066b50 2024-04-24T23:10:38.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.209+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61180fc2c0 con 0x7f6118066b50 2024-04-24T23:10:38.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.209+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6118115c60 con 0x7f6118066b50 2024-04-24T23:10:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.209+0000 7f610f7fe700 1 -- 172.21.15.31:0/1781861478 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f610801c9f0 con 0x7f6118066b50 2024-04-24T23:10:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.210+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611805f5c0 con 0x7f6118066b50 2024-04-24T23:10:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.210+0000 7f610f7fe700 1 -- 172.21.15.31:0/1781861478 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6108012a50 con 0x7f6118066b50 2024-04-24T23:10:38.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.211+0000 7f610f7fe700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f61040692a0 0x7f610406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.211+0000 7f610f7fe700 1 -- 172.21.15.31:0/1781861478 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f6108088b40 con 0x7f6118066b50 2024-04-24T23:10:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.211+0000 7f61167fc700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f61040692a0 0x7f610406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:38.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.211+0000 7f61167fc700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f61040692a0 0x7f610406b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6100006fd0 tx=0x7f6100008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:38.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.216+0000 7f610f7fe700 1 -- 172.21.15.31:0/1781861478 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6108055130 con 0x7f6118066b50 2024-04-24T23:10:38.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:38.438+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f61180f8fb0 con 0x7f61040692a0 2024-04-24T23:10:39.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:38 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-24T23:10:39.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:38 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-24T23:10:39.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.920+0000 7f610f7fe700 1 -- 172.21.15.31:0/1781861478 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f61180f8fb0 con 0x7f61040692a0 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f61040692a0 msgr2=0x7f610406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f61040692a0 0x7f610406b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6100006fd0 tx=0x7f6100008040 comp rx=0 tx=0).stop 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 msgr2=0x7f61180fd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 0x7f61180fd3c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6108009380 tx=0x7f6108004d10 comp rx=0 tx=0).stop 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 shutdown_connections 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f61040692a0 0x7f610406b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6118066b50 0x7f61180fd3c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 --2- 172.21.15.31:0/1781861478 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f61180fff70 0x7f61180fb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:39.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 >> 172.21.15.31:0/1781861478 conn(0x7f61180f74c0 msgr2=0x7f61180f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:39.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 shutdown_connections 2024-04-24T23:10:39.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:39.923+0000 7f611d569700 1 -- 172.21.15.31:0/1781861478 wait complete. 2024-04-24T23:10:40.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:39 smithi040 ceph-mon[33128]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:39 smithi040 ceph-mon[33128]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:40.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:39 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-24T23:10:40.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:39 smithi040 ceph-mon[33128]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:10:40.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:39 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-24T23:10:40.223 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:39 smithi031 ceph-mon[26411]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:10:40.224 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:39 smithi031 ceph-mon[26411]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:40.224 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:39 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-24T23:10:40.224 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:39 smithi031 ceph-mon[26411]: osdmap e41: 6 total, 6 up, 6 in 2024-04-24T23:10:40.224 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:39 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-24T23:10:40.224 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:39 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[26407]: 2024-04-24T23:10:39.902+0000 7f0d7e217700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:10:40.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T23:10:41.153 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-24T23:10:41.153 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:10:41.153 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-24T23:10:41.153 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: fsmap cephfs:0 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: Saving service mds.cephfs spec with placement count:4 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.xxejxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.xxejxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:41.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:40 smithi031 ceph-mon[26411]: Deploying daemon mds.cephfs.smithi031.xxejxr on smithi031 2024-04-24T23:10:41.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-24T23:10:41.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: osdmap e42: 6 total, 6 up, 6 in 2024-04-24T23:10:41.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-24T23:10:41.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:10:41.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-24T23:10:41.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: osdmap e43: 6 total, 6 up, 6 in 2024-04-24T23:10:41.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: fsmap cephfs:0 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: Saving service mds.cephfs spec with placement count:4 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.xxejxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.xxejxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:40 smithi040 ceph-mon[33128]: Deploying daemon mds.cephfs.smithi031.xxejxr on smithi031 2024-04-24T23:10:41.924 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:42.048 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:41 smithi031 ceph-mon[26411]: pgmap v178: 65 pgs: 7 creating+peering, 10 active+clean, 48 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:42.049 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:41 smithi031 ceph-mon[26411]: osdmap e44: 6 total, 6 up, 6 in 2024-04-24T23:10:42.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:41 smithi040 ceph-mon[33128]: pgmap v178: 65 pgs: 7 creating+peering, 10 active+clean, 48 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:42.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:41 smithi040 ceph-mon[33128]: osdmap e44: 6 total, 6 up, 6 in 2024-04-24T23:10:43.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:42 smithi040 ceph-mon[33128]: osdmap e45: 6 total, 6 up, 6 in 2024-04-24T23:10:43.227 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:42 smithi031 ceph-mon[26411]: osdmap e45: 6 total, 6 up, 6 in 2024-04-24T23:10:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.634+0000 7fb71533c700 1 -- 172.21.15.31:0/2846897761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 msgr2=0x7fb7100fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.634+0000 7fb71533c700 1 --2- 172.21.15.31:0/2846897761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 0x7fb7100fc0c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb700004940 tx=0x7fb700009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.634+0000 7fb71533c700 1 -- 172.21.15.31:0/2846897761 shutdown_connections 2024-04-24T23:10:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.634+0000 7fb71533c700 1 --2- 172.21.15.31:0/2846897761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7100fcf60 0x7fb7100fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.634+0000 7fb71533c700 1 --2- 172.21.15.31:0/2846897761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 0x7fb7100fc0c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.634+0000 7fb71533c700 1 -- 172.21.15.31:0/2846897761 >> 172.21.15.31:0/2846897761 conn(0x7fb7100f7430 msgr2=0x7fb7100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.634+0000 7fb71533c700 1 -- 172.21.15.31:0/2846897761 shutdown_connections 2024-04-24T23:10:43.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.634+0000 7fb71533c700 1 -- 172.21.15.31:0/2846897761 wait complete. 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb71533c700 1 Processor -- start 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb71533c700 1 -- start start 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb71533c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 0x7fb7100ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb71533c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7100fcf60 0x7fb7100fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb71533c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100fe7e0 con 0x7fb7100fbcb0 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb71533c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100fe920 con 0x7fb7100fcf60 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb70effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 0x7fb7100ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb70effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 0x7fb7100ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54908/0 (socket says 172.21.15.31:54908) 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb70effd700 1 -- 172.21.15.31:0/3484601183 learned_addr learned my addr 172.21.15.31:0/3484601183 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:43.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb70e7fc700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7100fcf60 0x7fb7100fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb70effd700 1 -- 172.21.15.31:0/3484601183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7100fcf60 msgr2=0x7fb7100fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb70effd700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7100fcf60 0x7fb7100fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb70effd700 1 -- 172.21.15.31:0/3484601183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb700005040 con 0x7fb7100fbcb0 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.635+0000 7fb70effd700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 0x7fb7100ffd30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b3b0 tx=0x7fb700009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.636+0000 7fb6f77fe700 1 -- 172.21.15.31:0/3484601183 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb70000e040 con 0x7fb7100fbcb0 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.636+0000 7fb6f77fe700 1 -- 172.21.15.31:0/3484601183 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb70000bbf0 con 0x7fb7100fbcb0 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.636+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7100feba0 con 0x7fb7100fbcb0 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.636+0000 7fb6f77fe700 1 -- 172.21.15.31:0/3484601183 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb70001c630 con 0x7fb7100fbcb0 2024-04-24T23:10:43.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.636+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb710115ae0 con 0x7fb7100fbcb0 2024-04-24T23:10:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.637+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb710101af0 con 0x7fb7100fbcb0 2024-04-24T23:10:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.637+0000 7fb6f77fe700 1 -- 172.21.15.31:0/3484601183 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb70000bd60 con 0x7fb7100fbcb0 2024-04-24T23:10:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.638+0000 7fb6f77fe700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6f80692a0 0x7fb6f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.638+0000 7fb6f77fe700 1 -- 172.21.15.31:0/3484601183 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb700087d80 con 0x7fb7100fbcb0 2024-04-24T23:10:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.638+0000 7fb70e7fc700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6f80692a0 0x7fb6f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:43.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.638+0000 7fb70e7fc700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6f80692a0 0x7fb6f806b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb704004f40 tx=0x7fb70400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:43.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.642+0000 7fb6f77fe700 1 -- 172.21.15.31:0/3484601183 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb700054150 con 0x7fb7100fbcb0 2024-04-24T23:10:43.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.890+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb71004bed0 con 0x7fb7100fbcb0 2024-04-24T23:10:43.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.890+0000 7fb6f77fe700 1 -- 172.21.15.31:0/3484601183 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fb700057770 con 0x7fb7100fbcb0 2024-04-24T23:10:43.892 INFO:teuthology.orchestra.run.smithi031.stdout:e2 2024-04-24T23:10:43.892 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 2 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:39.902838+0000 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:10:43.893 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:in 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:up {} 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:10:43.894 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:43.895 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.892+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6f80692a0 msgr2=0x7fb6f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.892+0000 7fb71533c700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6f80692a0 0x7fb6f806b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb704004f40 tx=0x7fb70400b040 comp rx=0 tx=0).stop 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.892+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 msgr2=0x7fb7100ffd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.892+0000 7fb71533c700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 0x7fb7100ffd30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb70000b3b0 tx=0x7fb700009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.893+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 shutdown_connections 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.893+0000 7fb71533c700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb6f80692a0 0x7fb6f806b750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.893+0000 7fb71533c700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7100fbcb0 0x7fb7100ffd30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.893+0000 7fb71533c700 1 --2- 172.21.15.31:0/3484601183 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7100fcf60 0x7fb7100fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.893+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 >> 172.21.15.31:0/3484601183 conn(0x7fb7100f7430 msgr2=0x7fb710065ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.893+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 shutdown_connections 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:43.893+0000 7fb71533c700 1 -- 172.21.15.31:0/3484601183 wait complete. 2024-04-24T23:10:43.896 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-04-24T23:10:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:43 smithi031 ceph-mon[26411]: pgmap v181: 65 pgs: 7 creating+peering, 40 active+clean, 18 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:44.205 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:43 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3484601183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:10:44.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:43 smithi040 ceph-mon[33128]: pgmap v181: 65 pgs: 7 creating+peering, 40 active+clean, 18 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:44.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:43 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3484601183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:10:44.716 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:10:44.726 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:10:44.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-24T23:10:44.999 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:45.435 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.dicevu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.dicevu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: Deploying daemon mds.cephfs.smithi040.dicevu on smithi040 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.31:6826/2779330105,v1:172.21.15.31:6827/2779330105] up:boot 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: daemon mds.cephfs.smithi031.xxejxr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: Cluster is now healthy 2024-04-24T23:10:45.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: fsmap cephfs:0 1 up:standby 2024-04-24T23:10:45.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:10:45.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:creating} 2024-04-24T23:10:45.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:45 smithi031 ceph-mon[26411]: daemon mds.cephfs.smithi031.xxejxr is now active in filesystem cephfs as rank 0 2024-04-24T23:10:45.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:45.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:45.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:45.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.dicevu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:45.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.dicevu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:45.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:45.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: Deploying daemon mds.cephfs.smithi040.dicevu on smithi040 2024-04-24T23:10:45.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: pgmap v182: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:10:45.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.31:6826/2779330105,v1:172.21.15.31:6827/2779330105] up:boot 2024-04-24T23:10:45.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: daemon mds.cephfs.smithi031.xxejxr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-24T23:10:45.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:10:45.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-24T23:10:45.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: Cluster is now healthy 2024-04-24T23:10:45.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: fsmap cephfs:0 1 up:standby 2024-04-24T23:10:45.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:10:45.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:creating} 2024-04-24T23:10:45.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:45 smithi040 ceph-mon[33128]: daemon mds.cephfs.smithi031.xxejxr is now active in filesystem cephfs as rank 0 2024-04-24T23:10:46.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.631+0000 7f8247bba700 1 -- 172.21.15.31:0/957539113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fce60 msgr2=0x7f82400fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:46.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.631+0000 7f8247bba700 1 --2- 172.21.15.31:0/957539113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fce60 0x7f82400fd270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f823c004940 tx=0x7f823c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:46.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.631+0000 7f8247bba700 1 -- 172.21.15.31:0/957539113 shutdown_connections 2024-04-24T23:10:46.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.631+0000 7f8247bba700 1 --2- 172.21.15.31:0/957539113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82400fdfd0 0x7f82400fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:46.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.631+0000 7f8247bba700 1 --2- 172.21.15.31:0/957539113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fce60 0x7f82400fd270 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:46.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.631+0000 7f8247bba700 1 -- 172.21.15.31:0/957539113 >> 172.21.15.31:0/957539113 conn(0x7f82400f8520 msgr2=0x7f82400fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:46.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.631+0000 7f8247bba700 1 -- 172.21.15.31:0/957539113 shutdown_connections 2024-04-24T23:10:46.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.631+0000 7f8247bba700 1 -- 172.21.15.31:0/957539113 wait complete. 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.632+0000 7f8247bba700 1 Processor -- start 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.632+0000 7f8247bba700 1 -- start start 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.632+0000 7f8247bba700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82400fce60 0x7f824010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.632+0000 7f8247bba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fdfd0 0x7f824010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.632+0000 7f8247bba700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824010e530 con 0x7f82400fdfd0 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.632+0000 7f8247bba700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240112ed0 con 0x7f82400fce60 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.632+0000 7f8245155700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fdfd0 0x7f824010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8245956700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82400fce60 0x7f824010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8245155700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fdfd0 0x7f824010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54940/0 (socket says 172.21.15.31:54940) 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8245155700 1 -- 172.21.15.31:0/3658182732 learned_addr learned my addr 172.21.15.31:0/3658182732 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8245155700 1 -- 172.21.15.31:0/3658182732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82400fce60 msgr2=0x7f824010d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8245155700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82400fce60 0x7f824010d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8245155700 1 -- 172.21.15.31:0/3658182732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c005040 con 0x7f82400fdfd0 2024-04-24T23:10:46.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8245155700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fdfd0 0x7f824010dea0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f823000b790 tx=0x7f8230009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f82367fc700 1 -- 172.21.15.31:0/3658182732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f823000f6f0 con 0x7f82400fdfd0 2024-04-24T23:10:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8240113130 con 0x7f82400fdfd0 2024-04-24T23:10:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f82367fc700 1 -- 172.21.15.31:0/3658182732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f823000fd20 con 0x7f82400fdfd0 2024-04-24T23:10:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f82367fc700 1 -- 172.21.15.31:0/3658182732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82300103a0 con 0x7f82400fdfd0 2024-04-24T23:10:46.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.633+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82401136a0 con 0x7f82400fdfd0 2024-04-24T23:10:46.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.634+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824004c710 con 0x7f82400fdfd0 2024-04-24T23:10:46.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.635+0000 7f82367fc700 1 -- 172.21.15.31:0/3658182732 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8230011400 con 0x7f82400fdfd0 2024-04-24T23:10:46.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.635+0000 7f82367fc700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f822c0692a0 0x7f822c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:46.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.635+0000 7f82367fc700 1 -- 172.21.15.31:0/3658182732 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8230086f80 con 0x7f82400fdfd0 2024-04-24T23:10:46.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.635+0000 7f8245956700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f822c0692a0 0x7f822c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:46.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.636+0000 7f8245956700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f822c0692a0 0x7f822c06b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f823c009fd0 tx=0x7f823c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:46.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.640+0000 7f82367fc700 1 -- 172.21.15.31:0/3658182732 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8230056d40 con 0x7f82400fdfd0 2024-04-24T23:10:46.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.882+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f8240113b00 con 0x7f82400fdfd0 2024-04-24T23:10:46.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.930+0000 7f82367fc700 1 -- 172.21.15.31:0/3658182732 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f82300568d0 con 0x7f82400fdfd0 2024-04-24T23:10:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f822c0692a0 msgr2=0x7f822c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f822c0692a0 0x7f822c06b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f823c009fd0 tx=0x7f823c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fdfd0 msgr2=0x7f824010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fdfd0 0x7f824010dea0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f823000b790 tx=0x7f8230009250 comp rx=0 tx=0).stop 2024-04-24T23:10:46.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 shutdown_connections 2024-04-24T23:10:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f82400fce60 0x7f824010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f822c0692a0 0x7f822c06b750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 --2- 172.21.15.31:0/3658182732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82400fdfd0 0x7f824010dea0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 >> 172.21.15.31:0/3658182732 conn(0x7f82400f8520 msgr2=0x7f82400f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 shutdown_connections 2024-04-24T23:10:46.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:46.933+0000 7f8247bba700 1 -- 172.21.15.31:0/3658182732 wait complete. 2024-04-24T23:10:47.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:46 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.31:6826/2779330105,v1:172.21.15.31:6827/2779330105] up:active 2024-04-24T23:10:47.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:46 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 2024-04-24T23:10:47.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:46 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3658182732' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-24T23:10:47.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:46 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.31:6826/2779330105,v1:172.21.15.31:6827/2779330105] up:active 2024-04-24T23:10:47.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:46 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 2024-04-24T23:10:47.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:46 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3658182732' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-24T23:10:47.597 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:10:47.607 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:10:47.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-24T23:10:48.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 5 op/s 2024-04-24T23:10:48.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3658182732' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-24T23:10:48.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:boot 2024-04-24T23:10:48.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 1 up:standby 2024-04-24T23:10:48.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:10:48.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 1 up:standby 2024-04-24T23:10:48.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:48.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:48.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:48.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.acqiox", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:48.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.acqiox", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:48.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:48.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:47 smithi031 ceph-mon[26411]: Deploying daemon mds.cephfs.smithi031.acqiox on smithi031 2024-04-24T23:10:48.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 5 op/s 2024-04-24T23:10:48.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3658182732' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-24T23:10:48.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:boot 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 1 up:standby 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 1 up:standby 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.acqiox", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.acqiox", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:48.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:47 smithi040 ceph-mon[33128]: Deploying daemon mds.cephfs.smithi031.acqiox on smithi031 2024-04-24T23:10:48.661 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:50.213 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:49 smithi031 ceph-mon[26411]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-24T23:10:50.213 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:10:50.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:49 smithi040 ceph-mon[33128]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-24T23:10:50.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:10:50.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 -- 172.21.15.31:0/3935016258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fdfd0 msgr2=0x7fc9040fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 --2- 172.21.15.31:0/3935016258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fdfd0 0x7fc9040fe440 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8004940 tx=0x7fc8f8009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 -- 172.21.15.31:0/3935016258 shutdown_connections 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 --2- 172.21.15.31:0/3935016258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fdfd0 0x7fc9040fe440 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 --2- 172.21.15.31:0/3935016258 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9040fce60 0x7fc9040fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 -- 172.21.15.31:0/3935016258 >> 172.21.15.31:0/3935016258 conn(0x7fc9040f8540 msgr2=0x7fc9040fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 -- 172.21.15.31:0/3935016258 shutdown_connections 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 -- 172.21.15.31:0/3935016258 wait complete. 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 Processor -- start 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.852+0000 7fc908895700 1 -- start start 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc908895700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fce60 0x7fc904071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc908895700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9040fdfd0 0x7fc9040702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc908895700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc904111620 con 0x7fc9040fce60 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc908895700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc904111790 con 0x7fc9040fdfd0 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc90259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fce60 0x7fc904071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc90259c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fce60 0x7fc904071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41932/0 (socket says 172.21.15.31:41932) 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc90259c700 1 -- 172.21.15.31:0/862257922 learned_addr learned my addr 172.21.15.31:0/862257922 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc901d9b700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9040fdfd0 0x7fc9040702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc90259c700 1 -- 172.21.15.31:0/862257922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9040fdfd0 msgr2=0x7fc9040702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc90259c700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9040fdfd0 0x7fc9040702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc90259c700 1 -- 172.21.15.31:0/862257922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f8005040 con 0x7fc9040fce60 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc90259c700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fce60 0x7fc904071c00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400bbb0 tx=0x7fc8f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.853+0000 7fc8f2ffd700 1 -- 172.21.15.31:0/862257922 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f40092e0 con 0x7fc9040fce60 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.854+0000 7fc8f2ffd700 1 -- 172.21.15.31:0/862257922 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f400cb20 con 0x7fc9040fce60 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.854+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9040707f0 con 0x7fc9040fce60 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.854+0000 7fc8f2ffd700 1 -- 172.21.15.31:0/862257922 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f4015d70 con 0x7fc9040fce60 2024-04-24T23:10:50.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.854+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc904070db0 con 0x7fc9040fce60 2024-04-24T23:10:50.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.855+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90404c710 con 0x7fc9040fce60 2024-04-24T23:10:50.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.855+0000 7fc8f2ffd700 1 -- 172.21.15.31:0/862257922 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc8f4012070 con 0x7fc9040fce60 2024-04-24T23:10:50.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.856+0000 7fc8f2ffd700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc8ec0692a0 0x7fc8ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:50.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.856+0000 7fc901d9b700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc8ec0692a0 0x7fc8ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:50.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.856+0000 7fc8f2ffd700 1 -- 172.21.15.31:0/862257922 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc8f4086c70 con 0x7fc9040fce60 2024-04-24T23:10:50.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.856+0000 7fc901d9b700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc8ec0692a0 0x7fc8ec06b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800b970 tx=0x7fc8f8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:50.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:50.860+0000 7fc8f2ffd700 1 -- 172.21.15.31:0/862257922 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc8f4056a30 con 0x7fc9040fce60 2024-04-24T23:10:51.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.104+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc904071020 con 0x7fc9040fce60 2024-04-24T23:10:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.osunbp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.osunbp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: Deploying daemon mds.cephfs.smithi040.osunbp on smithi040 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:boot 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 2 up:standby 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:10:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:51 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/862257922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-24T23:10:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.osunbp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:10:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.osunbp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-24T23:10:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:10:51.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: Deploying daemon mds.cephfs.smithi040.osunbp on smithi040 2024-04-24T23:10:51.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-24T23:10:51.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:boot 2024-04-24T23:10:51.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 2 up:standby 2024-04-24T23:10:51.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:10:51.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:51 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/862257922' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-24T23:10:51.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.936+0000 7fc8f2ffd700 1 -- 172.21.15.31:0/862257922 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc8f40565c0 con 0x7fc9040fce60 2024-04-24T23:10:51.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc8ec0692a0 msgr2=0x7fc8ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc8ec0692a0 0x7fc8ec06b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc8f800b970 tx=0x7fc8f8009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fce60 msgr2=0x7fc904071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fce60 0x7fc904071c00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400bbb0 tx=0x7fc8f4009840 comp rx=0 tx=0).stop 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 shutdown_connections 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc8ec0692a0 0x7fc8ec06b750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9040fce60 0x7fc904071c00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 --2- 172.21.15.31:0/862257922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9040fdfd0 0x7fc9040702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 >> 172.21.15.31:0/862257922 conn(0x7fc9040f8540 msgr2=0x7fc9040f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 shutdown_connections 2024-04-24T23:10:51.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:51.939+0000 7fc908895700 1 -- 172.21.15.31:0/862257922 wait complete. 2024-04-24T23:10:52.527 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:10:52.538 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:10:52.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-24T23:10:52.815 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:52.985 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:52 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/862257922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-24T23:10:52.985 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:52 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 2 up:standby 2024-04-24T23:10:53.231 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:52 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/862257922' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-24T23:10:53.231 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:52 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 2 up:standby 2024-04-24T23:10:54.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-04-24T23:10:54.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:54.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:boot 2024-04-24T23:10:54.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 3 up:standby 2024-04-24T23:10:54.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:10:54.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:54 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:boot 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 3 up:standby 2024-04-24T23:10:54.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:10:54.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:54 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.397+0000 7f0666cf9700 1 -- 172.21.15.31:0/2287364456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fdfd0 msgr2=0x7f06600fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.397+0000 7f0666cf9700 1 --2- 172.21.15.31:0/2287364456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fdfd0 0x7f06600fe440 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f064c004940 tx=0x7f064c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.397+0000 7f0666cf9700 1 -- 172.21.15.31:0/2287364456 shutdown_connections 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.397+0000 7f0666cf9700 1 --2- 172.21.15.31:0/2287364456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fdfd0 0x7f06600fe440 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.397+0000 7f0666cf9700 1 --2- 172.21.15.31:0/2287364456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600fce60 0x7f06600fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.398+0000 7f0666cf9700 1 -- 172.21.15.31:0/2287364456 >> 172.21.15.31:0/2287364456 conn(0x7f06600f8540 msgr2=0x7f06600fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.398+0000 7f0666cf9700 1 -- 172.21.15.31:0/2287364456 shutdown_connections 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.398+0000 7f0666cf9700 1 -- 172.21.15.31:0/2287364456 wait complete. 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.398+0000 7f0666cf9700 1 Processor -- start 2024-04-24T23:10:54.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.398+0000 7f0666cf9700 1 -- start start 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0666cf9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fce60 0x7f066010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0666cf9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600fdfd0 0x7f066010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0666cf9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066010e470 con 0x7f06600fce60 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0666cf9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066010e5e0 con 0x7f06600fdfd0 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0664a95700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fce60 0x7f066010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0664a95700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fce60 0x7f066010d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41958/0 (socket says 172.21.15.31:41958) 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0664a95700 1 -- 172.21.15.31:0/3096002180 learned_addr learned my addr 172.21.15.31:0/3096002180 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f065ffff700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600fdfd0 0x7f066010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0664a95700 1 -- 172.21.15.31:0/3096002180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600fdfd0 msgr2=0x7f066010de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:54.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0664a95700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600fdfd0 0x7f066010de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0664a95700 1 -- 172.21.15.31:0/3096002180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c005040 con 0x7f06600fce60 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f065ffff700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600fdfd0 0x7f066010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0664a95700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fce60 0x7f066010d930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0654002a50 tx=0x7f06540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f065d7fa700 1 -- 172.21.15.31:0/3096002180 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0654007cc0 con 0x7f06600fce60 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f065d7fa700 1 -- 172.21.15.31:0/3096002180 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f065400f040 con 0x7f06600fce60 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0660112fe0 con 0x7f06600fce60 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f065d7fa700 1 -- 172.21.15.31:0/3096002180 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f065400db70 con 0x7f06600fce60 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.399+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0660113570 con 0x7f06600fce60 2024-04-24T23:10:54.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.400+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066004c710 con 0x7f06600fce60 2024-04-24T23:10:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.401+0000 7f065d7fa700 1 -- 172.21.15.31:0/3096002180 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f065400dd10 con 0x7f06600fce60 2024-04-24T23:10:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.401+0000 7f065d7fa700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f06500692a0 0x7f065006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.401+0000 7f065d7fa700 1 -- 172.21.15.31:0/3096002180 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0654086970 con 0x7f06600fce60 2024-04-24T23:10:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.401+0000 7f065ffff700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f06500692a0 0x7f065006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:54.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.402+0000 7f065ffff700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f06500692a0 0x7f065006b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b970 tx=0x7f064c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:54.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.406+0000 7f065d7fa700 1 -- 172.21.15.31:0/3096002180 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0654056610 con 0x7f06600fce60 2024-04-24T23:10:54.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:54.669+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f06601138a0 con 0x7f06600fce60 2024-04-24T23:10:55.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.126+0000 7f065d7fa700 1 -- 172.21.15.31:0/3096002180 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7f06540561a0 con 0x7f06600fce60 2024-04-24T23:10:55.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.128+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f06500692a0 msgr2=0x7f065006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:55.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.128+0000 7f0666cf9700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f06500692a0 0x7f065006b750 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b970 tx=0x7f064c009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:55.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.128+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fce60 msgr2=0x7f066010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:55.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.128+0000 7f0666cf9700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fce60 0x7f066010d930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0654002a50 tx=0x7f06540046b0 comp rx=0 tx=0).stop 2024-04-24T23:10:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.129+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 shutdown_connections 2024-04-24T23:10:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.129+0000 7f0666cf9700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f06500692a0 0x7f065006b750 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.129+0000 7f0666cf9700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06600fce60 0x7f066010d930 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.129+0000 7f0666cf9700 1 --2- 172.21.15.31:0/3096002180 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06600fdfd0 0x7f066010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.129+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 >> 172.21.15.31:0/3096002180 conn(0x7f06600f8540 msgr2=0x7f06600f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.129+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 shutdown_connections 2024-04-24T23:10:55.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:55.129+0000 7f0666cf9700 1 -- 172.21.15.31:0/3096002180 wait complete. 2024-04-24T23:10:55.132 INFO:teuthology.orchestra.run.smithi031.stderr:inline data disabled 2024-04-24T23:10:55.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:55 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3096002180' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-24T23:10:55.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:55 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3096002180' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-24T23:10:55.842 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:10:55.852 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:10:55.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T23:10:56.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:standby 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3096002180' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:standby 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: Dropping low affinity active daemon mds.cephfs.smithi031.xxejxr in favor of higher affinity standby. 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: Replacing daemon mds.cephfs.smithi031.xxejxr as rank 0 with standby daemon mds.cephfs.smithi031.acqiox 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 3 up:standby 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: osdmap e46: 6 total, 6 up, 6 in 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: fsmap cephfs:1/1 {0=cephfs.smithi031.acqiox=up:replay} 2 up:standby 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:56.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:56 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:56.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-24T23:10:56.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:standby 2024-04-24T23:10:56.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3096002180' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-24T23:10:56.436 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:standby 2024-04-24T23:10:56.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: Dropping low affinity active daemon mds.cephfs.smithi031.xxejxr in favor of higher affinity standby. 2024-04-24T23:10:56.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: Replacing daemon mds.cephfs.smithi031.xxejxr as rank 0 with standby daemon mds.cephfs.smithi031.acqiox 2024-04-24T23:10:56.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:10:56.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 3 up:standby 2024-04-24T23:10:56.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: osdmap e46: 6 total, 6 up, 6 in 2024-04-24T23:10:56.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: fsmap cephfs:1/1 {0=cephfs.smithi031.acqiox=up:replay} 2 up:standby 2024-04-24T23:10:56.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:56.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:56 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:10:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:57 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:reconnect 2024-04-24T23:10:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:57 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] up:boot 2024-04-24T23:10:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:57 smithi031 ceph-mon[26411]: fsmap cephfs:1/1 {0=cephfs.smithi031.acqiox=up:reconnect} 3 up:standby 2024-04-24T23:10:57.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:57 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:10:57.409 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:10:57.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:57 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:reconnect 2024-04-24T23:10:57.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:57 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] up:boot 2024-04-24T23:10:57.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:57 smithi040 ceph-mon[33128]: fsmap cephfs:1/1 {0=cephfs.smithi031.acqiox=up:reconnect} 3 up:standby 2024-04-24T23:10:57.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:57 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:10:58.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:58 smithi040 ceph-mon[33128]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 307 B/s wr, 1 op/s 2024-04-24T23:10:58.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:58 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:rejoin 2024-04-24T23:10:58.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:58 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:standby 2024-04-24T23:10:58.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:58 smithi040 ceph-mon[33128]: fsmap cephfs:1/1 {0=cephfs.smithi031.acqiox=up:rejoin} 3 up:standby 2024-04-24T23:10:58.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:58 smithi040 ceph-mon[33128]: daemon mds.cephfs.smithi031.acqiox is now active in filesystem cephfs as rank 0 2024-04-24T23:10:58.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:58 smithi040 ceph-mon[33128]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:10:58.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:58 smithi040 ceph-mon[33128]: Cluster is now healthy 2024-04-24T23:10:58.475 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:58 smithi031 ceph-mon[26411]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 716 B/s rd, 307 B/s wr, 1 op/s 2024-04-24T23:10:58.475 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:58 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:rejoin 2024-04-24T23:10:58.475 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:58 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:standby 2024-04-24T23:10:58.475 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:58 smithi031 ceph-mon[26411]: fsmap cephfs:1/1 {0=cephfs.smithi031.acqiox=up:rejoin} 3 up:standby 2024-04-24T23:10:58.476 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:58 smithi031 ceph-mon[26411]: daemon mds.cephfs.smithi031.acqiox is now active in filesystem cephfs as rank 0 2024-04-24T23:10:58.476 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:58 smithi031 ceph-mon[26411]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:10:58.476 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:58 smithi031 ceph-mon[26411]: Cluster is now healthy 2024-04-24T23:10:59.399 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:59 smithi031 ceph-mon[26411]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:active 2024-04-24T23:10:59.399 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:10:59 smithi031 ceph-mon[26411]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:10:59.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:59 smithi040 ceph-mon[33128]: mds.? [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] up:active 2024-04-24T23:10:59.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:10:59 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:10:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.498+0000 7fe393a43700 1 -- 172.21.15.31:0/1835866269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c070810 msgr2=0x7fe38c070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.498+0000 7fe393a43700 1 --2- 172.21.15.31:0/1835866269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c070810 0x7fe38c070c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe380004940 tx=0x7fe380009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:59.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.498+0000 7fe393a43700 1 -- 172.21.15.31:0/1835866269 shutdown_connections 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.498+0000 7fe393a43700 1 --2- 172.21.15.31:0/1835866269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c070810 0x7fe38c070c80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.498+0000 7fe393a43700 1 --2- 172.21.15.31:0/1835866269 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c071dc0 0x7fe38c0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.498+0000 7fe393a43700 1 -- 172.21.15.31:0/1835866269 >> 172.21.15.31:0/1835866269 conn(0x7fe38c0f8590 msgr2=0x7fe38c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.498+0000 7fe393a43700 1 -- 172.21.15.31:0/1835866269 shutdown_connections 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.498+0000 7fe393a43700 1 -- 172.21.15.31:0/1835866269 wait complete. 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe393a43700 1 Processor -- start 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe393a43700 1 -- start start 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe393a43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c070810 0x7fe38c10fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe393a43700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c071dc0 0x7fe38c110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe393a43700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c110810 con 0x7fe38c071dc0 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe393a43700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c111000 con 0x7fe38c070810 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe390fde700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c071dc0 0x7fe38c110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe390fde700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c071dc0 0x7fe38c110180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45228/0 (socket says 172.21.15.31:45228) 2024-04-24T23:10:59.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe390fde700 1 -- 172.21.15.31:0/3018291033 learned_addr learned my addr 172.21.15.31:0/3018291033 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe390fde700 1 -- 172.21.15.31:0/3018291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c070810 msgr2=0x7fe38c10fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe3917df700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c070810 0x7fe38c10fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe390fde700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c070810 0x7fe38c10fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe390fde700 1 -- 172.21.15.31:0/3018291033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe380005040 con 0x7fe38c071dc0 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe3917df700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c070810 0x7fe38c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.499+0000 7fe390fde700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c071dc0 0x7fe38c110180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe38000b3b0 tx=0x7fe380009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.500+0000 7fe389ffb700 1 -- 172.21.15.31:0/3018291033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe38000e040 con 0x7fe38c071dc0 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.500+0000 7fe389ffb700 1 -- 172.21.15.31:0/3018291033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe38000bcc0 con 0x7fe38c071dc0 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.500+0000 7fe389ffb700 1 -- 172.21.15.31:0/3018291033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe38001c630 con 0x7fe38c071dc0 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.500+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38c111280 con 0x7fe38c071dc0 2024-04-24T23:10:59.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.500+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38c111840 con 0x7fe38c071dc0 2024-04-24T23:10:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.501+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38c05f5c0 con 0x7fe38c071dc0 2024-04-24T23:10:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.501+0000 7fe389ffb700 1 -- 172.21.15.31:0/3018291033 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe380013980 con 0x7fe38c071dc0 2024-04-24T23:10:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.502+0000 7fe389ffb700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fe374069370 0x7fe37406b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:10:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.502+0000 7fe389ffb700 1 -- 172.21.15.31:0/3018291033 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fe380087fe0 con 0x7fe38c071dc0 2024-04-24T23:10:59.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.502+0000 7fe3917df700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fe374069370 0x7fe37406b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:10:59.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.502+0000 7fe3917df700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fe374069370 0x7fe37406b820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe38c071b00 tx=0x7fe3840074f0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:10:59.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.507+0000 7fe389ffb700 1 -- 172.21.15.31:0/3018291033 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe380057cb0 con 0x7fe38c071dc0 2024-04-24T23:10:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.756+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe38c111ab0 con 0x7fe38c071dc0 2024-04-24T23:10:59.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.757+0000 7fe389ffb700 1 -- 172.21.15.31:0/3018291033 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1808 (secure 0 0 0) 0x7fe380057840 con 0x7fe38c071dc0 2024-04-24T23:10:59.758 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:10:59.759 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:10:59.760 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:10:59.761 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fe374069370 msgr2=0x7fe37406b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fe374069370 0x7fe37406b820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe38c071b00 tx=0x7fe3840074f0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c071dc0 msgr2=0x7fe38c110180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c071dc0 0x7fe38c110180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe38000b3b0 tx=0x7fe380009e30 comp rx=0 tx=0).stop 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 shutdown_connections 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe38c070810 0x7fe38c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fe374069370 0x7fe37406b820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 --2- 172.21.15.31:0/3018291033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe38c071dc0 0x7fe38c110180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 >> 172.21.15.31:0/3018291033 conn(0x7fe38c0f8590 msgr2=0x7fe38c0f9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:10:59.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 shutdown_connections 2024-04-24T23:10:59.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:10:59.759+0000 7fe393a43700 1 -- 172.21.15.31:0/3018291033 wait complete. 2024-04-24T23:10:59.763 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:11:00.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:00 smithi040 ceph-mon[33128]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 819 B/s rd, 204 B/s wr, 1 op/s 2024-04-24T23:11:00.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:00 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3018291033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:11:00.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:00 smithi031 ceph-mon[26411]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 819 B/s rd, 204 B/s wr, 1 op/s 2024-04-24T23:11:00.437 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:00 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3018291033' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:11:00.466 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-24T23:11:00.921 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:02.351 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:02 smithi031 ceph-mon[26411]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-24T23:11:02.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:02 smithi040 ceph-mon[33128]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-24T23:11:02.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.589+0000 7f005198b700 1 -- 172.21.15.31:0/2777707035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 msgr2=0x7f004c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.589+0000 7f005198b700 1 --2- 172.21.15.31:0/2777707035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 0x7f004c0fc0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f003c004940 tx=0x7f003c009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:02.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.589+0000 7f005198b700 1 -- 172.21.15.31:0/2777707035 shutdown_connections 2024-04-24T23:11:02.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.589+0000 7f005198b700 1 --2- 172.21.15.31:0/2777707035 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004c0fcf70 0x7f004c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.589+0000 7f005198b700 1 --2- 172.21.15.31:0/2777707035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 0x7f004c0fc0d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.589+0000 7f005198b700 1 -- 172.21.15.31:0/2777707035 >> 172.21.15.31:0/2777707035 conn(0x7f004c0f7420 msgr2=0x7f004c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.589+0000 7f005198b700 1 -- 172.21.15.31:0/2777707035 shutdown_connections 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.589+0000 7f005198b700 1 -- 172.21.15.31:0/2777707035 wait complete. 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f005198b700 1 Processor -- start 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f005198b700 1 -- start start 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f005198b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 0x7f004c10da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f005198b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004c0fcf70 0x7f004c10df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f005198b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c10e540 con 0x7f004c0fbcc0 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f005198b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c10ed40 con 0x7f004c0fcf70 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f004affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 0x7f004c10da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f004affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 0x7f004c10da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45248/0 (socket says 172.21.15.31:45248) 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f004affd700 1 -- 172.21.15.31:0/2265381116 learned_addr learned my addr 172.21.15.31:0/2265381116 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:02.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.590+0000 7f004a7fc700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004c0fcf70 0x7f004c10df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f004a7fc700 1 -- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 msgr2=0x7f004c10da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f004a7fc700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 0x7f004c10da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f004a7fc700 1 -- 172.21.15.31:0/2265381116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c005040 con 0x7f004c0fcf70 2024-04-24T23:11:02.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f004a7fc700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004c0fcf70 0x7f004c10df40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0040009fd0 tx=0x7f004000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f00337fe700 1 -- 172.21.15.31:0/2265381116 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0040004e90 con 0x7f004c0fcf70 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c10efc0 con 0x7f004c0fcf70 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f00337fe700 1 -- 172.21.15.31:0/2265381116 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f004000f040 con 0x7f004c0fcf70 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f00337fe700 1 -- 172.21.15.31:0/2265381116 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f004000ca50 con 0x7f004c0fcf70 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.591+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c10f500 con 0x7f004c0fcf70 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.592+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c101b00 con 0x7f004c0fcf70 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.593+0000 7f00337fe700 1 -- 172.21.15.31:0/2265381116 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f004000cc00 con 0x7f004c0fcf70 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.593+0000 7f00337fe700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f00340692a0 0x7f003406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:02.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.593+0000 7f004affd700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f00340692a0 0x7f003406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:02.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.593+0000 7f00337fe700 1 -- 172.21.15.31:0/2265381116 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0040086c30 con 0x7f004c0fcf70 2024-04-24T23:11:02.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.594+0000 7f004affd700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f00340692a0 0x7f003406b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f003c009b30 tx=0x7f003c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:02.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.598+0000 7f00337fe700 1 -- 172.21.15.31:0/2265381116 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f004004e250 con 0x7f004c0fcf70 2024-04-24T23:11:02.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.869+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f004c04bed0 con 0x7f004c0fcf70 2024-04-24T23:11:02.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.869+0000 7f00337fe700 1 -- 172.21.15.31:0/2265381116 <== mon.1 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4688 (secure 0 0 0) 0x7f00400569d0 con 0x7f004c0fcf70 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f00340692a0 msgr2=0x7f003406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f00340692a0 0x7f003406b750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f003c009b30 tx=0x7f003c009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004c0fcf70 msgr2=0x7f004c10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004c0fcf70 0x7f004c10df40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0040009fd0 tx=0x7f004000b040 comp rx=0 tx=0).stop 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 shutdown_connections 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f00340692a0 0x7f003406b750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c0fbcc0 0x7f004c10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 --2- 172.21.15.31:0/2265381116 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f004c0fcf70 0x7f004c10df40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.871+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 >> 172.21.15.31:0/2265381116 conn(0x7f004c0f7420 msgr2=0x7f004c0663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:02.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.872+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 shutdown_connections 2024-04-24T23:11:02.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:02.872+0000 7f005198b700 1 -- 172.21.15.31:0/2265381116 wait complete. 2024-04-24T23:11:02.874 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:11:02.883 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:11:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:03.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:03.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:03 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2265381116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:03.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:03.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:03.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:03.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:03.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:03.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:03.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:03 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2265381116' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:03.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-24T23:11:03.968 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:04.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:04 smithi031 ceph-mon[26411]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T23:11:04.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:11:04.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:04 smithi040 ceph-mon[33128]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T23:11:04.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:11:05.584 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:05.584 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:05 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:05.584 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:05 smithi031 ceph-mon[26411]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T23:11:05.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:05 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:05.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:05 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:05.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:05 smithi040 ceph-mon[33128]: pgmap v193: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 5.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T23:11:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 -- 172.21.15.31:0/2987951107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fcf60 msgr2=0x7fd7c00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 --2- 172.21.15.31:0/2987951107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fcf60 0x7fd7c00fd3d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0004940 tx=0x7fd7b0009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 -- 172.21.15.31:0/2987951107 shutdown_connections 2024-04-24T23:11:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 --2- 172.21.15.31:0/2987951107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fcf60 0x7fd7c00fd3d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 --2- 172.21.15.31:0/2987951107 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c00fbcb0 0x7fd7c00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 -- 172.21.15.31:0/2987951107 >> 172.21.15.31:0/2987951107 conn(0x7fd7c00f7430 msgr2=0x7fd7c00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:06.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 -- 172.21.15.31:0/2987951107 shutdown_connections 2024-04-24T23:11:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 -- 172.21.15.31:0/2987951107 wait complete. 2024-04-24T23:11:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.113+0000 7fd7c711a700 1 Processor -- start 2024-04-24T23:11:06.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c711a700 1 -- start start 2024-04-24T23:11:06.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c711a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fbcb0 0x7fd7c010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c711a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c00fcf60 0x7fd7c010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c711a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c010d470 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c711a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c010dcc0 con 0x7fd7c00fcf60 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c4eb6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fbcb0 0x7fd7c010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c4eb6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fbcb0 0x7fd7c010c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45272/0 (socket says 172.21.15.31:45272) 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c4eb6700 1 -- 172.21.15.31:0/851542888 learned_addr learned my addr 172.21.15.31:0/851542888 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c4eb6700 1 -- 172.21.15.31:0/851542888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c00fcf60 msgr2=0x7fd7c010ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7bffff700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c00fcf60 0x7fd7c010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c4eb6700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c00fcf60 0x7fd7c010ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c4eb6700 1 -- 172.21.15.31:0/851542888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0005040 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.114+0000 7fd7c4eb6700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fbcb0 0x7fd7c010c930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8001d80 tx=0x7fd7b8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.115+0000 7fd7bd7fa700 1 -- 172.21.15.31:0/851542888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7b8007d10 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.115+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c010dfa0 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.115+0000 7fd7bd7fa700 1 -- 172.21.15.31:0/851542888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7b8007e70 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.115+0000 7fd7bd7fa700 1 -- 172.21.15.31:0/851542888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7b8014640 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.115+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c010e4e0 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.116+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c005f5c0 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.116+0000 7fd7bd7fa700 1 -- 172.21.15.31:0/851542888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd7b800e960 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.116+0000 7fd7bd7fa700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd7ac0692a0 0x7fd7ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:06.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.117+0000 7fd7bd7fa700 1 -- 172.21.15.31:0/851542888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd7b8086fd0 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.117+0000 7fd7bffff700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd7ac0692a0 0x7fd7ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:06.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.117+0000 7fd7bffff700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd7ac0692a0 0x7fd7ac06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000c00 tx=0x7fd7b0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:06.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.121+0000 7fd7bd7fa700 1 -- 172.21.15.31:0/851542888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd7b8056d70 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.406+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fd7c004bed0 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.407+0000 7fd7bd7fa700 1 -- 172.21.15.31:0/851542888 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+99 (secure 0 0 0) 0x7fd7b8056900 con 0x7fd7c00fbcb0 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd7ac0692a0 msgr2=0x7fd7ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd7ac0692a0 0x7fd7ac06b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000c00 tx=0x7fd7b0009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fbcb0 msgr2=0x7fd7c010c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fbcb0 0x7fd7c010c930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8001d80 tx=0x7fd7b8009ec0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 shutdown_connections 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd7ac0692a0 0x7fd7ac06b750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7c00fbcb0 0x7fd7c010c930 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 --2- 172.21.15.31:0/851542888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7c00fcf60 0x7fd7c010ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 >> 172.21.15.31:0/851542888 conn(0x7fd7c00f7430 msgr2=0x7fd7c00f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:06.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 shutdown_connections 2024-04-24T23:11:06.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:06.409+0000 7fd7c711a700 1 -- 172.21.15.31:0/851542888 wait complete. 2024-04-24T23:11:06.422 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:11:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:06 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/851542888' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-24T23:11:06.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:06 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/851542888' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-24T23:11:07.047 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-24T23:11:07.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:11:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.196+0000 7f35fc261640 1 -- 172.21.15.31:0/2009749920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 msgr2=0x7f35f40fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.196+0000 7f35fc261640 1 --2- 172.21.15.31:0/2009749920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 0x7f35f40fee40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f35e40048d0 tx=0x7f35e4009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.196+0000 7f35fc261640 1 -- 172.21.15.31:0/2009749920 shutdown_connections 2024-04-24T23:11:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.196+0000 7f35fc261640 1 --2- 172.21.15.31:0/2009749920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f40ff380 0x7f35f4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.196+0000 7f35fc261640 1 --2- 172.21.15.31:0/2009749920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 0x7f35f40fee40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.196+0000 7f35fc261640 1 -- 172.21.15.31:0/2009749920 >> 172.21.15.31:0/2009749920 conn(0x7f35f40f6810 msgr2=0x7f35f40f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.196+0000 7f35fc261640 1 -- 172.21.15.31:0/2009749920 shutdown_connections 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.196+0000 7f35fc261640 1 -- 172.21.15.31:0/2009749920 wait complete. 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35fc261640 1 Processor -- start 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35fc261640 1 -- start start 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35fc261640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 0x7f35f410ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35fc261640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f40ff380 0x7f35f410f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35fc261640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f410faa0 con 0x7f35f40660d0 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35fc261640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f41144a0 con 0x7f35f40ff380 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35f9ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 0x7f35f410ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35f9ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 0x7f35f410ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45300/0 (socket says 172.21.15.31:45300) 2024-04-24T23:11:07.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35f9ffd640 1 -- 172.21.15.31:0/642978770 learned_addr learned my addr 172.21.15.31:0/642978770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35f97fc640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f40ff380 0x7f35f410f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35f9ffd640 1 -- 172.21.15.31:0/642978770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f40ff380 msgr2=0x7f35f410f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35f9ffd640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f40ff380 0x7f35f410f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.197+0000 7f35f9ffd640 1 -- 172.21.15.31:0/642978770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4005040 con 0x7f35f40660d0 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.198+0000 7f35f9ffd640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 0x7f35f410ef10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35e402bda0 tx=0x7f35e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.198+0000 7f35ea7fc640 1 -- 172.21.15.31:0/642978770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e40094c0 con 0x7f35f40660d0 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.198+0000 7f35ea7fc640 1 -- 172.21.15.31:0/642978770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35e4009620 con 0x7f35f40660d0 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.198+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f41145e0 con 0x7f35f40660d0 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.198+0000 7f35ea7fc640 1 -- 172.21.15.31:0/642978770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35e403b640 con 0x7f35f40660d0 2024-04-24T23:11:07.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.198+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f4114ad0 con 0x7f35f40660d0 2024-04-24T23:11:07.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.199+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f404bcc0 con 0x7f35f40660d0 2024-04-24T23:11:07.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.199+0000 7f35ea7fc640 1 -- 172.21.15.31:0/642978770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f35e402f070 con 0x7f35f40660d0 2024-04-24T23:11:07.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.200+0000 7f35ea7fc640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f35e0069210 0x7f35e006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.200+0000 7f35f97fc640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f35e0069210 0x7f35e006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.200+0000 7f35ea7fc640 1 -- 172.21.15.31:0/642978770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f35e40a9750 con 0x7f35f40660d0 2024-04-24T23:11:07.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.200+0000 7f35f97fc640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f35e0069210 0x7f35e006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f35f0007fd0 tx=0x7f35f0009e90 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.204+0000 7f35ea7fc640 1 -- 172.21.15.31:0/642978770 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f35e4075a30 con 0x7f35f40660d0 2024-04-24T23:11:07.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.384+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35f4104ce0 con 0x7f35f40660d0 2024-04-24T23:11:07.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.385+0000 7f35ea7fc640 1 -- 172.21.15.31:0/642978770 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14348 (secure 0 0 0) 0x7f35e4079050 con 0x7f35f40660d0 2024-04-24T23:11:07.386 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:11:07.386 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":46,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:03:41.065781+0000","modified":"2024-04-24T23:10:55.127049+0000","last_up_change":"2024-04-24T23:09:22.852760+0000","last_in_change":"2024-04-24T23:09:04.280370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:08:05.473853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T23:10:38.874551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T23:10:39.325676+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f1ff679-6b2e-4059-a299-2d2356bbe91c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6803","nonce":1397311567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6805","nonce":1397311567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6809","nonce":1397311567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1397311567},{"type":"v1","addr":"172.21.15.31:6807","nonce":1397311567}]},"public_addr":"172.21.15.31:6803/1397311567","cluster_addr":"172.21.15.31:6805/1397311567","heartbeat_back_addr":"172.21.15.31:6809/1397311567","heartbeat_front_addr":"172.21.15.31:6807/1397311567","state":["exists","up"]},{"osd":1,"uuid":"6885f0d2-70e8-4c54-a381-0b3683a0bc49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6811","nonce":1768854803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6813","nonce":1768854803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6817","nonce":1768854803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1768854803},{"type":"v1","addr":"172.21.15.31:6815","nonce":1768854803}]},"public_addr":"172.21.15.31:6811/1768854803","cluster_addr":"172.21.15.31:6813/1768854803","heartbeat_back_addr":"172.21.15.31:6817/1768854803","heartbeat_front_addr":"172.21.15.31:6815/1768854803","state":["exists","up"]},{"osd":2,"uuid":"9b57c743-5826-4bf9-ac00-0cb539b30224","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6819","nonce":563563256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6821","nonce":563563256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6825","nonce":563563256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":563563256},{"type":"v1","addr":"172.21.15.31:6823","nonce":563563256}]},"public_addr":"172.21.15.31:6819/563563256","cluster_addr":"172.21.15.31:6821/563563256","heartbeat_back_addr":"172.21.15.31:6825/563563256","heartbeat_front_addr":"172.21.15.31:6823/563563256","state":["exists","up"]},{"osd":3,"uuid":"1dc47b54-65da-4579-bcac-2258be1f4cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6801","nonce":824499182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6803","nonce":824499182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6807","nonce":824499182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":824499182},{"type":"v1","addr":"172.21.15.40:6805","nonce":824499182}]},"public_addr":"172.21.15.40:6801/824499182","cluster_addr":"172.21.15.40:6803/824499182","heartbeat_back_addr":"172.21.15.40:6807/824499182","heartbeat_front_addr":"172.21.15.40:6805/824499182","state":["exists","up"]},{"osd":4,"uuid":"e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6809","nonce":1464558178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6811","nonce":1464558178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6815","nonce":1464558178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1464558178},{"type":"v1","addr":"172.21.15.40:6813","nonce":1464558178}]},"public_addr":"172.21.15.40:6809/1464558178","cluster_addr":"172.21.15.40:6811/1464558178","heartbeat_back_addr":"172.21.15.40:6815/1464558178","heartbeat_front_addr":"172.21.15.40:6813/1464558178","state":["exists","up"]},{"osd":5,"uuid":"027b7b7a-afb6-4507-8cb2-274ebadbb2ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6817","nonce":2359746556}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6819","nonce":2359746556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6823","nonce":2359746556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":2359746556},{"type":"v1","addr":"172.21.15.40:6821","nonce":2359746556}]},"public_addr":"172.21.15.40:6817/2359746556","cluster_addr":"172.21.15.40:6819/2359746556","heartbeat_back_addr":"172.21.15.40:6823/2359746556","heartbeat_front_addr":"172.21.15.40:6821/2359746556","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:08.221856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:36.407326+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:03.612047+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:29.069233+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:55.660200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:21.507297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/611668999":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/3645410271":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6827/2779330105":"2024-04-25T23:10:55.127017+0000","172.21.15.31:0/1340983696":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/382933206":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6801/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/2308793407":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6800/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6800/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/1654686211":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6801/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/2457576751":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/39610504":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6800/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6801/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/1805904019":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6826/2779330105":"2024-04-25T23:10:55.127017+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:11:07.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f35e0069210 msgr2=0x7f35e006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f35e0069210 0x7f35e006b6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f35f0007fd0 tx=0x7f35f0009e90 comp rx=0 tx=0).stop 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 msgr2=0x7f35f410ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 0x7f35f410ef10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f35e402bda0 tx=0x7f35e4009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 shutdown_connections 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f35e0069210 0x7f35e006b6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f40ff380 0x7f35f410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 --2- 172.21.15.31:0/642978770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f40660d0 0x7f35f410ef10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 >> 172.21.15.31:0/642978770 conn(0x7f35f40f6810 msgr2=0x7f35f40faf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 shutdown_connections 2024-04-24T23:11:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.387+0000 7f35fc261640 1 -- 172.21.15.31:0/642978770 wait complete. 2024-04-24T23:11:07.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:08:05.473853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T23:10:38.874551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T23:10:39.325676+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-24T23:11:07.401 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:11:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 -- 172.21.15.31:0/331873448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fc160 msgr2=0x7fc4540fc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 --2- 172.21.15.31:0/331873448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fc160 0x7fc4540fc570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc4500048d0 tx=0x7fc450009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 -- 172.21.15.31:0/331873448 shutdown_connections 2024-04-24T23:11:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 --2- 172.21.15.31:0/331873448 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4540fd430 0x7fc4540fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 --2- 172.21.15.31:0/331873448 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fc160 0x7fc4540fc570 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 -- 172.21.15.31:0/331873448 >> 172.21.15.31:0/331873448 conn(0x7fc4540f7920 msgr2=0x7fc4540f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 -- 172.21.15.31:0/331873448 shutdown_connections 2024-04-24T23:11:07.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 -- 172.21.15.31:0/331873448 wait complete. 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 Processor -- start 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.507+0000 7fc45c46e640 1 -- start start 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc45c46e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4540fc160 0x7fc45410cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc45c46e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fd430 0x7fc45410d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc45c46e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45410d800 con 0x7fc4540fd430 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc45c46e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4541121c0 con 0x7fc4540fc160 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc459a09640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fd430 0x7fc45410d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc459a09640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fd430 0x7fc45410d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45326/0 (socket says 172.21.15.31:45326) 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc459a09640 1 -- 172.21.15.31:0/3824984962 learned_addr learned my addr 172.21.15.31:0/3824984962 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc45a20a640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4540fc160 0x7fc45410cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc459a09640 1 -- 172.21.15.31:0/3824984962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4540fc160 msgr2=0x7fc45410cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc459a09640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4540fc160 0x7fc45410cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc459a09640 1 -- 172.21.15.31:0/3824984962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc450005040 con 0x7fc4540fd430 2024-04-24T23:11:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc459a09640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fd430 0x7fc45410d1b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc444002a50 tx=0x7fc444004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc44affd640 1 -- 172.21.15.31:0/3824984962 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc444007c30 con 0x7fc4540fd430 2024-04-24T23:11:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc44affd640 1 -- 172.21.15.31:0/3824984962 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc44400f040 con 0x7fc4540fd430 2024-04-24T23:11:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc454112360 con 0x7fc4540fd430 2024-04-24T23:11:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc44affd640 1 -- 172.21.15.31:0/3824984962 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc44400dbe0 con 0x7fc4540fd430 2024-04-24T23:11:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.508+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc454112830 con 0x7fc4540fd430 2024-04-24T23:11:07.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.509+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45404bcc0 con 0x7fc4540fd430 2024-04-24T23:11:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.510+0000 7fc44affd640 1 -- 172.21.15.31:0/3824984962 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc44400dd40 con 0x7fc4540fd430 2024-04-24T23:11:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.510+0000 7fc44affd640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc440069210 0x7fc44006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.510+0000 7fc44affd640 1 -- 172.21.15.31:0/3824984962 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc444087240 con 0x7fc4540fd430 2024-04-24T23:11:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.510+0000 7fc45a20a640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc440069210 0x7fc44006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.511+0000 7fc45a20a640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc440069210 0x7fc44006b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc450031640 tx=0x7fc450009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.515+0000 7fc44affd640 1 -- 172.21.15.31:0/3824984962 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc444053400 con 0x7fc4540fd430 2024-04-24T23:11:07.656 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:07 smithi031 ceph-mon[26411]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T23:11:07.657 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:07 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/642978770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:07 smithi040 ceph-mon[33128]: pgmap v194: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 4.4 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-24T23:11:07.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:07 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/642978770' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:11:07.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.693+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc454112a50 con 0x7fc4540fd430 2024-04-24T23:11:07.695 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.694+0000 7fc44affd640 1 -- 172.21.15.31:0/3824984962 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc444056a20 con 0x7fc4540fd430 2024-04-24T23:11:07.696 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-24T23:11:07.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc440069210 msgr2=0x7fc44006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc440069210 0x7fc44006b6c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc450031640 tx=0x7fc450009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fd430 msgr2=0x7fc45410d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fd430 0x7fc45410d1b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc444002a50 tx=0x7fc444004600 comp rx=0 tx=0).stop 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 shutdown_connections 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc440069210 0x7fc44006b6c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4540fd430 0x7fc45410d1b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 --2- 172.21.15.31:0/3824984962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc4540fc160 0x7fc45410cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 >> 172.21.15.31:0/3824984962 conn(0x7fc4540f7920 msgr2=0x7fc4540f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 shutdown_connections 2024-04-24T23:11:07.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.696+0000 7fc45c46e640 1 -- 172.21.15.31:0/3824984962 wait complete. 2024-04-24T23:11:07.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T23:11:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 -- 172.21.15.31:0/482915786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 msgr2=0x7f56ec0fc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 --2- 172.21.15.31:0/482915786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 0x7f56ec0fc7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f56e80048d0 tx=0x7f56e8009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 -- 172.21.15.31:0/482915786 shutdown_connections 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 --2- 172.21.15.31:0/482915786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 0x7f56ec0fc7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 --2- 172.21.15.31:0/482915786 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0fb060 0x7f56ec0fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 -- 172.21.15.31:0/482915786 >> 172.21.15.31:0/482915786 conn(0x7f56ec0f6820 msgr2=0x7f56ec0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 -- 172.21.15.31:0/482915786 shutdown_connections 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 -- 172.21.15.31:0/482915786 wait complete. 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 Processor -- start 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.815+0000 7f56f391d640 1 -- start start 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f391d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0fb060 0x7f56ec0fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f391d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 0x7f56ec0fd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f391d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0fdbb0 con 0x7f56ec0fc330 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f391d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ec0fdd20 con 0x7f56ec0fb060 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f0eb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 0x7f56ec0fd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f0eb8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 0x7f56ec0fd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45334/0 (socket says 172.21.15.31:45334) 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f0eb8640 1 -- 172.21.15.31:0/3210752638 learned_addr learned my addr 172.21.15.31:0/3210752638 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f16b9640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0fb060 0x7f56ec0fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f0eb8640 1 -- 172.21.15.31:0/3210752638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0fb060 msgr2=0x7f56ec0fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f0eb8640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0fb060 0x7f56ec0fefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f0eb8640 1 -- 172.21.15.31:0/3210752638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e8005040 con 0x7f56ec0fc330 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f0eb8640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 0x7f56ec0fd670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f56e80048a0 tx=0x7f56e8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56e1ffb640 1 -- 172.21.15.31:0/3210752638 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e803d630 con 0x7f56ec0fc330 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ec0fdfa0 con 0x7f56ec0fc330 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56e1ffb640 1 -- 172.21.15.31:0/3210752638 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56e8009530 con 0x7f56ec0fc330 2024-04-24T23:11:07.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.816+0000 7f56e1ffb640 1 -- 172.21.15.31:0/3210752638 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56e803bb30 con 0x7f56ec0fc330 2024-04-24T23:11:07.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.817+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ec0fe4c0 con 0x7f56ec0fc330 2024-04-24T23:11:07.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.818+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ec04bcc0 con 0x7f56ec0fc330 2024-04-24T23:11:07.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.818+0000 7f56e1ffb640 1 -- 172.21.15.31:0/3210752638 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f56e803d790 con 0x7f56ec0fc330 2024-04-24T23:11:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.819+0000 7f56e1ffb640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f56d8069210 0x7f56d806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.819+0000 7f56e1ffb640 1 -- 172.21.15.31:0/3210752638 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f56e802d040 con 0x7f56ec0fc330 2024-04-24T23:11:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.819+0000 7f56f16b9640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f56d8069210 0x7f56d806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:07.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.819+0000 7f56f16b9640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f56d8069210 0x7f56d806b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56dc006fd0 tx=0x7f56dc009060 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:07.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:07.823+0000 7f56e1ffb640 1 -- 172.21.15.31:0/3210752638 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f56e8076520 con 0x7f56ec0fc330 2024-04-24T23:11:08.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.002+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f56ec02a8e0 con 0x7f56ec0fc330 2024-04-24T23:11:08.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.003+0000 7f56e1ffb640 1 -- 172.21.15.31:0/3210752638 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f56e807f020 con 0x7f56ec0fc330 2024-04-24T23:11:08.005 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-24T23:11:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f56d8069210 msgr2=0x7f56d806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f56d8069210 0x7f56d806b6c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f56dc006fd0 tx=0x7f56dc009060 comp rx=0 tx=0).stop 2024-04-24T23:11:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 msgr2=0x7f56ec0fd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 0x7f56ec0fd670 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f56e80048a0 tx=0x7f56e8009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 shutdown_connections 2024-04-24T23:11:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f56d8069210 0x7f56d806b6c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56ec0fc330 0x7f56ec0fd670 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 --2- 172.21.15.31:0/3210752638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56ec0fb060 0x7f56ec0fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 >> 172.21.15.31:0/3210752638 conn(0x7f56ec0f6820 msgr2=0x7f56ec064900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 shutdown_connections 2024-04-24T23:11:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.005+0000 7f56f391d640 1 -- 172.21.15.31:0/3210752638 wait complete. 2024-04-24T23:11:08.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T23:11:08.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 -- 172.21.15.31:0/3330198437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d0064f90 msgr2=0x7f22d00fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 --2- 172.21.15.31:0/3330198437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d0064f90 0x7f22d00fedf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f22c00048d0 tx=0x7f22c0009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 -- 172.21.15.31:0/3330198437 shutdown_connections 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 --2- 172.21.15.31:0/3330198437 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d00ff330 0x7f22d0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 --2- 172.21.15.31:0/3330198437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d0064f90 0x7f22d00fedf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 -- 172.21.15.31:0/3330198437 >> 172.21.15.31:0/3330198437 conn(0x7f22d00f6820 msgr2=0x7f22d00f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 -- 172.21.15.31:0/3330198437 shutdown_connections 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 -- 172.21.15.31:0/3330198437 wait complete. 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 Processor -- start 2024-04-24T23:11:08.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.117+0000 7f22d722e640 1 -- start start 2024-04-24T23:11:08.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22d722e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d0064f90 0x7f22d010cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22d722e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d00ff330 0x7f22d010d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22d722e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d010d8d0 con 0x7f22d00ff330 2024-04-24T23:11:08.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22d722e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d0112290 con 0x7f22d0064f90 2024-04-24T23:11:08.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d00ff330 0x7f22d010d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d00ff330 0x7f22d010d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45358/0 (socket says 172.21.15.31:45358) 2024-04-24T23:11:08.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cffff640 1 -- 172.21.15.31:0/1350125611 learned_addr learned my addr 172.21.15.31:0/1350125611 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cffff640 1 -- 172.21.15.31:0/1350125611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d0064f90 msgr2=0x7f22d010cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22d4fca640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d0064f90 0x7f22d010cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cffff640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d0064f90 0x7f22d010cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cffff640 1 -- 172.21.15.31:0/1350125611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0005040 con 0x7f22d00ff330 2024-04-24T23:11:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cffff640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d00ff330 0x7f22d010d280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f22c4002870 tx=0x7f22c4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cd7fa640 1 -- 172.21.15.31:0/1350125611 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22c4007c30 con 0x7f22d00ff330 2024-04-24T23:11:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cd7fa640 1 -- 172.21.15.31:0/1350125611 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22c400f040 con 0x7f22d00ff330 2024-04-24T23:11:08.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d0112430 con 0x7f22d00ff330 2024-04-24T23:11:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22cd7fa640 1 -- 172.21.15.31:0/1350125611 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22c40095f0 con 0x7f22d00ff330 2024-04-24T23:11:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.118+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d0112900 con 0x7f22d00ff330 2024-04-24T23:11:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.119+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d004bcc0 con 0x7f22d00ff330 2024-04-24T23:11:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.120+0000 7f22cd7fa640 1 -- 172.21.15.31:0/1350125611 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f22c4009750 con 0x7f22d00ff330 2024-04-24T23:11:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.120+0000 7f22cd7fa640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22b8069210 0x7f22b806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.120+0000 7f22cd7fa640 1 -- 172.21.15.31:0/1350125611 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f22c4086970 con 0x7f22d00ff330 2024-04-24T23:11:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.120+0000 7f22d4fca640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22b8069210 0x7f22b806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.121+0000 7f22d4fca640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22b8069210 0x7f22b806b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f22c00317e0 tx=0x7f22c0009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:08.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.125+0000 7f22cd7fa640 1 -- 172.21.15.31:0/1350125611 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f22c404d250 con 0x7f22d00ff330 2024-04-24T23:11:08.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.316+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f22d0104d00 con 0x7f22d00ff330 2024-04-24T23:11:08.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.316+0000 7f22cd7fa640 1 -- 172.21.15.31:0/1350125611 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f22c4052c50 con 0x7f22d00ff330 2024-04-24T23:11:08.318 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22b8069210 msgr2=0x7f22b806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22b8069210 0x7f22b806b6c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f22c00317e0 tx=0x7f22c0009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d00ff330 msgr2=0x7f22d010d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d00ff330 0x7f22d010d280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f22c4002870 tx=0x7f22c4004600 comp rx=0 tx=0).stop 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 shutdown_connections 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22b8069210 0x7f22b806b6c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22d00ff330 0x7f22d010d280 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 --2- 172.21.15.31:0/1350125611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22d0064f90 0x7f22d010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 >> 172.21.15.31:0/1350125611 conn(0x7f22d00f6820 msgr2=0x7f22d00f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 shutdown_connections 2024-04-24T23:11:08.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.318+0000 7f22d722e640 1 -- 172.21.15.31:0/1350125611 wait complete. 2024-04-24T23:11:08.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:11:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.426+0000 7fc3df59d640 1 -- 172.21.15.31:0/1998042954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 msgr2=0x7fc3d81016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.426+0000 7fc3df59d640 1 --2- 172.21.15.31:0/1998042954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 0x7fc3d81016f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0048d0 tx=0x7fc3cc009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 -- 172.21.15.31:0/1998042954 shutdown_connections 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 --2- 172.21.15.31:0/1998042954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 0x7fc3d81016f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 --2- 172.21.15.31:0/1998042954 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d8064fb0 0x7fc3d80fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 -- 172.21.15.31:0/1998042954 >> 172.21.15.31:0/1998042954 conn(0x7fc3d80f6820 msgr2=0x7fc3d80f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 -- 172.21.15.31:0/1998042954 shutdown_connections 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 -- 172.21.15.31:0/1998042954 wait complete. 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 Processor -- start 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 -- start start 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d8064fb0 0x7fc3d810cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 0x7fc3d810d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d810d8f0 con 0x7fc3d80ff310 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3df59d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d8112260 con 0x7fc3d8064fb0 2024-04-24T23:11:08.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3dcb38640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 0x7fc3d810d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.427+0000 7fc3dcb38640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 0x7fc3d810d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45378/0 (socket says 172.21.15.31:45378) 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3dcb38640 1 -- 172.21.15.31:0/3757817197 learned_addr learned my addr 172.21.15.31:0/3757817197 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3dd339640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d8064fb0 0x7fc3d810cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3dcb38640 1 -- 172.21.15.31:0/3757817197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d8064fb0 msgr2=0x7fc3d810cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3dcb38640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d8064fb0 0x7fc3d810cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3dcb38640 1 -- 172.21.15.31:0/3757817197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc005040 con 0x7fc3d80ff310 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3dd339640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d8064fb0 0x7fc3d810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3dcb38640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 0x7fc3d810d2a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0043d0 tx=0x7fc3cc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3d5ffb640 1 -- 172.21.15.31:0/3757817197 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3cc030e60 con 0x7fc3d80ff310 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3d5ffb640 1 -- 172.21.15.31:0/3757817197 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3cc0092e0 con 0x7fc3d80ff310 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d81123a0 con 0x7fc3d80ff310 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3d5ffb640 1 -- 172.21.15.31:0/3757817197 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3cc043640 con 0x7fc3d80ff310 2024-04-24T23:11:08.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.428+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d8112870 con 0x7fc3d80ff310 2024-04-24T23:11:08.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.429+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d804bcc0 con 0x7fc3d80ff310 2024-04-24T23:11:08.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.430+0000 7fc3d5ffb640 1 -- 172.21.15.31:0/3757817197 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc3cc03b9f0 con 0x7fc3d80ff310 2024-04-24T23:11:08.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.430+0000 7fc3d5ffb640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc3c0069210 0x7fc3c006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:08.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.430+0000 7fc3d5ffb640 1 -- 172.21.15.31:0/3757817197 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fc3cc0a9ff0 con 0x7fc3d80ff310 2024-04-24T23:11:08.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.430+0000 7fc3dd339640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc3c0069210 0x7fc3c006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:08.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.431+0000 7fc3dd339640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc3c0069210 0x7fc3c006b6c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8007fd0 tx=0x7fc3c800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:08.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.435+0000 7fc3d5ffb640 1 -- 172.21.15.31:0/3757817197 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc3cc0762d0 con 0x7fc3d80ff310 2024-04-24T23:11:08.615 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:08 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3824984962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:11:08.615 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:08 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3210752638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T23:11:08.615 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:08 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1350125611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T23:11:08.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.650+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3d8112a90 con 0x7fc3d80ff310 2024-04-24T23:11:08.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.651+0000 7fc3d5ffb640 1 -- 172.21.15.31:0/3757817197 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4688 (secure 0 0 0) 0x7fc3d8112a90 con 0x7fc3d80ff310 2024-04-24T23:11:08.652 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:11:08.652 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/1388844653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":1388844653},{"type":"v1","addr":"172.21.15.31:6827","nonce":1388844653}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:10:58.131406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14464":{"gid":14464,"name":"cephfs.smithi031.acqiox","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6829/1027887614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1027887614},{"type":"v1","addr":"172.21.15.31:6829","nonce":1027887614}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-24T23:11:08.653 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:11:08.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc3c0069210 msgr2=0x7fc3c006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc3c0069210 0x7fc3c006b6c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8007fd0 tx=0x7fc3c800b040 comp rx=0 tx=0).stop 2024-04-24T23:11:08.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 msgr2=0x7fc3d810d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 0x7fc3d810d2a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0043d0 tx=0x7fc3cc009da0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 shutdown_connections 2024-04-24T23:11:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fc3c0069210 0x7fc3c006b6c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3d80ff310 0x7fc3d810d2a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 --2- 172.21.15.31:0/3757817197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3d8064fb0 0x7fc3d810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.652+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 >> 172.21.15.31:0/3757817197 conn(0x7fc3d80f6820 msgr2=0x7fc3d80f6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.653+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 shutdown_connections 2024-04-24T23:11:08.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:08.653+0000 7fc3df59d640 1 -- 172.21.15.31:0/3757817197 wait complete. 2024-04-24T23:11:08.665 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 18} 2024-04-24T23:11:08.665 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-24T23:11:08.690 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-24T23:11:08.691 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-24T23:11:08.691 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-24T23:11:08.691 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:11:08.691 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:08.691 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-24T23:11:08.691 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-24T23:11:08.691 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:08.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:08.692 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-24T23:11:08.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:08.712 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2024-04-24T23:11:08.777 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:08 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3824984962' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:11:08.778 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:08 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3210752638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T23:11:08.778 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:08 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1350125611' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T23:11:08.785 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2024-04-24T23:11:08.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:11:08.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:08.787 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-24T23:11:08.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:08.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-04-24T23:11:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-04-24T23:11:08.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:11:08.871 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-24T23:11:08.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:11:09.209 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:09.585 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:09 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3757817197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:09.585 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:09 smithi031 ceph-mon[26411]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-24T23:11:09.585 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:09.585 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:09.585 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:09.585 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:09.586 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:09 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:09 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3757817197' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:11:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:09 smithi040 ceph-mon[33128]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-24T23:11:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:09 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:09 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:09 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:09.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:09 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:09.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:09 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 -- 172.21.15.31:0/2445470506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 msgr2=0x7f0e3c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 --2- 172.21.15.31:0/2445470506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 0x7f0e3c0fcdf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e28004940 tx=0x7f0e28009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 -- 172.21.15.31:0/2445470506 shutdown_connections 2024-04-24T23:11:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 --2- 172.21.15.31:0/2445470506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 0x7f0e3c0fcdf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 --2- 172.21.15.31:0/2445470506 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e3c102de0 0x7f0e3c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 -- 172.21.15.31:0/2445470506 >> 172.21.15.31:0/2445470506 conn(0x7f0e3c0f8990 msgr2=0x7f0e3c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:10.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 -- 172.21.15.31:0/2445470506 shutdown_connections 2024-04-24T23:11:10.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 -- 172.21.15.31:0/2445470506 wait complete. 2024-04-24T23:11:10.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 Processor -- start 2024-04-24T23:11:10.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.754+0000 7f0e42e83700 1 -- start start 2024-04-24T23:11:10.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e42e83700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e3c102de0 0x7f0e3c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e42e83700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 0x7f0e3c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e42e83700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0721b0 con 0x7f0e3c103780 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e42e83700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c070ce0 con 0x7f0e3c102de0 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 0x7f0e3c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 0x7f0e3c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33452/0 (socket says 172.21.15.31:33452) 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e3bfff700 1 -- 172.21.15.31:0/2251696123 learned_addr learned my addr 172.21.15.31:0/2251696123 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e40c1f700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e3c102de0 0x7f0e3c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e3bfff700 1 -- 172.21.15.31:0/2251696123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e3c102de0 msgr2=0x7f0e3c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e3bfff700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e3c102de0 0x7f0e3c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e3bfff700 1 -- 172.21.15.31:0/2251696123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e28005040 con 0x7f0e3c103780 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.755+0000 7f0e3bfff700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 0x7f0e3c0707a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009180 tx=0x7f0e28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.756+0000 7f0e397fa700 1 -- 172.21.15.31:0/2251696123 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e2800e040 con 0x7f0e3c103780 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.756+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c070f60 con 0x7f0e3c103780 2024-04-24T23:11:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.756+0000 7f0e397fa700 1 -- 172.21.15.31:0/2251696123 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e28013420 con 0x7f0e3c103780 2024-04-24T23:11:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.756+0000 7f0e397fa700 1 -- 172.21.15.31:0/2251696123 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e2801c610 con 0x7f0e3c103780 2024-04-24T23:11:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.756+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c11a0d0 con 0x7f0e3c103780 2024-04-24T23:11:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.757+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c1052b0 con 0x7f0e3c103780 2024-04-24T23:11:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.757+0000 7f0e397fa700 1 -- 172.21.15.31:0/2251696123 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0e28013a00 con 0x7f0e3c103780 2024-04-24T23:11:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.757+0000 7f0e397fa700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0e2c0692a0 0x7f0e2c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.758+0000 7f0e397fa700 1 -- 172.21.15.31:0/2251696123 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0e28088340 con 0x7f0e3c103780 2024-04-24T23:11:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.758+0000 7f0e40c1f700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0e2c0692a0 0x7f0e2c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:10.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.758+0000 7f0e40c1f700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0e2c0692a0 0x7f0e2c06b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e300041d0 tx=0x7f0e300043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:10.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:10.762+0000 7f0e397fa700 1 -- 172.21.15.31:0/2251696123 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0e28054620 con 0x7f0e3c103780 2024-04-24T23:11:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.004+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0e3c109ee0 con 0x7f0e3c103780 2024-04-24T23:11:11.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.004+0000 7f0e397fa700 1 -- 172.21.15.31:0/2251696123 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f0e28057c40 con 0x7f0e3c103780 2024-04-24T23:11:11.006 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-24T23:11:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.006+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0e2c0692a0 msgr2=0x7f0e2c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.006+0000 7f0e42e83700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0e2c0692a0 0x7f0e2c06b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0e300041d0 tx=0x7f0e300043c0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.006+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 msgr2=0x7f0e3c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.006+0000 7f0e42e83700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 0x7f0e3c0707a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009180 tx=0x7f0e28009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.007+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 shutdown_connections 2024-04-24T23:11:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.007+0000 7f0e42e83700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e3c102de0 0x7f0e3c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.007+0000 7f0e42e83700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0e2c0692a0 0x7f0e2c06b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.007+0000 7f0e42e83700 1 --2- 172.21.15.31:0/2251696123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e3c103780 0x7f0e3c0707a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.007+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 >> 172.21.15.31:0/2251696123 conn(0x7f0e3c0f8990 msgr2=0x7f0e3c102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.007+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 shutdown_connections 2024-04-24T23:11:11.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11.007+0000 7f0e42e83700 1 -- 172.21.15.31:0/2251696123 wait complete. 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:11:11.572 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:11:11.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:11.573 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-04-24T23:11:11.590 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-04-24T23:11:11.591 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:11:11.592 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42276sec preferred_lft 42276sec 2024-04-24T23:11:11.592 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link noprefixroute 2024-04-24T23:11:11.592 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:11:11.592 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:11:11.592 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-04-24T23:11:11.592 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-04-24T23:11:11.592 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:11:11.593 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:11.593 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-24T23:11:11.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:11:11.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-04-24T23:11:11.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-04-24T23:11:11.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:11:11.593 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-24T23:11:11.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:11.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:11.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:11.776 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:11:11.801 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-04-24T23:11:11.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:11.803 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-04-24T23:11:11.875 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-04-24T23:11:11.875 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:11:11.875 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:11:11.875 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:11:11.875 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:11:11.876 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:11.876 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-24T23:11:11.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:11:11.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:11:11.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:11:11.876 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-24T23:11:11.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:12.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:11 smithi031 ceph-mon[26411]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-24T23:11:12.135 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:11 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2251696123' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:11:12.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:12.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:12.143 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-04-24T23:11:12.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:12.158 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-04-24T23:11:12.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:11 smithi040 ceph-mon[33128]: pgmap v196: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-24T23:11:12.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:11 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2251696123' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:11:12.214 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:12.214 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-24T23:11:12.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:11:12.214 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-24T23:11:12.214 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-24T23:11:12.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:12.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:12.331 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-24T23:11:12.332 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:12.332 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-24T23:11:12.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-24T23:11:12.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:11:12.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-24T23:11:12.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-24T23:11:12.332 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-24T23:11:12.332 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-24T23:11:12.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:12.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:12.453 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:12.453 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-04-24T23:11:12.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-04-24T23:11:12.453 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:11:12.453 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-04-24T23:11:12.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:12.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:12.529 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-24T23:11:12.530 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:11:12.530 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:12.545 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-24T23:11:12.545 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:11:12.545 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:12.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-04-24T23:11:12.667 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:11:12.725 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-04-24T23:11:12.725 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-04-24T23:11:12.725 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-04-24T23:11:12.725 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-04-24T23:11:12.726 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-04-24T23:11:12.726 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-04-24T23:11:12.726 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-04-24T23:11:12.726 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-04-24T23:11:12.726 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-04-24T23:11:12.726 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-04-24T23:11:12.727 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-04-24T23:11:12.727 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-04-24T23:11:12.727 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:11:12.727 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-04-24T23:11:12.727 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-04-24T23:11:12.727 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-04-24T23:11:12.728 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-04-24T23:11:12.728 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:11:12.728 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-04-24T23:11:12.728 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-04-24T23:11:12.728 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:11:12.728 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:11:12.729 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:11:12.729 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:11:12.729 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-04-24T23:11:12.729 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-04-24T23:11:12.729 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:11:12.729 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/5222175f8b661930645f0b9dfaaf2d15d1de51ec47dd44da32d9911166ba9b0d/merged 2024-04-24T23:11:12.729 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/c210b179126ba4f32c54f343acf45566d415462f08982af746bfbefbfb0a2bf1/merged 2024-04-24T23:11:12.729 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/40cb85857d117eac095d5cff86dbbc2b5e589a950176fd515a6aca555a82dd69/merged 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/2452d829ffc095ecd51daf412aee6f5f841e3f702cb6864d1dae64dc16a126df/merged 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/9c3d2ac4674c586d57bd10a9888c8399e6fea15b7f78821cae6829143cadb579/merged 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/df7cbbba3e7ed81750d6bcc78ca38c0e5530a3af03bdba9dd7f75e1e98856234/merged 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/ad25f021cdf0e15c29e573b475143c631500f90fefa826e646fbf5bbb75922f4/merged 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/88a863db91d02ee03096aeaa87a2b6d7e8d7b8c03cf72adf58f13c9c46233a4b/merged 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/0e7d39f2d92bd6b24ae6a4dba85d98a0c2fa822c2d6b557142da470debb8899e/merged 2024-04-24T23:11:12.730 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/37ea5ef4479f17a748fcdd7c925b59c9c59d4276673f3318417fb2d930d97d3a/merged 2024-04-24T23:11:12.731 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/3d22a0d56e41768ae79f16fb6cc547f46068fb5991e1d33fbc56406cdeab154c/merged 2024-04-24T23:11:12.731 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/a5fc794595cf4ad587a9d7435525f7f07554ede8b9a29b9fd3f5e20cf2ffd791/merged 2024-04-24T23:11:12.731 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-04-24T23:11:12.731 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:11:12.731 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:11:12.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:12.732 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-04-24T23:11:12.781 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T23:11:12.781 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-24T23:11:12.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo modprobe fuse 2024-04-24T23:11:12.847 DEBUG:teuthology.orchestra.run.smithi031:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:11:12.903 INFO:teuthology.orchestra.run.smithi031.stdout:/proc 2024-04-24T23:11:12.903 INFO:teuthology.orchestra.run.smithi031.stdout:/sys 2024-04-24T23:11:12.903 INFO:teuthology.orchestra.run.smithi031.stdout:/dev 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/security 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/shm 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/pts 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/run 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/cgroup 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/pstore 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/bpf 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/ 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/selinux 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/mqueue 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/dev/hugepages 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/debug 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/tracing 2024-04-24T23:11:12.904 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/fs/fuse/connections 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/sys/kernel/config 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/1000 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/ceph 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/5222175f8b661930645f0b9dfaaf2d15d1de51ec47dd44da32d9911166ba9b0d/merged 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/c210b179126ba4f32c54f343acf45566d415462f08982af746bfbefbfb0a2bf1/merged 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/run/user/0 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:11:12.905 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/40cb85857d117eac095d5cff86dbbc2b5e589a950176fd515a6aca555a82dd69/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/2452d829ffc095ecd51daf412aee6f5f841e3f702cb6864d1dae64dc16a126df/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/9c3d2ac4674c586d57bd10a9888c8399e6fea15b7f78821cae6829143cadb579/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/df7cbbba3e7ed81750d6bcc78ca38c0e5530a3af03bdba9dd7f75e1e98856234/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/ad25f021cdf0e15c29e573b475143c631500f90fefa826e646fbf5bbb75922f4/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/88a863db91d02ee03096aeaa87a2b6d7e8d7b8c03cf72adf58f13c9c46233a4b/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/0e7d39f2d92bd6b24ae6a4dba85d98a0c2fa822c2d6b557142da470debb8899e/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/37ea5ef4479f17a748fcdd7c925b59c9c59d4276673f3318417fb2d930d97d3a/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/3d22a0d56e41768ae79f16fb6cc547f46068fb5991e1d33fbc56406cdeab154c/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/var/lib/containers/storage/overlay/a5fc794595cf4ad587a9d7435525f7f07554ede8b9a29b9fd3f5e20cf2ffd791/merged 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run.smithi031.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-24T23:11:12.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:12.907 DEBUG:teuthology.orchestra.run.smithi031:> ls /sys/fs/fuse/connections 2024-04-24T23:11:12.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:2024-04-24T23:11:12.907+0000 7f1eb46ce300 -1 init, newargv = 0x55d8b88514e0 newargc=15 2024-04-24T23:11:12.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[73971]: starting ceph client 2024-04-24T23:11:12.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[73971]: starting fuse 2024-04-24T23:11:12.924 INFO:teuthology.orchestra.run.smithi031.stdout:145 2024-04-24T23:11:12.924 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [145] 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo stdin-killer -- python3 -c ' 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> import glob 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> import re 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> import os 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> import subprocess 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> def _find_admin_socket(client_name): 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> files = glob.glob(asok_path) 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-24T23:11:12.925 DEBUG:teuthology.orchestra.run.smithi031:> # Given a non-glob path, it better be there 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> if "*" not in asok_path: 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> assert(len(files) == 1) 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> return files[0] 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> for f in files: 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> contents = proc_f.read() 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> if mountpoint in contents: 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> return f 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> print(_find_admin_socket("client.0")) 2024-04-24T23:11:12.926 DEBUG:teuthology.orchestra.run.smithi031:> ' 2024-04-24T23:11:13.043 INFO:teuthology.orchestra.run.smithi031.stdout:/var/run/ceph/ceph-client.0.73971.asok 2024-04-24T23:11:13.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:13.054 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.73971.asok 2024-04-24T23:11:13.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:13.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.73971.asok status 2024-04-24T23:11:13.186 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:11:13.186 INFO:teuthology.orchestra.run.smithi031.stdout: "metadata": { 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_sha1": "c61472c83ce76869b0fe076c19ed2f4bcc3fb4af", 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "entity_id": "0", 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "hostname": "smithi031", 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "pid": "73971", 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "root": "/" 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_count": 0, 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "dentry_pinned_count": 0, 2024-04-24T23:11:13.187 INFO:teuthology.orchestra.run.smithi031.stdout: "id": 14518, 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "inst": { 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "name": { 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "client", 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "num": 14518 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2806865259 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:11:13.188 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": { 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2806865259 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "inst_str": "client.14518 192.168.0.1:0/2806865259", 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "addr_str": "192.168.0.1:0/2806865259", 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "inode_count": 1, 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_epoch": 15, 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch": 46, 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_epoch_barrier": 0, 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklisted": false, 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout: "fs_name": "cephfs" 2024-04-24T23:11:13.189 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:11:13.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-24T23:11:13.461 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:14.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:13 smithi040 ceph-mon[33128]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-24T23:11:14.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:13 smithi031 ceph-mon[26411]: pgmap v197: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-24T23:11:15.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 -- 172.21.15.31:0/3496939251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2310102e00 msgr2=0x7f23101031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 --2- 172.21.15.31:0/3496939251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2310102e00 0x7f23101031d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2300004940 tx=0x7f2300009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 -- 172.21.15.31:0/3496939251 shutdown_connections 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 --2- 172.21.15.31:0/3496939251 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23101037a0 0x7f23100fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 --2- 172.21.15.31:0/3496939251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2310102e00 0x7f23101031d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 -- 172.21.15.31:0/3496939251 >> 172.21.15.31:0/3496939251 conn(0x7f23100f89b0 msgr2=0x7f23100fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 -- 172.21.15.31:0/3496939251 shutdown_connections 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 -- 172.21.15.31:0/3496939251 wait complete. 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 Processor -- start 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.084+0000 7f23148ca700 1 -- start start 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f23148ca700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2310102e00 0x7f2310070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f23148ca700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23101037a0 0x7f23100707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f23148ca700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23100721b0 con 0x7f23101037a0 2024-04-24T23:11:15.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f23148ca700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2310070ce0 con 0x7f2310102e00 2024-04-24T23:11:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f230dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23101037a0 0x7f23100707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f230dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23101037a0 0x7f23100707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33468/0 (socket says 172.21.15.31:33468) 2024-04-24T23:11:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f230dd9b700 1 -- 172.21.15.31:0/293463578 learned_addr learned my addr 172.21.15.31:0/293463578 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f230e59c700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2310102e00 0x7f2310070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f230dd9b700 1 -- 172.21.15.31:0/293463578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2310102e00 msgr2=0x7f2310070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f230dd9b700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2310102e00 0x7f2310070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f230dd9b700 1 -- 172.21.15.31:0/293463578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2300005040 con 0x7f23101037a0 2024-04-24T23:11:15.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f230dd9b700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23101037a0 0x7f23100707a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f230400c890 tx=0x7f23040044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f22feffd700 1 -- 172.21.15.31:0/293463578 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f230400a7d0 con 0x7f23101037a0 2024-04-24T23:11:15.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f22feffd700 1 -- 172.21.15.31:0/293463578 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f230400ae00 con 0x7f23101037a0 2024-04-24T23:11:15.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.085+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2310070fc0 con 0x7f23101037a0 2024-04-24T23:11:15.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.086+0000 7f22feffd700 1 -- 172.21.15.31:0/293463578 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23040113a0 con 0x7f23101037a0 2024-04-24T23:11:15.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.086+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231011a150 con 0x7f23101037a0 2024-04-24T23:11:15.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.087+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2310100110 con 0x7f23101037a0 2024-04-24T23:11:15.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.087+0000 7f22feffd700 1 -- 172.21.15.31:0/293463578 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2304012400 con 0x7f23101037a0 2024-04-24T23:11:15.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.088+0000 7f22feffd700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22f80692a0 0x7f22f806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:15.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.088+0000 7f22feffd700 1 -- 172.21.15.31:0/293463578 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f2304087320 con 0x7f23101037a0 2024-04-24T23:11:15.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.088+0000 7f230e59c700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22f80692a0 0x7f22f806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:15.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.088+0000 7f230e59c700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22f80692a0 0x7f22f806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2300004f90 tx=0x7f2300009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:15.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.092+0000 7f22feffd700 1 -- 172.21.15.31:0/293463578 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2304053600 con 0x7f23101037a0 2024-04-24T23:11:15.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.336+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2310109f00 con 0x7f23101037a0 2024-04-24T23:11:15.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.336+0000 7f22feffd700 1 -- 172.21.15.31:0/293463578 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f2304056c20 con 0x7f23101037a0 2024-04-24T23:11:15.338 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-24T23:11:15.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22f80692a0 msgr2=0x7f22f806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22f80692a0 0x7f22f806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2300004f90 tx=0x7f2300009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:15.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23101037a0 msgr2=0x7f23100707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:15.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23101037a0 0x7f23100707a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f230400c890 tx=0x7f23040044c0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 shutdown_connections 2024-04-24T23:11:15.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2310102e00 0x7f2310070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f22f80692a0 0x7f22f806b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 --2- 172.21.15.31:0/293463578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23101037a0 0x7f23100707a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:15.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 >> 172.21.15.31:0/293463578 conn(0x7f23100f89b0 msgr2=0x7f2310107350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:15.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 shutdown_connections 2024-04-24T23:11:15.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:15.339+0000 7f23148ca700 1 -- 172.21.15.31:0/293463578 wait complete. 2024-04-24T23:11:16.162 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:15 smithi031 ceph-mon[26411]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-24T23:11:16.162 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:15 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/293463578' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:11:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:15 smithi040 ceph-mon[33128]: pgmap v198: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-24T23:11:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/293463578' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-24T23:11:16.193 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:11:16.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:16.194 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2024-04-24T23:11:16.213 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-24T23:11:16.213 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-24T23:11:16.213 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2024-04-24T23:11:16.213 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:11:16.213 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2024-04-24T23:11:16.213 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-24T23:11:16.214 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42285sec preferred_lft 42285sec 2024-04-24T23:11:16.215 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link noprefixroute 2024-04-24T23:11:16.215 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2024-04-24T23:11:16.215 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-24T23:11:16.215 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2024-04-24T23:11:16.215 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2024-04-24T23:11:16.215 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-24T23:11:16.215 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:16.215 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-24T23:11:16.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2024-04-24T23:11:16.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2024-04-24T23:11:16.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2024-04-24T23:11:16.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-24T23:11:16.216 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-24T23:11:16.311 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:16.420 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:16.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:16.427 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-24T23:11:16.452 INFO:teuthology.orchestra.run.smithi040.stdout:1 2024-04-24T23:11:16.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:16.454 DEBUG:teuthology.orchestra.run.smithi040:> route 2024-04-24T23:11:16.524 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2024-04-24T23:11:16.524 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-24T23:11:16.524 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-24T23:11:16.524 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-24T23:11:16.524 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-24T23:11:16.525 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:16.525 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-24T23:11:16.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-24T23:11:16.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-24T23:11:16.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-24T23:11:16.525 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-24T23:11:16.614 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:16.804 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:16.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:16.810 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2024-04-24T23:11:16.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:16.826 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2024-04-24T23:11:16.881 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:16.881 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-24T23:11:16.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:11:16.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-24T23:11:16.882 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-24T23:11:16.971 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:16.994 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:17.000 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-24T23:11:17.000 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:17.000 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-24T23:11:17.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-24T23:11:17.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-24T23:11:17.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-24T23:11:17.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-24T23:11:17.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-24T23:11:17.001 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-24T23:11:17.050 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:17.117 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:17.123 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-24T23:11:17.123 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2024-04-24T23:11:17.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2024-04-24T23:11:17.124 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2024-04-24T23:11:17.124 DEBUG:teuthology.orchestra.run.smithi040:> ') 2024-04-24T23:11:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-24T23:11:17.193 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:17.200 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-24T23:11:17.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:11:17.200 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:17.215 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-24T23:11:17.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-24T23:11:17.216 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:17.271 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-24T23:11:17.336 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:11:17.393 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-24T23:11:17.393 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-24T23:11:17.393 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-24T23:11:17.393 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-24T23:11:17.393 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-24T23:11:17.393 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/selinux 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:11:17.394 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/0 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/99dd63db7e7c39b66caceb4dabef320095990b14c0ac6fae567a7067ddc04ae6/merged 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/a55dc3acb255108d872f72ea251fcd1031b02354440d73025cc0e2c03d87983d/merged 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/c492320147ff7834e6c8cd0de7f9c061e63579e2cf97d575d073d0f1b15ed713/merged 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/d146096c46100c1a2d3316df02afd475ad23dfbc0dcd08ee6077fa5618ca3da2/merged 2024-04-24T23:11:17.395 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/a6f80b464c54e400dec566cf227af8dde1425ca61fb8f165ce99cd19afa7cede/merged 2024-04-24T23:11:17.396 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/7fa409917e1c0ab74de40cbea4dfc269f86a4d112a2dc282daa91784646a32a7/merged 2024-04-24T23:11:17.396 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/3f154d9a34931cdfe8e5d4174d2cc158c5376cc19fc59dd15bc0b093fd1963a2/merged 2024-04-24T23:11:17.396 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/b2025a6058a230ee70343b2910c614529ee1fdb3f6816dc353c576c085f6ef59/merged 2024-04-24T23:11:17.396 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/514ef6f0f1ca3d5e24917a6947da79f4b6de2dc9fc022404e02f7d0422949c16/merged 2024-04-24T23:11:17.396 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-24T23:11:17.396 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:11:17.396 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:11:17.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:17.396 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-24T23:11:17.449 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-24T23:11:17.449 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-24T23:11:17.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2024-04-24T23:11:17.515 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2024-04-24T23:11:17.573 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/selinux 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysctl.service 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-24T23:11:17.574 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/nfs/rpc_pipefs 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/0 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/99dd63db7e7c39b66caceb4dabef320095990b14c0ac6fae567a7067ddc04ae6/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/a55dc3acb255108d872f72ea251fcd1031b02354440d73025cc0e2c03d87983d/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/c492320147ff7834e6c8cd0de7f9c061e63579e2cf97d575d073d0f1b15ed713/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/d146096c46100c1a2d3316df02afd475ad23dfbc0dcd08ee6077fa5618ca3da2/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/a6f80b464c54e400dec566cf227af8dde1425ca61fb8f165ce99cd19afa7cede/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/7fa409917e1c0ab74de40cbea4dfc269f86a4d112a2dc282daa91784646a32a7/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/3f154d9a34931cdfe8e5d4174d2cc158c5376cc19fc59dd15bc0b093fd1963a2/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/b2025a6058a230ee70343b2910c614529ee1fdb3f6816dc353c576c085f6ef59/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/containers/storage/overlay/514ef6f0f1ca3d5e24917a6947da79f4b6de2dc9fc022404e02f7d0422949c16/merged 2024-04-24T23:11:17.575 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2024-04-24T23:11:17.576 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:11:17.576 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-24T23:11:17.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:17.576 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2024-04-24T23:11:17.595 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi040.stderr:2024-04-24T23:11:17.594+0000 7f05e46c4300 -1 init, newargv = 0x5606664b38d0 newargc=15 2024-04-24T23:11:17.595 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi040.stderr:ceph-fuse[58439]: starting ceph client 2024-04-24T23:11:17.600 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi040.stderr:ceph-fuse[58439]: starting fuse 2024-04-24T23:11:17.611 INFO:teuthology.orchestra.run.smithi040.stdout:112 2024-04-24T23:11:17.611 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [112] 2024-04-24T23:11:17.611 DEBUG:teuthology.orchestra.run.smithi040:> sudo stdin-killer -- python3 -c ' 2024-04-24T23:11:17.611 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2024-04-24T23:11:17.611 DEBUG:teuthology.orchestra.run.smithi040:> import re 2024-04-24T23:11:17.611 DEBUG:teuthology.orchestra.run.smithi040:> import os 2024-04-24T23:11:17.611 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2024-04-24T23:11:17.611 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-24T23:11:17.611 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2024-04-24T23:11:17.611 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> return f 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> 2024-04-24T23:11:17.612 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.1")) 2024-04-24T23:11:17.613 DEBUG:teuthology.orchestra.run.smithi040:> ' 2024-04-24T23:11:17.734 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.1.58439.asok 2024-04-24T23:11:17.739 INFO:teuthology.orchestra.run.smithi040.stderr:2024-04-24T23:11:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-24T23:11:17.747 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.58439.asok 2024-04-24T23:11:17.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:17.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.58439.asok status 2024-04-24T23:11:17.880 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2024-04-24T23:11:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2024-04-24T23:11:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "c61472c83ce76869b0fe076c19ed2f4bcc3fb4af", 2024-04-24T23:11:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-24T23:11:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "1", 2024-04-24T23:11:17.880 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "58439", 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 24327, 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 24327 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-24T23:11:17.881 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3971968853 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: } 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3971968853 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.24327 192.168.0.1:0/3971968853", 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/3971968853", 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 15, 2024-04-24T23:11:17.882 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 46, 2024-04-24T23:11:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2024-04-24T23:11:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2024-04-24T23:11:17.883 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2024-04-24T23:11:17.883 INFO:teuthology.orchestra.run.smithi040.stdout:} 2024-04-24T23:11:17.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:17.890 DEBUG:teuthology.orchestra.run.smithi031:> stat --file-system '--printf=%T 2024-04-24T23:11:17.890 DEBUG:teuthology.orchestra.run.smithi031:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:17.910 INFO:teuthology.orchestra.run.smithi031.stdout:fuseblk 2024-04-24T23:11:17.910 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:17.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:17.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:17.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:17.983 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2024-04-24T23:11:17.983 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:17.999 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2024-04-24T23:11:17.999 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:17.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:11:17.999 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:18.067 INFO:teuthology.run_tasks:Running task print... 2024-04-24T23:11:18.078 INFO:teuthology.task.print:**** done client 2024-04-24T23:11:18.078 INFO:teuthology.run_tasks:Running task parallel... 2024-04-24T23:11:18.099 INFO:teuthology.task.parallel:starting parallel... 2024-04-24T23:11:18.099 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-24T23:11:18.100 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T23:11:18.101 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:11:18.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-24T23:11:18.101 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-24T23:11:18.101 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-24T23:11:18.105 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:11:18.105 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-24T23:11:18.105 INFO:tasks.workunit:timeout=3h 2024-04-24T23:11:18.105 INFO:tasks.workunit:cleanup=True 2024-04-24T23:11:18.106 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 91h/145d Inode: 1 Links: 2 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-04-24 23:10:44.924154040 +0000 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-04-24 23:11:17.979773836 +0000 2024-04-24T23:11:18.126 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2024-04-24T23:11:18.126 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-24T23:11:18.127 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-24T23:11:18.131 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:17 smithi031 ceph-mon[26411]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T23:11:18.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:17 smithi040 ceph-mon[33128]: pgmap v199: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T23:11:18.163 DEBUG:teuthology.orchestra.run.smithi040:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:18.179 INFO:teuthology.orchestra.run.smithi040.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:18.179 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-24T23:11:18.180 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 70h/112d Inode: 1 Links: 3 2024-04-24T23:11:18.180 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T23:11:18.180 INFO:teuthology.orchestra.run.smithi040.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-24T23:11:18.180 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-24T23:11:18.180 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2024-04-24 23:11:18.156795464 +0000 2024-04-24T23:11:18.180 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2024-04-24 23:11:18.156795464 +0000 2024-04-24T23:11:18.180 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2024-04-24T23:11:18.180 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-24T23:11:18.180 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-24T23:11:18.249 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:11:18.250 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:11:18.266 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-24T23:11:18.305 INFO:tasks.workunit.client.1.smithi040.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-24T23:11:18.385 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.953+0000 7f421cb88700 1 -- 172.21.15.31:0/68543611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fb420 msgr2=0x7f42180fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.953+0000 7f421cb88700 1 --2- 172.21.15.31:0/68543611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fb420 0x7f42180fb830 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4210009ce0 tx=0x7f4210004d10 comp rx=0 tx=0).stop 2024-04-24T23:11:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.953+0000 7f421cb88700 1 -- 172.21.15.31:0/68543611 shutdown_connections 2024-04-24T23:11:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.953+0000 7f421cb88700 1 --2- 172.21.15.31:0/68543611 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180fc6d0 0x7f42180fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.953+0000 7f421cb88700 1 --2- 172.21.15.31:0/68543611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fb420 0x7f42180fb830 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.953+0000 7f421cb88700 1 -- 172.21.15.31:0/68543611 >> 172.21.15.31:0/68543611 conn(0x7f42180f6b80 msgr2=0x7f42180f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.953+0000 7f421cb88700 1 -- 172.21.15.31:0/68543611 shutdown_connections 2024-04-24T23:11:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.953+0000 7f421cb88700 1 -- 172.21.15.31:0/68543611 wait complete. 2024-04-24T23:11:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f421cb88700 1 Processor -- start 2024-04-24T23:11:19.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f421cb88700 1 -- start start 2024-04-24T23:11:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f421cb88700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180fb420 0x7f421810db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f421cb88700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fc6d0 0x7f421810e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f421cb88700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421810e6b0 con 0x7f42180fc6d0 2024-04-24T23:11:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f421cb88700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421810ef00 con 0x7f42180fb420 2024-04-24T23:11:19.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f4215d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fc6d0 0x7f421810e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f4215d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fc6d0 0x7f421810e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44246/0 (socket says 172.21.15.31:44246) 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f4215d9b700 1 -- 172.21.15.31:0/3164570838 learned_addr learned my addr 172.21.15.31:0/3164570838 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f421659c700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180fb420 0x7f421810db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f4215d9b700 1 -- 172.21.15.31:0/3164570838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180fb420 msgr2=0x7f421810db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f4215d9b700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180fb420 0x7f421810db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.954+0000 7f4215d9b700 1 -- 172.21.15.31:0/3164570838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210009940 con 0x7f42180fc6d0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.955+0000 7f4215d9b700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fc6d0 0x7f421810e0b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4200002a50 tx=0x7f42000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.955+0000 7f420effd700 1 -- 172.21.15.31:0/3164570838 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4200007cc0 con 0x7f42180fc6d0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.955+0000 7f420effd700 1 -- 172.21.15.31:0/3164570838 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f420000f040 con 0x7f42180fc6d0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.955+0000 7f420effd700 1 -- 172.21.15.31:0/3164570838 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420000da30 con 0x7f42180fc6d0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.955+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421810f1e0 con 0x7f42180fc6d0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.955+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421810f720 con 0x7f42180fc6d0 2024-04-24T23:11:19.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.956+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f421805f5c0 con 0x7f42180fc6d0 2024-04-24T23:11:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.956+0000 7f420effd700 1 -- 172.21.15.31:0/3164570838 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f420000db90 con 0x7f42180fc6d0 2024-04-24T23:11:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.957+0000 7f420effd700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f42040692a0 0x7f420406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.957+0000 7f420effd700 1 -- 172.21.15.31:0/3164570838 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f4200086f60 con 0x7f42180fc6d0 2024-04-24T23:11:19.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.957+0000 7f421659c700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f42040692a0 0x7f420406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:19.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.957+0000 7f421659c700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f42040692a0 0x7f420406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4210015040 tx=0x7f4210004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:19.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:19.961+0000 7f420effd700 1 -- 172.21.15.31:0/3164570838 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4200056c30 con 0x7f42180fc6d0 2024-04-24T23:11:19.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:19 smithi031 ceph-mon[26411]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T23:11:19.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:11:20.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:19 smithi040 ceph-mon[33128]: pgmap v200: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T23:11:20.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:11:20.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.161+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f421810fa30 con 0x7f42180fc6d0 2024-04-24T23:11:20.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.162+0000 7f420effd700 1 -- 172.21.15.31:0/3164570838 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f42000567c0 con 0x7f42180fc6d0 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f42040692a0 msgr2=0x7f420406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f42040692a0 0x7f420406b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4210015040 tx=0x7f4210004d10 comp rx=0 tx=0).stop 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fc6d0 msgr2=0x7f421810e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fc6d0 0x7f421810e0b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4200002a50 tx=0x7f42000046b0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 shutdown_connections 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f42180fb420 0x7f421810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f42040692a0 0x7f420406b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 --2- 172.21.15.31:0/3164570838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42180fc6d0 0x7f421810e0b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 >> 172.21.15.31:0/3164570838 conn(0x7f42180f6b80 msgr2=0x7f42180f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 shutdown_connections 2024-04-24T23:11:20.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:20.164+0000 7f421cb88700 1 -- 172.21.15.31:0/3164570838 wait complete. 2024-04-24T23:11:20.858 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-24T23:11:21.169 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:22.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:21 smithi040 ceph-mon[33128]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T23:11:22.232 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:21 smithi031 ceph-mon[26411]: pgmap v201: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.744+0000 7f152f252700 1 -- 172.21.15.31:0/3795179897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 msgr2=0x7f15280fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.744+0000 7f152f252700 1 --2- 172.21.15.31:0/3795179897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 0x7f15280fd3d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f151c004940 tx=0x7f151c009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.744+0000 7f152f252700 1 -- 172.21.15.31:0/3795179897 shutdown_connections 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.744+0000 7f152f252700 1 --2- 172.21.15.31:0/3795179897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 0x7f15280fd3d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.744+0000 7f152f252700 1 --2- 172.21.15.31:0/3795179897 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15280fbcb0 0x7f15280fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.744+0000 7f152f252700 1 -- 172.21.15.31:0/3795179897 >> 172.21.15.31:0/3795179897 conn(0x7f15280f7430 msgr2=0x7f15280f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.744+0000 7f152f252700 1 -- 172.21.15.31:0/3795179897 shutdown_connections 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.744+0000 7f152f252700 1 -- 172.21.15.31:0/3795179897 wait complete. 2024-04-24T23:11:22.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.745+0000 7f152f252700 1 Processor -- start 2024-04-24T23:11:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.745+0000 7f152f252700 1 -- start start 2024-04-24T23:11:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.745+0000 7f152f252700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15280fbcb0 0x7f1528110660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.745+0000 7f152f252700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 0x7f1528112bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.745+0000 7f152f252700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1528066390 con 0x7f15280fcf60 2024-04-24T23:11:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.745+0000 7f152f252700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280664d0 con 0x7f15280fbcb0 2024-04-24T23:11:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f1527fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 0x7f1528112bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f1527fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 0x7f1528112bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44274/0 (socket says 172.21.15.31:44274) 2024-04-24T23:11:22.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f1527fff700 1 -- 172.21.15.31:0/2898216825 learned_addr learned my addr 172.21.15.31:0/2898216825 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f152cfee700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15280fbcb0 0x7f1528110660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f1527fff700 1 -- 172.21.15.31:0/2898216825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15280fbcb0 msgr2=0x7f1528110660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f1527fff700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15280fbcb0 0x7f1528110660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f1527fff700 1 -- 172.21.15.31:0/2898216825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c005040 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f1527fff700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 0x7f1528112bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b3b0 tx=0x7f151c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f15257fa700 1 -- 172.21.15.31:0/2898216825 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f151c00e040 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1528113150 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f15257fa700 1 -- 172.21.15.31:0/2898216825 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f151c004290 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f15257fa700 1 -- 172.21.15.31:0/2898216825 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f151c01c630 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.746+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1528113710 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.747+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152805f5c0 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.748+0000 7f15257fa700 1 -- 172.21.15.31:0/2898216825 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f151c00bbf0 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.748+0000 7f15257fa700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f15100692b0 0x7f151006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.748+0000 7f15257fa700 1 -- 172.21.15.31:0/2898216825 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f151c088640 con 0x7f15280fcf60 2024-04-24T23:11:22.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.748+0000 7f152cfee700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f15100692b0 0x7f151006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:22.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.749+0000 7f152cfee700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f15100692b0 0x7f151006b760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1528066010 tx=0x7f152000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:22.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.753+0000 7f15257fa700 1 -- 172.21.15.31:0/2898216825 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f151c054920 con 0x7f15280fcf60 2024-04-24T23:11:22.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.949+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f15281139d0 con 0x7f15280fcf60 2024-04-24T23:11:22.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.949+0000 7f15257fa700 1 -- 172.21.15.31:0/2898216825 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f151c057f40 con 0x7f15280fcf60 2024-04-24T23:11:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.951+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f15100692b0 msgr2=0x7f151006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.951+0000 7f152f252700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f15100692b0 0x7f151006b760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1528066010 tx=0x7f152000b040 comp rx=0 tx=0).stop 2024-04-24T23:11:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.951+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 msgr2=0x7f1528112bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.951+0000 7f152f252700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 0x7f1528112bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f151c00b3b0 tx=0x7f151c009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:22.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.952+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 shutdown_connections 2024-04-24T23:11:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.952+0000 7f152f252700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15280fbcb0 0x7f1528110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.952+0000 7f152f252700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f15100692b0 0x7f151006b760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.952+0000 7f152f252700 1 --2- 172.21.15.31:0/2898216825 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15280fcf60 0x7f1528112bb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.952+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 >> 172.21.15.31:0/2898216825 conn(0x7f15280f7430 msgr2=0x7f15280f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.952+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 shutdown_connections 2024-04-24T23:11:22.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:22.952+0000 7f152f252700 1 -- 172.21.15.31:0/2898216825 wait complete. 2024-04-24T23:11:23.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-24T23:11:23.876 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:24.147 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:23 smithi031 ceph-mon[26411]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-24T23:11:24.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:23 smithi040 ceph-mon[33128]: pgmap v202: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-24T23:11:25.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.592+0000 7f24896be700 1 -- 172.21.15.31:0/3453373774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 msgr2=0x7f24840fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.592+0000 7f24896be700 1 --2- 172.21.15.31:0/3453373774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 0x7f24840fd280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2474004940 tx=0x7f2474009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.593+0000 7f24896be700 1 -- 172.21.15.31:0/3453373774 shutdown_connections 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.593+0000 7f24896be700 1 --2- 172.21.15.31:0/3453373774 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24840fdfe0 0x7f24840fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.593+0000 7f24896be700 1 --2- 172.21.15.31:0/3453373774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 0x7f24840fd280 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.593+0000 7f24896be700 1 -- 172.21.15.31:0/3453373774 >> 172.21.15.31:0/3453373774 conn(0x7f24840f8550 msgr2=0x7f24840fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.593+0000 7f24896be700 1 -- 172.21.15.31:0/3453373774 shutdown_connections 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.593+0000 7f24896be700 1 -- 172.21.15.31:0/3453373774 wait complete. 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.593+0000 7f24896be700 1 Processor -- start 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.593+0000 7f24896be700 1 -- start start 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f24896be700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 0x7f2484071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f24896be700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24840fdfe0 0x7f24840702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f24896be700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840707f0 con 0x7f24840fce70 2024-04-24T23:11:25.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f24896be700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484070960 con 0x7f24840fdfe0 2024-04-24T23:11:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f2482ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 0x7f2484071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f2482ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 0x7f2484071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44292/0 (socket says 172.21.15.31:44292) 2024-04-24T23:11:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f2482ffd700 1 -- 172.21.15.31:0/1828559391 learned_addr learned my addr 172.21.15.31:0/1828559391 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f2482ffd700 1 -- 172.21.15.31:0/1828559391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24840fdfe0 msgr2=0x7f24840702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f2482ffd700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24840fdfe0 0x7f24840702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f2482ffd700 1 -- 172.21.15.31:0/1828559391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2474005040 con 0x7f24840fce70 2024-04-24T23:11:25.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f2482ffd700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 0x7f2484071c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2474009150 tx=0x7f2474009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f246b7fe700 1 -- 172.21.15.31:0/1828559391 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247400e040 con 0x7f24840fce70 2024-04-24T23:11:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f246b7fe700 1 -- 172.21.15.31:0/1828559391 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f247400bbf0 con 0x7f24840fce70 2024-04-24T23:11:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.594+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2484070be0 con 0x7f24840fce70 2024-04-24T23:11:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.595+0000 7f246b7fe700 1 -- 172.21.15.31:0/1828559391 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247401c630 con 0x7f24840fce70 2024-04-24T23:11:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.595+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24840710e0 con 0x7f24840fce70 2024-04-24T23:11:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.596+0000 7f246b7fe700 1 -- 172.21.15.31:0/1828559391 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f247401c8f0 con 0x7f24840fce70 2024-04-24T23:11:25.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.596+0000 7f246b7fe700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f246c0690a0 0x7f246c06b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:25.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.597+0000 7f246b7fe700 1 -- 172.21.15.31:0/1828559391 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f247408c170 con 0x7f24840fce70 2024-04-24T23:11:25.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.597+0000 7f24827fc700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f246c0690a0 0x7f246c06b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:25.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.597+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248404c710 con 0x7f24840fce70 2024-04-24T23:11:25.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.597+0000 7f24827fc700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f246c0690a0 0x7f246c06b550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2478006fd0 tx=0x7f2478006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:25.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.601+0000 7f246b7fe700 1 -- 172.21.15.31:0/1828559391 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f247408d060 con 0x7f24840fce70 2024-04-24T23:11:25.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.797+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f2484105950 con 0x7f24840fce70 2024-04-24T23:11:25.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.797+0000 7f246b7fe700 1 -- 172.21.15.31:0/1828559391 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f2484105950 con 0x7f24840fce70 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f246c0690a0 msgr2=0x7f246c06b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f246c0690a0 0x7f246c06b550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2478006fd0 tx=0x7f2478006db0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 msgr2=0x7f2484071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 0x7f2484071c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2474009150 tx=0x7f2474009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 shutdown_connections 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f246c0690a0 0x7f246c06b550 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24840fce70 0x7f2484071c00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 --2- 172.21.15.31:0/1828559391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f24840fdfe0 0x7f24840702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:25.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 >> 172.21.15.31:0/1828559391 conn(0x7f24840f8550 msgr2=0x7f24840f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 shutdown_connections 2024-04-24T23:11:25.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:25.799+0000 7f24896be700 1 -- 172.21.15.31:0/1828559391 wait complete. 2024-04-24T23:11:26.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:25 smithi040 ceph-mon[33128]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-24T23:11:26.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:25 smithi031 ceph-mon[26411]: pgmap v203: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-24T23:11:26.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-24T23:11:26.817 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:28.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:27 smithi031 ceph-mon[26411]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-04-24T23:11:28.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:27 smithi040 ceph-mon[33128]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 0 op/s 2024-04-24T23:11:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.670+0000 7f0cf95a9700 1 -- 172.21.15.31:0/2874176780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 msgr2=0x7f0cf40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.670+0000 7f0cf95a9700 1 --2- 172.21.15.31:0/2874176780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 0x7f0cf40fd270 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4004940 tx=0x7f0ce4009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.670+0000 7f0cf95a9700 1 -- 172.21.15.31:0/2874176780 shutdown_connections 2024-04-24T23:11:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.670+0000 7f0cf95a9700 1 --2- 172.21.15.31:0/2874176780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40fdfd0 0x7f0cf40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.670+0000 7f0cf95a9700 1 --2- 172.21.15.31:0/2874176780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 0x7f0cf40fd270 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.670+0000 7f0cf95a9700 1 -- 172.21.15.31:0/2874176780 >> 172.21.15.31:0/2874176780 conn(0x7f0cf40f8540 msgr2=0x7f0cf40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.670+0000 7f0cf95a9700 1 -- 172.21.15.31:0/2874176780 shutdown_connections 2024-04-24T23:11:28.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.670+0000 7f0cf95a9700 1 -- 172.21.15.31:0/2874176780 wait complete. 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf95a9700 1 Processor -- start 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf95a9700 1 -- start start 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf95a9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 0x7f0cf410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf95a9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40fdfd0 0x7f0cf410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf95a9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf410e510 con 0x7f0cf40fce60 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf95a9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf4112eb0 con 0x7f0cf40fdfd0 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 0x7f0cf410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 0x7f0cf410d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53564/0 (socket says 172.21.15.31:53564) 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf2ffd700 1 -- 172.21.15.31:0/733000416 learned_addr learned my addr 172.21.15.31:0/733000416 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.671+0000 7f0cf27fc700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40fdfd0 0x7f0cf410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cf2ffd700 1 -- 172.21.15.31:0/733000416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40fdfd0 msgr2=0x7f0cf410de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cf2ffd700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40fdfd0 0x7f0cf410de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cf2ffd700 1 -- 172.21.15.31:0/733000416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4005040 con 0x7f0cf40fce60 2024-04-24T23:11:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cf2ffd700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 0x7f0cf410d940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009150 tx=0x7f0ce4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cdb7fe700 1 -- 172.21.15.31:0/733000416 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ce400e040 con 0x7f0cf40fce60 2024-04-24T23:11:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cdb7fe700 1 -- 172.21.15.31:0/733000416 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ce400bbf0 con 0x7f0cf40fce60 2024-04-24T23:11:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf41130b0 con 0x7f0cf40fce60 2024-04-24T23:11:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cdb7fe700 1 -- 172.21.15.31:0/733000416 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ce401c630 con 0x7f0cf40fce60 2024-04-24T23:11:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.672+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf4113620 con 0x7f0cf40fce60 2024-04-24T23:11:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.673+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf404c710 con 0x7f0cf40fce60 2024-04-24T23:11:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.674+0000 7f0cdb7fe700 1 -- 172.21.15.31:0/733000416 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0ce400bd60 con 0x7f0cf40fce60 2024-04-24T23:11:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.674+0000 7f0cdb7fe700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0cdc0692a0 0x7f0cdc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:11:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.674+0000 7f0cdb7fe700 1 -- 172.21.15.31:0/733000416 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f0ce4088520 con 0x7f0cf40fce60 2024-04-24T23:11:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.674+0000 7f0cf27fc700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0cdc0692a0 0x7f0cdc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:11:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.675+0000 7f0cf27fc700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0cdc0692a0 0x7f0cdc06b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009bf0 tx=0x7f0ce80091a0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:11:28.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.679+0000 7f0cdb7fe700 1 -- 172.21.15.31:0/733000416 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0ce4058340 con 0x7f0cf40fce60 2024-04-24T23:11:28.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:28.892+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cf41022a0 con 0x7f0cdc0692a0 2024-04-24T23:11:30.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:29 smithi040 ceph-mon[33128]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-24T23:11:30.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:29 smithi040 ceph-mon[33128]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:30.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:29 smithi031 ceph-mon[26411]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-24T23:11:30.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:29 smithi031 ceph-mon[26411]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:11:32.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:31 smithi040 ceph-mon[33128]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-24T23:11:32.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:31 smithi031 ceph-mon[26411]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-24T23:11:34.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:33 smithi040 ceph-mon[33128]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-24T23:11:34.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:33 smithi031 ceph-mon[26411]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-24T23:11:35.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:34 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:11:35.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:34 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:11:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:35 smithi040 ceph-mon[33128]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-24T23:11:36.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:35 smithi031 ceph-mon[26411]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-24T23:11:38.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:37 smithi040 ceph-mon[33128]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:11:38.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:37 smithi031 ceph-mon[26411]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-24T23:11:40.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:39 smithi031 ceph-mon[26411]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-24T23:11:40.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:39 smithi040 ceph-mon[33128]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-24T23:11:42.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:41 smithi031 ceph-mon[26411]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-24T23:11:42.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:41 smithi040 ceph-mon[33128]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-24T23:11:44.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:43 smithi031 ceph-mon[26411]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:44.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:43 smithi040 ceph-mon[33128]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:46.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:45 smithi031 ceph-mon[26411]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:46.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:45 smithi040 ceph-mon[33128]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:48.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:48 smithi040 ceph-mon[33128]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:48.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:48 smithi031 ceph-mon[26411]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:49.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:49 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:11:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:49 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:11:50.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:50 smithi040 ceph-mon[33128]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:50 smithi031 ceph-mon[26411]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:52.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:52 smithi040 ceph-mon[33128]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:52.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:52 smithi031 ceph-mon[26411]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:54.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:54 smithi040 ceph-mon[33128]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:54 smithi031 ceph-mon[26411]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:56.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:56 smithi040 ceph-mon[33128]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:56.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:56 smithi031 ceph-mon[26411]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:58.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.351+0000 7f0cdb7fe700 1 -- 172.21.15.31:0/733000416 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f0cf41022a0 con 0x7f0cdc0692a0 2024-04-24T23:11:58.353 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:58 smithi031 ceph-mon[26411]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:58.354 INFO:teuthology.orchestra.run.smithi031.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:11:58.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0cdc0692a0 msgr2=0x7f0cdc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0cdc0692a0 0x7f0cdc06b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009bf0 tx=0x7f0ce80091a0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 msgr2=0x7f0cf410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 0x7f0cf410d940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009150 tx=0x7f0ce4009e30 comp rx=0 tx=0).stop 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 shutdown_connections 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f0cdc0692a0 0x7f0cdc06b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cf40fce60 0x7f0cf410d940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 --2- 172.21.15.31:0/733000416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0cf40fdfd0 0x7f0cf410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 >> 172.21.15.31:0/733000416 conn(0x7f0cf40f8540 msgr2=0x7f0cf40f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 shutdown_connections 2024-04-24T23:11:58.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:11:58.354+0000 7f0cf95a9700 1 -- 172.21.15.31:0/733000416 wait complete. 2024-04-24T23:11:58.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:58 smithi040 ceph-mon[33128]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:59.251 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-24T23:11:59.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:59 smithi031 ceph-mon[26411]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:11:59.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:59.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:59.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:59 smithi031 ceph-mon[26411]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:59.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:59.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:59.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:59 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:59.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:11:59 smithi031 ceph-mon[26411]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:11:59.565 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:11:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:59 smithi040 ceph-mon[33128]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:11:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:11:59.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:59 smithi040 ceph-mon[33128]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:11:59.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:11:59.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:11:59.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:59 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:11:59.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:11:59 smithi040 ceph-mon[33128]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:12:01.967 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:01 smithi031 ceph-mon[26411]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:02.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:01 smithi040 ceph-mon[33128]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.527+0000 7fa5b8394700 1 -- 172.21.15.31:0/3833361648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fcf90 msgr2=0x7fa5b00fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.527+0000 7fa5b8394700 1 --2- 172.21.15.31:0/3833361648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fcf90 0x7fa5b00fd400 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0004940 tx=0x7fa5a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.527+0000 7fa5b8394700 1 -- 172.21.15.31:0/3833361648 shutdown_connections 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.527+0000 7fa5b8394700 1 --2- 172.21.15.31:0/3833361648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fcf90 0x7fa5b00fd400 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.527+0000 7fa5b8394700 1 --2- 172.21.15.31:0/3833361648 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b00fbce0 0x7fa5b00fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.527+0000 7fa5b8394700 1 -- 172.21.15.31:0/3833361648 >> 172.21.15.31:0/3833361648 conn(0x7fa5b00f7420 msgr2=0x7fa5b00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.527+0000 7fa5b8394700 1 -- 172.21.15.31:0/3833361648 shutdown_connections 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.527+0000 7fa5b8394700 1 -- 172.21.15.31:0/3833361648 wait complete. 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.528+0000 7fa5b8394700 1 Processor -- start 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.528+0000 7fa5b8394700 1 -- start start 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.528+0000 7fa5b8394700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fbce0 0x7fa5b0109440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.528+0000 7fa5b8394700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b00fcf90 0x7fa5b0109980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.528+0000 7fa5b8394700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b0105f30 con 0x7fa5b00fbce0 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.528+0000 7fa5b8394700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b01060a0 con 0x7fa5b00fcf90 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5b6130700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fbce0 0x7fa5b0109440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5b6130700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fbce0 0x7fa5b0109440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57248/0 (socket says 172.21.15.31:57248) 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5b6130700 1 -- 172.21.15.31:0/1306568888 learned_addr learned my addr 172.21.15.31:0/1306568888 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5b592f700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b00fcf90 0x7fa5b0109980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5b6130700 1 -- 172.21.15.31:0/1306568888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b00fcf90 msgr2=0x7fa5b0109980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5b6130700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b00fcf90 0x7fa5b0109980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5b6130700 1 -- 172.21.15.31:0/1306568888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a0005040 con 0x7fa5b00fbce0 2024-04-24T23:12:02.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5b6130700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fbce0 0x7fa5b0109440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00eb40 tx=0x7fa5ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.529+0000 7fa5a6ffd700 1 -- 172.21.15.31:0/1306568888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5ac007cc0 con 0x7fa5b00fbce0 2024-04-24T23:12:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.530+0000 7fa5a6ffd700 1 -- 172.21.15.31:0/1306568888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5ac00f040 con 0x7fa5b00fbce0 2024-04-24T23:12:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.530+0000 7fa5a6ffd700 1 -- 172.21.15.31:0/1306568888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5ac00ca10 con 0x7fa5b00fbce0 2024-04-24T23:12:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.530+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b0106320 con 0x7fa5b00fbce0 2024-04-24T23:12:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.530+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b0106860 con 0x7fa5b00fbce0 2024-04-24T23:12:02.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.531+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b0101920 con 0x7fa5b00fbce0 2024-04-24T23:12:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.532+0000 7fa5a6ffd700 1 -- 172.21.15.31:0/1306568888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa5ac00cb70 con 0x7fa5b00fbce0 2024-04-24T23:12:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.533+0000 7fa5a6ffd700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa59c0692a0 0x7fa59c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.533+0000 7fa5a6ffd700 1 -- 172.21.15.31:0/1306568888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fa5ac086f00 con 0x7fa5b00fbce0 2024-04-24T23:12:02.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.534+0000 7fa5b592f700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa59c0692a0 0x7fa59c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.534+0000 7fa5b592f700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa59c0692a0 0x7fa59c06b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009b30 tx=0x7fa5a0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.538+0000 7fa5a6ffd700 1 -- 172.21.15.31:0/1306568888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa5ac056dc0 con 0x7fa5b00fbce0 2024-04-24T23:12:02.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.818+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5b0104940 con 0x7fa59c0692a0 2024-04-24T23:12:02.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.819+0000 7fa5a6ffd700 1 -- 172.21.15.31:0/1306568888 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fa5b0104940 con 0x7fa59c0692a0 2024-04-24T23:12:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa59c0692a0 msgr2=0x7fa59c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa59c0692a0 0x7fa59c06b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0009b30 tx=0x7fa5a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fbce0 msgr2=0x7fa5b0109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fbce0 0x7fa5b0109440 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00eb40 tx=0x7fa5ac00b040 comp rx=0 tx=0).stop 2024-04-24T23:12:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 shutdown_connections 2024-04-24T23:12:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fa59c0692a0 0x7fa59c06b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa5b00fbce0 0x7fa5b0109440 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 --2- 172.21.15.31:0/1306568888 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa5b00fcf90 0x7fa5b0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 >> 172.21.15.31:0/1306568888 conn(0x7fa5b00f7420 msgr2=0x7fa5b0066720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 shutdown_connections 2024-04-24T23:12:02.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.822+0000 7fa5b8394700 1 -- 172.21.15.31:0/1306568888 wait complete. 2024-04-24T23:12:02.838 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:12:02.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.956+0000 7f8aefec0700 1 -- 172.21.15.31:0/3622139029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 msgr2=0x7f8ae80ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.956+0000 7f8aefec0700 1 --2- 172.21.15.31:0/3622139029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 0x7f8ae80ff9a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8ae4004940 tx=0x7f8ae4009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.957+0000 7f8aefec0700 1 -- 172.21.15.31:0/3622139029 shutdown_connections 2024-04-24T23:12:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.957+0000 7f8aefec0700 1 --2- 172.21.15.31:0/3622139029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ae80ffee0 0x7f8ae8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.957+0000 7f8aefec0700 1 --2- 172.21.15.31:0/3622139029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 0x7f8ae80ff9a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.957+0000 7f8aefec0700 1 -- 172.21.15.31:0/3622139029 >> 172.21.15.31:0/3622139029 conn(0x7f8ae80f7430 msgr2=0x7f8ae80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.957+0000 7f8aefec0700 1 -- 172.21.15.31:0/3622139029 shutdown_connections 2024-04-24T23:12:02.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.957+0000 7f8aefec0700 1 -- 172.21.15.31:0/3622139029 wait complete. 2024-04-24T23:12:02.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aefec0700 1 Processor -- start 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aefec0700 1 -- start start 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aefec0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 0x7f8ae810da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aefec0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ae80ffee0 0x7f8ae810df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aefec0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae810e540 con 0x7f8ae80662c0 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aefec0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae810ed40 con 0x7f8ae80ffee0 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aedc5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 0x7f8ae810da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aed45b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ae80ffee0 0x7f8ae810df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aedc5c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 0x7f8ae810da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57260/0 (socket says 172.21.15.31:57260) 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aedc5c700 1 -- 172.21.15.31:0/1521857185 learned_addr learned my addr 172.21.15.31:0/1521857185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aed45b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ae80ffee0 0x7f8ae810df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40912/0 (socket says 172.21.15.31:40912) 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aed45b700 1 -- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 msgr2=0x7f8ae810da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:02.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.958+0000 7f8aed45b700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 0x7f8ae810da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.959+0000 7f8aed45b700 1 -- 172.21.15.31:0/1521857185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae4005040 con 0x7f8ae80ffee0 2024-04-24T23:12:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.959+0000 7f8aedc5c700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 0x7f8ae810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.959+0000 7f8aed45b700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ae80ffee0 0x7f8ae810df40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009fd0 tx=0x7f8ad800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.959+0000 7f8ade7fc700 1 -- 172.21.15.31:0/1521857185 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad8015070 con 0x7f8ae80ffee0 2024-04-24T23:12:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.959+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae810efc0 con 0x7f8ae80ffee0 2024-04-24T23:12:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.959+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae810f500 con 0x7f8ae80ffee0 2024-04-24T23:12:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.959+0000 7f8ade7fc700 1 -- 172.21.15.31:0/1521857185 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ad8010e50 con 0x7f8ae80ffee0 2024-04-24T23:12:02.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.959+0000 7f8ade7fc700 1 -- 172.21.15.31:0/1521857185 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ad801b470 con 0x7f8ae80ffee0 2024-04-24T23:12:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.960+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8acc0052e0 con 0x7f8ae80ffee0 2024-04-24T23:12:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.961+0000 7f8ade7fc700 1 -- 172.21.15.31:0/1521857185 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8ad80090d0 con 0x7f8ae80ffee0 2024-04-24T23:12:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.961+0000 7f8ade7fc700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ad40692a0 0x7f8ad406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.961+0000 7f8ade7fc700 1 -- 172.21.15.31:0/1521857185 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8ad808b3c0 con 0x7f8ae80ffee0 2024-04-24T23:12:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.961+0000 7f8aedc5c700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ad40692a0 0x7f8ad406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:02.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.962+0000 7f8aedc5c700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ad40692a0 0x7f8ad406b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40045a0 tx=0x7f8ae4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:02.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:02.969+0000 7f8ade7fc700 1 -- 172.21.15.31:0/1521857185 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8ad805b0d0 con 0x7f8ae80ffee0 2024-04-24T23:12:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:03 smithi031 ceph-mon[26411]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (unknown) 2024-04-24T23:12:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:03 smithi031 ceph-mon[26411]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-24T23:12:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:03 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:03 smithi031 ceph-mon[26411]: Upgrade: Need to upgrade myself (mgr.smithi031.rrnmwr) 2024-04-24T23:12:03.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.262+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8acc000bf0 con 0x7f8ad40692a0 2024-04-24T23:12:03.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.263+0000 7f8ade7fc700 1 -- 172.21.15.31:0/1521857185 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f8acc000bf0 con 0x7f8ad40692a0 2024-04-24T23:12:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ad40692a0 msgr2=0x7f8ad406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ad40692a0 0x7f8ad406b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40045a0 tx=0x7f8ae4009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ae80ffee0 msgr2=0x7f8ae810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ae80ffee0 0x7f8ae810df40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009fd0 tx=0x7f8ad800b040 comp rx=0 tx=0).stop 2024-04-24T23:12:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 shutdown_connections 2024-04-24T23:12:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8ad40692a0 0x7f8ad406b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ae80662c0 0x7f8ae810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 --2- 172.21.15.31:0/1521857185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ae80ffee0 0x7f8ae810df40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.266+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 >> 172.21.15.31:0/1521857185 conn(0x7f8ae80f7430 msgr2=0x7f8ae80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.267+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 shutdown_connections 2024-04-24T23:12:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.267+0000 7f8aefec0700 1 -- 172.21.15.31:0/1521857185 wait complete. 2024-04-24T23:12:03.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.389+0000 7f450ebcb700 1 -- 172.21.15.31:0/3486144818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45080fff80 msgr2=0x7f4508102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.389+0000 7f450ebcb700 1 --2- 172.21.15.31:0/3486144818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45080fff80 0x7f4508102360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f44fc004940 tx=0x7f44fc009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.389+0000 7f450ebcb700 1 -- 172.21.15.31:0/3486144818 shutdown_connections 2024-04-24T23:12:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.389+0000 7f450ebcb700 1 --2- 172.21.15.31:0/3486144818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45080fff80 0x7f4508102360 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.389+0000 7f450ebcb700 1 --2- 172.21.15.31:0/3486144818 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4508066220 0x7f45080ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.389+0000 7f450ebcb700 1 -- 172.21.15.31:0/3486144818 >> 172.21.15.31:0/3486144818 conn(0x7f45080f7430 msgr2=0x7f45080f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.389+0000 7f450ebcb700 1 -- 172.21.15.31:0/3486144818 shutdown_connections 2024-04-24T23:12:03.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.389+0000 7f450ebcb700 1 -- 172.21.15.31:0/3486144818 wait complete. 2024-04-24T23:12:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.390+0000 7f450ebcb700 1 Processor -- start 2024-04-24T23:12:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.391+0000 7f450ebcb700 1 -- start start 2024-04-24T23:12:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.391+0000 7f450ebcb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508066220 0x7f450810d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.391+0000 7f450ebcb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45080fff80 0x7f450810ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.391+0000 7f450c967700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508066220 0x7f450810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.391+0000 7f450c967700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508066220 0x7f450810d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57274/0 (socket says 172.21.15.31:57274) 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.392+0000 7f450c967700 1 -- 172.21.15.31:0/2912356349 learned_addr learned my addr 172.21.15.31:0/2912356349 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.392+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450810e4d0 con 0x7f4508066220 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.392+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450810ed20 con 0x7f45080fff80 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.392+0000 7f4507fff700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45080fff80 0x7f450810ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.392+0000 7f450c967700 1 -- 172.21.15.31:0/2912356349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45080fff80 msgr2=0x7f450810ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.392+0000 7f450c967700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45080fff80 0x7f450810ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.392+0000 7f450c967700 1 -- 172.21.15.31:0/2912356349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc005040 con 0x7f4508066220 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.393+0000 7f450c967700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508066220 0x7f450810d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f44f800eb70 tx=0x7f44f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.393+0000 7f45057fa700 1 -- 172.21.15.31:0/2912356349 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44f8007cc0 con 0x7f4508066220 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.393+0000 7f45057fa700 1 -- 172.21.15.31:0/2912356349 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44f800f040 con 0x7f4508066220 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.393+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450810efa0 con 0x7f4508066220 2024-04-24T23:12:03.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.393+0000 7f45057fa700 1 -- 172.21.15.31:0/2912356349 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44f800cb20 con 0x7f4508066220 2024-04-24T23:12:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.393+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450810f4e0 con 0x7f4508066220 2024-04-24T23:12:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.394+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450805f5c0 con 0x7f4508066220 2024-04-24T23:12:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.395+0000 7f45057fa700 1 -- 172.21.15.31:0/2912356349 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f44f800cc80 con 0x7f4508066220 2024-04-24T23:12:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.395+0000 7f45057fa700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44f00692a0 0x7f44f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.396+0000 7f45057fa700 1 -- 172.21.15.31:0/2912356349 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f44f8086b90 con 0x7f4508066220 2024-04-24T23:12:03.397 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.396+0000 7f4507fff700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44f00692a0 0x7f44f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.398 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.396+0000 7f4507fff700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44f00692a0 0x7f44f006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00b0d0 tx=0x7f44fc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.404+0000 7f45057fa700 1 -- 172.21.15.31:0/2912356349 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f44f804e250 con 0x7f4508066220 2024-04-24T23:12:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:03 smithi040 ceph-mon[33128]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (unknown) 2024-04-24T23:12:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:03 smithi040 ceph-mon[33128]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-24T23:12:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:03 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:03.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:03 smithi040 ceph-mon[33128]: Upgrade: Need to upgrade myself (mgr.smithi031.rrnmwr) 2024-04-24T23:12:03.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.783+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f450810f8c0 con 0x7f44f00692a0 2024-04-24T23:12:03.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.800+0000 7f45057fa700 1 -- 172.21.15.31:0/2912356349 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f450810f8c0 con 0x7f44f00692a0 2024-04-24T23:12:03.802 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:12:03.802 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (5m) 54s ago 6m 24.4M - 0.25.0 c8568f914cd2 87840bb14366 2024-04-24T23:12:03.803 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (7m) 54s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:12:03.803 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (6m) 59s ago 6m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:12:03.803 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 54s ago 6m 84.3M - 9.4.7 954c08fa6188 44ecf082adf8 2024-04-24T23:12:03.803 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (73s) 54s ago 73s 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:12:03.803 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (80s) 54s ago 79s 10.4M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:12:03.803 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (77s) 59s ago 76s 11.2M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:12:03.803 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (71s) 59s ago 70s 12.1M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:12:03.804 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:9283 running (8m) 54s ago 8m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 7b9a4ca2c955 2024-04-24T23:12:03.804 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (5m) 59s ago 5m 435M - 17.2.7-902-gc61472c8 0de407c1b3af bb3f989b92a5 2024-04-24T23:12:03.804 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (8m) 54s ago 8m 42.6M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:12:03.804 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (5m) 59s ago 5m 32.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:12:03.804 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (6m) 54s ago 6m 15.7M - 1.5.0 0da6a335fe13 0541b481dfad 2024-04-24T23:12:03.804 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (5m) 59s ago 5m 16.0M - 1.5.0 0da6a335fe13 7bd2678617bc 2024-04-24T23:12:03.804 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 54s ago 5m 45.4M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:12:03.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 54s ago 4m 44.4M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:12:03.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 54s ago 4m 43.9M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:12:03.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (3m) 59s ago 3m 47.5M 5313M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:12:03.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (3m) 59s ago 3m 42.1M 5313M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:12:03.805 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (2m) 59s ago 2m 41.8M 5313M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:12:03.805 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 54s ago 6m 49.6M - 2.43.0 a07b618ecd1d aac36626882b 2024-04-24T23:12:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.802+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44f00692a0 msgr2=0x7f44f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.802+0000 7f450ebcb700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44f00692a0 0x7f44f006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00b0d0 tx=0x7f44fc009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:03.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.802+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508066220 msgr2=0x7f450810d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.802+0000 7f450ebcb700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508066220 0x7f450810d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f44f800eb70 tx=0x7f44f800b040 comp rx=0 tx=0).stop 2024-04-24T23:12:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.803+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 shutdown_connections 2024-04-24T23:12:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.803+0000 7f450ebcb700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f44f00692a0 0x7f44f006b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.803+0000 7f450ebcb700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4508066220 0x7f450810d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.803+0000 7f450ebcb700 1 --2- 172.21.15.31:0/2912356349 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45080fff80 0x7f450810ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.803+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 >> 172.21.15.31:0/2912356349 conn(0x7f45080f7430 msgr2=0x7f45080f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.803+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 shutdown_connections 2024-04-24T23:12:03.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.803+0000 7f450ebcb700 1 -- 172.21.15.31:0/2912356349 wait complete. 2024-04-24T23:12:03.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.939+0000 7f8c1dda1700 1 -- 172.21.15.31:0/3479516703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 msgr2=0x7f8c180fbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.940+0000 7f8c1dda1700 1 --2- 172.21.15.31:0/3479516703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 0x7f8c180fbf00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c004880 tx=0x7f8c0c009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:03.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.940+0000 7f8c1dda1700 1 -- 172.21.15.31:0/3479516703 shutdown_connections 2024-04-24T23:12:03.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.940+0000 7f8c1dda1700 1 --2- 172.21.15.31:0/3479516703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c180fcda0 0x7f8c180fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.940+0000 7f8c1dda1700 1 --2- 172.21.15.31:0/3479516703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 0x7f8c180fbf00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.941+0000 7f8c1dda1700 1 -- 172.21.15.31:0/3479516703 >> 172.21.15.31:0/3479516703 conn(0x7f8c180f7250 msgr2=0x7f8c180f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:03.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.941+0000 7f8c1dda1700 1 -- 172.21.15.31:0/3479516703 shutdown_connections 2024-04-24T23:12:03.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.941+0000 7f8c1dda1700 1 -- 172.21.15.31:0/3479516703 wait complete. 2024-04-24T23:12:03.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.942+0000 7f8c1dda1700 1 Processor -- start 2024-04-24T23:12:03.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.942+0000 7f8c1dda1700 1 -- start start 2024-04-24T23:12:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.943+0000 7f8c1dda1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 0x7f8c1810c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.943+0000 7f8c1dda1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c180fcda0 0x7f8c1810cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.943+0000 7f8c1dda1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1810d260 con 0x7f8c180fcda0 2024-04-24T23:12:03.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.943+0000 7f8c1cd9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 0x7f8c1810c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.944+0000 7f8c1cd9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 0x7f8c1810c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:40946/0 (socket says 172.21.15.31:40946) 2024-04-24T23:12:03.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.944+0000 7f8c1cd9f700 1 -- 172.21.15.31:0/4074338086 learned_addr learned my addr 172.21.15.31:0/4074338086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:03.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.944+0000 7f8c17fff700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c180fcda0 0x7f8c1810cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.944+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1810dab0 con 0x7f8c180fbaf0 2024-04-24T23:12:03.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.944+0000 7f8c1cd9f700 1 -- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c180fcda0 msgr2=0x7f8c1810cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:03.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.944+0000 7f8c1cd9f700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c180fcda0 0x7f8c1810cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:03.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.944+0000 7f8c1cd9f700 1 -- 172.21.15.31:0/4074338086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c04005040 con 0x7f8c180fbaf0 2024-04-24T23:12:03.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.945+0000 7f8c1cd9f700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 0x7f8c1810c720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c013720 tx=0x7f8c0c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.945+0000 7f8c157fa700 1 -- 172.21.15.31:0/4074338086 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c0c00e030 con 0x7f8c180fbaf0 2024-04-24T23:12:03.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.945+0000 7f8c157fa700 1 -- 172.21.15.31:0/4074338086 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c0c013e60 con 0x7f8c180fbaf0 2024-04-24T23:12:03.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.945+0000 7f8c157fa700 1 -- 172.21.15.31:0/4074338086 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c0c0223f0 con 0x7f8c180fbaf0 2024-04-24T23:12:03.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.945+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c005040 con 0x7f8c180fbaf0 2024-04-24T23:12:03.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.945+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1810e100 con 0x7f8c180fbaf0 2024-04-24T23:12:03.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.946+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1805f5c0 con 0x7f8c180fbaf0 2024-04-24T23:12:03.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.947+0000 7f8c157fa700 1 -- 172.21.15.31:0/4074338086 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8c0c013990 con 0x7f8c180fbaf0 2024-04-24T23:12:03.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.948+0000 7f8c157fa700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8c080692b0 0x7f8c0806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:03.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.948+0000 7f8c157fa700 1 -- 172.21.15.31:0/4074338086 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f8c0c088710 con 0x7f8c180fbaf0 2024-04-24T23:12:03.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.948+0000 7f8c17fff700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8c080692b0 0x7f8c0806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:03.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.949+0000 7f8c17fff700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8c080692b0 0x7f8c0806b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009fd0 tx=0x7f8c0400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:03.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:03.956+0000 7f8c157fa700 1 -- 172.21.15.31:0/4074338086 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8c0c058520 con 0x7f8c180fbaf0 2024-04-24T23:12:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:04 smithi031 ceph-mon[26411]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 on smithi040 2024-04-24T23:12:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:04 smithi031 ceph-mon[26411]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:04 smithi031 ceph-mon[26411]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:04 smithi031 ceph-mon[26411]: from='client.24345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:04 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:12:04.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.361+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8c1810e3a0 con 0x7f8c180fbaf0 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.362+0000 7f8c157fa700 1 -- 172.21.15.31:0/4074338086 <== mon.1 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f8c0c0580b0 con 0x7f8c180fbaf0 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:12:04.365 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:12:04.366 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:12:04.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.365+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8c080692b0 msgr2=0x7f8c0806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.365+0000 7f8c1dda1700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8c080692b0 0x7f8c0806b760 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009fd0 tx=0x7f8c0400b040 comp rx=0 tx=0).stop 2024-04-24T23:12:04.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.365+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 msgr2=0x7f8c1810c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.365+0000 7f8c1dda1700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 0x7f8c1810c720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c013720 tx=0x7f8c0c009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.366+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 shutdown_connections 2024-04-24T23:12:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.366+0000 7f8c1dda1700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c180fbaf0 0x7f8c1810c720 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.366+0000 7f8c1dda1700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f8c080692b0 0x7f8c0806b760 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.366+0000 7f8c1dda1700 1 --2- 172.21.15.31:0/4074338086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c180fcda0 0x7f8c1810cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.366+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 >> 172.21.15.31:0/4074338086 conn(0x7f8c180f7250 msgr2=0x7f8c180f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.366+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 shutdown_connections 2024-04-24T23:12:04.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.366+0000 7f8c1dda1700 1 -- 172.21.15.31:0/4074338086 wait complete. 2024-04-24T23:12:04.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:04 smithi040 ceph-mon[33128]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 on smithi040 2024-04-24T23:12:04.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:04 smithi040 ceph-mon[33128]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:04.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:04 smithi040 ceph-mon[33128]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:04.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:04 smithi040 ceph-mon[33128]: from='client.24345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:04.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:04 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:12:04.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.501+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3035894869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 msgr2=0x7ffa18107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.501+0000 7ffa1eae4700 1 --2- 172.21.15.31:0/3035894869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 0x7ffa18107340 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c004940 tx=0x7ffa0c009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:04.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.501+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3035894869 shutdown_connections 2024-04-24T23:12:04.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.501+0000 7ffa1eae4700 1 --2- 172.21.15.31:0/3035894869 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa18107880 0x7ffa18109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.501+0000 7ffa1eae4700 1 --2- 172.21.15.31:0/3035894869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 0x7ffa18107340 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.501+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3035894869 >> 172.21.15.31:0/3035894869 conn(0x7ffa1806a360 msgr2=0x7ffa1806c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.502+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3035894869 shutdown_connections 2024-04-24T23:12:04.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.502+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3035894869 wait complete. 2024-04-24T23:12:04.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.502+0000 7ffa1eae4700 1 Processor -- start 2024-04-24T23:12:04.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.502+0000 7ffa1eae4700 1 -- start start 2024-04-24T23:12:04.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa1eae4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 0x7ffa181180a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa1eae4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa18107880 0x7ffa181185e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa1eae4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa18118be0 con 0x7ffa18104f60 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa1eae4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa18118d50 con 0x7ffa18107880 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa1c880700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 0x7ffa181180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa1c880700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 0x7ffa181180a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57316/0 (socket says 172.21.15.31:57316) 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa1c880700 1 -- 172.21.15.31:0/3379558007 learned_addr learned my addr 172.21.15.31:0/3379558007 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa17fff700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa18107880 0x7ffa181185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa17fff700 1 -- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 msgr2=0x7ffa181180a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa17fff700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 0x7ffa181180a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.503+0000 7ffa17fff700 1 -- 172.21.15.31:0/3379558007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c005040 con 0x7ffa18107880 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.504+0000 7ffa17fff700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa18107880 0x7ffa181185e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009fd0 tx=0x7ffa0400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.504+0000 7ffa157fa700 1 -- 172.21.15.31:0/3379558007 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa04004e90 con 0x7ffa18107880 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.504+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1811d6e0 con 0x7ffa18107880 2024-04-24T23:12:04.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.504+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1811dc20 con 0x7ffa18107880 2024-04-24T23:12:04.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.504+0000 7ffa157fa700 1 -- 172.21.15.31:0/3379558007 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa0400f040 con 0x7ffa18107880 2024-04-24T23:12:04.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.504+0000 7ffa157fa700 1 -- 172.21.15.31:0/3379558007 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa0400ca50 con 0x7ffa18107880 2024-04-24T23:12:04.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.505+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc0052e0 con 0x7ffa18107880 2024-04-24T23:12:04.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.506+0000 7ffa157fa700 1 -- 172.21.15.31:0/3379558007 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ffa04007cc0 con 0x7ffa18107880 2024-04-24T23:12:04.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.506+0000 7ffa157fa700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ffa08069310 0x7ffa0806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:04.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.506+0000 7ffa157fa700 1 -- 172.21.15.31:0/3379558007 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ffa04086a90 con 0x7ffa18107880 2024-04-24T23:12:04.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.507+0000 7ffa1c880700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ffa08069310 0x7ffa0806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:04.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.507+0000 7ffa1c880700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ffa08069310 0x7ffa0806b7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009b30 tx=0x7ffa0c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:04.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.512+0000 7ffa157fa700 1 -- 172.21.15.31:0/3379558007 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffa0404e250 con 0x7ffa18107880 2024-04-24T23:12:04.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.891+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9fc000bf0 con 0x7ffa08069310 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.892+0000 7ffa157fa700 1 -- 172.21.15.31:0/3379558007 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7ff9fc000bf0 con 0x7ffa08069310 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "0/2 daemons upgraded", 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 image on host smithi040", 2024-04-24T23:12:04.894 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:12:04.895 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.895+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ffa08069310 msgr2=0x7ffa0806b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.895+0000 7ffa1eae4700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ffa08069310 0x7ffa0806b7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009b30 tx=0x7ffa0c009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.895+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa18107880 msgr2=0x7ffa181185e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.895+0000 7ffa1eae4700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa18107880 0x7ffa181185e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009fd0 tx=0x7ffa0400b040 comp rx=0 tx=0).stop 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.896+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 shutdown_connections 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.896+0000 7ffa1eae4700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ffa08069310 0x7ffa0806b7c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.896+0000 7ffa1eae4700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa18104f60 0x7ffa181180a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.896+0000 7ffa1eae4700 1 --2- 172.21.15.31:0/3379558007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffa18107880 0x7ffa181185e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.896+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 >> 172.21.15.31:0/3379558007 conn(0x7ffa1806a360 msgr2=0x7ffa1806a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.896+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 shutdown_connections 2024-04-24T23:12:04.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:04.896+0000 7ffa1eae4700 1 -- 172.21.15.31:0/3379558007 wait complete. 2024-04-24T23:12:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:05 smithi031 ceph-mon[26411]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:05.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:05 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/4074338086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:12:05.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:05 smithi040 ceph-mon[33128]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:05.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:05 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/4074338086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:12:06.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:06 smithi040 ceph-mon[33128]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:06.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:06 smithi040 ceph-mon[33128]: from='client.24357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:06.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:06 smithi031 ceph-mon[26411]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:06.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:06 smithi031 ceph-mon[26411]: from='client.24357 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:08.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:08 smithi040 ceph-mon[33128]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:08.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:08 smithi031 ceph-mon[26411]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:10.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:10 smithi040 ceph-mon[33128]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:10.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:10 smithi031 ceph-mon[26411]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:12.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:12 smithi040 ceph-mon[33128]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:12.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:12 smithi031 ceph-mon[26411]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:14.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:14 smithi040 ceph-mon[33128]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:14.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:14 smithi031 ceph-mon[26411]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:16.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:16 smithi040 ceph-mon[33128]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:16.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:16 smithi031 ceph-mon[26411]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:18.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:18 smithi040 ceph-mon[33128]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:18.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:18 smithi031 ceph-mon[26411]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:19.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:19 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:12:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:19 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:12:20.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:20 smithi040 ceph-mon[33128]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:20.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:20 smithi031 ceph-mon[26411]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:22.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:22 smithi040 ceph-mon[33128]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:22.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:22 smithi031 ceph-mon[26411]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:24.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:24 smithi040 ceph-mon[33128]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:24.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:24 smithi031 ceph-mon[26411]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:26.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:26 smithi040 ceph-mon[33128]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:26 smithi031 ceph-mon[26411]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:28.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:28 smithi040 ceph-mon[33128]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:28.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:28 smithi031 ceph-mon[26411]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:29.052 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 96% (12098/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-24T23:12:29.274 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T23:12:29.274 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-24T23:12:29.274 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:12:29.274 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:12:29.274 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-04-24T23:12:29.275 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T23:12:29.278 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-24T23:12:29.307 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T23:12:29.308 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:12:29.309 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T23:12:29.990 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T23:12:30.050 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T23:12:30.092 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-24T23:12:30.094 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:12:30.094 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T23:12:30.134 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-24T23:12:30.138 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:12:30.138 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-24T23:12:30.194 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-24T23:12:30.196 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-24T23:12:30.196 DEBUG:teuthology.orchestra.run.smithi031:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-24T23:12:30.259 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir -p fsstress 2024-04-24T23:12:30.261 INFO:tasks.workunit.client.0.smithi031.stderr:+ pushd fsstress 2024-04-24T23:12:30.262 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T23:12:30.263 INFO:tasks.workunit.client.0.smithi031.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-24T23:12:30.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:30 smithi040 ceph-mon[33128]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:30.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:30 smithi031 ceph-mon[26411]: pgmap v235: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:30.767 INFO:tasks.workunit.client.0.smithi031.stderr:+ tar xzf ltp-full.tgz 2024-04-24T23:12:32.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:32 smithi040 ceph-mon[33128]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:32.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:32 smithi031 ceph-mon[26411]: pgmap v236: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:12:34.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:34 smithi040 ceph-mon[33128]: pgmap v237: 65 pgs: 65 active+clean; 3.2 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 230 KiB/s wr, 7 op/s 2024-04-24T23:12:34.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:34 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:12:34.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:34 smithi031 ceph-mon[26411]: pgmap v237: 65 pgs: 65 active+clean; 3.2 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 230 KiB/s wr, 7 op/s 2024-04-24T23:12:34.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:34 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:12:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.032+0000 7fd525a26700 1 -- 172.21.15.31:0/1655172691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fcf10 msgr2=0x7fd5200fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.032+0000 7fd525a26700 1 --2- 172.21.15.31:0/1655172691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fcf10 0x7fd5200fd320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd510004940 tx=0x7fd510009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.032+0000 7fd525a26700 1 -- 172.21.15.31:0/1655172691 shutdown_connections 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.032+0000 7fd525a26700 1 --2- 172.21.15.31:0/1655172691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5200fe080 0x7fd5200fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.032+0000 7fd525a26700 1 --2- 172.21.15.31:0/1655172691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fcf10 0x7fd5200fd320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.032+0000 7fd525a26700 1 -- 172.21.15.31:0/1655172691 >> 172.21.15.31:0/1655172691 conn(0x7fd5200f8560 msgr2=0x7fd5200fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.032+0000 7fd525a26700 1 -- 172.21.15.31:0/1655172691 shutdown_connections 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.032+0000 7fd525a26700 1 -- 172.21.15.31:0/1655172691 wait complete. 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd525a26700 1 Processor -- start 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd525a26700 1 -- start start 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd525a26700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5200fcf10 0x7fd52010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd525a26700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fe080 0x7fd52010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd525a26700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52010e4a0 con 0x7fd5200fe080 2024-04-24T23:12:35.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd525a26700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52010ecf0 con 0x7fd5200fcf10 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd517fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fe080 0x7fd52010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd517fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fe080 0x7fd52010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41990/0 (socket says 172.21.15.31:41990) 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd517fff700 1 -- 172.21.15.31:0/4206064768 learned_addr learned my addr 172.21.15.31:0/4206064768 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.033+0000 7fd51effd700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5200fcf10 0x7fd52010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd517fff700 1 -- 172.21.15.31:0/4206064768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5200fcf10 msgr2=0x7fd52010d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd517fff700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5200fcf10 0x7fd52010d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd517fff700 1 -- 172.21.15.31:0/4206064768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510005040 con 0x7fd5200fe080 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd517fff700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fe080 0x7fd52010dea0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd50800eb40 tx=0x7fd50800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd5177fe700 1 -- 172.21.15.31:0/4206064768 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd508007cc0 con 0x7fd5200fe080 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52010ef70 con 0x7fd5200fe080 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52010f4b0 con 0x7fd5200fe080 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd5177fe700 1 -- 172.21.15.31:0/4206064768 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd50800f040 con 0x7fd5200fe080 2024-04-24T23:12:35.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.034+0000 7fd5177fe700 1 -- 172.21.15.31:0/4206064768 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd50800ca10 con 0x7fd5200fe080 2024-04-24T23:12:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.035+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52005f5c0 con 0x7fd5200fe080 2024-04-24T23:12:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.036+0000 7fd5177fe700 1 -- 172.21.15.31:0/4206064768 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd50800cb70 con 0x7fd5200fe080 2024-04-24T23:12:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.037+0000 7fd5177fe700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd50c06d6c0 0x7fd50c06fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.037+0000 7fd5177fe700 1 -- 172.21.15.31:0/4206064768 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fd508086d00 con 0x7fd5200fe080 2024-04-24T23:12:35.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.037+0000 7fd51effd700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd50c06d6c0 0x7fd50c06fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.038+0000 7fd51effd700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd50c06d6c0 0x7fd50c06fb70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd51000b0d0 tx=0x7fd510009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:35.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.043+0000 7fd5177fe700 1 -- 172.21.15.31:0/4206064768 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd50804e250 con 0x7fd5200fe080 2024-04-24T23:12:35.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.359+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd52010f830 con 0x7fd50c06d6c0 2024-04-24T23:12:35.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.361+0000 7fd5177fe700 1 -- 172.21.15.31:0/4206064768 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fd52010f830 con 0x7fd50c06d6c0 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd50c06d6c0 msgr2=0x7fd50c06fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd50c06d6c0 0x7fd50c06fb70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd51000b0d0 tx=0x7fd510009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fe080 msgr2=0x7fd52010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fe080 0x7fd52010dea0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd50800eb40 tx=0x7fd50800b040 comp rx=0 tx=0).stop 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 shutdown_connections 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5200fcf10 0x7fd52010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fd50c06d6c0 0x7fd50c06fb70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 --2- 172.21.15.31:0/4206064768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5200fe080 0x7fd52010dea0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 >> 172.21.15.31:0/4206064768 conn(0x7fd5200f8560 msgr2=0x7fd5200f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.364+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 shutdown_connections 2024-04-24T23:12:35.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.365+0000 7fd525a26700 1 -- 172.21.15.31:0/4206064768 wait complete. 2024-04-24T23:12:35.380 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:12:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.500+0000 7ff1dc97c700 1 -- 172.21.15.31:0/1258309466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fc270 msgr2=0x7ff1d80fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.500+0000 7ff1dc97c700 1 --2- 172.21.15.31:0/1258309466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fc270 0x7ff1d80fc680 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8004940 tx=0x7ff1c8009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.500+0000 7ff1dc97c700 1 -- 172.21.15.31:0/1258309466 shutdown_connections 2024-04-24T23:12:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.500+0000 7ff1dc97c700 1 --2- 172.21.15.31:0/1258309466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d80fcbc0 0x7ff1d80fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.500+0000 7ff1dc97c700 1 --2- 172.21.15.31:0/1258309466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fc270 0x7ff1d80fc680 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.500+0000 7ff1dc97c700 1 -- 172.21.15.31:0/1258309466 >> 172.21.15.31:0/1258309466 conn(0x7ff1d80f6b80 msgr2=0x7ff1d80f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.500+0000 7ff1dc97c700 1 -- 172.21.15.31:0/1258309466 shutdown_connections 2024-04-24T23:12:35.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.500+0000 7ff1dc97c700 1 -- 172.21.15.31:0/1258309466 wait complete. 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1dc97c700 1 Processor -- start 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1dc97c700 1 -- start start 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1dc97c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d80fc270 0x7ff1d810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1dc97c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fcbc0 0x7ff1d810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1dc97c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d810e4c0 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1dc97c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d810ed10 con 0x7ff1d80fc270 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1d5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fcbc0 0x7ff1d810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1d5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fcbc0 0x7ff1d810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42012/0 (socket says 172.21.15.31:42012) 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1d5d9b700 1 -- 172.21.15.31:0/405343892 learned_addr learned my addr 172.21.15.31:0/405343892 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.501+0000 7ff1d659c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d80fc270 0x7ff1d810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1d5d9b700 1 -- 172.21.15.31:0/405343892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d80fc270 msgr2=0x7ff1d810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1d5d9b700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d80fc270 0x7ff1d810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1d5d9b700 1 -- 172.21.15.31:0/405343892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc005040 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1d659c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d80fc270 0x7ff1d810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:12:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1d5d9b700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fcbc0 0x7ff1d810dec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0096d0 tx=0x7ff1cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1c6ffd700 1 -- 172.21.15.31:0/405343892 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1cc00e840 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c8005040 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1c6ffd700 1 -- 172.21.15.31:0/405343892 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1cc00f040 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1c6ffd700 1 -- 172.21.15.31:0/405343892 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1cc013dc0 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.502+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d810f340 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.504+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d8101bc0 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.504+0000 7ff1c6ffd700 1 -- 172.21.15.31:0/405343892 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff1cc00e9a0 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.505+0000 7ff1c6ffd700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff1c00692a0 0x7ff1c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.505+0000 7ff1c6ffd700 1 -- 172.21.15.31:0/405343892 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7ff1cc086fa0 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.505+0000 7ff1d659c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff1c00692a0 0x7ff1c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.505+0000 7ff1d659c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff1c00692a0 0x7ff1c006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800b0d0 tx=0x7ff1c8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:35.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.510+0000 7ff1c6ffd700 1 -- 172.21.15.31:0/405343892 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff1cc056c90 con 0x7ff1d80fcbc0 2024-04-24T23:12:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:35 smithi031 ceph-mon[26411]: Upgrade: Updating mgr.smithi040.qtdwuq 2024-04-24T23:12:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:12:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:12:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:35 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:35 smithi031 ceph-mon[26411]: Deploying daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:12:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:35 smithi031 ceph-mon[26411]: pgmap v238: 65 pgs: 65 active+clean; 8.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 718 KiB/s wr, 43 op/s 2024-04-24T23:12:35.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.805+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1d810f560 con 0x7ff1c00692a0 2024-04-24T23:12:35.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.807+0000 7ff1c6ffd700 1 -- 172.21.15.31:0/405343892 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7ff1d810f560 con 0x7ff1c00692a0 2024-04-24T23:12:35.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff1c00692a0 msgr2=0x7ff1c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff1c00692a0 0x7ff1c006b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800b0d0 tx=0x7ff1c8009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:35.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fcbc0 msgr2=0x7ff1d810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fcbc0 0x7ff1d810dec0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0096d0 tx=0x7ff1cc009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:35.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 shutdown_connections 2024-04-24T23:12:35.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1d80fc270 0x7ff1d810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7ff1c00692a0 0x7ff1c006b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 --2- 172.21.15.31:0/405343892 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80fcbc0 0x7ff1d810dec0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 >> 172.21.15.31:0/405343892 conn(0x7ff1d80f6b80 msgr2=0x7ff1d80fb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 shutdown_connections 2024-04-24T23:12:35.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.809+0000 7ff1dc97c700 1 -- 172.21.15.31:0/405343892 wait complete. 2024-04-24T23:12:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:35 smithi040 ceph-mon[33128]: Upgrade: Updating mgr.smithi040.qtdwuq 2024-04-24T23:12:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:12:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:12:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:35 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:12:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:35 smithi040 ceph-mon[33128]: Deploying daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:12:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:35 smithi040 ceph-mon[33128]: pgmap v238: 65 pgs: 65 active+clean; 8.7 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 718 KiB/s wr, 43 op/s 2024-04-24T23:12:35.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.946+0000 7fb06f5c7700 1 -- 172.21.15.31:0/2014580322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fcf70 msgr2=0x7fb0680fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.946+0000 7fb06f5c7700 1 --2- 172.21.15.31:0/2014580322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fcf70 0x7fb0680fd3e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb058004940 tx=0x7fb058009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.946+0000 7fb06f5c7700 1 -- 172.21.15.31:0/2014580322 shutdown_connections 2024-04-24T23:12:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.946+0000 7fb06f5c7700 1 --2- 172.21.15.31:0/2014580322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fcf70 0x7fb0680fd3e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.946+0000 7fb06f5c7700 1 --2- 172.21.15.31:0/2014580322 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0680fbcc0 0x7fb0680fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.946+0000 7fb06f5c7700 1 -- 172.21.15.31:0/2014580322 >> 172.21.15.31:0/2014580322 conn(0x7fb0680f7420 msgr2=0x7fb0680f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.946+0000 7fb06f5c7700 1 -- 172.21.15.31:0/2014580322 shutdown_connections 2024-04-24T23:12:35.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.946+0000 7fb06f5c7700 1 -- 172.21.15.31:0/2014580322 wait complete. 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06f5c7700 1 Processor -- start 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06f5c7700 1 -- start start 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06f5c7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fbcc0 0x7fb06810fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06f5c7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0680fcf70 0x7fb0681100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06f5c7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0681106c0 con 0x7fb0680fbcc0 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06f5c7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068110f10 con 0x7fb0680fcf70 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06d363700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fbcc0 0x7fb06810fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06d363700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fbcc0 0x7fb06810fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42038/0 (socket says 172.21.15.31:42038) 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06d363700 1 -- 172.21.15.31:0/55335979 learned_addr learned my addr 172.21.15.31:0/55335979 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.947+0000 7fb06cb62700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0680fcf70 0x7fb0681100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb06d363700 1 -- 172.21.15.31:0/55335979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0680fcf70 msgr2=0x7fb0681100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb06d363700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0680fcf70 0x7fb0681100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:35.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb06d363700 1 -- 172.21.15.31:0/55335979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058005040 con 0x7fb0680fbcc0 2024-04-24T23:12:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb06d363700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fbcc0 0x7fb06810fb80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb06400ba50 tx=0x7fb06400d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb05dffb700 1 -- 172.21.15.31:0/55335979 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb064010040 con 0x7fb0680fbcc0 2024-04-24T23:12:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb068111190 con 0x7fb0680fbcc0 2024-04-24T23:12:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb05dffb700 1 -- 172.21.15.31:0/55335979 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb06400ee10 con 0x7fb0680fbcc0 2024-04-24T23:12:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb05dffb700 1 -- 172.21.15.31:0/55335979 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb064014400 con 0x7fb0680fbcc0 2024-04-24T23:12:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.948+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0681116d0 con 0x7fb0680fbcc0 2024-04-24T23:12:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.949+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06805f5c0 con 0x7fb0680fbcc0 2024-04-24T23:12:35.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.950+0000 7fb05dffb700 1 -- 172.21.15.31:0/55335979 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fb06400e940 con 0x7fb0680fbcc0 2024-04-24T23:12:35.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.950+0000 7fb05dffb700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0540692a0 0x7fb05406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.951+0000 7fb05dffb700 1 -- 172.21.15.31:0/55335979 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7fb064086e80 con 0x7fb0680fbcc0 2024-04-24T23:12:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.951+0000 7fb06cb62700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0540692a0 0x7fb05406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:35.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.951+0000 7fb06cb62700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0540692a0 0x7fb05406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb058009b30 tx=0x7fb058009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:35.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:35.955+0000 7fb05dffb700 1 -- 172.21.15.31:0/55335979 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb06404e250 con 0x7fb0680fbcc0 2024-04-24T23:12:36.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.261+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb068111990 con 0x7fb0540692a0 2024-04-24T23:12:36.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.274+0000 7fb05dffb700 1 -- 172.21.15.31:0/55335979 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7fb068111990 con 0x7fb0540692a0 2024-04-24T23:12:36.276 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:12:36.276 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 87s ago 7m 24.4M - 0.25.0 c8568f914cd2 87840bb14366 2024-04-24T23:12:36.276 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (7m) 87s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:12:36.276 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (6m) 91s ago 6m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:12:36.276 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 87s ago 7m 84.3M - 9.4.7 954c08fa6188 44ecf082adf8 2024-04-24T23:12:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (106s) 87s ago 105s 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:12:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (112s) 87s ago 111s 10.4M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:12:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (109s) 91s ago 109s 11.2M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:12:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (103s) 91s ago 103s 12.1M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:12:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:9283 running (8m) 87s ago 8m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 7b9a4ca2c955 2024-04-24T23:12:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (6m) 91s ago 6m 435M - 17.2.7-902-gc61472c8 0de407c1b3af bb3f989b92a5 2024-04-24T23:12:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (8m) 87s ago 8m 42.6M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:12:36.277 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (6m) 91s ago 6m 32.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:12:36.278 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 87s ago 7m 15.7M - 1.5.0 0da6a335fe13 0541b481dfad 2024-04-24T23:12:36.278 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (6m) 91s ago 6m 16.0M - 1.5.0 0da6a335fe13 7bd2678617bc 2024-04-24T23:12:36.278 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 87s ago 5m 45.4M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:12:36.278 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 87s ago 5m 44.4M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:12:36.279 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 87s ago 4m 43.9M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:12:36.279 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (4m) 91s ago 4m 47.5M 5313M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:12:36.279 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (3m) 91s ago 3m 42.1M 5313M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:12:36.279 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (3m) 91s ago 3m 41.8M 5313M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:12:36.279 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 87s ago 6m 49.6M - 2.43.0 a07b618ecd1d aac36626882b 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0540692a0 msgr2=0x7fb05406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0540692a0 0x7fb05406b750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb058009b30 tx=0x7fb058009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fbcc0 msgr2=0x7fb06810fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fbcc0 0x7fb06810fb80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb06400ba50 tx=0x7fb06400d040 comp rx=0 tx=0).stop 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 shutdown_connections 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7fb0540692a0 0x7fb05406b750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0680fbcc0 0x7fb06810fb80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 --2- 172.21.15.31:0/55335979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0680fcf70 0x7fb0681100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 >> 172.21.15.31:0/55335979 conn(0x7fb0680f7420 msgr2=0x7fb0680f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 shutdown_connections 2024-04-24T23:12:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.276+0000 7fb06f5c7700 1 -- 172.21.15.31:0/55335979 wait complete. 2024-04-24T23:12:36.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.407+0000 7f57f4333700 1 -- 172.21.15.31:0/2540417922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57ec0fcf60 msgr2=0x7f57ec0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.407+0000 7f57f4333700 1 --2- 172.21.15.31:0/2540417922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57ec0fcf60 0x7f57ec0fd3d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f57dc004880 tx=0x7f57dc009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.408+0000 7f57f4333700 1 -- 172.21.15.31:0/2540417922 shutdown_connections 2024-04-24T23:12:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.408+0000 7f57f4333700 1 --2- 172.21.15.31:0/2540417922 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57ec0fcf60 0x7f57ec0fd3d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.408+0000 7f57f4333700 1 --2- 172.21.15.31:0/2540417922 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec0fbcb0 0x7f57ec0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.408+0000 7f57f4333700 1 -- 172.21.15.31:0/2540417922 >> 172.21.15.31:0/2540417922 conn(0x7f57ec0f7430 msgr2=0x7f57ec0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.408+0000 7f57f4333700 1 -- 172.21.15.31:0/2540417922 shutdown_connections 2024-04-24T23:12:36.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.408+0000 7f57f4333700 1 -- 172.21.15.31:0/2540417922 wait complete. 2024-04-24T23:12:36.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.408+0000 7f57f4333700 1 Processor -- start 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.408+0000 7f57f4333700 1 -- start start 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f4333700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57ec0fbcb0 0x7f57ec10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f4333700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec0fcf60 0x7f57ec10df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f4333700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec10e570 con 0x7f57ec0fcf60 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f4333700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec10edc0 con 0x7f57ec0fbcb0 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f18ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec0fcf60 0x7f57ec10df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f18ce700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec0fcf60 0x7f57ec10df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42058/0 (socket says 172.21.15.31:42058) 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f18ce700 1 -- 172.21.15.31:0/276524948 learned_addr learned my addr 172.21.15.31:0/276524948 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f20cf700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57ec0fbcb0 0x7f57ec10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f18ce700 1 -- 172.21.15.31:0/276524948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57ec0fbcb0 msgr2=0x7f57ec10da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f18ce700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57ec0fbcb0 0x7f57ec10da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f18ce700 1 -- 172.21.15.31:0/276524948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc005040 con 0x7f57ec0fcf60 2024-04-24T23:12:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.409+0000 7f57f18ce700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec0fcf60 0x7f57ec10df70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f57dc008cd0 tx=0x7f57dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.410+0000 7f57e2ffd700 1 -- 172.21.15.31:0/276524948 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57dc00e030 con 0x7f57ec0fcf60 2024-04-24T23:12:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.410+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec10f040 con 0x7f57ec0fcf60 2024-04-24T23:12:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.410+0000 7f57e2ffd700 1 -- 172.21.15.31:0/276524948 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57dc013e60 con 0x7f57ec0fcf60 2024-04-24T23:12:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.410+0000 7f57e2ffd700 1 -- 172.21.15.31:0/276524948 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57dc0123c0 con 0x7f57ec0fcf60 2024-04-24T23:12:36.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.410+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec10f580 con 0x7f57ec0fcf60 2024-04-24T23:12:36.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.411+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec101af0 con 0x7f57ec0fcf60 2024-04-24T23:12:36.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.412+0000 7f57e2ffd700 1 -- 172.21.15.31:0/276524948 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f57dc01c400 con 0x7f57ec0fcf60 2024-04-24T23:12:36.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.412+0000 7f57e2ffd700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f57d80692b0 0x7f57d806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.412+0000 7f57e2ffd700 1 -- 172.21.15.31:0/276524948 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f57dc088f70 con 0x7f57ec0fcf60 2024-04-24T23:12:36.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.413+0000 7f57f20cf700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f57d80692b0 0x7f57d806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.413+0000 7f57f20cf700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f57d80692b0 0x7f57d806b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009b80 tx=0x7f57e800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.418+0000 7f57e2ffd700 1 -- 172.21.15.31:0/276524948 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f57dc058e40 con 0x7f57ec0fcf60 2024-04-24T23:12:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.794+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f57ec104930 con 0x7f57ec0fcf60 2024-04-24T23:12:36.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.794+0000 7f57e2ffd700 1 -- 172.21.15.31:0/276524948 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f57ec104930 con 0x7f57ec0fcf60 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:12:36.797 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:12:36.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:12:36.798 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:12:36.798 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:12:36.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:12:36.798 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:12:36.798 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-24T23:12:36.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:12:36.799 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:12:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f57d80692b0 msgr2=0x7f57d806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f57d80692b0 0x7f57d806b760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009b80 tx=0x7f57e800b040 comp rx=0 tx=0).stop 2024-04-24T23:12:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec0fcf60 msgr2=0x7f57ec10df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec0fcf60 0x7f57ec10df70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f57dc008cd0 tx=0x7f57dc009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:36.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 shutdown_connections 2024-04-24T23:12:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57ec0fbcb0 0x7f57ec10da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f57d80692b0 0x7f57d806b760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 --2- 172.21.15.31:0/276524948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f57ec0fcf60 0x7f57ec10df70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 >> 172.21.15.31:0/276524948 conn(0x7f57ec0f7430 msgr2=0x7f57ec065bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 shutdown_connections 2024-04-24T23:12:36.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.797+0000 7f57f4333700 1 -- 172.21.15.31:0/276524948 wait complete. 2024-04-24T23:12:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:36 smithi040 ceph-mon[33128]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:36 smithi040 ceph-mon[33128]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:36 smithi040 ceph-mon[33128]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.928+0000 7f8681c7a700 1 -- 172.21.15.31:0/2545533589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867c066eb0 msgr2=0x7f867c0ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.928+0000 7f8681c7a700 1 --2- 172.21.15.31:0/2545533589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867c066eb0 0x7f867c0ffa50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f866c004940 tx=0x7f866c009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:36.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.928+0000 7f8681c7a700 1 -- 172.21.15.31:0/2545533589 shutdown_connections 2024-04-24T23:12:36.930 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:36 smithi031 ceph-mon[26411]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:36.930 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:36 smithi031 ceph-mon[26411]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:36.930 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:36 smithi031 ceph-mon[26411]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.928+0000 7f8681c7a700 1 --2- 172.21.15.31:0/2545533589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f867c0fff90 0x7f867c102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.928+0000 7f8681c7a700 1 --2- 172.21.15.31:0/2545533589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867c066eb0 0x7f867c0ffa50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.928+0000 7f8681c7a700 1 -- 172.21.15.31:0/2545533589 >> 172.21.15.31:0/2545533589 conn(0x7f867c0f7440 msgr2=0x7f867c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.928+0000 7f8681c7a700 1 -- 172.21.15.31:0/2545533589 shutdown_connections 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.928+0000 7f8681c7a700 1 -- 172.21.15.31:0/2545533589 wait complete. 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.929+0000 7f8681c7a700 1 Processor -- start 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.929+0000 7f8681c7a700 1 -- start start 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.929+0000 7f8681c7a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f867c066eb0 0x7f867c10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.929+0000 7f8681c7a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867c0fff90 0x7f867c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.929+0000 7f8681c7a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c10e490 con 0x7f867c0fff90 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.929+0000 7f8681c7a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c10e600 con 0x7f867c066eb0 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.930+0000 7f867b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f867c066eb0 0x7f867c10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.930+0000 7f867b7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f867c066eb0 0x7f867c10d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:48834/0 (socket says 172.21.15.31:48834) 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.930+0000 7f867b7fe700 1 -- 172.21.15.31:0/3548078935 learned_addr learned my addr 172.21.15.31:0/3548078935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.930+0000 7f867affd700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867c0fff90 0x7f867c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.930+0000 7f867b7fe700 1 -- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867c0fff90 msgr2=0x7f867c10de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.930+0000 7f867b7fe700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867c0fff90 0x7f867c10de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.930+0000 7f867b7fe700 1 -- 172.21.15.31:0/3548078935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c005040 con 0x7f867c066eb0 2024-04-24T23:12:36.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.931+0000 7f867b7fe700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f867c066eb0 0x7f867c10d950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f866c000c00 tx=0x7f866c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.931+0000 7f8663fff700 1 -- 172.21.15.31:0/3548078935 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f866c00e030 con 0x7f867c066eb0 2024-04-24T23:12:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.931+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c113000 con 0x7f867c066eb0 2024-04-24T23:12:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.931+0000 7f8663fff700 1 -- 172.21.15.31:0/3548078935 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f866c01c400 con 0x7f867c066eb0 2024-04-24T23:12:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.931+0000 7f8663fff700 1 -- 172.21.15.31:0/3548078935 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f866c00b650 con 0x7f867c066eb0 2024-04-24T23:12:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.931+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c113540 con 0x7f867c066eb0 2024-04-24T23:12:36.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.933+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c04c710 con 0x7f867c066eb0 2024-04-24T23:12:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.933+0000 7f8663fff700 1 -- 172.21.15.31:0/3548078935 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f866c013af0 con 0x7f867c066eb0 2024-04-24T23:12:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.934+0000 7f8663fff700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f86640692a0 0x7f866406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:12:36.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.934+0000 7f8663fff700 1 -- 172.21.15.31:0/3548078935 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5550+0+0 (secure 0 0 0) 0x7f866c088bb0 con 0x7f867c066eb0 2024-04-24T23:12:36.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.934+0000 7f867affd700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f86640692a0 0x7f866406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:12:36.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.934+0000 7f867affd700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f86640692a0 0x7f866406b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8670009b80 tx=0x7f867000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:12:36.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:36.938+0000 7f8663fff700 1 -- 172.21.15.31:0/3548078935 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f866c08f080 con 0x7f867c066eb0 2024-04-24T23:12:37.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.170+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 --> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f867c0f8f30 con 0x7f86640692a0 2024-04-24T23:12:37.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.171+0000 7f8663fff700 1 -- 172.21.15.31:0/3548078935 <== mgr.14208 v2:172.21.15.31:6800/1061697388 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f867c0f8f30 con 0x7f86640692a0 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "0/2 daemons upgraded", 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:12:37.173 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f86640692a0 msgr2=0x7f866406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f86640692a0 0x7f866406b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8670009b80 tx=0x7f867000b040 comp rx=0 tx=0).stop 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f867c066eb0 msgr2=0x7f867c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f867c066eb0 0x7f867c10d950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f866c000c00 tx=0x7f866c009e30 comp rx=0 tx=0).stop 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 shutdown_connections 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f867c066eb0 0x7f867c10d950 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:6800/1061697388,v1:172.21.15.31:6801/1061697388] conn(0x7f86640692a0 0x7f866406b750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 --2- 172.21.15.31:0/3548078935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f867c0fff90 0x7f867c10de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 >> 172.21.15.31:0/3548078935 conn(0x7f867c0f7440 msgr2=0x7f867c0f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 shutdown_connections 2024-04-24T23:12:37.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:12:37.173+0000 7f8681c7a700 1 -- 172.21.15.31:0/3548078935 wait complete. 2024-04-24T23:12:37.898 INFO:tasks.workunit.client.1.smithi040.stderr:Updating files: 84% (10600/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-24T23:12:37.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:37 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/276524948' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:12:37.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:37 smithi040 ceph-mon[33128]: pgmap v239: 65 pgs: 65 active+clean; 19 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s wr, 110 op/s 2024-04-24T23:12:37.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:37 smithi040 ceph-mon[33128]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:37.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:37 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/276524948' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:12:37.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:37 smithi031 ceph-mon[26411]: pgmap v239: 65 pgs: 65 active+clean; 19 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s wr, 110 op/s 2024-04-24T23:12:37.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:37 smithi031 ceph-mon[26411]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:12:38.736 INFO:tasks.workunit.client.1.smithi040.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-24T23:12:38.736 INFO:tasks.workunit.client.1.smithi040.stderr: 2024-04-24T23:12:38.736 INFO:tasks.workunit.client.1.smithi040.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-24T23:12:38.736 INFO:tasks.workunit.client.1.smithi040.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-24T23:12:38.736 INFO:tasks.workunit.client.1.smithi040.stderr:state without impacting any branches by switching back to a branch. 2024-04-24T23:12:38.736 INFO:tasks.workunit.client.1.smithi040.stderr: 2024-04-24T23:12:38.736 INFO:tasks.workunit.client.1.smithi040.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr: 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr: git switch -c 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr: 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr:Or undo this operation with: 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr: 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr: git switch - 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr: 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr: 2024-04-24T23:12:38.737 INFO:tasks.workunit.client.1.smithi040.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-24T23:12:38.740 DEBUG:teuthology.orchestra.run.smithi040:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-24T23:12:38.804 INFO:tasks.workunit.client.1.smithi040.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-24T23:12:38.806 INFO:tasks.workunit.client.1.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-24T23:12:38.806 INFO:tasks.workunit.client.1.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-24T23:12:40.140 INFO:tasks.workunit.client.1.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-24T23:12:40.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:39 smithi040 ceph-mon[33128]: pgmap v240: 65 pgs: 65 active+clean; 20 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s wr, 123 op/s 2024-04-24T23:12:40.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:39 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:40.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:39 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:40.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:39 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:40.199 INFO:tasks.workunit.client.1.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-24T23:12:40.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:39 smithi031 ceph-mon[26411]: pgmap v240: 65 pgs: 65 active+clean; 20 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s wr, 123 op/s 2024-04-24T23:12:40.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:39 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:40.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:39 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:40.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:39 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:12:40.254 INFO:tasks.workunit.client.1.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-24T23:12:40.257 INFO:tasks.workunit.client.1.smithi040.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-24T23:12:40.257 INFO:tasks.workunit.client.1.smithi040.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-24T23:12:40.309 INFO:tasks.workunit.client.1.smithi040.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-24T23:12:40.313 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2024-04-24T23:12:40.313 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-24T23:12:40.373 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-24T23:12:40.374 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-24T23:12:40.375 DEBUG:teuthology.orchestra.run.smithi040:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-24T23:12:40.443 INFO:tasks.workunit.client.1.smithi040.stderr:+ mkdir -p fsstress 2024-04-24T23:12:40.448 INFO:tasks.workunit.client.1.smithi040.stderr:+ pushd fsstress 2024-04-24T23:12:40.450 INFO:tasks.workunit.client.1.smithi040.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T23:12:40.450 INFO:tasks.workunit.client.1.smithi040.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-24T23:12:41.160 INFO:tasks.workunit.client.1.smithi040.stderr:+ tar xzf ltp-full.tgz 2024-04-24T23:12:42.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:41 smithi040 ceph-mon[33128]: pgmap v241: 65 pgs: 65 active+clean; 28 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s wr, 201 op/s 2024-04-24T23:12:42.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:41 smithi031 ceph-mon[26411]: pgmap v241: 65 pgs: 65 active+clean; 28 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s wr, 201 op/s 2024-04-24T23:12:44.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:43 smithi040 ceph-mon[33128]: pgmap v242: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s wr, 254 op/s 2024-04-24T23:12:44.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:43 smithi031 ceph-mon[26411]: pgmap v242: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 2.8 MiB/s wr, 254 op/s 2024-04-24T23:12:46.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:45 smithi040 ceph-mon[33128]: pgmap v243: 65 pgs: 65 active+clean; 48 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 3.9 MiB/s wr, 314 op/s 2024-04-24T23:12:46.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:45 smithi031 ceph-mon[26411]: pgmap v243: 65 pgs: 65 active+clean; 48 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 3.9 MiB/s wr, 314 op/s 2024-04-24T23:12:48.155 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:47 smithi040 ceph-mon[33128]: pgmap v244: 65 pgs: 65 active+clean; 63 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.8 MiB/s wr, 375 op/s 2024-04-24T23:12:48.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:47 smithi031 ceph-mon[26411]: pgmap v244: 65 pgs: 65 active+clean; 63 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.8 MiB/s wr, 375 op/s 2024-04-24T23:12:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: Standby manager daemon smithi040.qtdwuq restarted 2024-04-24T23:12:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:12:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:12:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:12:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:12:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:12:49.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:49.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:48 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: Standby manager daemon smithi040.qtdwuq restarted 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:49.350 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:48 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:12:50.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:50 smithi040 ceph-mon[33128]: pgmap v245: 65 pgs: 65 active+clean; 64 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 326 op/s 2024-04-24T23:12:50.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:50 smithi040 ceph-mon[33128]: mgrmap e20: smithi031.rrnmwr(active, since 7m), standbys: smithi040.qtdwuq 2024-04-24T23:12:50.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:50 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:12:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:50 smithi031 ceph-mon[26411]: pgmap v245: 65 pgs: 65 active+clean; 64 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s wr, 326 op/s 2024-04-24T23:12:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:50 smithi031 ceph-mon[26411]: mgrmap e20: smithi031.rrnmwr(active, since 7m), standbys: smithi040.qtdwuq 2024-04-24T23:12:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:50 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:12:52.117 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:52 smithi040 ceph-mon[33128]: pgmap v246: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.2 MiB/s wr, 415 op/s 2024-04-24T23:12:52.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:52 smithi031 ceph-mon[26411]: pgmap v246: 65 pgs: 65 active+clean; 67 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.2 MiB/s wr, 415 op/s 2024-04-24T23:12:54.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:54 smithi040 ceph-mon[33128]: pgmap v247: 65 pgs: 65 active+clean; 78 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.5 MiB/s wr, 414 op/s 2024-04-24T23:12:54.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:54 smithi031 ceph-mon[26411]: pgmap v247: 65 pgs: 65 active+clean; 78 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.5 MiB/s wr, 414 op/s 2024-04-24T23:12:56.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:56 smithi040 ceph-mon[33128]: pgmap v248: 65 pgs: 65 active+clean; 81 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.3 MiB/s wr, 431 op/s 2024-04-24T23:12:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:56 smithi031 ceph-mon[26411]: pgmap v248: 65 pgs: 65 active+clean; 81 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.3 MiB/s wr, 431 op/s 2024-04-24T23:12:58.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:12:58 smithi031 ceph-mon[26411]: pgmap v249: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s wr, 453 op/s 2024-04-24T23:12:58.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:12:58 smithi040 ceph-mon[33128]: pgmap v249: 65 pgs: 65 active+clean; 93 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 4.0 MiB/s wr, 453 op/s 2024-04-24T23:13:00.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: pgmap v250: 65 pgs: 65 active+clean; 93 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 375 op/s 2024-04-24T23:13:00.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr fail", "who": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: Activating manager daemon smithi040.qtdwuq 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: osdmap e47: 6 total, 6 up, 6 in 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "mgr fail", "who": "smithi031.rrnmwr"}]': finished 2024-04-24T23:13:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: mgrmap e21: smithi040.qtdwuq(active, starting, since 0.00921313s) 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr metadata", "who": "smithi040.qtdwuq", "id": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: Manager daemon smithi040.qtdwuq is now available 2024-04-24T23:13:00.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:00.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:13:00.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.qtdwuq/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:13:00.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.qtdwuq/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:13:00.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.qtdwuq/trash_purge_schedule"}]: dispatch 2024-04-24T23:13:00.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.qtdwuq/trash_purge_schedule"}]: dispatch 2024-04-24T23:13:00.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:00.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:00 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:00.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: pgmap v250: 65 pgs: 65 active+clean; 93 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 MiB/s wr, 375 op/s 2024-04-24T23:13:00.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr fail", "who": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: Activating manager daemon smithi040.qtdwuq 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: osdmap e47: 6 total, 6 up, 6 in 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:13:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.14208 172.21.15.31:0/1762739926' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "mgr fail", "who": "smithi031.rrnmwr"}]': finished 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: mgrmap e21: smithi040.qtdwuq(active, starting, since 0.00921313s) 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr metadata", "who": "smithi040.qtdwuq", "id": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: Manager daemon smithi040.qtdwuq is now available 2024-04-24T23:13:00.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:13:00.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:13:00.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.qtdwuq/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:13:00.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.qtdwuq/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:13:00.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.qtdwuq/trash_purge_schedule"}]: dispatch 2024-04-24T23:13:00.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi040.qtdwuq/trash_purge_schedule"}]: dispatch 2024-04-24T23:13:00.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:00.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:00 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:01.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:01 smithi040 ceph-mon[33128]: Deploying cephadm binary to smithi040 2024-04-24T23:13:01.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:01 smithi040 ceph-mon[33128]: Deploying cephadm binary to smithi031 2024-04-24T23:13:01.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:01 smithi040 ceph-mon[33128]: mgrmap e22: smithi040.qtdwuq(active, since 1.01056s) 2024-04-24T23:13:01.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:01 smithi040 ceph-mon[33128]: pgmap v3: 65 pgs: 65 active+clean; 120 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:13:01.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:01 smithi040 ceph-mon[33128]: [24/Apr/2024:23:13:00] ENGINE Bus STARTING 2024-04-24T23:13:01.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:01 smithi031 ceph-mon[26411]: Deploying cephadm binary to smithi040 2024-04-24T23:13:01.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:01 smithi031 ceph-mon[26411]: Deploying cephadm binary to smithi031 2024-04-24T23:13:01.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:01 smithi031 ceph-mon[26411]: mgrmap e22: smithi040.qtdwuq(active, since 1.01056s) 2024-04-24T23:13:01.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:01 smithi031 ceph-mon[26411]: pgmap v3: 65 pgs: 65 active+clean; 120 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:13:01.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:01 smithi031 ceph-mon[26411]: [24/Apr/2024:23:13:00] ENGINE Bus STARTING 2024-04-24T23:13:02.649 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:02 smithi031 ceph-mon[26411]: [24/Apr/2024:23:13:00] ENGINE Serving on https://172.21.15.40:7150 2024-04-24T23:13:02.650 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:02 smithi031 ceph-mon[26411]: [24/Apr/2024:23:13:00] ENGINE Serving on http://172.21.15.40:8765 2024-04-24T23:13:02.650 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:02 smithi031 ceph-mon[26411]: [24/Apr/2024:23:13:00] ENGINE Bus STARTED 2024-04-24T23:13:02.650 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:02 smithi031 ceph-mon[26411]: pgmap v4: 65 pgs: 65 active+clean; 120 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:13:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:02 smithi040 ceph-mon[33128]: [24/Apr/2024:23:13:00] ENGINE Serving on https://172.21.15.40:7150 2024-04-24T23:13:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:02 smithi040 ceph-mon[33128]: [24/Apr/2024:23:13:00] ENGINE Serving on http://172.21.15.40:8765 2024-04-24T23:13:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:02 smithi040 ceph-mon[33128]: [24/Apr/2024:23:13:00] ENGINE Bus STARTED 2024-04-24T23:13:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:02 smithi040 ceph-mon[33128]: pgmap v4: 65 pgs: 65 active+clean; 120 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:13:03.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:03 smithi040 ceph-mon[33128]: mgrmap e23: smithi040.qtdwuq(active, since 3s) 2024-04-24T23:13:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:03 smithi031 ceph-mon[26411]: mgrmap e23: smithi040.qtdwuq(active, since 3s) 2024-04-24T23:13:04.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:04 smithi040 ceph-mon[33128]: pgmap v5: 65 pgs: 65 active+clean; 120 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:13:04.709 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:04 smithi031 ceph-mon[26411]: pgmap v5: 65 pgs: 65 active+clean; 120 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:13:06.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:06 smithi031 ceph-mon[26411]: pgmap v6: 65 pgs: 65 active+clean; 121 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 289 KiB/s wr, 69 op/s 2024-04-24T23:13:06.772 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:06 smithi040 ceph-mon[33128]: pgmap v6: 65 pgs: 65 active+clean; 121 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 289 KiB/s wr, 69 op/s 2024-04-24T23:13:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.335+0000 7f9143213700 1 -- 172.21.15.31:0/1469945761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f913c100280 msgr2=0x7f913c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.335+0000 7f9143213700 1 --2- 172.21.15.31:0/1469945761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f913c100280 0x7f913c1006f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9128004880 tx=0x7f9128009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.335+0000 7f9143213700 1 -- 172.21.15.31:0/1469945761 shutdown_connections 2024-04-24T23:13:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.335+0000 7f9143213700 1 --2- 172.21.15.31:0/1469945761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f913c100280 0x7f913c1006f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.335+0000 7f9143213700 1 --2- 172.21.15.31:0/1469945761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f913c0fefd0 0x7f913c0ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.335+0000 7f9143213700 1 -- 172.21.15.31:0/1469945761 >> 172.21.15.31:0/1469945761 conn(0x7f913c0fa8f0 msgr2=0x7f913c0fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.335+0000 7f9143213700 1 -- 172.21.15.31:0/1469945761 shutdown_connections 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.335+0000 7f9143213700 1 -- 172.21.15.31:0/1469945761 wait complete. 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.336+0000 7f9143213700 1 Processor -- start 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.336+0000 7f9143213700 1 -- start start 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.336+0000 7f9143213700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f913c0fefd0 0x7f913c10b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.336+0000 7f9143213700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f913c100280 0x7f913c10bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.336+0000 7f9143213700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c10c390 con 0x7f913c100280 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.336+0000 7f9143213700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c10cbe0 con 0x7f913c0fefd0 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.337+0000 7f913bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f913c100280 0x7f913c10bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.337+0000 7f913bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f913c100280 0x7f913c10bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49826/0 (socket says 172.21.15.31:49826) 2024-04-24T23:13:07.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.337+0000 7f913bfff700 1 -- 172.21.15.31:0/2229023104 learned_addr learned my addr 172.21.15.31:0/2229023104 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.337+0000 7f9140faf700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f913c0fefd0 0x7f913c10b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.337+0000 7f913bfff700 1 -- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f913c0fefd0 msgr2=0x7f913c10b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.337+0000 7f913bfff700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f913c0fefd0 0x7f913c10b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.337+0000 7f913bfff700 1 -- 172.21.15.31:0/2229023104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128005040 con 0x7f913c100280 2024-04-24T23:13:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.338+0000 7f913bfff700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f913c100280 0x7f913c10bd40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9128008d00 tx=0x7f9128009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.339+0000 7f91397fa700 1 -- 172.21.15.31:0/2229023104 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912800e030 con 0x7f913c100280 2024-04-24T23:13:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.339+0000 7f91397fa700 1 -- 172.21.15.31:0/2229023104 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9128013e60 con 0x7f913c100280 2024-04-24T23:13:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.339+0000 7f91397fa700 1 -- 172.21.15.31:0/2229023104 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91280123c0 con 0x7f913c100280 2024-04-24T23:13:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.339+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c10ce60 con 0x7f913c100280 2024-04-24T23:13:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.339+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c10d420 con 0x7f913c100280 2024-04-24T23:13:07.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.340+0000 7f91397fa700 1 -- 172.21.15.31:0/2229023104 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f912801c400 con 0x7f913c100280 2024-04-24T23:13:07.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.340+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c05f5c0 con 0x7f913c100280 2024-04-24T23:13:07.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.341+0000 7f91397fa700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f912c038f60 0x7f912c03b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:07.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.341+0000 7f91397fa700 1 -- 172.21.15.31:0/2229023104 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f912804e5a0 con 0x7f913c100280 2024-04-24T23:13:07.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.341+0000 7f9140faf700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f912c038f60 0x7f912c03b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:07.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.342+0000 7f9140faf700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f912c038f60 0x7f912c03b410 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f91300046c0 tx=0x7f913000b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:07.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.348+0000 7f91397fa700 1 -- 172.21.15.31:0/2229023104 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f912801b750 con 0x7f913c100280 2024-04-24T23:13:07.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.702+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f913c10d750 con 0x7f912c038f60 2024-04-24T23:13:07.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.703+0000 7f91397fa700 1 -- 172.21.15.31:0/2229023104 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f913c10d750 con 0x7f912c038f60 2024-04-24T23:13:07.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.706+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f912c038f60 msgr2=0x7f912c03b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:07.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.706+0000 7f9143213700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f912c038f60 0x7f912c03b410 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f91300046c0 tx=0x7f913000b040 comp rx=0 tx=0).stop 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.706+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f913c100280 msgr2=0x7f913c10bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.706+0000 7f9143213700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f913c100280 0x7f913c10bd40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9128008d00 tx=0x7f9128009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.707+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 shutdown_connections 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.707+0000 7f9143213700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f912c038f60 0x7f912c03b410 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.707+0000 7f9143213700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f913c0fefd0 0x7f913c10b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.707+0000 7f9143213700 1 --2- 172.21.15.31:0/2229023104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f913c100280 0x7f913c10bd40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.707+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 >> 172.21.15.31:0/2229023104 conn(0x7f913c0fa8f0 msgr2=0x7f913c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.707+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 shutdown_connections 2024-04-24T23:13:07.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.707+0000 7f9143213700 1 -- 172.21.15.31:0/2229023104 wait complete. 2024-04-24T23:13:07.720 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:13:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.847+0000 7f3fdeffe700 1 -- 172.21.15.31:0/226817769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd806f8f0 msgr2=0x7f3fd806e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.847+0000 7f3fdeffe700 1 --2- 172.21.15.31:0/226817769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd806f8f0 0x7f3fd806e3d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc004880 tx=0x7f3fcc009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.847+0000 7f3fdeffe700 1 -- 172.21.15.31:0/226817769 shutdown_connections 2024-04-24T23:13:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.847+0000 7f3fdeffe700 1 --2- 172.21.15.31:0/226817769 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806e910 0x7f3fd806ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.847+0000 7f3fdeffe700 1 --2- 172.21.15.31:0/226817769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd806f8f0 0x7f3fd806e3d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.847+0000 7f3fdeffe700 1 -- 172.21.15.31:0/226817769 >> 172.21.15.31:0/226817769 conn(0x7f3fd8069d50 msgr2=0x7f3fd806c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.847+0000 7f3fdeffe700 1 -- 172.21.15.31:0/226817769 shutdown_connections 2024-04-24T23:13:07.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.847+0000 7f3fdeffe700 1 -- 172.21.15.31:0/226817769 wait complete. 2024-04-24T23:13:07.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.848+0000 7f3fdeffe700 1 Processor -- start 2024-04-24T23:13:07.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.848+0000 7f3fdeffe700 1 -- start start 2024-04-24T23:13:07.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.848+0000 7f3fdeffe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806e910 0x7f3fd811c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdeffe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd806f8f0 0x7f3fd811cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdeffe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd811d310 con 0x7f3fd806f8f0 2024-04-24T23:13:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdeffe700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd811db60 con 0x7f3fd806e910 2024-04-24T23:13:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdcd9a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806e910 0x7f3fd811c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdcd9a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806e910 0x7f3fd811c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33542/0 (socket says 172.21.15.31:33542) 2024-04-24T23:13:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1817477935 learned_addr learned my addr 172.21.15.31:0/1817477935 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:07.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fd7fff700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd806f8f0 0x7f3fd811cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:07.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1817477935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd806f8f0 msgr2=0x7f3fd811cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd806f8f0 0x7f3fd811cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.849+0000 7f3fdcd9a700 1 -- 172.21.15.31:0/1817477935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc005040 con 0x7f3fd806e910 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.850+0000 7f3fdcd9a700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806e910 0x7f3fd811c7d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc000c00 tx=0x7f3fcc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.850+0000 7f3fd57fa700 1 -- 172.21.15.31:0/1817477935 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fcc00e030 con 0x7f3fd806e910 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.850+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd811dde0 con 0x7f3fd806e910 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.850+0000 7f3fd57fa700 1 -- 172.21.15.31:0/1817477935 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fcc013b70 con 0x7f3fd806e910 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.850+0000 7f3fd57fa700 1 -- 172.21.15.31:0/1817477935 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fcc0123c0 con 0x7f3fd806e910 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.850+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd811e320 con 0x7f3fd806e910 2024-04-24T23:13:07.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.851+0000 7f3fd57fa700 1 -- 172.21.15.31:0/1817477935 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 23) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f3fcc012520 con 0x7f3fd806e910 2024-04-24T23:13:07.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.851+0000 7f3fd57fa700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fc0038f60 0x7f3fc003b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:07.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.852+0000 7f3fd57fa700 1 -- 172.21.15.31:0/1817477935 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f3fcc04e680 con 0x7f3fd806e910 2024-04-24T23:13:07.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.852+0000 7f3fd7fff700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fc0038f60 0x7f3fc003b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:07.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.852+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd806d950 con 0x7f3fd806e910 2024-04-24T23:13:07.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.852+0000 7f3fd7fff700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fc0038f60 0x7f3fc003b410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009b80 tx=0x7f3fc800b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:07.860 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:07.859+0000 7f3fd57fa700 1 -- 172.21.15.31:0/1817477935 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3fcc011960 con 0x7f3fd806e910 2024-04-24T23:13:08.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.281+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fd811e5a0 con 0x7f3fc0038f60 2024-04-24T23:13:08.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.283+0000 7f3fd57fa700 1 -- 172.21.15.31:0/1817477935 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f3fd811e5a0 con 0x7f3fc0038f60 2024-04-24T23:13:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fc0038f60 msgr2=0x7f3fc003b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fc0038f60 0x7f3fc003b410 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009b80 tx=0x7f3fc800b040 comp rx=0 tx=0).stop 2024-04-24T23:13:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806e910 msgr2=0x7f3fd811c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806e910 0x7f3fd811c7d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc000c00 tx=0x7f3fcc009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 shutdown_connections 2024-04-24T23:13:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fc0038f60 0x7f3fc003b410 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fd806e910 0x7f3fd811c7d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 --2- 172.21.15.31:0/1817477935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fd806f8f0 0x7f3fd811cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.286+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 >> 172.21.15.31:0/1817477935 conn(0x7f3fd8069d50 msgr2=0x7f3fd806b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:08.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.287+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 shutdown_connections 2024-04-24T23:13:08.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.287+0000 7f3fdeffe700 1 -- 172.21.15.31:0/1817477935 wait complete. 2024-04-24T23:13:08.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.464+0000 7fdbba08c700 1 -- 172.21.15.31:0/290389534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fbcb0 msgr2=0x7fdbb40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.464+0000 7fdbba08c700 1 --2- 172.21.15.31:0/290389534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fbcb0 0x7fdbb40fc0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdba4004880 tx=0x7fdba4009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.464+0000 7fdbba08c700 1 -- 172.21.15.31:0/290389534 shutdown_connections 2024-04-24T23:13:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.464+0000 7fdbba08c700 1 --2- 172.21.15.31:0/290389534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbb40fcf60 0x7fdbb40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.464+0000 7fdbba08c700 1 --2- 172.21.15.31:0/290389534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fbcb0 0x7fdbb40fc0c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.464+0000 7fdbba08c700 1 -- 172.21.15.31:0/290389534 >> 172.21.15.31:0/290389534 conn(0x7fdbb40f7430 msgr2=0x7fdbb40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.464+0000 7fdbba08c700 1 -- 172.21.15.31:0/290389534 shutdown_connections 2024-04-24T23:13:08.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.465+0000 7fdbba08c700 1 -- 172.21.15.31:0/290389534 wait complete. 2024-04-24T23:13:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.466+0000 7fdbba08c700 1 Processor -- start 2024-04-24T23:13:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.466+0000 7fdbba08c700 1 -- start start 2024-04-24T23:13:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.466+0000 7fdbba08c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbb40fbcb0 0x7fdbb410c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.466+0000 7fdbba08c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fcf60 0x7fdbb410cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.466+0000 7fdbba08c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb410d3b0 con 0x7fdbb40fbcb0 2024-04-24T23:13:08.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.466+0000 7fdbba08c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb410dc00 con 0x7fdbb40fcf60 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbaaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fcf60 0x7fdbb410cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbb37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbb40fbcb0 0x7fdbb410c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbaaffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fcf60 0x7fdbb410cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:33564/0 (socket says 172.21.15.31:33564) 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbaaffd700 1 -- 172.21.15.31:0/1812653015 learned_addr learned my addr 172.21.15.31:0/1812653015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbaaffd700 1 -- 172.21.15.31:0/1812653015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbb40fbcb0 msgr2=0x7fdbb410c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbaaffd700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbb40fbcb0 0x7fdbb410c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbaaffd700 1 -- 172.21.15.31:0/1812653015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba4005040 con 0x7fdbb40fcf60 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbb37fe700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbb40fbcb0 0x7fdbb410c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbaaffd700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fcf60 0x7fdbb410cdb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0041d0 tx=0x7fdb9c00c060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.467+0000 7fdbb0ff9700 1 -- 172.21.15.31:0/1812653015 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb9c007ea0 con 0x7fdbb40fcf60 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.468+0000 7fdbb0ff9700 1 -- 172.21.15.31:0/1812653015 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb9c00dc20 con 0x7fdbb40fcf60 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.468+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb410de80 con 0x7fdbb40fcf60 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.468+0000 7fdbb0ff9700 1 -- 172.21.15.31:0/1812653015 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb9c011450 con 0x7fdbb40fcf60 2024-04-24T23:13:08.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.468+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb410e440 con 0x7fdbb40fcf60 2024-04-24T23:13:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.469+0000 7fdbb0ff9700 1 -- 172.21.15.31:0/1812653015 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 23) v1 ==== 46295+0+0 (secure 0 0 0) 0x7fdb9c0116c0 con 0x7fdbb40fcf60 2024-04-24T23:13:08.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.469+0000 7fdbb0ff9700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdba0038f10 0x7fdba003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.469+0000 7fdbb0ff9700 1 -- 172.21.15.31:0/1812653015 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fdb9c00d750 con 0x7fdbb40fcf60 2024-04-24T23:13:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.469+0000 7fdbb37fe700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdba0038f10 0x7fdba003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.469+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb405f5c0 con 0x7fdbb40fcf60 2024-04-24T23:13:08.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.470+0000 7fdbb37fe700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdba0038f10 0x7fdba003b3c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b040 tx=0x7fdba401d040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:08.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.475+0000 7fdbb0ff9700 1 -- 172.21.15.31:0/1812653015 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fdb9c01a400 con 0x7fdbb40fcf60 2024-04-24T23:13:08.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:08 smithi040 ceph-mon[33128]: pgmap v7: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.0 MiB/s wr, 267 op/s 2024-04-24T23:13:08.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:08 smithi031 ceph-mon[26411]: pgmap v7: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 2.0 MiB/s wr, 267 op/s 2024-04-24T23:13:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.866+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbb410e7d0 con 0x7fdba0038f10 2024-04-24T23:13:08.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.882+0000 7fdbb0ff9700 1 -- 172.21.15.31:0/1812653015 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fdbb410e7d0 con 0x7fdba0038f10 2024-04-24T23:13:08.883 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 119s ago 7m 24.4M - 0.25.0 c8568f914cd2 87840bb14366 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (8m) 119s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (7m) 20s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (6m) 119s ago 7m 84.3M - 9.4.7 954c08fa6188 44ecf082adf8 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (2m) 119s ago 2m 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (2m) 119s ago 2m 10.4M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (2m) 20s ago 2m 11.6M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (2m) 20s ago 2m 12.5M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:13:08.884 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:9283 running (9m) 119s ago 9m 480M - 17.2.7-902-gc61472c8 0de407c1b3af 7b9a4ca2c955 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (29s) 20s ago 7m 56.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8b888d3a97fe 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (9m) 119s ago 9m 42.6M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (6m) 20s ago 6m 34.8M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (8m) 119s ago 8m 15.7M - 1.5.0 0da6a335fe13 0541b481dfad 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (7m) 20s ago 7m 16.1M - 1.5.0 0da6a335fe13 7bd2678617bc 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 119s ago 6m 45.4M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 119s ago 5m 44.4M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 119s ago 5m 43.9M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (4m) 20s ago 4m 67.9M 5313M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (4m) 20s ago 4m 61.9M 5313M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (3m) 20s ago 3m 59.4M 5313M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:13:08.885 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 119s ago 7m 49.6M - 2.43.0 a07b618ecd1d aac36626882b 2024-04-24T23:13:08.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdba0038f10 msgr2=0x7fdba003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdba0038f10 0x7fdba003b3c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b040 tx=0x7fdba401d040 comp rx=0 tx=0).stop 2024-04-24T23:13:08.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fcf60 msgr2=0x7fdbb410cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fcf60 0x7fdbb410cdb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0041d0 tx=0x7fdb9c00c060 comp rx=0 tx=0).stop 2024-04-24T23:13:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 shutdown_connections 2024-04-24T23:13:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdba0038f10 0x7fdba003b3c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdbb40fbcb0 0x7fdbb410c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 --2- 172.21.15.31:0/1812653015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdbb40fcf60 0x7fdbb410cdb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.885+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 >> 172.21.15.31:0/1812653015 conn(0x7fdbb40f7430 msgr2=0x7fdbb40f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.886+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 shutdown_connections 2024-04-24T23:13:08.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:08.886+0000 7fdbba08c700 1 -- 172.21.15.31:0/1812653015 wait complete. 2024-04-24T23:13:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.038+0000 7f7c54b53700 1 -- 172.21.15.31:0/1886605283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fcf10 msgr2=0x7f7c500fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.038+0000 7f7c54b53700 1 --2- 172.21.15.31:0/1886605283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fcf10 0x7f7c500fd320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7c40004940 tx=0x7f7c40009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.038+0000 7f7c54b53700 1 -- 172.21.15.31:0/1886605283 shutdown_connections 2024-04-24T23:13:09.041 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.038+0000 7f7c54b53700 1 --2- 172.21.15.31:0/1886605283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c500fe080 0x7f7c500fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.038+0000 7f7c54b53700 1 --2- 172.21.15.31:0/1886605283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fcf10 0x7f7c500fd320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.038+0000 7f7c54b53700 1 -- 172.21.15.31:0/1886605283 >> 172.21.15.31:0/1886605283 conn(0x7f7c500f8560 msgr2=0x7f7c500fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.038+0000 7f7c54b53700 1 -- 172.21.15.31:0/1886605283 shutdown_connections 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.038+0000 7f7c54b53700 1 -- 172.21.15.31:0/1886605283 wait complete. 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c54b53700 1 Processor -- start 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c54b53700 1 -- start start 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c54b53700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c500fcf10 0x7f7c5010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c54b53700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fe080 0x7f7c5010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c54b53700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5010e4b0 con 0x7f7c500fe080 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c54b53700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5010ed00 con 0x7f7c500fcf10 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c4dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fe080 0x7f7c5010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c4dd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fe080 0x7f7c5010deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53402/0 (socket says 172.21.15.31:53402) 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c4dd9b700 1 -- 172.21.15.31:0/3064093213 learned_addr learned my addr 172.21.15.31:0/3064093213 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:09.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c4e59c700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c500fcf10 0x7f7c5010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c4dd9b700 1 -- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c500fcf10 msgr2=0x7f7c5010d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c4dd9b700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c500fcf10 0x7f7c5010d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c4dd9b700 1 -- 172.21.15.31:0/3064093213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c40005040 con 0x7f7c500fe080 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.039+0000 7f7c4dd9b700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fe080 0x7f7c5010deb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400eb70 tx=0x7f7c4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.040+0000 7f7c3effd700 1 -- 172.21.15.31:0/3064093213 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c44007cc0 con 0x7f7c500fe080 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.040+0000 7f7c3effd700 1 -- 172.21.15.31:0/3064093213 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c4400f040 con 0x7f7c500fe080 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.040+0000 7f7c3effd700 1 -- 172.21.15.31:0/3064093213 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c4400ca90 con 0x7f7c500fe080 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.040+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5010ef80 con 0x7f7c500fe080 2024-04-24T23:13:09.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.040+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5010f440 con 0x7f7c500fe080 2024-04-24T23:13:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.041+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5005f5c0 con 0x7f7c500fe080 2024-04-24T23:13:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.041+0000 7f7c3effd700 1 -- 172.21.15.31:0/3064093213 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f7c4400cbf0 con 0x7f7c500fe080 2024-04-24T23:13:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.041+0000 7f7c3effd700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f7c38038f10 0x7f7c3803b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.041+0000 7f7c3effd700 1 -- 172.21.15.31:0/3064093213 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f7c4404cb60 con 0x7f7c500fe080 2024-04-24T23:13:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.042+0000 7f7c4e59c700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f7c38038f10 0x7f7c3803b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:09.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.042+0000 7f7c4e59c700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f7c38038f10 0x7f7c3803b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009b30 tx=0x7f7c40009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:09.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.046+0000 7f7c3effd700 1 -- 172.21.15.31:0/3064093213 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f7c4401c540 con 0x7f7c500fe080 2024-04-24T23:13:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.492+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7c50105990 con 0x7f7c500fe080 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.493+0000 7f7c3effd700 1 -- 172.21.15.31:0/3064093213 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f7c44019020 con 0x7f7c500fe080 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:13:09.495 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:13:09.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:13:09.496 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:13:09.496 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T23:13:09.496 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:13:09.496 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:13:09.496 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f7c38038f10 msgr2=0x7f7c3803b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f7c38038f10 0x7f7c3803b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009b30 tx=0x7f7c40009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fe080 msgr2=0x7f7c5010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fe080 0x7f7c5010deb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400eb70 tx=0x7f7c4400b040 comp rx=0 tx=0).stop 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 shutdown_connections 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f7c38038f10 0x7f7c3803b3c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c500fcf10 0x7f7c5010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 --2- 172.21.15.31:0/3064093213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c500fe080 0x7f7c5010deb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 >> 172.21.15.31:0/3064093213 conn(0x7f7c500f8560 msgr2=0x7f7c500f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 shutdown_connections 2024-04-24T23:13:09.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.495+0000 7f7c54b53700 1 -- 172.21.15.31:0/3064093213 wait complete. 2024-04-24T23:13:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.615+0000 7f3bae72f700 1 -- 172.21.15.31:0/3867052609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fcf60 msgr2=0x7f3ba80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.615+0000 7f3bae72f700 1 --2- 172.21.15.31:0/3867052609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fcf60 0x7f3ba80fd3d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c004940 tx=0x7f3b9c009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.615+0000 7f3bae72f700 1 -- 172.21.15.31:0/3867052609 shutdown_connections 2024-04-24T23:13:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.615+0000 7f3bae72f700 1 --2- 172.21.15.31:0/3867052609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fcf60 0x7f3ba80fd3d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.615+0000 7f3bae72f700 1 --2- 172.21.15.31:0/3867052609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba80fbcb0 0x7f3ba80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.615+0000 7f3bae72f700 1 -- 172.21.15.31:0/3867052609 >> 172.21.15.31:0/3867052609 conn(0x7f3ba80f7430 msgr2=0x7f3ba80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.615+0000 7f3bae72f700 1 -- 172.21.15.31:0/3867052609 shutdown_connections 2024-04-24T23:13:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.615+0000 7f3bae72f700 1 -- 172.21.15.31:0/3867052609 wait complete. 2024-04-24T23:13:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.616+0000 7f3bae72f700 1 Processor -- start 2024-04-24T23:13:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.616+0000 7f3bae72f700 1 -- start start 2024-04-24T23:13:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.616+0000 7f3bae72f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fbcb0 0x7f3ba80ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.616+0000 7f3bae72f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba80fcf60 0x7f3ba80fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.616+0000 7f3bae72f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80fe830 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.616+0000 7f3bae72f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80fe970 con 0x7f3ba80fcf60 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fbcb0 0x7f3ba80ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba77fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba80fcf60 0x7f3ba80fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fbcb0 0x7f3ba80ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53428/0 (socket says 172.21.15.31:53428) 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba7fff700 1 -- 172.21.15.31:0/1172885909 learned_addr learned my addr 172.21.15.31:0/1172885909 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba7fff700 1 -- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba80fcf60 msgr2=0x7f3ba80fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba7fff700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba80fcf60 0x7f3ba80fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba7fff700 1 -- 172.21.15.31:0/1172885909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c005040 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba7fff700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fbcb0 0x7f3ba80ffd80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b98002a80 tx=0x7f3b980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.617+0000 7f3ba4ff9700 1 -- 172.21.15.31:0/1172885909 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b98007cc0 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.618+0000 7f3ba4ff9700 1 -- 172.21.15.31:0/1172885909 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b9800f040 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.618+0000 7f3ba4ff9700 1 -- 172.21.15.31:0/1172885909 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b9800dac0 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.618+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba80fec50 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.618+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba81159f0 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.619+0000 7f3ba4ff9700 1 -- 172.21.15.31:0/1172885909 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 23) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f3b9800dc20 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.619+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba8101af0 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.620+0000 7f3ba4ff9700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3b90038f60 0x7f3b9003b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:09.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.620+0000 7f3ba4ff9700 1 -- 172.21.15.31:0/1172885909 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f3b9804c7f0 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.620+0000 7f3ba77fe700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3b90038f60 0x7f3b9003b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:09.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.620+0000 7f3ba77fe700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3b90038f60 0x7f3b9003b410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8100360 tx=0x7f3b9c009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:09.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.627+0000 7f3ba4ff9700 1 -- 172.21.15.31:0/1172885909 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3b98019b40 con 0x7f3ba80fbcb0 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: Standby manager daemon smithi031.rrnmwr started 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.31:0/18616835' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.rrnmwr/crt"}]: dispatch 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.31:0/18616835' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.31:0/18616835' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.rrnmwr/key"}]: dispatch 2024-04-24T23:13:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:09 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.31:0/18616835' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:13:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: Standby manager daemon smithi031.rrnmwr started 2024-04-24T23:13:09.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.31:0/18616835' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.rrnmwr/crt"}]: dispatch 2024-04-24T23:13:09.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.31:0/18616835' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:13:09.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.31:0/18616835' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.rrnmwr/key"}]: dispatch 2024-04-24T23:13:09.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:09 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.31:0/18616835' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:13:09.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.949+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ba805e3a0 con 0x7f3b90038f60 2024-04-24T23:13:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.951+0000 7f3ba4ff9700 1 -- 172.21.15.31:0/1172885909 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f3ba805e3a0 con 0x7f3b90038f60 2024-04-24T23:13:09.952 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:13:09.952 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:13:09.952 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:13:09.952 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:13:09.952 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-24T23:13:09.952 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T23:13:09.953 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-04-24T23:13:09.953 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:13:09.953 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:13:09.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.952+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3b90038f60 msgr2=0x7f3b9003b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:09.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.952+0000 7f3bae72f700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3b90038f60 0x7f3b9003b410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8100360 tx=0x7f3b9c009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:09.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.952+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fbcb0 msgr2=0x7f3ba80ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:09.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.952+0000 7f3bae72f700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fbcb0 0x7f3ba80ffd80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b98002a80 tx=0x7f3b980046b0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.953+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 shutdown_connections 2024-04-24T23:13:09.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.953+0000 7f3bae72f700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3b90038f60 0x7f3b9003b410 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.953+0000 7f3bae72f700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80fbcb0 0x7f3ba80ffd80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.953+0000 7f3bae72f700 1 --2- 172.21.15.31:0/1172885909 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ba80fcf60 0x7f3ba80fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.953+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 >> 172.21.15.31:0/1172885909 conn(0x7f3ba80f7430 msgr2=0x7f3ba8065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.953+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 shutdown_connections 2024-04-24T23:13:09.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:09.953+0000 7f3bae72f700 1 -- 172.21.15.31:0/1172885909 wait complete. 2024-04-24T23:13:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:10 smithi031 ceph-mon[26411]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:10 smithi031 ceph-mon[26411]: pgmap v8: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.5 MiB/s wr, 207 op/s 2024-04-24T23:13:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:10 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3064093213' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:13:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:10 smithi031 ceph-mon[26411]: mgrmap e24: smithi040.qtdwuq(active, since 10s), standbys: smithi031.rrnmwr 2024-04-24T23:13:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:10 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:13:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:10 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:10.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:10 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:10.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:10 smithi040 ceph-mon[33128]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:10.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:10 smithi040 ceph-mon[33128]: pgmap v8: 65 pgs: 65 active+clean; 133 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.5 MiB/s wr, 207 op/s 2024-04-24T23:13:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:10 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3064093213' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:13:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:10 smithi040 ceph-mon[33128]: mgrmap e24: smithi040.qtdwuq(active, since 10s), standbys: smithi031.rrnmwr 2024-04-24T23:13:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:10 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:13:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:10 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:10 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:11 smithi040 ceph-mon[33128]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:11.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:11 smithi031 ceph-mon[26411]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:12.724 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:12 smithi040 ceph-mon[33128]: pgmap v9: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-04-24T23:13:12.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:12 smithi031 ceph-mon[26411]: pgmap v9: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 2.4 MiB/s wr, 314 op/s 2024-04-24T23:13:14.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:14 smithi040 ceph-mon[33128]: pgmap v10: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-04-24T23:13:14.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:14 smithi031 ceph-mon[26411]: pgmap v10: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.2 MiB/s wr, 288 op/s 2024-04-24T23:13:15.483 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: Adjusting osd_memory_target on smithi040 to 2583M 2024-04-24T23:13:15.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:15 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:13:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:15.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: Adjusting osd_memory_target on smithi040 to 2583M 2024-04-24T23:13:15.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:15 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:16.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:16 smithi040 ceph-mon[33128]: pgmap v11: 65 pgs: 65 active+clean; 146 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.3 MiB/s wr, 321 op/s 2024-04-24T23:13:16.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:16 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:16.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:16 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:16.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:16 smithi031 ceph-mon[26411]: pgmap v11: 65 pgs: 65 active+clean; 146 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 2.3 MiB/s wr, 321 op/s 2024-04-24T23:13:16.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:16 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:16.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:16 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:18.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:18.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:18.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: Adjusting osd_memory_target on smithi031 to 1559M 2024-04-24T23:13:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:13:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:13:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: pgmap v12: 65 pgs: 65 active+clean; 162 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-04-24T23:13:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:18 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:18.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:18.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:18.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: Adjusting osd_memory_target on smithi031 to 1559M 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: pgmap v12: 65 pgs: 65 active+clean; 162 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:18 smithi031 ceph-mon[26411]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:13:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: Reconfiguring node-exporter.smithi031 (dependencies changed)... 2024-04-24T23:13:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:19 smithi031 ceph-mon[26411]: Reconfiguring daemon node-exporter.smithi031 on smithi031 2024-04-24T23:13:19.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:19.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:13:19.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: Reconfiguring node-exporter.smithi031 (dependencies changed)... 2024-04-24T23:13:19.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:19 smithi040 ceph-mon[33128]: Reconfiguring daemon node-exporter.smithi031 on smithi031 2024-04-24T23:13:19.998 INFO:tasks.workunit.client.0.smithi031.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-24T23:13:20.005 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T23:13:20.005 INFO:tasks.workunit.client.0.smithi031.stderr:+ make 2024-04-24T23:13:20.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:20 smithi040 ceph-mon[33128]: pgmap v13: 65 pgs: 65 active+clean; 162 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s wr, 285 op/s 2024-04-24T23:13:20.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:20 smithi031 ceph-mon[26411]: pgmap v13: 65 pgs: 65 active+clean; 162 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.6 MiB/s wr, 285 op/s 2024-04-24T23:13:21.071 INFO:tasks.workunit.client.0.smithi031.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-24T23:13:21.548 INFO:tasks.workunit.client.0.smithi031.stderr:++ readlink -f fsstress 2024-04-24T23:13:21.550 INFO:tasks.workunit.client.0.smithi031.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-24T23:13:21.550 INFO:tasks.workunit.client.0.smithi031.stderr:+ popd 2024-04-24T23:13:21.552 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-24T23:13:21.552 INFO:tasks.workunit.client.0.smithi031.stderr:+ popd 2024-04-24T23:13:21.553 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-24T23:13:21.553 INFO:tasks.workunit.client.0.smithi031.stderr:++ mktemp -d -p . 2024-04-24T23:13:21.672 INFO:tasks.workunit.client.0.smithi031.stderr:+ T=./tmp.77cF6t9Mox 2024-04-24T23:13:21.672 INFO:tasks.workunit.client.0.smithi031.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.77cF6t9Mox -l 1 -n 1000 -p 10 -v 2024-04-24T23:13:21.679 INFO:tasks.workunit.client.0.smithi031.stdout:seed = 1713622611 2024-04-24T23:13:21.686 INFO:tasks.workunit.client.0.smithi031.stdout:0/0: truncate - no filename 2024-04-24T23:13:21.687 INFO:tasks.workunit.client.0.smithi031.stdout:0/1: chown . 13773 1 2024-04-24T23:13:21.687 INFO:tasks.workunit.client.0.smithi031.stdout:0/2: chown . 696497 1 2024-04-24T23:13:21.687 INFO:tasks.workunit.client.0.smithi031.stdout:0/3: rename - no filename 2024-04-24T23:13:21.687 INFO:tasks.workunit.client.0.smithi031.stdout:0/4: rename - no filename 2024-04-24T23:13:21.687 INFO:tasks.workunit.client.0.smithi031.stdout:0/5: rmdir - no directory 2024-04-24T23:13:21.687 INFO:tasks.workunit.client.0.smithi031.stdout:0/6: dread - no filename 2024-04-24T23:13:21.688 INFO:tasks.workunit.client.0.smithi031.stdout:1/0: dwrite - no filename 2024-04-24T23:13:21.688 INFO:tasks.workunit.client.0.smithi031.stdout:1/1: stat - no entries 2024-04-24T23:13:21.692 INFO:tasks.workunit.client.0.smithi031.stdout:0/7: mknod c0 0 2024-04-24T23:13:21.696 INFO:tasks.workunit.client.0.smithi031.stdout:3/0: write - no filename 2024-04-24T23:13:21.696 INFO:tasks.workunit.client.0.smithi031.stdout:3/1: truncate - no filename 2024-04-24T23:13:21.697 INFO:tasks.workunit.client.0.smithi031.stdout:1/2: creat f0 x:0 0 0 2024-04-24T23:13:21.697 INFO:tasks.workunit.client.0.smithi031.stdout:1/3: dread - f0 zero size 2024-04-24T23:13:21.698 INFO:tasks.workunit.client.0.smithi031.stdout:1/4: dread - f0 zero size 2024-04-24T23:13:21.698 INFO:tasks.workunit.client.0.smithi031.stdout:2/0: mknod c0 0 2024-04-24T23:13:21.698 INFO:tasks.workunit.client.0.smithi031.stdout:2/1: fdatasync - no filename 2024-04-24T23:13:21.702 INFO:tasks.workunit.client.0.smithi031.stdout:4/0: chown . 11640 1 2024-04-24T23:13:21.702 INFO:tasks.workunit.client.0.smithi031.stdout:4/1: unlink - no file 2024-04-24T23:13:21.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/5: mkdir d1 0 2024-04-24T23:13:21.706 INFO:tasks.workunit.client.0.smithi031.stdout:2/2: rename c0 to c1 0 2024-04-24T23:13:21.708 INFO:tasks.workunit.client.0.smithi031.stdout:3/2: creat f0 x:0 0 0 2024-04-24T23:13:21.724 INFO:tasks.workunit.client.0.smithi031.stdout:3/3: creat f1 x:0 0 0 2024-04-24T23:13:21.724 INFO:tasks.workunit.client.0.smithi031.stdout:4/2: symlink l0 0 2024-04-24T23:13:21.724 INFO:tasks.workunit.client.0.smithi031.stdout:6/0: link - no file 2024-04-24T23:13:21.725 INFO:tasks.workunit.client.0.smithi031.stdout:1/6: creat d1/f2 x:0 0 0 2024-04-24T23:13:21.725 INFO:tasks.workunit.client.0.smithi031.stdout:1/7: read - d1/f2 zero size 2024-04-24T23:13:21.839 INFO:tasks.workunit.client.0.smithi031.stdout:5/0: getdents . 0 2024-04-24T23:13:21.839 INFO:tasks.workunit.client.0.smithi031.stdout:5/1: readlink - no filename 2024-04-24T23:13:21.841 INFO:tasks.workunit.client.0.smithi031.stdout:3/4: creat f2 x:0 0 0 2024-04-24T23:13:21.842 INFO:tasks.workunit.client.0.smithi031.stdout:3/5: write f1 [412996,40738] 0 2024-04-24T23:13:21.844 INFO:tasks.workunit.client.0.smithi031.stdout:2/3: rename c1 to c2 0 2024-04-24T23:13:21.844 INFO:tasks.workunit.client.0.smithi031.stdout:2/4: write - no filename 2024-04-24T23:13:21.847 INFO:tasks.workunit.client.0.smithi031.stdout:8/0: write - no filename 2024-04-24T23:13:21.847 INFO:tasks.workunit.client.0.smithi031.stdout:8/1: rename - no filename 2024-04-24T23:13:21.847 INFO:tasks.workunit.client.0.smithi031.stdout:6/1: symlink l0 0 2024-04-24T23:13:21.847 INFO:tasks.workunit.client.0.smithi031.stdout:6/2: dwrite - no filename 2024-04-24T23:13:21.847 INFO:tasks.workunit.client.0.smithi031.stdout:6/3: write - no filename 2024-04-24T23:13:21.851 INFO:tasks.workunit.client.0.smithi031.stdout:1/8: rename f0 to d1/f3 0 2024-04-24T23:13:21.851 INFO:tasks.workunit.client.0.smithi031.stdout:5/2: mkdir d0 0 2024-04-24T23:13:21.854 INFO:tasks.workunit.client.0.smithi031.stdout:7/0: creat f0 x:0 0 0 2024-04-24T23:13:21.855 INFO:tasks.workunit.client.0.smithi031.stdout:7/1: read - f0 zero size 2024-04-24T23:13:21.858 INFO:tasks.workunit.client.0.smithi031.stdout:6/4: creat f1 x:0 0 0 2024-04-24T23:13:21.859 INFO:tasks.workunit.client.0.smithi031.stdout:4/3: link l0 l1 0 2024-04-24T23:13:21.859 INFO:tasks.workunit.client.0.smithi031.stdout:4/4: fsync - no filename 2024-04-24T23:13:21.859 INFO:tasks.workunit.client.0.smithi031.stdout:4/5: dwrite - no filename 2024-04-24T23:13:21.859 INFO:tasks.workunit.client.0.smithi031.stdout:4/6: fdatasync - no filename 2024-04-24T23:13:21.862 INFO:tasks.workunit.client.0.smithi031.stdout:3/6: link f2 f3 0 2024-04-24T23:13:21.863 INFO:tasks.workunit.client.0.smithi031.stdout:1/9: mknod d1/c4 0 2024-04-24T23:13:21.865 INFO:tasks.workunit.client.0.smithi031.stdout:8/2: creat f0 x:0 0 0 2024-04-24T23:13:21.867 INFO:tasks.workunit.client.0.smithi031.stdout:7/2: dwrite f0 [0,4194304] 0 2024-04-24T23:13:21.867 INFO:tasks.workunit.client.0.smithi031.stdout:7/3: readlink - no filename 2024-04-24T23:13:21.867 INFO:tasks.workunit.client.0.smithi031.stdout:1/10: truncate d1/f3 33260 0 2024-04-24T23:13:21.867 INFO:tasks.workunit.client.0.smithi031.stdout:1/11: rename d1 to d1/d5 22 2024-04-24T23:13:21.868 INFO:tasks.workunit.client.0.smithi031.stdout:3/7: dread f1 [0,4194304] 0 2024-04-24T23:13:21.868 INFO:tasks.workunit.client.0.smithi031.stdout:1/12: fsync d1/f2 0 2024-04-24T23:13:21.868 INFO:tasks.workunit.client.0.smithi031.stdout:7/4: write f0 [1197924,64457] 0 2024-04-24T23:13:21.870 INFO:tasks.workunit.client.0.smithi031.stdout:6/5: symlink l2 0 2024-04-24T23:13:21.871 INFO:tasks.workunit.client.0.smithi031.stdout:6/6: write f1 [493990,4050] 0 2024-04-24T23:13:21.872 INFO:tasks.workunit.client.0.smithi031.stdout:6/7: chown l2 6 1 2024-04-24T23:13:21.877 INFO:tasks.workunit.client.0.smithi031.stdout:8/3: mknod c1 0 2024-04-24T23:13:21.878 INFO:tasks.workunit.client.0.smithi031.stdout:8/4: truncate f0 155001 0 2024-04-24T23:13:21.881 INFO:tasks.workunit.client.0.smithi031.stdout:9/0: mknod c0 0 2024-04-24T23:13:21.881 INFO:tasks.workunit.client.0.smithi031.stdout:7/5: mkdir d1 0 2024-04-24T23:13:21.882 INFO:tasks.workunit.client.0.smithi031.stdout:8/5: dread f0 [0,4194304] 0 2024-04-24T23:13:21.883 INFO:tasks.workunit.client.0.smithi031.stdout:1/13: mknod d1/c6 0 2024-04-24T23:13:21.884 INFO:tasks.workunit.client.0.smithi031.stdout:1/14: chown d1 4930 1 2024-04-24T23:13:21.884 INFO:tasks.workunit.client.0.smithi031.stdout:1/15: readlink - no filename 2024-04-24T23:13:21.889 INFO:tasks.workunit.client.0.smithi031.stdout:6/8: symlink l3 0 2024-04-24T23:13:21.893 INFO:tasks.workunit.client.0.smithi031.stdout:1/16: dwrite d1/f2 [0,4194304] 0 2024-04-24T23:13:21.895 INFO:tasks.workunit.client.0.smithi031.stdout:1/17: write d1/f2 [3675408,125250] 0 2024-04-24T23:13:21.895 INFO:tasks.workunit.client.0.smithi031.stdout:9/1: rename c0 to c1 0 2024-04-24T23:13:21.902 INFO:tasks.workunit.client.0.smithi031.stdout:8/6: unlink c1 0 2024-04-24T23:13:21.907 INFO:tasks.workunit.client.0.smithi031.stdout:6/9: symlink l4 0 2024-04-24T23:13:21.913 INFO:tasks.workunit.client.0.smithi031.stdout:1/18: rename d1/c4 to d1/c7 0 2024-04-24T23:13:21.915 INFO:tasks.workunit.client.0.smithi031.stdout:7/6: mkdir d1/d2 0 2024-04-24T23:13:21.917 INFO:tasks.workunit.client.0.smithi031.stdout:8/7: dread f0 [0,4194304] 0 2024-04-24T23:13:21.919 INFO:tasks.workunit.client.0.smithi031.stdout:8/8: dread f0 [0,4194304] 0 2024-04-24T23:13:21.920 INFO:tasks.workunit.client.0.smithi031.stdout:8/9: dread f0 [0,4194304] 0 2024-04-24T23:13:21.920 INFO:tasks.workunit.client.0.smithi031.stdout:8/10: stat f0 0 2024-04-24T23:13:21.920 INFO:tasks.workunit.client.0.smithi031.stdout:6/10: mknod c5 0 2024-04-24T23:13:21.921 INFO:tasks.workunit.client.0.smithi031.stdout:6/11: write f1 [1034520,116731] 0 2024-04-24T23:13:21.922 INFO:tasks.workunit.client.0.smithi031.stdout:6/12: chown f1 24959 1 2024-04-24T23:13:21.924 INFO:tasks.workunit.client.0.smithi031.stdout:6/13: write f1 [1218056,1102] 0 2024-04-24T23:13:21.926 INFO:tasks.workunit.client.0.smithi031.stdout:6/14: write f1 [126308,48074] 0 2024-04-24T23:13:21.929 INFO:tasks.workunit.client.0.smithi031.stdout:8/11: creat f2 x:0 0 0 2024-04-24T23:13:21.930 INFO:tasks.workunit.client.0.smithi031.stdout:8/12: dread - f2 zero size 2024-04-24T23:13:21.931 INFO:tasks.workunit.client.0.smithi031.stdout:8/13: chown f2 24198074 1 2024-04-24T23:13:21.932 INFO:tasks.workunit.client.0.smithi031.stdout:1/19: dwrite d1/f3 [0,4194304] 0 2024-04-24T23:13:21.933 INFO:tasks.workunit.client.0.smithi031.stdout:7/7: link f0 d1/d2/f3 0 2024-04-24T23:13:21.934 INFO:tasks.workunit.client.0.smithi031.stdout:7/8: read f0 [3172605,69407] 0 2024-04-24T23:13:21.939 INFO:tasks.workunit.client.0.smithi031.stdout:8/14: creat f3 x:0 0 0 2024-04-24T23:13:21.949 INFO:tasks.workunit.client.0.smithi031.stdout:8/15: write f0 [616753,16300] 0 2024-04-24T23:13:21.950 INFO:tasks.workunit.client.0.smithi031.stdout:6/15: link l4 l6 0 2024-04-24T23:13:21.951 INFO:tasks.workunit.client.0.smithi031.stdout:6/16: write f1 [1559323,76070] 0 2024-04-24T23:13:21.954 INFO:tasks.workunit.client.0.smithi031.stdout:7/9: link d1/d2/f3 d1/d2/f4 0 2024-04-24T23:13:21.954 INFO:tasks.workunit.client.0.smithi031.stdout:7/10: read d1/d2/f4 [3095741,57482] 0 2024-04-24T23:13:21.954 INFO:tasks.workunit.client.0.smithi031.stdout:7/11: stat d1/d2/f3 0 2024-04-24T23:13:21.955 INFO:tasks.workunit.client.0.smithi031.stdout:8/16: symlink l4 0 2024-04-24T23:13:21.957 INFO:tasks.workunit.client.0.smithi031.stdout:6/17: creat f7 x:0 0 0 2024-04-24T23:13:21.962 INFO:tasks.workunit.client.0.smithi031.stdout:7/12: write d1/d2/f3 [2142043,29464] 0 2024-04-24T23:13:21.963 INFO:tasks.workunit.client.0.smithi031.stdout:8/17: symlink l5 0 2024-04-24T23:13:21.963 INFO:tasks.workunit.client.0.smithi031.stdout:8/18: rmdir - no directory 2024-04-24T23:13:21.964 INFO:tasks.workunit.client.0.smithi031.stdout:8/19: chown f0 0 1 2024-04-24T23:13:21.965 INFO:tasks.workunit.client.0.smithi031.stdout:6/18: rename l2 to l8 0 2024-04-24T23:13:21.968 INFO:tasks.workunit.client.0.smithi031.stdout:8/20: mknod c6 0 2024-04-24T23:13:21.970 INFO:tasks.workunit.client.0.smithi031.stdout:8/21: mknod c7 0 2024-04-24T23:13:21.970 INFO:tasks.workunit.client.0.smithi031.stdout:8/22: dread - f2 zero size 2024-04-24T23:13:21.974 INFO:tasks.workunit.client.0.smithi031.stdout:8/23: mkdir d8 0 2024-04-24T23:13:21.974 INFO:tasks.workunit.client.0.smithi031.stdout:8/24: readlink l4 0 2024-04-24T23:13:21.975 INFO:tasks.workunit.client.0.smithi031.stdout:8/25: truncate f2 674922 0 2024-04-24T23:13:21.977 INFO:tasks.workunit.client.0.smithi031.stdout:8/26: rename c6 to d8/c9 0 2024-04-24T23:13:21.977 INFO:tasks.workunit.client.0.smithi031.stdout:8/27: chown f0 13098890 1 2024-04-24T23:13:21.977 INFO:tasks.workunit.client.0.smithi031.stdout:8/28: chown l4 2 1 2024-04-24T23:13:21.978 INFO:tasks.workunit.client.0.smithi031.stdout:8/29: fdatasync f2 0 2024-04-24T23:13:21.986 INFO:tasks.workunit.client.0.smithi031.stdout:8/30: dwrite f0 [0,4194304] 0 2024-04-24T23:13:21.990 INFO:tasks.workunit.client.0.smithi031.stdout:8/31: mknod d8/ca 0 2024-04-24T23:13:21.994 INFO:tasks.workunit.client.0.smithi031.stdout:8/32: rename f2 to d8/fb 0 2024-04-24T23:13:21.995 INFO:tasks.workunit.client.0.smithi031.stdout:8/33: write f0 [2025265,97439] 0 2024-04-24T23:13:21.997 INFO:tasks.workunit.client.0.smithi031.stdout:8/34: write f0 [481470,121752] 0 2024-04-24T23:13:22.001 INFO:tasks.workunit.client.0.smithi031.stdout:8/35: creat d8/fc x:0 0 0 2024-04-24T23:13:22.002 INFO:tasks.workunit.client.0.smithi031.stdout:8/36: write f0 [3922504,23674] 0 2024-04-24T23:13:22.002 INFO:tasks.workunit.client.0.smithi031.stdout:8/37: readlink l4 0 2024-04-24T23:13:22.024 INFO:tasks.workunit.client.0.smithi031.stdout:2/5: chown c2 439434 1 2024-04-24T23:13:22.027 INFO:tasks.workunit.client.0.smithi031.stdout:2/6: symlink l3 0 2024-04-24T23:13:22.028 INFO:tasks.workunit.client.0.smithi031.stdout:4/7: getdents . 0 2024-04-24T23:13:22.028 INFO:tasks.workunit.client.0.smithi031.stdout:4/8: write - no filename 2024-04-24T23:13:22.031 INFO:tasks.workunit.client.0.smithi031.stdout:2/7: creat f4 x:0 0 0 2024-04-24T23:13:22.034 INFO:tasks.workunit.client.0.smithi031.stdout:2/8: mkdir d5 0 2024-04-24T23:13:22.035 INFO:tasks.workunit.client.0.smithi031.stdout:2/9: truncate f4 39412 0 2024-04-24T23:13:22.036 INFO:tasks.workunit.client.0.smithi031.stdout:2/10: dread f4 [0,4194304] 0 2024-04-24T23:13:22.041 INFO:tasks.workunit.client.0.smithi031.stdout:4/9: link l0 l2 0 2024-04-24T23:13:22.041 INFO:tasks.workunit.client.0.smithi031.stdout:4/10: truncate - no filename 2024-04-24T23:13:22.043 INFO:tasks.workunit.client.0.smithi031.stdout:3/8: getdents . 0 2024-04-24T23:13:22.045 INFO:tasks.workunit.client.0.smithi031.stdout:2/11: dwrite f4 [0,4194304] 0 2024-04-24T23:13:22.045 INFO:tasks.workunit.client.0.smithi031.stdout:3/9: read f1 [110168,84413] 0 2024-04-24T23:13:22.050 INFO:tasks.workunit.client.0.smithi031.stdout:3/10: write f0 [638674,69288] 0 2024-04-24T23:13:22.051 INFO:tasks.workunit.client.0.smithi031.stdout:7/13: write d1/d2/f3 [5164692,49604] 0 2024-04-24T23:13:22.052 INFO:tasks.workunit.client.0.smithi031.stdout:4/11: unlink l2 0 2024-04-24T23:13:22.053 INFO:tasks.workunit.client.0.smithi031.stdout:4/12: write - no filename 2024-04-24T23:13:22.056 INFO:tasks.workunit.client.0.smithi031.stdout:3/11: unlink f2 0 2024-04-24T23:13:22.056 INFO:tasks.workunit.client.0.smithi031.stdout:2/12: mkdir d5/d6 0 2024-04-24T23:13:22.057 INFO:tasks.workunit.client.0.smithi031.stdout:2/13: rename d5 to d5/d7 22 2024-04-24T23:13:22.065 INFO:tasks.workunit.client.0.smithi031.stdout:2/14: dwrite f4 [0,4194304] 0 2024-04-24T23:13:22.068 INFO:tasks.workunit.client.0.smithi031.stdout:9/2: chown c1 5261268 1 2024-04-24T23:13:22.069 INFO:tasks.workunit.client.0.smithi031.stdout:4/13: chown l0 20 1 2024-04-24T23:13:22.075 INFO:tasks.workunit.client.0.smithi031.stdout:2/15: mkdir d5/d8 0 2024-04-24T23:13:22.077 INFO:tasks.workunit.client.0.smithi031.stdout:9/3: unlink c1 0 2024-04-24T23:13:22.077 INFO:tasks.workunit.client.0.smithi031.stdout:9/4: write - no filename 2024-04-24T23:13:22.077 INFO:tasks.workunit.client.0.smithi031.stdout:9/5: read - no filename 2024-04-24T23:13:22.079 INFO:tasks.workunit.client.0.smithi031.stdout:4/14: mkdir d3 0 2024-04-24T23:13:22.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/15: stat d3 0 2024-04-24T23:13:22.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/16: dread - no filename 2024-04-24T23:13:22.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/17: dwrite - no filename 2024-04-24T23:13:22.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/18: dwrite - no filename 2024-04-24T23:13:22.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/19: dread - no filename 2024-04-24T23:13:22.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/20: dwrite - no filename 2024-04-24T23:13:22.082 INFO:tasks.workunit.client.0.smithi031.stdout:2/16: dwrite f4 [0,4194304] 0 2024-04-24T23:13:22.086 INFO:tasks.workunit.client.0.smithi031.stdout:9/6: mkdir d2 0 2024-04-24T23:13:22.086 INFO:tasks.workunit.client.0.smithi031.stdout:9/7: write - no filename 2024-04-24T23:13:22.086 INFO:tasks.workunit.client.0.smithi031.stdout:9/8: readlink - no filename 2024-04-24T23:13:22.087 INFO:tasks.workunit.client.0.smithi031.stdout:2/17: dread f4 [0,4194304] 0 2024-04-24T23:13:22.092 INFO:tasks.workunit.client.0.smithi031.stdout:4/21: symlink d3/l4 0 2024-04-24T23:13:22.093 INFO:tasks.workunit.client.0.smithi031.stdout:2/18: dread f4 [0,4194304] 0 2024-04-24T23:13:22.095 INFO:tasks.workunit.client.0.smithi031.stdout:9/9: creat d2/f3 x:0 0 0 2024-04-24T23:13:22.096 INFO:tasks.workunit.client.0.smithi031.stdout:9/10: write d2/f3 [120403,21455] 0 2024-04-24T23:13:22.096 INFO:tasks.workunit.client.0.smithi031.stdout:4/22: symlink d3/l5 0 2024-04-24T23:13:22.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/23: dread - no filename 2024-04-24T23:13:22.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/24: dread - no filename 2024-04-24T23:13:22.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/25: dwrite - no filename 2024-04-24T23:13:22.099 INFO:tasks.workunit.client.0.smithi031.stdout:1/20: write d1/f3 [4332371,69688] 0 2024-04-24T23:13:22.104 INFO:tasks.workunit.client.0.smithi031.stdout:1/21: dread d1/f3 [0,4194304] 0 2024-04-24T23:13:22.104 INFO:tasks.workunit.client.0.smithi031.stdout:9/11: read d2/f3 [38274,60932] 0 2024-04-24T23:13:22.105 INFO:tasks.workunit.client.0.smithi031.stdout:4/26: chown l0 1411959411 1 2024-04-24T23:13:22.105 INFO:tasks.workunit.client.0.smithi031.stdout:4/27: dwrite - no filename 2024-04-24T23:13:22.108 INFO:tasks.workunit.client.0.smithi031.stdout:7/14: getdents d1/d2 0 2024-04-24T23:13:22.109 INFO:tasks.workunit.client.0.smithi031.stdout:7/15: chown d1/d2/f4 11713 1 2024-04-24T23:13:22.111 INFO:tasks.workunit.client.0.smithi031.stdout:9/12: write d2/f3 [90359,357] 0 2024-04-24T23:13:22.112 INFO:tasks.workunit.client.0.smithi031.stdout:4/28: symlink d3/l6 0 2024-04-24T23:13:22.112 INFO:tasks.workunit.client.0.smithi031.stdout:4/29: write - no filename 2024-04-24T23:13:22.115 INFO:tasks.workunit.client.0.smithi031.stdout:7/16: dread d1/d2/f4 [4194304,4194304] 0 2024-04-24T23:13:22.119 INFO:tasks.workunit.client.0.smithi031.stdout:9/13: dwrite d2/f3 [0,4194304] 0 2024-04-24T23:13:22.119 INFO:tasks.workunit.client.0.smithi031.stdout:4/30: creat d3/f7 x:0 0 0 2024-04-24T23:13:22.122 INFO:tasks.workunit.client.0.smithi031.stdout:7/17: fdatasync f0 0 2024-04-24T23:13:22.123 INFO:tasks.workunit.client.0.smithi031.stdout:6/19: getdents . 0 2024-04-24T23:13:22.129 INFO:tasks.workunit.client.0.smithi031.stdout:6/20: creat f9 x:0 0 0 2024-04-24T23:13:22.130 INFO:tasks.workunit.client.0.smithi031.stdout:4/31: dwrite d3/f7 [0,4194304] 0 2024-04-24T23:13:22.131 INFO:tasks.workunit.client.0.smithi031.stdout:6/21: write f1 [79033,28269] 0 2024-04-24T23:13:22.132 INFO:tasks.workunit.client.0.smithi031.stdout:6/22: write f1 [1916799,34019] 0 2024-04-24T23:13:22.143 INFO:tasks.workunit.client.0.smithi031.stdout:7/18: link f0 d1/d2/f5 0 2024-04-24T23:13:22.147 INFO:tasks.workunit.client.0.smithi031.stdout:7/19: unlink d1/d2/f3 0 2024-04-24T23:13:22.147 INFO:tasks.workunit.client.0.smithi031.stdout:7/20: readlink - no filename 2024-04-24T23:13:22.149 INFO:tasks.workunit.client.0.smithi031.stdout:7/21: symlink d1/l6 0 2024-04-24T23:13:22.152 INFO:tasks.workunit.client.0.smithi031.stdout:8/38: truncate f0 3593334 0 2024-04-24T23:13:22.155 INFO:tasks.workunit.client.0.smithi031.stdout:7/22: write d1/d2/f4 [4977424,14678] 0 2024-04-24T23:13:22.156 INFO:tasks.workunit.client.0.smithi031.stdout:7/23: write f0 [1313899,19121] 0 2024-04-24T23:13:22.166 INFO:tasks.workunit.client.0.smithi031.stdout:7/24: link d1/d2/f5 d1/d2/f7 0 2024-04-24T23:13:22.170 INFO:tasks.workunit.client.0.smithi031.stdout:7/25: write d1/d2/f4 [202326,75345] 0 2024-04-24T23:13:22.189 INFO:tasks.workunit.client.0.smithi031.stdout:3/12: dwrite f3 [0,4194304] 0 2024-04-24T23:13:22.204 INFO:tasks.workunit.client.0.smithi031.stdout:1/22: truncate d1/f3 206501 0 2024-04-24T23:13:22.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/23: creat d1/f8 x:0 0 0 2024-04-24T23:13:22.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/24: rename d1 to d1/d9 22 2024-04-24T23:13:22.208 INFO:tasks.workunit.client.0.smithi031.stdout:1/25: fsync d1/f2 0 2024-04-24T23:13:22.209 INFO:tasks.workunit.client.0.smithi031.stdout:1/26: read d1/f2 [632306,83002] 0 2024-04-24T23:13:22.210 INFO:tasks.workunit.client.0.smithi031.stdout:9/14: truncate d2/f3 2614003 0 2024-04-24T23:13:22.211 INFO:tasks.workunit.client.0.smithi031.stdout:9/15: rename d2 to d2/d4 22 2024-04-24T23:13:22.211 INFO:tasks.workunit.client.0.smithi031.stdout:6/23: fsync f9 0 2024-04-24T23:13:22.217 INFO:tasks.workunit.client.0.smithi031.stdout:6/24: mknod ca 0 2024-04-24T23:13:22.220 INFO:tasks.workunit.client.0.smithi031.stdout:4/32: dwrite d3/f7 [4194304,4194304] 0 2024-04-24T23:13:22.225 INFO:tasks.workunit.client.0.smithi031.stdout:6/25: symlink lb 0 2024-04-24T23:13:22.227 INFO:tasks.workunit.client.0.smithi031.stdout:4/33: mknod d3/c8 0 2024-04-24T23:13:22.229 INFO:tasks.workunit.client.0.smithi031.stdout:6/26: rename f7 to fc 0 2024-04-24T23:13:22.230 INFO:tasks.workunit.client.0.smithi031.stdout:6/27: truncate fc 771310 0 2024-04-24T23:13:22.231 INFO:tasks.workunit.client.0.smithi031.stdout:4/34: creat d3/f9 x:0 0 0 2024-04-24T23:13:22.238 INFO:tasks.workunit.client.0.smithi031.stdout:4/35: chown l0 31 1 2024-04-24T23:13:22.238 INFO:tasks.workunit.client.0.smithi031.stdout:6/28: dwrite f9 [0,4194304] 0 2024-04-24T23:13:22.239 INFO:tasks.workunit.client.0.smithi031.stdout:4/36: chown d3/l5 24 1 2024-04-24T23:13:22.243 INFO:tasks.workunit.client.0.smithi031.stdout:6/29: write fc [573698,113714] 0 2024-04-24T23:13:22.244 INFO:tasks.workunit.client.0.smithi031.stdout:4/37: dread d3/f7 [4194304,4194304] 0 2024-04-24T23:13:22.245 INFO:tasks.workunit.client.0.smithi031.stdout:4/38: write d3/f7 [122864,116170] 0 2024-04-24T23:13:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:22 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:22 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:22 smithi031 ceph-mon[26411]: Reconfiguring alertmanager.smithi031 (dependencies changed)... 2024-04-24T23:13:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:22 smithi031 ceph-mon[26411]: Reconfiguring daemon alertmanager.smithi031 on smithi031 2024-04-24T23:13:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:22 smithi031 ceph-mon[26411]: pgmap v14: 65 pgs: 65 active+clean; 185 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 4.6 MiB/s wr, 452 op/s 2024-04-24T23:13:22.254 INFO:tasks.workunit.client.0.smithi031.stdout:6/30: dwrite fc [0,4194304] 0 2024-04-24T23:13:22.255 INFO:tasks.workunit.client.0.smithi031.stdout:4/39: creat d3/fa x:0 0 0 2024-04-24T23:13:22.255 INFO:tasks.workunit.client.0.smithi031.stdout:6/31: fdatasync f1 0 2024-04-24T23:13:22.263 INFO:tasks.workunit.client.0.smithi031.stdout:7/26: truncate d1/d2/f5 3956806 0 2024-04-24T23:13:22.266 INFO:tasks.workunit.client.0.smithi031.stdout:4/40: dwrite d3/f9 [0,4194304] 0 2024-04-24T23:13:22.267 INFO:tasks.workunit.client.0.smithi031.stdout:4/41: write d3/f9 [2552548,38730] 0 2024-04-24T23:13:22.276 INFO:tasks.workunit.client.0.smithi031.stdout:7/27: rmdir d1 39 2024-04-24T23:13:22.281 INFO:tasks.workunit.client.0.smithi031.stdout:4/42: creat d3/fb x:0 0 0 2024-04-24T23:13:22.286 INFO:tasks.workunit.client.0.smithi031.stdout:7/28: rename d1 to d1/d8 22 2024-04-24T23:13:22.295 INFO:tasks.workunit.client.0.smithi031.stdout:9/16: write d2/f3 [2468072,86859] 0 2024-04-24T23:13:22.296 INFO:tasks.workunit.client.0.smithi031.stdout:9/17: write d2/f3 [12152,51658] 0 2024-04-24T23:13:22.299 INFO:tasks.workunit.client.0.smithi031.stdout:9/18: fdatasync d2/f3 0 2024-04-24T23:13:22.300 INFO:tasks.workunit.client.0.smithi031.stdout:9/19: write d2/f3 [3137114,59530] 0 2024-04-24T23:13:22.306 INFO:tasks.workunit.client.0.smithi031.stdout:9/20: creat d2/f5 x:0 0 0 2024-04-24T23:13:22.309 INFO:tasks.workunit.client.0.smithi031.stdout:9/21: creat d2/f6 x:0 0 0 2024-04-24T23:13:22.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/22: creat d2/f7 x:0 0 0 2024-04-24T23:13:22.312 INFO:tasks.workunit.client.0.smithi031.stdout:8/39: truncate f0 992001 0 2024-04-24T23:13:22.313 INFO:tasks.workunit.client.0.smithi031.stdout:9/23: read - d2/f6 zero size 2024-04-24T23:13:22.316 INFO:tasks.workunit.client.0.smithi031.stdout:8/40: creat d8/fd x:0 0 0 2024-04-24T23:13:22.316 INFO:tasks.workunit.client.0.smithi031.stdout:8/41: readlink l4 0 2024-04-24T23:13:22.316 INFO:tasks.workunit.client.0.smithi031.stdout:8/42: chown d8/fd 18941 1 2024-04-24T23:13:22.318 INFO:tasks.workunit.client.0.smithi031.stdout:9/24: creat d2/f8 x:0 0 0 2024-04-24T23:13:22.318 INFO:tasks.workunit.client.0.smithi031.stdout:9/25: stat d2/f6 0 2024-04-24T23:13:22.319 INFO:tasks.workunit.client.0.smithi031.stdout:9/26: write d2/f5 [161096,95268] 0 2024-04-24T23:13:22.319 INFO:tasks.workunit.client.0.smithi031.stdout:8/43: creat d8/fe x:0 0 0 2024-04-24T23:13:22.321 INFO:tasks.workunit.client.0.smithi031.stdout:8/44: write f3 [488783,88928] 0 2024-04-24T23:13:22.322 INFO:tasks.workunit.client.0.smithi031.stdout:8/45: chown d8/fb 3945540 1 2024-04-24T23:13:22.322 INFO:tasks.workunit.client.0.smithi031.stdout:9/27: dread d2/f5 [0,4194304] 0 2024-04-24T23:13:22.325 INFO:tasks.workunit.client.0.smithi031.stdout:8/46: symlink d8/lf 0 2024-04-24T23:13:22.327 INFO:tasks.workunit.client.0.smithi031.stdout:6/32: truncate f1 804666 0 2024-04-24T23:13:22.328 INFO:tasks.workunit.client.0.smithi031.stdout:9/28: creat d2/f9 x:0 0 0 2024-04-24T23:13:22.333 INFO:tasks.workunit.client.0.smithi031.stdout:8/47: symlink d8/l10 0 2024-04-24T23:13:22.335 INFO:tasks.workunit.client.0.smithi031.stdout:6/33: creat fd x:0 0 0 2024-04-24T23:13:22.335 INFO:tasks.workunit.client.0.smithi031.stdout:4/43: write d3/f9 [4081260,114167] 0 2024-04-24T23:13:22.340 INFO:tasks.workunit.client.0.smithi031.stdout:6/34: creat fe x:0 0 0 2024-04-24T23:13:22.340 INFO:tasks.workunit.client.0.smithi031.stdout:6/35: chown fe 32 1 2024-04-24T23:13:22.341 INFO:tasks.workunit.client.0.smithi031.stdout:6/36: truncate fe 347191 0 2024-04-24T23:13:22.342 INFO:tasks.workunit.client.0.smithi031.stdout:4/44: creat d3/fc x:0 0 0 2024-04-24T23:13:22.342 INFO:tasks.workunit.client.0.smithi031.stdout:4/45: read d3/f7 [2450916,9768] 0 2024-04-24T23:13:22.344 INFO:tasks.workunit.client.0.smithi031.stdout:9/29: link d2/f6 d2/fa 0 2024-04-24T23:13:22.345 INFO:tasks.workunit.client.0.smithi031.stdout:1/27: write d1/f3 [401266,42254] 0 2024-04-24T23:13:22.346 INFO:tasks.workunit.client.0.smithi031.stdout:6/37: symlink lf 0 2024-04-24T23:13:22.347 INFO:tasks.workunit.client.0.smithi031.stdout:6/38: fsync f9 0 2024-04-24T23:13:22.351 INFO:tasks.workunit.client.0.smithi031.stdout:9/30: symlink d2/lb 0 2024-04-24T23:13:22.352 INFO:tasks.workunit.client.0.smithi031.stdout:9/31: readlink d2/lb 0 2024-04-24T23:13:22.352 INFO:tasks.workunit.client.0.smithi031.stdout:3/13: truncate f3 927032 0 2024-04-24T23:13:22.354 INFO:tasks.workunit.client.0.smithi031.stdout:1/28: mkdir d1/da 0 2024-04-24T23:13:22.355 INFO:tasks.workunit.client.0.smithi031.stdout:1/29: fsync d1/f3 0 2024-04-24T23:13:22.356 INFO:tasks.workunit.client.0.smithi031.stdout:1/30: truncate d1/f8 424565 0 2024-04-24T23:13:22.359 INFO:tasks.workunit.client.0.smithi031.stdout:6/39: creat f10 x:0 0 0 2024-04-24T23:13:22.360 INFO:tasks.workunit.client.0.smithi031.stdout:9/32: dwrite d2/f3 [0,4194304] 0 2024-04-24T23:13:22.361 INFO:tasks.workunit.client.0.smithi031.stdout:9/33: chown d2/f5 92 1 2024-04-24T23:13:22.495 INFO:tasks.workunit.client.0.smithi031.stdout:6/40: creat f11 x:0 0 0 2024-04-24T23:13:22.501 INFO:tasks.workunit.client.0.smithi031.stdout:3/14: fdatasync f1 0 2024-04-24T23:13:22.503 INFO:tasks.workunit.client.0.smithi031.stdout:1/31: symlink d1/da/lb 0 2024-04-24T23:13:22.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/34: dwrite d2/fa [0,4194304] 0 2024-04-24T23:13:22.512 INFO:tasks.workunit.client.0.smithi031.stdout:1/32: dwrite d1/f8 [0,4194304] 0 2024-04-24T23:13:22.513 INFO:tasks.workunit.client.0.smithi031.stdout:1/33: read d1/f2 [21400,30884] 0 2024-04-24T23:13:22.514 INFO:tasks.workunit.client.0.smithi031.stdout:1/34: chown d1/f3 28 1 2024-04-24T23:13:22.518 INFO:tasks.workunit.client.0.smithi031.stdout:6/41: link fc f12 0 2024-04-24T23:13:22.518 INFO:tasks.workunit.client.0.smithi031.stdout:6/42: readlink l3 0 2024-04-24T23:13:22.519 INFO:tasks.workunit.client.0.smithi031.stdout:7/29: write d1/d2/f7 [3084645,99708] 0 2024-04-24T23:13:22.521 INFO:tasks.workunit.client.0.smithi031.stdout:3/15: mknod c4 0 2024-04-24T23:13:22.525 INFO:tasks.workunit.client.0.smithi031.stdout:1/35: mkdir d1/dc 0 2024-04-24T23:13:22.526 INFO:tasks.workunit.client.0.smithi031.stdout:1/36: chown d1/c6 182069661 1 2024-04-24T23:13:22.528 INFO:tasks.workunit.client.0.smithi031.stdout:6/43: fdatasync fc 0 2024-04-24T23:13:22.531 INFO:tasks.workunit.client.0.smithi031.stdout:6/44: dread f12 [0,4194304] 0 2024-04-24T23:13:22.532 INFO:tasks.workunit.client.0.smithi031.stdout:4/46: rmdir d3 39 2024-04-24T23:13:22.532 INFO:tasks.workunit.client.0.smithi031.stdout:6/45: write fe [1267746,88598] 0 2024-04-24T23:13:22.540 INFO:tasks.workunit.client.0.smithi031.stdout:1/37: mknod d1/cd 0 2024-04-24T23:13:22.541 INFO:tasks.workunit.client.0.smithi031.stdout:1/38: write d1/f8 [3580566,82955] 0 2024-04-24T23:13:22.548 INFO:tasks.workunit.client.0.smithi031.stdout:6/46: mkdir d13 0 2024-04-24T23:13:22.551 INFO:tasks.workunit.client.0.smithi031.stdout:6/47: dread f12 [0,4194304] 0 2024-04-24T23:13:22.552 INFO:tasks.workunit.client.0.smithi031.stdout:6/48: dread - f11 zero size 2024-04-24T23:13:22.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/16: fsync f3 0 2024-04-24T23:13:22.553 INFO:tasks.workunit.client.0.smithi031.stdout:3/17: stat c4 0 2024-04-24T23:13:22.560 INFO:tasks.workunit.client.0.smithi031.stdout:4/47: unlink d3/l4 0 2024-04-24T23:13:22.560 INFO:tasks.workunit.client.0.smithi031.stdout:4/48: chown d3/l6 6 1 2024-04-24T23:13:22.561 INFO:tasks.workunit.client.0.smithi031.stdout:4/49: read - d3/fc zero size 2024-04-24T23:13:22.563 INFO:tasks.workunit.client.0.smithi031.stdout:3/18: chown f0 32078 1 2024-04-24T23:13:22.563 INFO:tasks.workunit.client.0.smithi031.stdout:3/19: rmdir - no directory 2024-04-24T23:13:22.564 INFO:tasks.workunit.client.0.smithi031.stdout:1/39: creat d1/dc/fe x:0 0 0 2024-04-24T23:13:22.565 INFO:tasks.workunit.client.0.smithi031.stdout:6/49: creat d13/f14 x:0 0 0 2024-04-24T23:13:22.566 INFO:tasks.workunit.client.0.smithi031.stdout:6/50: dread - f10 zero size 2024-04-24T23:13:22.567 INFO:tasks.workunit.client.0.smithi031.stdout:4/50: mkdir d3/dd 0 2024-04-24T23:13:22.569 INFO:tasks.workunit.client.0.smithi031.stdout:1/40: creat d1/ff x:0 0 0 2024-04-24T23:13:22.571 INFO:tasks.workunit.client.0.smithi031.stdout:6/51: mkdir d13/d15 0 2024-04-24T23:13:22.573 INFO:tasks.workunit.client.0.smithi031.stdout:3/20: mknod c5 0 2024-04-24T23:13:22.574 INFO:tasks.workunit.client.0.smithi031.stdout:3/21: chown f0 111 1 2024-04-24T23:13:22.577 INFO:tasks.workunit.client.0.smithi031.stdout:1/41: dwrite d1/dc/fe [0,4194304] 0 2024-04-24T23:13:22.579 INFO:tasks.workunit.client.0.smithi031.stdout:6/52: rename f1 to d13/d15/f16 0 2024-04-24T23:13:22.581 INFO:tasks.workunit.client.0.smithi031.stdout:1/42: mkdir d1/da/d10 0 2024-04-24T23:13:22.581 INFO:tasks.workunit.client.0.smithi031.stdout:1/43: chown d1/da/lb 720908878 1 2024-04-24T23:13:22.583 INFO:tasks.workunit.client.0.smithi031.stdout:6/53: creat d13/f17 x:0 0 0 2024-04-24T23:13:22.584 INFO:tasks.workunit.client.0.smithi031.stdout:6/54: write f10 [589402,20610] 0 2024-04-24T23:13:22.585 INFO:tasks.workunit.client.0.smithi031.stdout:1/44: mkdir d1/dc/d11 0 2024-04-24T23:13:22.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/45: readlink d1/da/lb 0 2024-04-24T23:13:22.587 INFO:tasks.workunit.client.0.smithi031.stdout:6/55: rmdir d13/d15 39 2024-04-24T23:13:22.587 INFO:tasks.workunit.client.0.smithi031.stdout:1/46: write d1/ff [658939,83056] 0 2024-04-24T23:13:22.589 INFO:tasks.workunit.client.0.smithi031.stdout:1/47: write d1/ff [183890,80573] 0 2024-04-24T23:13:22.592 INFO:tasks.workunit.client.0.smithi031.stdout:1/48: mkdir d1/da/d12 0 2024-04-24T23:13:22.595 INFO:tasks.workunit.client.0.smithi031.stdout:6/56: dwrite d13/f17 [0,4194304] 0 2024-04-24T23:13:22.597 INFO:tasks.workunit.client.0.smithi031.stdout:1/49: creat d1/dc/d11/f13 x:0 0 0 2024-04-24T23:13:22.601 INFO:tasks.workunit.client.0.smithi031.stdout:1/50: truncate d1/f8 5118964 0 2024-04-24T23:13:22.605 INFO:tasks.workunit.client.0.smithi031.stdout:6/57: dwrite f11 [0,4194304] 0 2024-04-24T23:13:22.607 INFO:tasks.workunit.client.0.smithi031.stdout:1/51: dread d1/ff [0,4194304] 0 2024-04-24T23:13:22.614 INFO:tasks.workunit.client.0.smithi031.stdout:1/52: dwrite d1/f2 [4194304,4194304] 0 2024-04-24T23:13:22.617 INFO:tasks.workunit.client.0.smithi031.stdout:1/53: unlink d1/c6 0 2024-04-24T23:13:22.622 INFO:tasks.workunit.client.0.smithi031.stdout:1/54: mknod d1/da/d12/c14 0 2024-04-24T23:13:22.623 INFO:tasks.workunit.client.0.smithi031.stdout:1/55: dread d1/ff [0,4194304] 0 2024-04-24T23:13:22.625 INFO:tasks.workunit.client.0.smithi031.stdout:1/56: symlink d1/da/d10/l15 0 2024-04-24T23:13:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:22 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:22 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:22 smithi040 ceph-mon[33128]: Reconfiguring alertmanager.smithi031 (dependencies changed)... 2024-04-24T23:13:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:22 smithi040 ceph-mon[33128]: Reconfiguring daemon alertmanager.smithi031 on smithi031 2024-04-24T23:13:22.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:22 smithi040 ceph-mon[33128]: pgmap v14: 65 pgs: 65 active+clean; 185 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 4.6 MiB/s wr, 452 op/s 2024-04-24T23:13:22.779 INFO:tasks.workunit.client.0.smithi031.stdout:3/22: dread f1 [0,4194304] 0 2024-04-24T23:13:22.797 INFO:tasks.workunit.client.0.smithi031.stdout:9/35: truncate d2/f3 3772175 0 2024-04-24T23:13:22.919 INFO:tasks.workunit.client.0.smithi031.stdout:7/30: truncate d1/d2/f4 89672 0 2024-04-24T23:13:23.086 INFO:tasks.workunit.client.0.smithi031.stdout:7/31: creat d1/f9 x:0 0 0 2024-04-24T23:13:23.088 INFO:tasks.workunit.client.0.smithi031.stdout:6/58: mknod d13/d15/c18 0 2024-04-24T23:13:23.090 INFO:tasks.workunit.client.0.smithi031.stdout:3/23: dwrite f3 [0,4194304] 0 2024-04-24T23:13:23.092 INFO:tasks.workunit.client.0.smithi031.stdout:3/24: stat f1 0 2024-04-24T23:13:23.094 INFO:tasks.workunit.client.0.smithi031.stdout:3/25: truncate f0 897338 0 2024-04-24T23:13:23.097 INFO:tasks.workunit.client.0.smithi031.stdout:7/32: dwrite d1/f9 [0,4194304] 0 2024-04-24T23:13:23.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/59: link l4 d13/d15/l19 0 2024-04-24T23:13:23.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/60: fdatasync f9 0 2024-04-24T23:13:23.113 INFO:tasks.workunit.client.0.smithi031.stdout:6/61: mknod d13/c1a 0 2024-04-24T23:13:23.113 INFO:tasks.workunit.client.0.smithi031.stdout:6/62: chown f10 89273503 1 2024-04-24T23:13:23.264 INFO:tasks.workunit.client.0.smithi031.stdout:6/63: unlink d13/f17 0 2024-04-24T23:13:23.268 INFO:tasks.workunit.client.0.smithi031.stdout:6/64: dread fe [0,4194304] 0 2024-04-24T23:13:23.269 INFO:tasks.workunit.client.0.smithi031.stdout:6/65: dread fe [0,4194304] 0 2024-04-24T23:13:23.289 INFO:tasks.workunit.client.0.smithi031.stdout:6/66: mkdir d13/d1b 0 2024-04-24T23:13:23.290 INFO:tasks.workunit.client.0.smithi031.stdout:6/67: fsync d13/f14 0 2024-04-24T23:13:23.293 INFO:tasks.workunit.client.0.smithi031.stdout:6/68: dread fc [0,4194304] 0 2024-04-24T23:13:23.299 INFO:tasks.workunit.client.0.smithi031.stdout:6/69: creat d13/d1b/f1c x:0 0 0 2024-04-24T23:13:23.300 INFO:tasks.workunit.client.0.smithi031.stdout:6/70: chown d13/f14 5752135 1 2024-04-24T23:13:23.304 INFO:tasks.workunit.client.0.smithi031.stdout:6/71: mkdir d13/d15/d1d 0 2024-04-24T23:13:23.308 INFO:tasks.workunit.client.0.smithi031.stdout:6/72: mkdir d13/d15/d1d/d1e 0 2024-04-24T23:13:23.311 INFO:tasks.workunit.client.0.smithi031.stdout:6/73: creat d13/d15/f1f x:0 0 0 2024-04-24T23:13:23.317 INFO:tasks.workunit.client.0.smithi031.stdout:6/74: rename l4 to d13/d15/d1d/l20 0 2024-04-24T23:13:23.317 INFO:tasks.workunit.client.0.smithi031.stdout:6/75: fdatasync f10 0 2024-04-24T23:13:23.320 INFO:tasks.workunit.client.0.smithi031.stdout:6/76: dread f10 [0,4194304] 0 2024-04-24T23:13:23.324 INFO:tasks.workunit.client.0.smithi031.stdout:6/77: mknod d13/d15/d1d/c21 0 2024-04-24T23:13:23.327 INFO:tasks.workunit.client.0.smithi031.stdout:6/78: creat d13/d1b/f22 x:0 0 0 2024-04-24T23:13:23.327 INFO:tasks.workunit.client.0.smithi031.stdout:6/79: stat ca 0 2024-04-24T23:13:23.330 INFO:tasks.workunit.client.0.smithi031.stdout:6/80: mkdir d13/d15/d1d/d23 0 2024-04-24T23:13:23.334 INFO:tasks.workunit.client.0.smithi031.stdout:6/81: mkdir d13/d15/d1d/d23/d24 0 2024-04-24T23:13:23.335 INFO:tasks.workunit.client.0.smithi031.stdout:6/82: chown d13/c1a 1579 1 2024-04-24T23:13:23.342 INFO:tasks.workunit.client.0.smithi031.stdout:6/83: truncate fc 2496045 0 2024-04-24T23:13:23.354 INFO:tasks.workunit.client.0.smithi031.stdout:6/84: link d13/d15/f16 d13/f25 0 2024-04-24T23:13:23.364 INFO:tasks.workunit.client.0.smithi031.stdout:6/85: truncate d13/f25 583934 0 2024-04-24T23:13:23.365 INFO:tasks.workunit.client.0.smithi031.stdout:6/86: truncate d13/d15/f1f 718405 0 2024-04-24T23:13:23.366 INFO:tasks.workunit.client.0.smithi031.stdout:6/87: dread - d13/d1b/f1c zero size 2024-04-24T23:13:23.369 INFO:tasks.workunit.client.0.smithi031.stdout:6/88: dread f9 [0,4194304] 0 2024-04-24T23:13:23.376 INFO:tasks.workunit.client.0.smithi031.stdout:6/89: dwrite d13/d1b/f1c [0,4194304] 0 2024-04-24T23:13:23.377 INFO:tasks.workunit.client.0.smithi031.stdout:6/90: truncate f11 4249859 0 2024-04-24T23:13:23.381 INFO:tasks.workunit.client.0.smithi031.stdout:6/91: rename d13/d15/c18 to d13/d15/d1d/d23/c26 0 2024-04-24T23:13:23.389 INFO:tasks.workunit.client.0.smithi031.stdout:6/92: rmdir d13/d15/d1d/d1e 0 2024-04-24T23:13:23.397 INFO:tasks.workunit.client.0.smithi031.stdout:6/93: dwrite f10 [0,4194304] 0 2024-04-24T23:13:23.401 INFO:tasks.workunit.client.0.smithi031.stdout:6/94: creat d13/d15/f27 x:0 0 0 2024-04-24T23:13:23.411 INFO:tasks.workunit.client.0.smithi031.stdout:6/95: link d13/d15/l19 d13/d1b/l28 0 2024-04-24T23:13:23.413 INFO:tasks.workunit.client.0.smithi031.stdout:6/96: fsync fe 0 2024-04-24T23:13:23.418 INFO:tasks.workunit.client.0.smithi031.stdout:6/97: mknod d13/c29 0 2024-04-24T23:13:23.533 INFO:tasks.workunit.client.0.smithi031.stdout:7/33: truncate d1/f9 1238816 0 2024-04-24T23:13:23.534 INFO:tasks.workunit.client.0.smithi031.stdout:3/26: dwrite f1 [0,4194304] 0 2024-04-24T23:13:23.538 INFO:tasks.workunit.client.0.smithi031.stdout:7/34: creat d1/d2/fa x:0 0 0 2024-04-24T23:13:23.541 INFO:tasks.workunit.client.0.smithi031.stdout:3/27: unlink f3 0 2024-04-24T23:13:23.541 INFO:tasks.workunit.client.0.smithi031.stdout:3/28: chown f1 243 1 2024-04-24T23:13:23.546 INFO:tasks.workunit.client.0.smithi031.stdout:3/29: dread f0 [0,4194304] 0 2024-04-24T23:13:23.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/30: write f1 [968333,76997] 0 2024-04-24T23:13:23.548 INFO:tasks.workunit.client.0.smithi031.stdout:3/31: dread f0 [0,4194304] 0 2024-04-24T23:13:23.554 INFO:tasks.workunit.client.0.smithi031.stdout:3/32: write f0 [1543430,104653] 0 2024-04-24T23:13:23.555 INFO:tasks.workunit.client.0.smithi031.stdout:3/33: read f1 [498338,14961] 0 2024-04-24T23:13:23.561 INFO:tasks.workunit.client.0.smithi031.stdout:3/34: mkdir d6 0 2024-04-24T23:13:23.565 INFO:tasks.workunit.client.0.smithi031.stdout:3/35: symlink d6/l7 0 2024-04-24T23:13:23.568 INFO:tasks.workunit.client.0.smithi031.stdout:3/36: creat d6/f8 x:0 0 0 2024-04-24T23:13:23.575 INFO:tasks.workunit.client.0.smithi031.stdout:3/37: dwrite d6/f8 [0,4194304] 0 2024-04-24T23:13:23.579 INFO:tasks.workunit.client.0.smithi031.stdout:3/38: symlink d6/l9 0 2024-04-24T23:13:23.592 INFO:tasks.workunit.client.0.smithi031.stdout:9/36: write d2/f3 [3599403,6476] 0 2024-04-24T23:13:23.599 INFO:tasks.workunit.client.0.smithi031.stdout:9/37: dwrite d2/f6 [0,4194304] 0 2024-04-24T23:13:23.603 INFO:tasks.workunit.client.0.smithi031.stdout:9/38: creat d2/fc x:0 0 0 2024-04-24T23:13:23.607 INFO:tasks.workunit.client.0.smithi031.stdout:9/39: rename d2/f7 to d2/fd 0 2024-04-24T23:13:23.610 INFO:tasks.workunit.client.0.smithi031.stdout:9/40: creat d2/fe x:0 0 0 2024-04-24T23:13:23.628 INFO:tasks.workunit.client.0.smithi031.stdout:6/98: rmdir d13/d1b 39 2024-04-24T23:13:23.628 INFO:tasks.workunit.client.0.smithi031.stdout:6/99: chown d13/d15/d1d/c21 1624864960 1 2024-04-24T23:13:23.873 INFO:tasks.workunit.client.0.smithi031.stdout:3/39: fsync d6/f8 0 2024-04-24T23:13:23.901 INFO:tasks.workunit.client.0.smithi031.stdout:9/41: getdents d2 0 2024-04-24T23:13:23.901 INFO:tasks.workunit.client.0.smithi031.stdout:9/42: fdatasync d2/f9 0 2024-04-24T23:13:23.912 INFO:tasks.workunit.client.0.smithi031.stdout:6/100: dwrite f12 [0,4194304] 0 2024-04-24T23:13:24.024 INFO:tasks.workunit.client.0.smithi031.stdout:7/35: dwrite d1/d2/f7 [0,4194304] 0 2024-04-24T23:13:24.025 INFO:tasks.workunit.client.0.smithi031.stdout:7/36: read - d1/d2/fa zero size 2024-04-24T23:13:24.025 INFO:tasks.workunit.client.0.smithi031.stdout:7/37: chown d1/l6 5 1 2024-04-24T23:13:24.058 INFO:tasks.workunit.client.0.smithi031.stdout:3/40: truncate f1 4531790 0 2024-04-24T23:13:24.061 INFO:tasks.workunit.client.0.smithi031.stdout:7/38: symlink d1/lb 0 2024-04-24T23:13:24.065 INFO:tasks.workunit.client.0.smithi031.stdout:7/39: mknod d1/d2/cc 0 2024-04-24T23:13:24.066 INFO:tasks.workunit.client.0.smithi031.stdout:3/41: chown f1 1016888 1 2024-04-24T23:13:24.073 INFO:tasks.workunit.client.0.smithi031.stdout:3/42: link d6/l9 d6/la 0 2024-04-24T23:13:24.075 INFO:tasks.workunit.client.0.smithi031.stdout:3/43: creat d6/fb x:0 0 0 2024-04-24T23:13:24.149 INFO:tasks.workunit.client.0.smithi031.stdout:3/44: rename f1 to d6/fc 0 2024-04-24T23:13:24.152 INFO:tasks.workunit.client.0.smithi031.stdout:9/43: truncate d2/f3 2412202 0 2024-04-24T23:13:24.152 INFO:tasks.workunit.client.0.smithi031.stdout:9/44: stat d2/f5 0 2024-04-24T23:13:24.153 INFO:tasks.workunit.client.0.smithi031.stdout:6/101: truncate f12 2013207 0 2024-04-24T23:13:24.155 INFO:tasks.workunit.client.0.smithi031.stdout:7/40: truncate f0 3997422 0 2024-04-24T23:13:24.160 INFO:tasks.workunit.client.0.smithi031.stdout:9/45: dwrite d2/fd [0,4194304] 0 2024-04-24T23:13:24.161 INFO:tasks.workunit.client.0.smithi031.stdout:9/46: fsync d2/fc 0 2024-04-24T23:13:24.162 INFO:tasks.workunit.client.0.smithi031.stdout:9/47: truncate d2/f8 733387 0 2024-04-24T23:13:24.165 INFO:tasks.workunit.client.0.smithi031.stdout:9/48: mkdir d2/df 0 2024-04-24T23:13:24.172 INFO:tasks.workunit.client.0.smithi031.stdout:7/41: getdents d1 0 2024-04-24T23:13:24.357 INFO:tasks.workunit.client.0.smithi031.stdout:3/45: fsync d6/fc 0 2024-04-24T23:13:24.363 INFO:tasks.workunit.client.0.smithi031.stdout:6/102: write d13/d15/f16 [233437,130769] 0 2024-04-24T23:13:24.364 INFO:tasks.workunit.client.0.smithi031.stdout:3/46: mknod d6/cd 0 2024-04-24T23:13:24.365 INFO:tasks.workunit.client.0.smithi031.stdout:3/47: chown d6 5815071 1 2024-04-24T23:13:24.365 INFO:tasks.workunit.client.0.smithi031.stdout:9/49: write d2/fd [4659817,47440] 0 2024-04-24T23:13:24.366 INFO:tasks.workunit.client.0.smithi031.stdout:9/50: chown d2/f6 12 1 2024-04-24T23:13:24.368 INFO:tasks.workunit.client.0.smithi031.stdout:9/51: fdatasync d2/fd 0 2024-04-24T23:13:24.371 INFO:tasks.workunit.client.0.smithi031.stdout:6/103: rename d13/c29 to d13/d15/d1d/d23/d24/c2a 0 2024-04-24T23:13:24.375 INFO:tasks.workunit.client.0.smithi031.stdout:3/48: dwrite f0 [0,4194304] 0 2024-04-24T23:13:24.378 INFO:tasks.workunit.client.0.smithi031.stdout:9/52: dwrite d2/fd [0,4194304] 0 2024-04-24T23:13:24.379 INFO:tasks.workunit.client.0.smithi031.stdout:9/53: write d2/fa [459386,26087] 0 2024-04-24T23:13:24.380 INFO:tasks.workunit.client.0.smithi031.stdout:9/54: readlink d2/lb 0 2024-04-24T23:13:24.380 INFO:tasks.workunit.client.0.smithi031.stdout:9/55: dread - d2/fe zero size 2024-04-24T23:13:24.381 INFO:tasks.workunit.client.0.smithi031.stdout:6/104: creat d13/d15/d1d/d23/f2b x:0 0 0 2024-04-24T23:13:24.383 INFO:tasks.workunit.client.0.smithi031.stdout:3/49: creat d6/fe x:0 0 0 2024-04-24T23:13:24.392 INFO:tasks.workunit.client.0.smithi031.stdout:3/50: mkdir d6/df 0 2024-04-24T23:13:24.393 INFO:tasks.workunit.client.0.smithi031.stdout:3/51: read d6/f8 [3505935,112481] 0 2024-04-24T23:13:24.395 INFO:tasks.workunit.client.0.smithi031.stdout:6/105: mknod d13/c2c 0 2024-04-24T23:13:24.397 INFO:tasks.workunit.client.0.smithi031.stdout:3/52: chown d6/la 2018 1 2024-04-24T23:13:24.398 INFO:tasks.workunit.client.0.smithi031.stdout:3/53: chown c4 9169 1 2024-04-24T23:13:24.398 INFO:tasks.workunit.client.0.smithi031.stdout:3/54: write d6/f8 [1042045,95218] 0 2024-04-24T23:13:24.400 INFO:tasks.workunit.client.0.smithi031.stdout:6/106: dread - d13/d1b/f22 zero size 2024-04-24T23:13:24.402 INFO:tasks.workunit.client.0.smithi031.stdout:3/55: symlink d6/l10 0 2024-04-24T23:13:24.402 INFO:tasks.workunit.client.0.smithi031.stdout:3/56: chown d6 119753 1 2024-04-24T23:13:24.403 INFO:tasks.workunit.client.0.smithi031.stdout:3/57: write d6/f8 [3956001,13698] 0 2024-04-24T23:13:24.409 INFO:tasks.workunit.client.0.smithi031.stdout:3/58: unlink d6/l9 0 2024-04-24T23:13:24.415 INFO:tasks.workunit.client.0.smithi031.stdout:3/59: rename d6/l7 to d6/l11 0 2024-04-24T23:13:24.420 INFO:tasks.workunit.client.0.smithi031.stdout:3/60: symlink d6/l12 0 2024-04-24T23:13:24.423 INFO:tasks.workunit.client.0.smithi031.stdout:3/61: readlink d6/la 0 2024-04-24T23:13:24.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/62: link c4 d6/c13 0 2024-04-24T23:13:24.433 INFO:tasks.workunit.client.0.smithi031.stdout:3/63: mkdir d6/d14 0 2024-04-24T23:13:24.436 INFO:tasks.workunit.client.0.smithi031.stdout:3/64: mkdir d6/d15 0 2024-04-24T23:13:24.436 INFO:tasks.workunit.client.0.smithi031.stdout:3/65: chown d6/fb 815 1 2024-04-24T23:13:24.441 INFO:tasks.workunit.client.0.smithi031.stdout:3/66: mkdir d6/df/d16 0 2024-04-24T23:13:24.441 INFO:tasks.workunit.client.0.smithi031.stdout:9/56: link d2/f3 d2/df/f10 0 2024-04-24T23:13:24.442 INFO:tasks.workunit.client.0.smithi031.stdout:9/57: chown d2/fd 43243 1 2024-04-24T23:13:24.449 INFO:tasks.workunit.client.0.smithi031.stdout:9/58: mknod d2/df/c11 0 2024-04-24T23:13:24.452 INFO:tasks.workunit.client.0.smithi031.stdout:9/59: creat d2/f12 x:0 0 0 2024-04-24T23:13:24.453 INFO:tasks.workunit.client.0.smithi031.stdout:9/60: truncate d2/f8 1360846 0 2024-04-24T23:13:24.453 INFO:tasks.workunit.client.0.smithi031.stdout:9/61: stat d2/df 0 2024-04-24T23:13:24.454 INFO:tasks.workunit.client.0.smithi031.stdout:9/62: chown d2/fa 50755 1 2024-04-24T23:13:24.456 INFO:tasks.workunit.client.0.smithi031.stdout:9/63: rmdir d2/df 39 2024-04-24T23:13:24.457 INFO:tasks.workunit.client.0.smithi031.stdout:9/64: write d2/f9 [1047843,103781] 0 2024-04-24T23:13:24.459 INFO:tasks.workunit.client.0.smithi031.stdout:9/65: fsync d2/f5 0 2024-04-24T23:13:24.686 INFO:tasks.workunit.client.0.smithi031.stdout:9/66: link d2/df/f10 d2/df/f13 0 2024-04-24T23:13:24.687 INFO:tasks.workunit.client.0.smithi031.stdout:9/67: write d2/f8 [438417,10923] 0 2024-04-24T23:13:24.691 INFO:tasks.workunit.client.0.smithi031.stdout:9/68: mkdir d2/df/d14 0 2024-04-24T23:13:24.701 INFO:tasks.workunit.client.0.smithi031.stdout:9/69: link d2/f6 d2/f15 0 2024-04-24T23:13:24.710 INFO:tasks.workunit.client.0.smithi031.stdout:3/67: dwrite d6/f8 [0,4194304] 0 2024-04-24T23:13:24.721 INFO:tasks.workunit.client.0.smithi031.stdout:3/68: creat d6/df/f17 x:0 0 0 2024-04-24T23:13:24.723 INFO:tasks.workunit.client.0.smithi031.stdout:3/69: truncate d6/fb 56402 0 2024-04-24T23:13:24.728 INFO:tasks.workunit.client.0.smithi031.stdout:3/70: dread d6/fc [0,4194304] 0 2024-04-24T23:13:24.733 INFO:tasks.workunit.client.0.smithi031.stdout:3/71: mknod d6/d14/c18 0 2024-04-24T23:13:24.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:24 smithi031 ceph-mon[26411]: pgmap v15: 65 pgs: 65 active+clean; 185 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.6 MiB/s wr, 319 op/s 2024-04-24T23:13:24.749 INFO:tasks.workunit.client.0.smithi031.stdout:3/72: rmdir d6/df 39 2024-04-24T23:13:24.749 INFO:tasks.workunit.client.0.smithi031.stdout:3/73: stat d6 0 2024-04-24T23:13:24.751 INFO:tasks.workunit.client.0.smithi031.stdout:3/74: chown d6/df/d16 287811569 1 2024-04-24T23:13:24.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:24 smithi040 ceph-mon[33128]: pgmap v15: 65 pgs: 65 active+clean; 185 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 3.6 MiB/s wr, 319 op/s 2024-04-24T23:13:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:26 smithi031 ceph-mon[26411]: pgmap v16: 65 pgs: 65 active+clean; 211 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 214 KiB/s rd, 5.6 MiB/s wr, 354 op/s 2024-04-24T23:13:26.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:26 smithi040 ceph-mon[33128]: pgmap v16: 65 pgs: 65 active+clean; 211 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 214 KiB/s rd, 5.6 MiB/s wr, 354 op/s 2024-04-24T23:13:28.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:28 smithi031 ceph-mon[26411]: pgmap v17: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 393 KiB/s rd, 16 MiB/s wr, 379 op/s 2024-04-24T23:13:28.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:28 smithi040 ceph-mon[33128]: pgmap v17: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 393 KiB/s rd, 16 MiB/s wr, 379 op/s 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:8/48: sync 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:6/107: sync 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:4/51: sync 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:1/57: sync 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:9/70: sync 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:7/42: sync 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:3/75: sync 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:0/8: sync 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:0/9: dread - no filename 2024-04-24T23:13:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:2/19: sync 2024-04-24T23:13:29.869 INFO:tasks.workunit.client.0.smithi031.stdout:5/3: sync 2024-04-24T23:13:29.869 INFO:tasks.workunit.client.0.smithi031.stdout:5/4: truncate - no filename 2024-04-24T23:13:29.869 INFO:tasks.workunit.client.0.smithi031.stdout:3/76: chown d6/c13 45410 1 2024-04-24T23:13:29.870 INFO:tasks.workunit.client.0.smithi031.stdout:0/10: mknod c1 0 2024-04-24T23:13:29.870 INFO:tasks.workunit.client.0.smithi031.stdout:0/11: dwrite - no filename 2024-04-24T23:13:29.870 INFO:tasks.workunit.client.0.smithi031.stdout:0/12: truncate - no filename 2024-04-24T23:13:29.870 INFO:tasks.workunit.client.0.smithi031.stdout:0/13: truncate - no filename 2024-04-24T23:13:29.873 INFO:tasks.workunit.client.0.smithi031.stdout:5/5: mkdir d0/d1 0 2024-04-24T23:13:29.873 INFO:tasks.workunit.client.0.smithi031.stdout:5/6: dread - no filename 2024-04-24T23:13:29.873 INFO:tasks.workunit.client.0.smithi031.stdout:5/7: stat d0/d1 0 2024-04-24T23:13:29.873 INFO:tasks.workunit.client.0.smithi031.stdout:5/8: readlink - no filename 2024-04-24T23:13:29.874 INFO:tasks.workunit.client.0.smithi031.stdout:3/77: fdatasync d6/fb 0 2024-04-24T23:13:29.875 INFO:tasks.workunit.client.0.smithi031.stdout:7/43: dread d1/d2/f4 [0,4194304] 0 2024-04-24T23:13:29.875 INFO:tasks.workunit.client.0.smithi031.stdout:0/14: symlink l2 0 2024-04-24T23:13:29.875 INFO:tasks.workunit.client.0.smithi031.stdout:7/44: readlink d1/lb 0 2024-04-24T23:13:29.877 INFO:tasks.workunit.client.0.smithi031.stdout:5/9: mkdir d0/d2 0 2024-04-24T23:13:29.877 INFO:tasks.workunit.client.0.smithi031.stdout:5/10: dwrite - no filename 2024-04-24T23:13:29.878 INFO:tasks.workunit.client.0.smithi031.stdout:4/52: chown d3/f7 1 1 2024-04-24T23:13:29.879 INFO:tasks.workunit.client.0.smithi031.stdout:0/15: chown c1 1051617800 1 2024-04-24T23:13:29.879 INFO:tasks.workunit.client.0.smithi031.stdout:7/45: mkdir d1/d2/dd 0 2024-04-24T23:13:29.880 INFO:tasks.workunit.client.0.smithi031.stdout:4/53: chown d3/l5 570 1 2024-04-24T23:13:29.880 INFO:tasks.workunit.client.0.smithi031.stdout:6/108: readlink d13/d1b/l28 0 2024-04-24T23:13:29.881 INFO:tasks.workunit.client.0.smithi031.stdout:4/54: readlink l1 0 2024-04-24T23:13:29.883 INFO:tasks.workunit.client.0.smithi031.stdout:9/71: fsync d2/fd 0 2024-04-24T23:13:29.884 INFO:tasks.workunit.client.0.smithi031.stdout:2/20: symlink d5/l9 0 2024-04-24T23:13:29.889 INFO:tasks.workunit.client.0.smithi031.stdout:0/16: creat f3 x:0 0 0 2024-04-24T23:13:29.889 INFO:tasks.workunit.client.0.smithi031.stdout:0/17: read - f3 zero size 2024-04-24T23:13:29.890 INFO:tasks.workunit.client.0.smithi031.stdout:1/58: truncate d1/dc/fe 4666838 0 2024-04-24T23:13:29.890 INFO:tasks.workunit.client.0.smithi031.stdout:1/59: chown d1/dc/d11 2445770 1 2024-04-24T23:13:29.893 INFO:tasks.workunit.client.0.smithi031.stdout:7/46: write d1/d2/fa [768868,107793] 0 2024-04-24T23:13:29.897 INFO:tasks.workunit.client.0.smithi031.stdout:6/109: mkdir d13/d15/d1d/d23/d2d 0 2024-04-24T23:13:29.898 INFO:tasks.workunit.client.0.smithi031.stdout:7/47: dread d1/d2/f4 [0,4194304] 0 2024-04-24T23:13:29.899 INFO:tasks.workunit.client.0.smithi031.stdout:7/48: truncate d1/f9 2207269 0 2024-04-24T23:13:29.900 INFO:tasks.workunit.client.0.smithi031.stdout:7/49: chown d1/d2/f7 133541 1 2024-04-24T23:13:29.900 INFO:tasks.workunit.client.0.smithi031.stdout:7/50: stat d1/f9 0 2024-04-24T23:13:29.901 INFO:tasks.workunit.client.0.smithi031.stdout:0/18: symlink l4 0 2024-04-24T23:13:29.906 INFO:tasks.workunit.client.0.smithi031.stdout:6/110: mknod d13/d1b/c2e 0 2024-04-24T23:13:29.909 INFO:tasks.workunit.client.0.smithi031.stdout:0/19: dwrite f3 [0,4194304] 0 2024-04-24T23:13:29.909 INFO:tasks.workunit.client.0.smithi031.stdout:9/72: mkdir d2/d16 0 2024-04-24T23:13:29.910 INFO:tasks.workunit.client.0.smithi031.stdout:0/20: readlink l4 0 2024-04-24T23:13:29.911 INFO:tasks.workunit.client.0.smithi031.stdout:7/51: symlink d1/le 0 2024-04-24T23:13:29.911 INFO:tasks.workunit.client.0.smithi031.stdout:0/21: write f3 [233851,34196] 0 2024-04-24T23:13:29.914 INFO:tasks.workunit.client.0.smithi031.stdout:6/111: rename l8 to d13/d15/d1d/l2f 0 2024-04-24T23:13:29.915 INFO:tasks.workunit.client.0.smithi031.stdout:9/73: dread - d2/f12 zero size 2024-04-24T23:13:29.920 INFO:tasks.workunit.client.0.smithi031.stdout:7/52: creat d1/ff x:0 0 0 2024-04-24T23:13:29.921 INFO:tasks.workunit.client.0.smithi031.stdout:0/22: creat f5 x:0 0 0 2024-04-24T23:13:29.926 INFO:tasks.workunit.client.0.smithi031.stdout:9/74: dread d2/df/f10 [0,4194304] 0 2024-04-24T23:13:29.926 INFO:tasks.workunit.client.0.smithi031.stdout:7/53: rename d1/d2/f5 to d1/f10 0 2024-04-24T23:13:29.928 INFO:tasks.workunit.client.0.smithi031.stdout:0/23: creat f6 x:0 0 0 2024-04-24T23:13:29.930 INFO:tasks.workunit.client.0.smithi031.stdout:9/75: fdatasync d2/df/f10 0 2024-04-24T23:13:29.931 INFO:tasks.workunit.client.0.smithi031.stdout:7/54: creat d1/d2/f11 x:0 0 0 2024-04-24T23:13:29.933 INFO:tasks.workunit.client.0.smithi031.stdout:7/55: write d1/ff [51591,18590] 0 2024-04-24T23:13:29.934 INFO:tasks.workunit.client.0.smithi031.stdout:9/76: dread d2/fa [0,4194304] 0 2024-04-24T23:13:29.938 INFO:tasks.workunit.client.0.smithi031.stdout:7/56: creat d1/d2/f12 x:0 0 0 2024-04-24T23:13:29.942 INFO:tasks.workunit.client.0.smithi031.stdout:7/57: unlink d1/le 0 2024-04-24T23:13:29.945 INFO:tasks.workunit.client.0.smithi031.stdout:7/58: creat d1/f13 x:0 0 0 2024-04-24T23:13:29.952 INFO:tasks.workunit.client.0.smithi031.stdout:7/59: truncate d1/d2/f4 275438 0 2024-04-24T23:13:29.952 INFO:tasks.workunit.client.0.smithi031.stdout:7/60: chown d1 9531 1 2024-04-24T23:13:30.250 INFO:tasks.workunit.client.0.smithi031.stdout:5/11: sync 2024-04-24T23:13:30.254 INFO:tasks.workunit.client.0.smithi031.stdout:5/12: symlink d0/d1/l3 0 2024-04-24T23:13:30.254 INFO:tasks.workunit.client.0.smithi031.stdout:5/13: dread - no filename 2024-04-24T23:13:30.254 INFO:tasks.workunit.client.0.smithi031.stdout:5/14: dwrite - no filename 2024-04-24T23:13:30.260 INFO:tasks.workunit.client.0.smithi031.stdout:5/15: creat d0/d2/f4 x:0 0 0 2024-04-24T23:13:30.260 INFO:tasks.workunit.client.0.smithi031.stdout:5/16: chown d0/d2/f4 93 1 2024-04-24T23:13:30.264 INFO:tasks.workunit.client.0.smithi031.stdout:5/17: creat d0/d2/f5 x:0 0 0 2024-04-24T23:13:30.268 INFO:tasks.workunit.client.0.smithi031.stdout:5/18: mkdir d0/d6 0 2024-04-24T23:13:30.269 INFO:tasks.workunit.client.0.smithi031.stdout:5/19: write d0/d2/f5 [318460,120595] 0 2024-04-24T23:13:30.324 INFO:tasks.workunit.client.0.smithi031.stdout:2/21: sync 2024-04-24T23:13:30.330 INFO:tasks.workunit.client.0.smithi031.stdout:2/22: dread f4 [0,4194304] 0 2024-04-24T23:13:30.336 INFO:tasks.workunit.client.0.smithi031.stdout:3/78: dwrite f0 [4194304,4194304] 0 2024-04-24T23:13:30.343 INFO:tasks.workunit.client.0.smithi031.stdout:8/49: dwrite f3 [0,4194304] 0 2024-04-24T23:13:30.343 INFO:tasks.workunit.client.0.smithi031.stdout:3/79: chown d6/cd 91277762 1 2024-04-24T23:13:30.343 INFO:tasks.workunit.client.0.smithi031.stdout:3/80: fdatasync d6/fb 0 2024-04-24T23:13:30.352 INFO:tasks.workunit.client.0.smithi031.stdout:4/55: dwrite d3/f9 [0,4194304] 0 2024-04-24T23:13:30.358 INFO:tasks.workunit.client.0.smithi031.stdout:0/24: write f3 [4919567,21972] 0 2024-04-24T23:13:30.358 INFO:tasks.workunit.client.0.smithi031.stdout:1/60: dwrite d1/dc/d11/f13 [0,4194304] 0 2024-04-24T23:13:30.359 INFO:tasks.workunit.client.0.smithi031.stdout:0/25: chown f5 901941427 1 2024-04-24T23:13:30.371 INFO:tasks.workunit.client.0.smithi031.stdout:4/56: symlink d3/dd/le 0 2024-04-24T23:13:30.374 INFO:tasks.workunit.client.0.smithi031.stdout:0/26: symlink l7 0 2024-04-24T23:13:30.375 INFO:tasks.workunit.client.0.smithi031.stdout:0/27: dread - f6 zero size 2024-04-24T23:13:30.377 INFO:tasks.workunit.client.0.smithi031.stdout:6/112: sync 2024-04-24T23:13:30.377 INFO:tasks.workunit.client.0.smithi031.stdout:7/61: sync 2024-04-24T23:13:30.377 INFO:tasks.workunit.client.0.smithi031.stdout:9/77: write d2/f6 [3975376,65077] 0 2024-04-24T23:13:30.379 INFO:tasks.workunit.client.0.smithi031.stdout:0/28: mknod c8 0 2024-04-24T23:13:30.387 INFO:tasks.workunit.client.0.smithi031.stdout:6/113: truncate d13/d15/f27 850369 0 2024-04-24T23:13:30.391 INFO:tasks.workunit.client.0.smithi031.stdout:0/29: symlink l9 0 2024-04-24T23:13:30.392 INFO:tasks.workunit.client.0.smithi031.stdout:4/57: link l1 d3/dd/lf 0 2024-04-24T23:13:30.395 INFO:tasks.workunit.client.0.smithi031.stdout:7/62: symlink d1/d2/dd/l14 0 2024-04-24T23:13:30.396 INFO:tasks.workunit.client.0.smithi031.stdout:7/63: readlink d1/d2/dd/l14 0 2024-04-24T23:13:30.396 INFO:tasks.workunit.client.0.smithi031.stdout:4/58: dread d3/f9 [0,4194304] 0 2024-04-24T23:13:30.397 INFO:tasks.workunit.client.0.smithi031.stdout:9/78: getdents d2/df/d14 0 2024-04-24T23:13:30.401 INFO:tasks.workunit.client.0.smithi031.stdout:9/79: dread d2/f15 [0,4194304] 0 2024-04-24T23:13:30.401 INFO:tasks.workunit.client.0.smithi031.stdout:0/30: mkdir da 0 2024-04-24T23:13:30.411 INFO:tasks.workunit.client.0.smithi031.stdout:9/80: creat d2/df/f17 x:0 0 0 2024-04-24T23:13:30.419 INFO:tasks.workunit.client.0.smithi031.stdout:9/81: fdatasync d2/f12 0 2024-04-24T23:13:30.421 INFO:tasks.workunit.client.0.smithi031.stdout:0/31: creat da/fb x:0 0 0 2024-04-24T23:13:30.421 INFO:tasks.workunit.client.0.smithi031.stdout:0/32: rename da to da/dc 22 2024-04-24T23:13:30.424 INFO:tasks.workunit.client.0.smithi031.stdout:4/59: link d3/c8 d3/c10 0 2024-04-24T23:13:30.428 INFO:tasks.workunit.client.0.smithi031.stdout:0/33: creat da/fd x:0 0 0 2024-04-24T23:13:30.429 INFO:tasks.workunit.client.0.smithi031.stdout:7/64: getdents d1 0 2024-04-24T23:13:30.434 INFO:tasks.workunit.client.0.smithi031.stdout:0/34: creat da/fe x:0 0 0 2024-04-24T23:13:30.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/35: write f3 [2623511,97738] 0 2024-04-24T23:13:30.438 INFO:tasks.workunit.client.0.smithi031.stdout:0/36: creat da/ff x:0 0 0 2024-04-24T23:13:30.439 INFO:tasks.workunit.client.0.smithi031.stdout:0/37: truncate f6 479668 0 2024-04-24T23:13:30.440 INFO:tasks.workunit.client.0.smithi031.stdout:0/38: write f6 [562504,113212] 0 2024-04-24T23:13:30.442 INFO:tasks.workunit.client.0.smithi031.stdout:0/39: creat da/f10 x:0 0 0 2024-04-24T23:13:30.443 INFO:tasks.workunit.client.0.smithi031.stdout:0/40: truncate da/f10 386763 0 2024-04-24T23:13:30.445 INFO:tasks.workunit.client.0.smithi031.stdout:0/41: mkdir da/d11 0 2024-04-24T23:13:30.445 INFO:tasks.workunit.client.0.smithi031.stdout:0/42: fsync f5 0 2024-04-24T23:13:30.449 INFO:tasks.workunit.client.0.smithi031.stdout:0/43: dread f3 [4194304,4194304] 0 2024-04-24T23:13:30.450 INFO:tasks.workunit.client.0.smithi031.stdout:0/44: stat da/fb 0 2024-04-24T23:13:30.452 INFO:tasks.workunit.client.0.smithi031.stdout:0/45: creat da/d11/f12 x:0 0 0 2024-04-24T23:13:30.456 INFO:tasks.workunit.client.0.smithi031.stdout:0/46: link f6 da/f13 0 2024-04-24T23:13:30.457 INFO:tasks.workunit.client.0.smithi031.stdout:0/47: chown da/f13 254 1 2024-04-24T23:13:30.457 INFO:tasks.workunit.client.0.smithi031.stdout:0/48: write da/fd [177617,62571] 0 2024-04-24T23:13:30.465 INFO:tasks.workunit.client.0.smithi031.stdout:0/49: dwrite da/d11/f12 [0,4194304] 0 2024-04-24T23:13:30.465 INFO:tasks.workunit.client.0.smithi031.stdout:0/50: stat da/d11 0 2024-04-24T23:13:30.472 INFO:tasks.workunit.client.0.smithi031.stdout:5/20: rename d0/d1 to d0/d7 0 2024-04-24T23:13:30.473 INFO:tasks.workunit.client.0.smithi031.stdout:5/21: write d0/d2/f5 [176530,116809] 0 2024-04-24T23:13:30.474 INFO:tasks.workunit.client.0.smithi031.stdout:5/22: dread - d0/d2/f4 zero size 2024-04-24T23:13:30.476 INFO:tasks.workunit.client.0.smithi031.stdout:5/23: creat d0/d2/f8 x:0 0 0 2024-04-24T23:13:30.501 INFO:tasks.workunit.client.0.smithi031.stdout:1/61: symlink d1/da/l16 0 2024-04-24T23:13:30.501 INFO:tasks.workunit.client.0.smithi031.stdout:1/62: chown d1/f2 162 1 2024-04-24T23:13:30.520 INFO:tasks.workunit.client.0.smithi031.stdout:3/81: symlink d6/l19 0 2024-04-24T23:13:30.522 INFO:tasks.workunit.client.0.smithi031.stdout:3/82: fsync d6/fe 0 2024-04-24T23:13:30.525 INFO:tasks.workunit.client.0.smithi031.stdout:3/83: dread - d6/df/f17 zero size 2024-04-24T23:13:30.530 INFO:tasks.workunit.client.0.smithi031.stdout:2/23: unlink l3 0 2024-04-24T23:13:30.530 INFO:tasks.workunit.client.0.smithi031.stdout:2/24: rmdir d5 39 2024-04-24T23:13:30.532 INFO:tasks.workunit.client.0.smithi031.stdout:2/25: truncate f4 775084 0 2024-04-24T23:13:30.547 INFO:tasks.workunit.client.0.smithi031.stdout:8/50: dwrite f0 [0,4194304] 0 2024-04-24T23:13:30.549 INFO:tasks.workunit.client.0.smithi031.stdout:8/51: truncate d8/fc 672871 0 2024-04-24T23:13:30.564 INFO:tasks.workunit.client.0.smithi031.stdout:6/114: write f10 [2075822,93978] 0 2024-04-24T23:13:30.566 INFO:tasks.workunit.client.0.smithi031.stdout:6/115: dread - fd zero size 2024-04-24T23:13:30.571 INFO:tasks.workunit.client.0.smithi031.stdout:6/116: link d13/d1b/c2e d13/d15/d1d/d23/c30 0 2024-04-24T23:13:30.571 INFO:tasks.workunit.client.0.smithi031.stdout:6/117: stat f11 0 2024-04-24T23:13:30.575 INFO:tasks.workunit.client.0.smithi031.stdout:6/118: dread f9 [0,4194304] 0 2024-04-24T23:13:30.579 INFO:tasks.workunit.client.0.smithi031.stdout:9/82: write d2/fc [283574,60879] 0 2024-04-24T23:13:30.580 INFO:tasks.workunit.client.0.smithi031.stdout:4/60: write d3/fc [107782,100696] 0 2024-04-24T23:13:30.585 INFO:tasks.workunit.client.0.smithi031.stdout:9/83: mkdir d2/df/d14/d18 0 2024-04-24T23:13:30.588 INFO:tasks.workunit.client.0.smithi031.stdout:4/61: link d3/f9 d3/f11 0 2024-04-24T23:13:30.598 INFO:tasks.workunit.client.0.smithi031.stdout:5/24: getdents d0/d2 0 2024-04-24T23:13:30.599 INFO:tasks.workunit.client.0.smithi031.stdout:5/25: truncate d0/d2/f8 877698 0 2024-04-24T23:13:30.618 INFO:tasks.workunit.client.0.smithi031.stdout:5/26: rename d0/d2/f8 to d0/f9 0 2024-04-24T23:13:30.625 INFO:tasks.workunit.client.0.smithi031.stdout:5/27: dwrite d0/d2/f5 [0,4194304] 0 2024-04-24T23:13:30.627 INFO:tasks.workunit.client.0.smithi031.stdout:5/28: truncate d0/d2/f4 253508 0 2024-04-24T23:13:30.641 INFO:tasks.workunit.client.0.smithi031.stdout:3/84: write d6/fc [1605590,11798] 0 2024-04-24T23:13:30.646 INFO:tasks.workunit.client.0.smithi031.stdout:3/85: unlink d6/l19 0 2024-04-24T23:13:30.646 INFO:tasks.workunit.client.0.smithi031.stdout:3/86: chown d6/d14/c18 322220 1 2024-04-24T23:13:30.647 INFO:tasks.workunit.client.0.smithi031.stdout:1/63: creat d1/f17 x:0 0 0 2024-04-24T23:13:30.648 INFO:tasks.workunit.client.0.smithi031.stdout:1/64: write d1/f17 [669986,96372] 0 2024-04-24T23:13:30.651 INFO:tasks.workunit.client.0.smithi031.stdout:1/65: creat d1/f18 x:0 0 0 2024-04-24T23:13:30.655 INFO:tasks.workunit.client.0.smithi031.stdout:1/66: creat d1/da/d12/f19 x:0 0 0 2024-04-24T23:13:30.664 INFO:tasks.workunit.client.0.smithi031.stdout:6/119: write f11 [632046,111375] 0 2024-04-24T23:13:30.664 INFO:tasks.workunit.client.0.smithi031.stdout:6/120: chown d13/d15/d1d/d23 189835710 1 2024-04-24T23:13:30.668 INFO:tasks.workunit.client.0.smithi031.stdout:6/121: symlink d13/d15/d1d/d23/d2d/l31 0 2024-04-24T23:13:30.671 INFO:tasks.workunit.client.0.smithi031.stdout:4/62: write d3/fa [36708,5886] 0 2024-04-24T23:13:30.671 INFO:tasks.workunit.client.0.smithi031.stdout:6/122: rmdir d13/d1b 39 2024-04-24T23:13:30.673 INFO:tasks.workunit.client.0.smithi031.stdout:4/63: readlink l0 0 2024-04-24T23:13:30.674 INFO:tasks.workunit.client.0.smithi031.stdout:9/84: dwrite d2/df/f10 [0,4194304] 0 2024-04-24T23:13:30.675 INFO:tasks.workunit.client.0.smithi031.stdout:4/64: chown d3/dd/le 178483161 1 2024-04-24T23:13:30.678 INFO:tasks.workunit.client.0.smithi031.stdout:6/123: dread f10 [0,4194304] 0 2024-04-24T23:13:30.681 INFO:tasks.workunit.client.0.smithi031.stdout:9/85: dread d2/f6 [0,4194304] 0 2024-04-24T23:13:30.687 INFO:tasks.workunit.client.0.smithi031.stdout:6/124: mkdir d13/d1b/d32 0 2024-04-24T23:13:30.702 INFO:tasks.workunit.client.0.smithi031.stdout:3/87: dwrite d6/fe [0,4194304] 0 2024-04-24T23:13:30.708 INFO:tasks.workunit.client.0.smithi031.stdout:4/65: write d3/f9 [3551577,48475] 0 2024-04-24T23:13:30.708 INFO:tasks.workunit.client.0.smithi031.stdout:9/86: write d2/f8 [1346413,3220] 0 2024-04-24T23:13:30.710 INFO:tasks.workunit.client.0.smithi031.stdout:8/52: dwrite d8/fc [0,4194304] 0 2024-04-24T23:13:30.712 INFO:tasks.workunit.client.0.smithi031.stdout:8/53: chown d8/lf 2993030 1 2024-04-24T23:13:30.712 INFO:tasks.workunit.client.0.smithi031.stdout:1/67: dwrite d1/dc/fe [0,4194304] 0 2024-04-24T23:13:30.713 INFO:tasks.workunit.client.0.smithi031.stdout:6/125: write d13/f25 [205474,72353] 0 2024-04-24T23:13:30.714 INFO:tasks.workunit.client.0.smithi031.stdout:4/66: rmdir d3 39 2024-04-24T23:13:30.714 INFO:tasks.workunit.client.0.smithi031.stdout:1/68: dread - d1/f18 zero size 2024-04-24T23:13:30.715 INFO:tasks.workunit.client.0.smithi031.stdout:1/69: chown d1/da/d10 6709023 1 2024-04-24T23:13:30.715 INFO:tasks.workunit.client.0.smithi031.stdout:3/88: mkdir d6/d15/d1a 0 2024-04-24T23:13:30.718 INFO:tasks.workunit.client.0.smithi031.stdout:2/26: truncate f4 228452 0 2024-04-24T23:13:30.720 INFO:tasks.workunit.client.0.smithi031.stdout:9/87: creat d2/df/d14/f19 x:0 0 0 2024-04-24T23:13:30.721 INFO:tasks.workunit.client.0.smithi031.stdout:8/54: symlink d8/l11 0 2024-04-24T23:13:30.722 INFO:tasks.workunit.client.0.smithi031.stdout:3/89: write d6/fe [4923857,2725] 0 2024-04-24T23:13:30.725 INFO:tasks.workunit.client.0.smithi031.stdout:6/126: mknod d13/d15/d1d/d23/c33 0 2024-04-24T23:13:30.726 INFO:tasks.workunit.client.0.smithi031.stdout:4/67: dread d3/fc [0,4194304] 0 2024-04-24T23:13:30.728 INFO:tasks.workunit.client.0.smithi031.stdout:1/70: symlink d1/da/l1a 0 2024-04-24T23:13:30.732 INFO:tasks.workunit.client.0.smithi031.stdout:8/55: mknod d8/c12 0 2024-04-24T23:13:30.735 INFO:tasks.workunit.client.0.smithi031.stdout:4/68: symlink d3/dd/l12 0 2024-04-24T23:13:30.736 INFO:tasks.workunit.client.0.smithi031.stdout:1/71: mknod d1/dc/d11/c1b 0 2024-04-24T23:13:30.737 INFO:tasks.workunit.client.0.smithi031.stdout:2/27: fdatasync f4 0 2024-04-24T23:13:30.738 INFO:tasks.workunit.client.0.smithi031.stdout:9/88: mknod d2/df/d14/d18/c1a 0 2024-04-24T23:13:30.738 INFO:tasks.workunit.client.0.smithi031.stdout:9/89: readlink d2/lb 0 2024-04-24T23:13:30.739 INFO:tasks.workunit.client.0.smithi031.stdout:9/90: readlink d2/lb 0 2024-04-24T23:13:30.744 INFO:tasks.workunit.client.0.smithi031.stdout:6/127: mknod d13/c34 0 2024-04-24T23:13:30.745 INFO:tasks.workunit.client.0.smithi031.stdout:4/69: creat d3/dd/f13 x:0 0 0 2024-04-24T23:13:30.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:30 smithi031 ceph-mon[26411]: pgmap v18: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 393 KiB/s rd, 14 MiB/s wr, 258 op/s 2024-04-24T23:13:30.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:30 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:13:30.751 INFO:tasks.workunit.client.0.smithi031.stdout:3/90: link c4 d6/d15/c1b 0 2024-04-24T23:13:30.753 INFO:tasks.workunit.client.0.smithi031.stdout:6/128: dread - d13/d15/d1d/d23/f2b zero size 2024-04-24T23:13:30.754 INFO:tasks.workunit.client.0.smithi031.stdout:4/70: creat d3/f14 x:0 0 0 2024-04-24T23:13:30.755 INFO:tasks.workunit.client.0.smithi031.stdout:4/71: chown d3/dd/f13 21893154 1 2024-04-24T23:13:30.762 INFO:tasks.workunit.client.0.smithi031.stdout:3/91: dread d6/f8 [0,4194304] 0 2024-04-24T23:13:30.765 INFO:tasks.workunit.client.0.smithi031.stdout:3/92: dread d6/fb [0,4194304] 0 2024-04-24T23:13:30.767 INFO:tasks.workunit.client.0.smithi031.stdout:9/91: rename d2/df to d2/d1b 0 2024-04-24T23:13:30.767 INFO:tasks.workunit.client.0.smithi031.stdout:3/93: write f0 [2841971,67120] 0 2024-04-24T23:13:30.768 INFO:tasks.workunit.client.0.smithi031.stdout:8/56: getdents d8 0 2024-04-24T23:13:30.771 INFO:tasks.workunit.client.0.smithi031.stdout:1/72: dwrite d1/f3 [0,4194304] 0 2024-04-24T23:13:30.779 INFO:tasks.workunit.client.0.smithi031.stdout:6/129: link d13/d15/d1d/d23/f2b d13/d15/d1d/d23/f35 0 2024-04-24T23:13:30.781 INFO:tasks.workunit.client.0.smithi031.stdout:9/92: dread d2/f9 [0,4194304] 0 2024-04-24T23:13:30.781 INFO:tasks.workunit.client.0.smithi031.stdout:3/94: creat d6/df/d16/f1c x:0 0 0 2024-04-24T23:13:30.783 INFO:tasks.workunit.client.0.smithi031.stdout:4/72: getdents d3/dd 0 2024-04-24T23:13:30.783 INFO:tasks.workunit.client.0.smithi031.stdout:4/73: fdatasync d3/fa 0 2024-04-24T23:13:30.784 INFO:tasks.workunit.client.0.smithi031.stdout:6/130: mkdir d13/d15/d1d/d23/d2d/d36 0 2024-04-24T23:13:30.786 INFO:tasks.workunit.client.0.smithi031.stdout:7/65: sync 2024-04-24T23:13:30.788 INFO:tasks.workunit.client.0.smithi031.stdout:3/95: readlink d6/l12 0 2024-04-24T23:13:30.792 INFO:tasks.workunit.client.0.smithi031.stdout:8/57: write d8/fe [532029,29442] 0 2024-04-24T23:13:30.793 INFO:tasks.workunit.client.0.smithi031.stdout:4/74: symlink d3/l15 0 2024-04-24T23:13:30.795 INFO:tasks.workunit.client.0.smithi031.stdout:3/96: dwrite d6/fe [0,4194304] 0 2024-04-24T23:13:30.798 INFO:tasks.workunit.client.0.smithi031.stdout:1/73: getdents d1/dc 0 2024-04-24T23:13:30.831 INFO:tasks.workunit.client.0.smithi031.stdout:3/97: mkdir d6/d14/d1d 0 2024-04-24T23:13:30.831 INFO:tasks.workunit.client.0.smithi031.stdout:4/75: rmdir d3 39 2024-04-24T23:13:30.832 INFO:tasks.workunit.client.0.smithi031.stdout:9/93: creat d2/f1c x:0 0 0 2024-04-24T23:13:30.832 INFO:tasks.workunit.client.0.smithi031.stdout:1/74: creat d1/da/d10/f1c x:0 0 0 2024-04-24T23:13:30.835 INFO:tasks.workunit.client.0.smithi031.stdout:3/98: readlink d6/l11 0 2024-04-24T23:13:30.835 INFO:tasks.workunit.client.0.smithi031.stdout:8/58: link d8/lf d8/l13 0 2024-04-24T23:13:30.836 INFO:tasks.workunit.client.0.smithi031.stdout:9/94: mkdir d2/d1b/d14/d1d 0 2024-04-24T23:13:30.837 INFO:tasks.workunit.client.0.smithi031.stdout:1/75: write d1/da/d12/f19 [839580,123456] 0 2024-04-24T23:13:30.838 INFO:tasks.workunit.client.0.smithi031.stdout:4/76: creat d3/f16 x:0 0 0 2024-04-24T23:13:30.840 INFO:tasks.workunit.client.0.smithi031.stdout:8/59: creat d8/f14 x:0 0 0 2024-04-24T23:13:30.840 INFO:tasks.workunit.client.0.smithi031.stdout:6/131: dwrite d13/d1b/f1c [4194304,4194304] 0 2024-04-24T23:13:30.842 INFO:tasks.workunit.client.0.smithi031.stdout:1/76: chown d1/dc/d11/f13 2930750 1 2024-04-24T23:13:30.844 INFO:tasks.workunit.client.0.smithi031.stdout:1/77: fsync d1/f17 0 2024-04-24T23:13:30.849 INFO:tasks.workunit.client.0.smithi031.stdout:4/77: rename l0 to d3/l17 0 2024-04-24T23:13:30.851 INFO:tasks.workunit.client.0.smithi031.stdout:9/95: symlink d2/d16/l1e 0 2024-04-24T23:13:30.853 INFO:tasks.workunit.client.0.smithi031.stdout:3/99: dwrite d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:30.854 INFO:tasks.workunit.client.0.smithi031.stdout:9/96: readlink d2/d16/l1e 0 2024-04-24T23:13:30.855 INFO:tasks.workunit.client.0.smithi031.stdout:9/97: readlink d2/lb 0 2024-04-24T23:13:30.856 INFO:tasks.workunit.client.0.smithi031.stdout:6/132: fsync d13/d1b/f22 0 2024-04-24T23:13:30.857 INFO:tasks.workunit.client.0.smithi031.stdout:8/60: dwrite d8/fe [0,4194304] 0 2024-04-24T23:13:30.862 INFO:tasks.workunit.client.0.smithi031.stdout:1/78: truncate d1/f2 5104170 0 2024-04-24T23:13:30.864 INFO:tasks.workunit.client.0.smithi031.stdout:1/79: write d1/f18 [153470,84190] 0 2024-04-24T23:13:30.864 INFO:tasks.workunit.client.0.smithi031.stdout:1/80: chown d1/f17 2 1 2024-04-24T23:13:30.865 INFO:tasks.workunit.client.0.smithi031.stdout:3/100: rmdir d6 39 2024-04-24T23:13:30.870 INFO:tasks.workunit.client.0.smithi031.stdout:9/98: creat d2/d1b/f1f x:0 0 0 2024-04-24T23:13:30.873 INFO:tasks.workunit.client.0.smithi031.stdout:8/61: unlink d8/ca 0 2024-04-24T23:13:30.874 INFO:tasks.workunit.client.0.smithi031.stdout:8/62: chown l4 40 1 2024-04-24T23:13:30.877 INFO:tasks.workunit.client.0.smithi031.stdout:1/81: mkdir d1/d1d 0 2024-04-24T23:13:30.879 INFO:tasks.workunit.client.0.smithi031.stdout:4/78: chown d3/l17 44 1 2024-04-24T23:13:30.880 INFO:tasks.workunit.client.0.smithi031.stdout:9/99: fdatasync d2/f5 0 2024-04-24T23:13:30.886 INFO:tasks.workunit.client.0.smithi031.stdout:1/82: mknod d1/c1e 0 2024-04-24T23:13:30.889 INFO:tasks.workunit.client.0.smithi031.stdout:4/79: symlink d3/l18 0 2024-04-24T23:13:30.890 INFO:tasks.workunit.client.0.smithi031.stdout:9/100: dwrite d2/fd [0,4194304] 0 2024-04-24T23:13:30.895 INFO:tasks.workunit.client.0.smithi031.stdout:3/101: creat d6/d14/d1d/f1e x:0 0 0 2024-04-24T23:13:30.899 INFO:tasks.workunit.client.0.smithi031.stdout:6/133: dwrite d13/d1b/f22 [0,4194304] 0 2024-04-24T23:13:30.899 INFO:tasks.workunit.client.0.smithi031.stdout:3/102: dread - d6/d14/d1d/f1e zero size 2024-04-24T23:13:30.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/83: dwrite d1/dc/d11/f13 [4194304,4194304] 0 2024-04-24T23:13:30.906 INFO:tasks.workunit.client.0.smithi031.stdout:4/80: chown d3/c8 2 1 2024-04-24T23:13:30.908 INFO:tasks.workunit.client.0.smithi031.stdout:9/101: rename d2/f15 to d2/d16/f20 0 2024-04-24T23:13:30.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:30 smithi040 ceph-mon[33128]: pgmap v18: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 393 KiB/s rd, 14 MiB/s wr, 258 op/s 2024-04-24T23:13:30.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:30 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:13:30.914 INFO:tasks.workunit.client.0.smithi031.stdout:3/103: dwrite d6/d14/d1d/f1e [0,4194304] 0 2024-04-24T23:13:30.918 INFO:tasks.workunit.client.0.smithi031.stdout:1/84: symlink d1/d1d/l1f 0 2024-04-24T23:13:30.925 INFO:tasks.workunit.client.0.smithi031.stdout:8/63: dwrite d8/fe [4194304,4194304] 0 2024-04-24T23:13:30.927 INFO:tasks.workunit.client.0.smithi031.stdout:8/64: readlink l5 0 2024-04-24T23:13:30.928 INFO:tasks.workunit.client.0.smithi031.stdout:4/81: getdents d3 0 2024-04-24T23:13:30.930 INFO:tasks.workunit.client.0.smithi031.stdout:3/104: getdents d6/df 0 2024-04-24T23:13:30.931 INFO:tasks.workunit.client.0.smithi031.stdout:9/102: dwrite d2/d1b/d14/f19 [0,4194304] 0 2024-04-24T23:13:30.932 INFO:tasks.workunit.client.0.smithi031.stdout:8/65: dread - d8/f14 zero size 2024-04-24T23:13:30.934 INFO:tasks.workunit.client.0.smithi031.stdout:3/105: chown d6/fe 5 1 2024-04-24T23:13:30.934 INFO:tasks.workunit.client.0.smithi031.stdout:3/106: readlink d6/l11 0 2024-04-24T23:13:30.939 INFO:tasks.workunit.client.0.smithi031.stdout:4/82: symlink d3/l19 0 2024-04-24T23:13:30.942 INFO:tasks.workunit.client.0.smithi031.stdout:8/66: mkdir d8/d15 0 2024-04-24T23:13:30.944 INFO:tasks.workunit.client.0.smithi031.stdout:4/83: truncate d3/f14 38803 0 2024-04-24T23:13:30.946 INFO:tasks.workunit.client.0.smithi031.stdout:4/84: rename d3 to d3/dd/d1a 22 2024-04-24T23:13:30.948 INFO:tasks.workunit.client.0.smithi031.stdout:9/103: dwrite d2/d1b/f10 [0,4194304] 0 2024-04-24T23:13:30.948 INFO:tasks.workunit.client.0.smithi031.stdout:4/85: write d3/f11 [4580140,119740] 0 2024-04-24T23:13:30.949 INFO:tasks.workunit.client.0.smithi031.stdout:9/104: readlink d2/lb 0 2024-04-24T23:13:30.950 INFO:tasks.workunit.client.0.smithi031.stdout:8/67: dread f3 [0,4194304] 0 2024-04-24T23:13:30.952 INFO:tasks.workunit.client.0.smithi031.stdout:9/105: write d2/d1b/f17 [45448,96892] 0 2024-04-24T23:13:30.956 INFO:tasks.workunit.client.0.smithi031.stdout:9/106: chown d2/d1b/d14 14522 1 2024-04-24T23:13:30.961 INFO:tasks.workunit.client.0.smithi031.stdout:4/86: dwrite d3/fa [0,4194304] 0 2024-04-24T23:13:30.964 INFO:tasks.workunit.client.0.smithi031.stdout:9/107: dread d2/d1b/f13 [0,4194304] 0 2024-04-24T23:13:30.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/68: link d8/fd d8/d15/f16 0 2024-04-24T23:13:30.968 INFO:tasks.workunit.client.0.smithi031.stdout:3/107: write d6/df/f17 [700424,71864] 0 2024-04-24T23:13:30.968 INFO:tasks.workunit.client.0.smithi031.stdout:4/87: creat d3/f1b x:0 0 0 2024-04-24T23:13:30.972 INFO:tasks.workunit.client.0.smithi031.stdout:8/69: rename d8/c12 to d8/d15/c17 0 2024-04-24T23:13:30.974 INFO:tasks.workunit.client.0.smithi031.stdout:3/108: creat d6/d14/d1d/f1f x:0 0 0 2024-04-24T23:13:30.977 INFO:tasks.workunit.client.0.smithi031.stdout:8/70: dread d8/fc [0,4194304] 0 2024-04-24T23:13:30.977 INFO:tasks.workunit.client.0.smithi031.stdout:9/108: getdents d2/d1b/d14/d1d 0 2024-04-24T23:13:30.977 INFO:tasks.workunit.client.0.smithi031.stdout:9/109: stat d2/lb 0 2024-04-24T23:13:30.982 INFO:tasks.workunit.client.0.smithi031.stdout:9/110: rmdir d2/d16 39 2024-04-24T23:13:31.009 INFO:tasks.workunit.client.0.smithi031.stdout:9/111: rename d2/fd to d2/d1b/d14/d18/f21 0 2024-04-24T23:13:31.010 INFO:tasks.workunit.client.0.smithi031.stdout:3/109: write d6/f8 [1150546,92249] 0 2024-04-24T23:13:31.011 INFO:tasks.workunit.client.0.smithi031.stdout:3/110: fsync d6/d14/d1d/f1e 0 2024-04-24T23:13:31.012 INFO:tasks.workunit.client.0.smithi031.stdout:3/111: write d6/fc [951873,92458] 0 2024-04-24T23:13:31.013 INFO:tasks.workunit.client.0.smithi031.stdout:4/88: truncate d3/fa 29349 0 2024-04-24T23:13:31.016 INFO:tasks.workunit.client.0.smithi031.stdout:9/112: unlink d2/f3 0 2024-04-24T23:13:31.017 INFO:tasks.workunit.client.0.smithi031.stdout:9/113: dread d2/f5 [0,4194304] 0 2024-04-24T23:13:31.021 INFO:tasks.workunit.client.0.smithi031.stdout:4/89: mkdir d3/d1c 0 2024-04-24T23:13:31.027 INFO:tasks.workunit.client.0.smithi031.stdout:3/112: symlink d6/d15/d1a/l20 0 2024-04-24T23:13:31.027 INFO:tasks.workunit.client.0.smithi031.stdout:3/113: write d6/f8 [2380882,84350] 0 2024-04-24T23:13:31.028 INFO:tasks.workunit.client.0.smithi031.stdout:4/90: rename d3/c8 to d3/d1c/c1d 0 2024-04-24T23:13:31.029 INFO:tasks.workunit.client.0.smithi031.stdout:3/114: write d6/d14/d1d/f1e [641863,16248] 0 2024-04-24T23:13:31.033 INFO:tasks.workunit.client.0.smithi031.stdout:9/114: write d2/fe [245115,16993] 0 2024-04-24T23:13:31.034 INFO:tasks.workunit.client.0.smithi031.stdout:4/91: fsync d3/fa 0 2024-04-24T23:13:31.037 INFO:tasks.workunit.client.0.smithi031.stdout:3/115: symlink d6/d14/l21 0 2024-04-24T23:13:31.046 INFO:tasks.workunit.client.0.smithi031.stdout:9/115: write d2/f9 [689231,25017] 0 2024-04-24T23:13:31.047 INFO:tasks.workunit.client.0.smithi031.stdout:3/116: rename d6/fb to d6/d15/d1a/f22 0 2024-04-24T23:13:31.049 INFO:tasks.workunit.client.0.smithi031.stdout:4/92: write d3/fc [633730,109580] 0 2024-04-24T23:13:31.053 INFO:tasks.workunit.client.0.smithi031.stdout:4/93: mknod d3/d1c/c1e 0 2024-04-24T23:13:31.054 INFO:tasks.workunit.client.0.smithi031.stdout:9/116: dread - d2/d1b/f1f zero size 2024-04-24T23:13:31.055 INFO:tasks.workunit.client.0.smithi031.stdout:3/117: dwrite d6/df/f17 [0,4194304] 0 2024-04-24T23:13:31.056 INFO:tasks.workunit.client.0.smithi031.stdout:4/94: mknod d3/c1f 0 2024-04-24T23:13:31.065 INFO:tasks.workunit.client.0.smithi031.stdout:3/118: dread d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:31.065 INFO:tasks.workunit.client.0.smithi031.stdout:4/95: dwrite d3/f14 [0,4194304] 0 2024-04-24T23:13:31.068 INFO:tasks.workunit.client.0.smithi031.stdout:3/119: rmdir d6/d14 39 2024-04-24T23:13:31.070 INFO:tasks.workunit.client.0.smithi031.stdout:3/120: readlink d6/d15/d1a/l20 0 2024-04-24T23:13:31.073 INFO:tasks.workunit.client.0.smithi031.stdout:3/121: mknod d6/df/d16/c23 0 2024-04-24T23:13:31.074 INFO:tasks.workunit.client.0.smithi031.stdout:9/117: write d2/f5 [339906,10311] 0 2024-04-24T23:13:31.076 INFO:tasks.workunit.client.0.smithi031.stdout:4/96: dwrite d3/fc [0,4194304] 0 2024-04-24T23:13:31.081 INFO:tasks.workunit.client.0.smithi031.stdout:9/118: creat d2/d1b/d14/f22 x:0 0 0 2024-04-24T23:13:31.086 INFO:tasks.workunit.client.0.smithi031.stdout:9/119: rename d2/f6 to d2/d1b/d14/d1d/f23 0 2024-04-24T23:13:31.092 INFO:tasks.workunit.client.0.smithi031.stdout:4/97: write d3/f7 [3938724,67349] 0 2024-04-24T23:13:31.095 INFO:tasks.workunit.client.0.smithi031.stdout:9/120: truncate d2/d1b/d14/d18/f21 3019172 0 2024-04-24T23:13:31.099 INFO:tasks.workunit.client.0.smithi031.stdout:9/121: dread d2/d1b/f17 [0,4194304] 0 2024-04-24T23:13:31.103 INFO:tasks.workunit.client.0.smithi031.stdout:0/51: sync 2024-04-24T23:13:31.104 INFO:tasks.workunit.client.0.smithi031.stdout:0/52: rename da/d11 to da/d11/d14 22 2024-04-24T23:13:31.105 INFO:tasks.workunit.client.0.smithi031.stdout:0/53: fsync da/ff 0 2024-04-24T23:13:31.107 INFO:tasks.workunit.client.0.smithi031.stdout:9/122: dwrite d2/d1b/d14/f19 [0,4194304] 0 2024-04-24T23:13:31.111 INFO:tasks.workunit.client.0.smithi031.stdout:0/54: dread f3 [0,4194304] 0 2024-04-24T23:13:31.120 INFO:tasks.workunit.client.0.smithi031.stdout:0/55: dwrite f5 [0,4194304] 0 2024-04-24T23:13:31.123 INFO:tasks.workunit.client.0.smithi031.stdout:0/56: fdatasync f6 0 2024-04-24T23:13:31.126 INFO:tasks.workunit.client.0.smithi031.stdout:0/57: write f3 [1747848,86545] 0 2024-04-24T23:13:31.127 INFO:tasks.workunit.client.0.smithi031.stdout:9/123: dwrite d2/fa [0,4194304] 0 2024-04-24T23:13:31.130 INFO:tasks.workunit.client.0.smithi031.stdout:0/58: mknod da/d11/c15 0 2024-04-24T23:13:31.131 INFO:tasks.workunit.client.0.smithi031.stdout:0/59: write da/fb [334314,105665] 0 2024-04-24T23:13:31.133 INFO:tasks.workunit.client.0.smithi031.stdout:0/60: dread da/f10 [0,4194304] 0 2024-04-24T23:13:31.136 INFO:tasks.workunit.client.0.smithi031.stdout:0/61: symlink da/d11/l16 0 2024-04-24T23:13:31.137 INFO:tasks.workunit.client.0.smithi031.stdout:0/62: readlink l4 0 2024-04-24T23:13:31.141 INFO:tasks.workunit.client.0.smithi031.stdout:0/63: creat da/d11/f17 x:0 0 0 2024-04-24T23:13:31.149 INFO:tasks.workunit.client.0.smithi031.stdout:0/64: dwrite f5 [0,4194304] 0 2024-04-24T23:13:31.152 INFO:tasks.workunit.client.0.smithi031.stdout:0/65: mkdir da/d18 0 2024-04-24T23:13:31.155 INFO:tasks.workunit.client.0.smithi031.stdout:0/66: creat da/f19 x:0 0 0 2024-04-24T23:13:31.162 INFO:tasks.workunit.client.0.smithi031.stdout:0/67: dwrite da/fd [0,4194304] 0 2024-04-24T23:13:31.165 INFO:tasks.workunit.client.0.smithi031.stdout:0/68: mkdir da/d18/d1a 0 2024-04-24T23:13:31.165 INFO:tasks.workunit.client.0.smithi031.stdout:0/69: stat da/f19 0 2024-04-24T23:13:31.175 INFO:tasks.workunit.client.0.smithi031.stdout:0/70: dwrite da/f19 [0,4194304] 0 2024-04-24T23:13:31.175 INFO:tasks.workunit.client.0.smithi031.stdout:0/71: stat l7 0 2024-04-24T23:13:31.175 INFO:tasks.workunit.client.0.smithi031.stdout:0/72: chown da/d11 65011 1 2024-04-24T23:13:31.177 INFO:tasks.workunit.client.0.smithi031.stdout:0/73: truncate da/d11/f17 15430 0 2024-04-24T23:13:31.177 INFO:tasks.workunit.client.0.smithi031.stdout:0/74: chown da/d11/f12 1037610 1 2024-04-24T23:13:31.187 INFO:tasks.workunit.client.0.smithi031.stdout:0/75: write da/f13 [667006,74722] 0 2024-04-24T23:13:31.189 INFO:tasks.workunit.client.0.smithi031.stdout:0/76: truncate da/f10 27254 0 2024-04-24T23:13:31.190 INFO:tasks.workunit.client.0.smithi031.stdout:0/77: dread - da/ff zero size 2024-04-24T23:13:31.191 INFO:tasks.workunit.client.0.smithi031.stdout:5/29: sync 2024-04-24T23:13:31.192 INFO:tasks.workunit.client.0.smithi031.stdout:0/78: write da/d11/f17 [308211,35949] 0 2024-04-24T23:13:31.193 INFO:tasks.workunit.client.0.smithi031.stdout:5/30: mknod d0/d6/ca 0 2024-04-24T23:13:31.194 INFO:tasks.workunit.client.0.smithi031.stdout:0/79: mkdir da/d18/d1b 0 2024-04-24T23:13:31.197 INFO:tasks.workunit.client.0.smithi031.stdout:5/31: dread d0/f9 [0,4194304] 0 2024-04-24T23:13:31.199 INFO:tasks.workunit.client.0.smithi031.stdout:0/80: symlink da/d18/d1a/l1c 0 2024-04-24T23:13:31.203 INFO:tasks.workunit.client.0.smithi031.stdout:0/81: dread da/fb [0,4194304] 0 2024-04-24T23:13:31.206 INFO:tasks.workunit.client.0.smithi031.stdout:5/32: dwrite d0/d2/f4 [0,4194304] 0 2024-04-24T23:13:31.207 INFO:tasks.workunit.client.0.smithi031.stdout:0/82: mknod da/d18/d1b/c1d 0 2024-04-24T23:13:31.207 INFO:tasks.workunit.client.0.smithi031.stdout:5/33: read d0/d2/f4 [3008618,96861] 0 2024-04-24T23:13:31.222 INFO:tasks.workunit.client.0.smithi031.stdout:5/34: truncate d0/d2/f5 3885412 0 2024-04-24T23:13:31.228 INFO:tasks.workunit.client.0.smithi031.stdout:5/35: read d0/d2/f5 [3007525,23202] 0 2024-04-24T23:13:31.230 INFO:tasks.workunit.client.0.smithi031.stdout:5/36: write d0/d2/f4 [2675148,103] 0 2024-04-24T23:13:31.232 INFO:tasks.workunit.client.0.smithi031.stdout:5/37: write d0/f9 [177177,123977] 0 2024-04-24T23:13:31.233 INFO:tasks.workunit.client.0.smithi031.stdout:5/38: fsync d0/d2/f4 0 2024-04-24T23:13:31.560 INFO:tasks.workunit.client.0.smithi031.stdout:2/28: sync 2024-04-24T23:13:31.586 INFO:tasks.workunit.client.0.smithi031.stdout:7/66: sync 2024-04-24T23:13:31.588 INFO:tasks.workunit.client.0.smithi031.stdout:7/67: creat d1/d2/dd/f15 x:0 0 0 2024-04-24T23:13:31.636 INFO:tasks.workunit.client.0.smithi031.stdout:4/98: sync 2024-04-24T23:13:31.636 INFO:tasks.workunit.client.0.smithi031.stdout:1/85: sync 2024-04-24T23:13:31.636 INFO:tasks.workunit.client.0.smithi031.stdout:6/134: sync 2024-04-24T23:13:31.636 INFO:tasks.workunit.client.0.smithi031.stdout:5/39: sync 2024-04-24T23:13:31.636 INFO:tasks.workunit.client.0.smithi031.stdout:9/124: sync 2024-04-24T23:13:31.636 INFO:tasks.workunit.client.0.smithi031.stdout:0/83: sync 2024-04-24T23:13:31.636 INFO:tasks.workunit.client.0.smithi031.stdout:3/122: sync 2024-04-24T23:13:31.636 INFO:tasks.workunit.client.0.smithi031.stdout:8/71: sync 2024-04-24T23:13:31.638 INFO:tasks.workunit.client.0.smithi031.stdout:8/72: dread - d8/f14 zero size 2024-04-24T23:13:31.638 INFO:tasks.workunit.client.0.smithi031.stdout:5/40: write d0/f9 [215806,44459] 0 2024-04-24T23:13:31.639 INFO:tasks.workunit.client.0.smithi031.stdout:0/84: read f6 [501223,87069] 0 2024-04-24T23:13:31.640 INFO:tasks.workunit.client.0.smithi031.stdout:0/85: chown da/d11/c15 3874435 1 2024-04-24T23:13:31.644 INFO:tasks.workunit.client.0.smithi031.stdout:4/99: unlink d3/l19 0 2024-04-24T23:13:31.646 INFO:tasks.workunit.client.0.smithi031.stdout:8/73: rmdir d8/d15 39 2024-04-24T23:13:31.648 INFO:tasks.workunit.client.0.smithi031.stdout:6/135: truncate f12 2149600 0 2024-04-24T23:13:31.648 INFO:tasks.workunit.client.0.smithi031.stdout:9/125: dread d2/d1b/f10 [0,4194304] 0 2024-04-24T23:13:31.651 INFO:tasks.workunit.client.0.smithi031.stdout:5/41: mkdir d0/d7/db 0 2024-04-24T23:13:31.654 INFO:tasks.workunit.client.0.smithi031.stdout:3/123: creat d6/d14/d1d/f24 x:0 0 0 2024-04-24T23:13:31.656 INFO:tasks.workunit.client.0.smithi031.stdout:4/100: mkdir d3/d20 0 2024-04-24T23:13:31.659 INFO:tasks.workunit.client.0.smithi031.stdout:8/74: dread d8/fc [0,4194304] 0 2024-04-24T23:13:31.660 INFO:tasks.workunit.client.0.smithi031.stdout:1/86: link d1/f17 d1/dc/d11/f20 0 2024-04-24T23:13:31.661 INFO:tasks.workunit.client.0.smithi031.stdout:1/87: fsync d1/da/d10/f1c 0 2024-04-24T23:13:31.661 INFO:tasks.workunit.client.0.smithi031.stdout:6/136: creat d13/d15/d1d/d23/d24/f37 x:0 0 0 2024-04-24T23:13:31.662 INFO:tasks.workunit.client.0.smithi031.stdout:6/137: stat d13/d1b/l28 0 2024-04-24T23:13:31.664 INFO:tasks.workunit.client.0.smithi031.stdout:9/126: write d2/d1b/f10 [1693332,130288] 0 2024-04-24T23:13:31.665 INFO:tasks.workunit.client.0.smithi031.stdout:9/127: chown d2/f1c 121 1 2024-04-24T23:13:31.665 INFO:tasks.workunit.client.0.smithi031.stdout:3/124: mkdir d6/d15/d1a/d25 0 2024-04-24T23:13:31.668 INFO:tasks.workunit.client.0.smithi031.stdout:4/101: symlink d3/d1c/l21 0 2024-04-24T23:13:31.669 INFO:tasks.workunit.client.0.smithi031.stdout:8/75: read - d8/fd zero size 2024-04-24T23:13:31.670 INFO:tasks.workunit.client.0.smithi031.stdout:1/88: mkdir d1/dc/d11/d21 0 2024-04-24T23:13:31.671 INFO:tasks.workunit.client.0.smithi031.stdout:6/138: dwrite d13/f25 [0,4194304] 0 2024-04-24T23:13:31.674 INFO:tasks.workunit.client.0.smithi031.stdout:5/42: mkdir d0/d7/db/dc 0 2024-04-24T23:13:31.676 INFO:tasks.workunit.client.0.smithi031.stdout:5/43: chown d0/d7/db 2822 1 2024-04-24T23:13:31.676 INFO:tasks.workunit.client.0.smithi031.stdout:3/125: mkdir d6/d15/d26 0 2024-04-24T23:13:31.677 INFO:tasks.workunit.client.0.smithi031.stdout:8/76: dread d8/fb [0,4194304] 0 2024-04-24T23:13:31.679 INFO:tasks.workunit.client.0.smithi031.stdout:9/128: creat d2/d1b/d14/d1d/f24 x:0 0 0 2024-04-24T23:13:31.682 INFO:tasks.workunit.client.0.smithi031.stdout:4/102: rename d3/l15 to d3/d20/l22 0 2024-04-24T23:13:31.683 INFO:tasks.workunit.client.0.smithi031.stdout:1/89: rmdir d1/d1d 39 2024-04-24T23:13:31.687 INFO:tasks.workunit.client.0.smithi031.stdout:4/103: write d3/f14 [1469425,35143] 0 2024-04-24T23:13:31.687 INFO:tasks.workunit.client.0.smithi031.stdout:1/90: stat d1/dc/d11/d21 0 2024-04-24T23:13:31.688 INFO:tasks.workunit.client.0.smithi031.stdout:3/126: mknod d6/d14/c27 0 2024-04-24T23:13:31.690 INFO:tasks.workunit.client.0.smithi031.stdout:8/77: dwrite d8/f14 [0,4194304] 0 2024-04-24T23:13:31.690 INFO:tasks.workunit.client.0.smithi031.stdout:3/127: fsync f0 0 2024-04-24T23:13:31.691 INFO:tasks.workunit.client.0.smithi031.stdout:5/44: dwrite d0/d2/f4 [0,4194304] 0 2024-04-24T23:13:31.693 INFO:tasks.workunit.client.0.smithi031.stdout:5/45: rename d0 to d0/d7/db/dc/dd 22 2024-04-24T23:13:31.703 INFO:tasks.workunit.client.0.smithi031.stdout:4/104: mkdir d3/d23 0 2024-04-24T23:13:31.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/46: dread d0/d2/f4 [0,4194304] 0 2024-04-24T23:13:31.703 INFO:tasks.workunit.client.0.smithi031.stdout:3/128: dwrite d6/d14/d1d/f1f [0,4194304] 0 2024-04-24T23:13:31.713 INFO:tasks.workunit.client.0.smithi031.stdout:3/129: dread f0 [4194304,4194304] 0 2024-04-24T23:13:31.714 INFO:tasks.workunit.client.0.smithi031.stdout:5/47: dwrite d0/f9 [0,4194304] 0 2024-04-24T23:13:31.719 INFO:tasks.workunit.client.0.smithi031.stdout:3/130: write d6/d14/d1d/f24 [69645,18369] 0 2024-04-24T23:13:31.720 INFO:tasks.workunit.client.0.smithi031.stdout:3/131: fsync f0 0 2024-04-24T23:13:31.721 INFO:tasks.workunit.client.0.smithi031.stdout:6/139: getdents d13/d15/d1d/d23 0 2024-04-24T23:13:31.721 INFO:tasks.workunit.client.0.smithi031.stdout:6/140: fsync f11 0 2024-04-24T23:13:31.723 INFO:tasks.workunit.client.0.smithi031.stdout:9/129: link d2/lb d2/d1b/l25 0 2024-04-24T23:13:31.729 INFO:tasks.workunit.client.0.smithi031.stdout:8/78: creat d8/f18 x:0 0 0 2024-04-24T23:13:31.735 INFO:tasks.workunit.client.0.smithi031.stdout:6/141: truncate d13/d15/f1f 678548 0 2024-04-24T23:13:31.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/91: getdents d1/da/d10 0 2024-04-24T23:13:31.739 INFO:tasks.workunit.client.0.smithi031.stdout:3/132: mkdir d6/d15/d1a/d25/d28 0 2024-04-24T23:13:31.740 INFO:tasks.workunit.client.0.smithi031.stdout:9/130: symlink d2/d16/l26 0 2024-04-24T23:13:31.745 INFO:tasks.workunit.client.0.smithi031.stdout:1/92: write d1/f17 [1673235,127013] 0 2024-04-24T23:13:31.746 INFO:tasks.workunit.client.0.smithi031.stdout:9/131: creat d2/d1b/d14/d18/f27 x:0 0 0 2024-04-24T23:13:31.749 INFO:tasks.workunit.client.0.smithi031.stdout:6/142: rename d13/d15/d1d/d23/c30 to d13/d15/d1d/d23/d24/c38 0 2024-04-24T23:13:31.749 INFO:tasks.workunit.client.0.smithi031.stdout:6/143: chown d13/d1b/l28 2696854 1 2024-04-24T23:13:31.757 INFO:tasks.workunit.client.0.smithi031.stdout:6/144: creat d13/d15/d1d/d23/d2d/d36/f39 x:0 0 0 2024-04-24T23:13:31.759 INFO:tasks.workunit.client.0.smithi031.stdout:6/145: symlink d13/d15/l3a 0 2024-04-24T23:13:31.762 INFO:tasks.workunit.client.0.smithi031.stdout:6/146: mkdir d13/d1b/d3b 0 2024-04-24T23:13:31.763 INFO:tasks.workunit.client.0.smithi031.stdout:6/147: creat d13/d1b/f3c x:0 0 0 2024-04-24T23:13:31.769 INFO:tasks.workunit.client.0.smithi031.stdout:6/148: dwrite f11 [0,4194304] 0 2024-04-24T23:13:31.770 INFO:tasks.workunit.client.0.smithi031.stdout:6/149: chown d13/d1b/l28 189 1 2024-04-24T23:13:31.812 INFO:tasks.workunit.client.0.smithi031.stdout:2/29: write f4 [535224,17381] 0 2024-04-24T23:13:31.817 INFO:tasks.workunit.client.0.smithi031.stdout:7/68: write d1/f10 [755677,91169] 0 2024-04-24T23:13:31.819 INFO:tasks.workunit.client.0.smithi031.stdout:7/69: symlink d1/l16 0 2024-04-24T23:13:31.826 INFO:tasks.workunit.client.0.smithi031.stdout:7/70: dwrite d1/f13 [0,4194304] 0 2024-04-24T23:13:31.827 INFO:tasks.workunit.client.0.smithi031.stdout:7/71: chown d1/d2/dd/l14 1436 1 2024-04-24T23:13:31.828 INFO:tasks.workunit.client.0.smithi031.stdout:7/72: fdatasync d1/d2/dd/f15 0 2024-04-24T23:13:31.833 INFO:tasks.workunit.client.0.smithi031.stdout:7/73: dread d1/f13 [0,4194304] 0 2024-04-24T23:13:31.836 INFO:tasks.workunit.client.0.smithi031.stdout:7/74: creat d1/d2/dd/f17 x:0 0 0 2024-04-24T23:13:31.837 INFO:tasks.workunit.client.0.smithi031.stdout:7/75: write d1/f10 [1605883,84825] 0 2024-04-24T23:13:31.838 INFO:tasks.workunit.client.0.smithi031.stdout:0/86: truncate f6 66386 0 2024-04-24T23:13:31.838 INFO:tasks.workunit.client.0.smithi031.stdout:0/87: chown da/d18/d1a 0 1 2024-04-24T23:13:31.842 INFO:tasks.workunit.client.0.smithi031.stdout:0/88: read da/f10 [25603,7470] 0 2024-04-24T23:13:31.843 INFO:tasks.workunit.client.0.smithi031.stdout:0/89: chown da/f19 49 1 2024-04-24T23:13:31.844 INFO:tasks.workunit.client.0.smithi031.stdout:0/90: truncate da/f19 5233634 0 2024-04-24T23:13:31.844 INFO:tasks.workunit.client.0.smithi031.stdout:0/91: chown c8 40244 1 2024-04-24T23:13:31.844 INFO:tasks.workunit.client.0.smithi031.stdout:0/92: chown l9 18 1 2024-04-24T23:13:31.847 INFO:tasks.workunit.client.0.smithi031.stdout:0/93: creat da/d18/d1a/f1e x:0 0 0 2024-04-24T23:13:31.849 INFO:tasks.workunit.client.0.smithi031.stdout:0/94: creat da/d18/d1a/f1f x:0 0 0 2024-04-24T23:13:31.850 INFO:tasks.workunit.client.0.smithi031.stdout:0/95: write da/fd [953467,59028] 0 2024-04-24T23:13:31.852 INFO:tasks.workunit.client.0.smithi031.stdout:0/96: rmdir da/d18 39 2024-04-24T23:13:31.866 INFO:tasks.workunit.client.0.smithi031.stdout:4/105: truncate d3/f7 1258753 0 2024-04-24T23:13:31.867 INFO:tasks.workunit.client.0.smithi031.stdout:8/79: fsync d8/f18 0 2024-04-24T23:13:31.868 INFO:tasks.workunit.client.0.smithi031.stdout:4/106: readlink d3/d20/l22 0 2024-04-24T23:13:31.871 INFO:tasks.workunit.client.0.smithi031.stdout:5/48: dwrite d0/d2/f5 [0,4194304] 0 2024-04-24T23:13:31.876 INFO:tasks.workunit.client.0.smithi031.stdout:8/80: dwrite d8/f14 [0,4194304] 0 2024-04-24T23:13:31.876 INFO:tasks.workunit.client.0.smithi031.stdout:8/81: dread - d8/f18 zero size 2024-04-24T23:13:31.892 INFO:tasks.workunit.client.0.smithi031.stdout:5/49: dwrite d0/d2/f4 [0,4194304] 0 2024-04-24T23:13:31.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/82: mknod d8/c19 0 2024-04-24T23:13:31.966 INFO:tasks.workunit.client.0.smithi031.stdout:5/50: mknod d0/d6/ce 0 2024-04-24T23:13:31.967 INFO:tasks.workunit.client.0.smithi031.stdout:9/132: truncate d2/d1b/d14/f19 1114445 0 2024-04-24T23:13:31.969 INFO:tasks.workunit.client.0.smithi031.stdout:1/93: dwrite d1/f8 [0,4194304] 0 2024-04-24T23:13:31.971 INFO:tasks.workunit.client.0.smithi031.stdout:5/51: symlink d0/d2/lf 0 2024-04-24T23:13:31.974 INFO:tasks.workunit.client.0.smithi031.stdout:9/133: unlink d2/fa 0 2024-04-24T23:13:31.974 INFO:tasks.workunit.client.0.smithi031.stdout:9/134: dread - d2/d1b/d14/f22 zero size 2024-04-24T23:13:31.975 INFO:tasks.workunit.client.0.smithi031.stdout:9/135: write d2/fe [722920,57540] 0 2024-04-24T23:13:31.976 INFO:tasks.workunit.client.0.smithi031.stdout:8/83: mkdir d8/d1a 0 2024-04-24T23:13:31.986 INFO:tasks.workunit.client.0.smithi031.stdout:9/136: dread d2/d1b/f13 [0,4194304] 0 2024-04-24T23:13:31.987 INFO:tasks.workunit.client.0.smithi031.stdout:8/84: mkdir d8/d1b 0 2024-04-24T23:13:31.988 INFO:tasks.workunit.client.0.smithi031.stdout:9/137: fdatasync d2/d1b/f13 0 2024-04-24T23:13:31.990 INFO:tasks.workunit.client.0.smithi031.stdout:9/138: chown d2/d1b/d14/d18/c1a 3356148 1 2024-04-24T23:13:31.995 INFO:tasks.workunit.client.0.smithi031.stdout:8/85: dwrite d8/fe [8388608,4194304] 0 2024-04-24T23:13:32.001 INFO:tasks.workunit.client.0.smithi031.stdout:7/76: write d1/f13 [4089242,124862] 0 2024-04-24T23:13:32.002 INFO:tasks.workunit.client.0.smithi031.stdout:9/139: dwrite d2/d1b/f1f [0,4194304] 0 2024-04-24T23:13:32.003 INFO:tasks.workunit.client.0.smithi031.stdout:7/77: write d1/d2/f12 [803806,7787] 0 2024-04-24T23:13:32.004 INFO:tasks.workunit.client.0.smithi031.stdout:7/78: write d1/d2/f11 [590859,79740] 0 2024-04-24T23:13:32.008 INFO:tasks.workunit.client.0.smithi031.stdout:9/140: creat d2/d1b/d14/d1d/f28 x:0 0 0 2024-04-24T23:13:32.010 INFO:tasks.workunit.client.0.smithi031.stdout:6/150: write fc [717590,24665] 0 2024-04-24T23:13:32.013 INFO:tasks.workunit.client.0.smithi031.stdout:6/151: fsync d13/d15/f16 0 2024-04-24T23:13:32.013 INFO:tasks.workunit.client.0.smithi031.stdout:9/141: truncate d2/d1b/d14/f22 943066 0 2024-04-24T23:13:32.015 INFO:tasks.workunit.client.0.smithi031.stdout:7/79: dwrite d1/d2/f11 [0,4194304] 0 2024-04-24T23:13:32.017 INFO:tasks.workunit.client.0.smithi031.stdout:7/80: chown d1/l6 7 1 2024-04-24T23:13:32.026 INFO:tasks.workunit.client.0.smithi031.stdout:7/81: mknod d1/c18 0 2024-04-24T23:13:32.028 INFO:tasks.workunit.client.0.smithi031.stdout:6/152: dwrite fc [0,4194304] 0 2024-04-24T23:13:32.030 INFO:tasks.workunit.client.0.smithi031.stdout:9/142: dwrite d2/f9 [0,4194304] 0 2024-04-24T23:13:32.033 INFO:tasks.workunit.client.0.smithi031.stdout:9/143: chown d2/fe 623 1 2024-04-24T23:13:32.039 INFO:tasks.workunit.client.0.smithi031.stdout:9/144: fdatasync d2/f12 0 2024-04-24T23:13:32.041 INFO:tasks.workunit.client.0.smithi031.stdout:7/82: dwrite d1/ff [0,4194304] 0 2024-04-24T23:13:32.044 INFO:tasks.workunit.client.0.smithi031.stdout:4/107: truncate d3/f9 3373600 0 2024-04-24T23:13:32.046 INFO:tasks.workunit.client.0.smithi031.stdout:9/145: dread d2/fc [0,4194304] 0 2024-04-24T23:13:32.047 INFO:tasks.workunit.client.0.smithi031.stdout:9/146: dread - d2/d1b/d14/d1d/f28 zero size 2024-04-24T23:13:32.048 INFO:tasks.workunit.client.0.smithi031.stdout:9/147: read d2/d1b/f13 [3429392,45397] 0 2024-04-24T23:13:32.049 INFO:tasks.workunit.client.0.smithi031.stdout:4/108: symlink d3/d23/l24 0 2024-04-24T23:13:32.050 INFO:tasks.workunit.client.0.smithi031.stdout:9/148: truncate d2/f5 1173151 0 2024-04-24T23:13:32.053 INFO:tasks.workunit.client.0.smithi031.stdout:4/109: mkdir d3/d20/d25 0 2024-04-24T23:13:32.055 INFO:tasks.workunit.client.0.smithi031.stdout:4/110: truncate d3/dd/f13 397595 0 2024-04-24T23:13:32.059 INFO:tasks.workunit.client.0.smithi031.stdout:4/111: truncate d3/fb 639112 0 2024-04-24T23:13:32.070 INFO:tasks.workunit.client.0.smithi031.stdout:0/97: write da/f13 [388242,73080] 0 2024-04-24T23:13:32.074 INFO:tasks.workunit.client.0.smithi031.stdout:8/86: dwrite d8/fd [0,4194304] 0 2024-04-24T23:13:32.074 INFO:tasks.workunit.client.0.smithi031.stdout:0/98: chown da/d18/d1a/f1e 2 1 2024-04-24T23:13:32.080 INFO:tasks.workunit.client.0.smithi031.stdout:8/87: dread d8/d15/f16 [0,4194304] 0 2024-04-24T23:13:32.082 INFO:tasks.workunit.client.0.smithi031.stdout:0/99: mkdir da/d20 0 2024-04-24T23:13:32.083 INFO:tasks.workunit.client.0.smithi031.stdout:0/100: dread - da/fe zero size 2024-04-24T23:13:32.088 INFO:tasks.workunit.client.0.smithi031.stdout:6/153: dwrite f9 [0,4194304] 0 2024-04-24T23:13:32.092 INFO:tasks.workunit.client.0.smithi031.stdout:0/101: symlink da/l21 0 2024-04-24T23:13:32.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/112: dwrite d3/f7 [0,4194304] 0 2024-04-24T23:13:32.097 INFO:tasks.workunit.client.0.smithi031.stdout:8/88: fdatasync d8/fd 0 2024-04-24T23:13:32.098 INFO:tasks.workunit.client.0.smithi031.stdout:4/113: readlink d3/dd/le 0 2024-04-24T23:13:32.103 INFO:tasks.workunit.client.0.smithi031.stdout:0/102: mkdir da/d22 0 2024-04-24T23:13:32.105 INFO:tasks.workunit.client.0.smithi031.stdout:9/149: write d2/d1b/d14/d18/f21 [1224466,34441] 0 2024-04-24T23:13:32.110 INFO:tasks.workunit.client.0.smithi031.stdout:0/103: creat da/d18/d1b/f23 x:0 0 0 2024-04-24T23:13:32.111 INFO:tasks.workunit.client.0.smithi031.stdout:0/104: chown da/d18/d1b/f23 9 1 2024-04-24T23:13:32.112 INFO:tasks.workunit.client.0.smithi031.stdout:0/105: write da/fd [5010430,3696] 0 2024-04-24T23:13:32.113 INFO:tasks.workunit.client.0.smithi031.stdout:9/150: symlink d2/d1b/d14/l29 0 2024-04-24T23:13:32.116 INFO:tasks.workunit.client.0.smithi031.stdout:0/106: dread f6 [0,4194304] 0 2024-04-24T23:13:32.116 INFO:tasks.workunit.client.0.smithi031.stdout:0/107: stat c8 0 2024-04-24T23:13:32.116 INFO:tasks.workunit.client.0.smithi031.stdout:8/89: mkdir d8/d1a/d1c 0 2024-04-24T23:13:32.117 INFO:tasks.workunit.client.0.smithi031.stdout:0/108: truncate da/d18/d1a/f1e 464399 0 2024-04-24T23:13:32.120 INFO:tasks.workunit.client.0.smithi031.stdout:9/151: stat d2/lb 0 2024-04-24T23:13:32.121 INFO:tasks.workunit.client.0.smithi031.stdout:9/152: dread - d2/d1b/d14/d1d/f28 zero size 2024-04-24T23:13:32.122 INFO:tasks.workunit.client.0.smithi031.stdout:8/90: rename d8/d1a/d1c to d8/d1a/d1d 0 2024-04-24T23:13:32.127 INFO:tasks.workunit.client.0.smithi031.stdout:6/154: write d13/d15/d1d/d23/f35 [734094,109797] 0 2024-04-24T23:13:32.130 INFO:tasks.workunit.client.0.smithi031.stdout:9/153: dwrite d2/d1b/d14/d18/f27 [0,4194304] 0 2024-04-24T23:13:32.130 INFO:tasks.workunit.client.0.smithi031.stdout:6/155: write d13/d15/d1d/d23/f2b [1028525,81253] 0 2024-04-24T23:13:32.131 INFO:tasks.workunit.client.0.smithi031.stdout:4/114: fsync d3/f11 0 2024-04-24T23:13:32.135 INFO:tasks.workunit.client.0.smithi031.stdout:0/109: symlink da/d20/l24 0 2024-04-24T23:13:32.135 INFO:tasks.workunit.client.0.smithi031.stdout:6/156: dread fc [0,4194304] 0 2024-04-24T23:13:32.140 INFO:tasks.workunit.client.0.smithi031.stdout:8/91: rename d8/l13 to d8/d1b/l1e 0 2024-04-24T23:13:32.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/133: sync 2024-04-24T23:13:32.145 INFO:tasks.workunit.client.0.smithi031.stdout:9/154: creat d2/d16/f2a x:0 0 0 2024-04-24T23:13:32.146 INFO:tasks.workunit.client.0.smithi031.stdout:0/110: dwrite da/d11/f12 [0,4194304] 0 2024-04-24T23:13:32.147 INFO:tasks.workunit.client.0.smithi031.stdout:4/115: rmdir d3/d1c 39 2024-04-24T23:13:32.148 INFO:tasks.workunit.client.0.smithi031.stdout:6/157: dwrite d13/d1b/f22 [4194304,4194304] 0 2024-04-24T23:13:32.149 INFO:tasks.workunit.client.0.smithi031.stdout:4/116: write d3/f16 [709404,10299] 0 2024-04-24T23:13:32.152 INFO:tasks.workunit.client.0.smithi031.stdout:8/92: dread d8/fe [8388608,4194304] 0 2024-04-24T23:13:32.153 INFO:tasks.workunit.client.0.smithi031.stdout:4/117: fdatasync d3/f7 0 2024-04-24T23:13:32.158 INFO:tasks.workunit.client.0.smithi031.stdout:8/93: dread d8/fd [0,4194304] 0 2024-04-24T23:13:32.163 INFO:tasks.workunit.client.0.smithi031.stdout:3/134: mkdir d6/d15/d29 0 2024-04-24T23:13:32.171 INFO:tasks.workunit.client.0.smithi031.stdout:3/135: dwrite d6/d14/d1d/f1e [0,4194304] 0 2024-04-24T23:13:32.175 INFO:tasks.workunit.client.0.smithi031.stdout:6/158: rename d13/d15/d1d/d23/d24/f37 to d13/d15/d1d/d23/d2d/f3d 0 2024-04-24T23:13:32.176 INFO:tasks.workunit.client.0.smithi031.stdout:9/155: dwrite d2/d1b/f17 [0,4194304] 0 2024-04-24T23:13:32.178 INFO:tasks.workunit.client.0.smithi031.stdout:8/94: mknod d8/d1a/c1f 0 2024-04-24T23:13:32.178 INFO:tasks.workunit.client.0.smithi031.stdout:8/95: dread - d8/f18 zero size 2024-04-24T23:13:32.189 INFO:tasks.workunit.client.0.smithi031.stdout:3/136: rename d6/d15/d26 to d6/df/d16/d2a 0 2024-04-24T23:13:32.193 INFO:tasks.workunit.client.0.smithi031.stdout:3/137: chown d6/d14/d1d 1078 1 2024-04-24T23:13:32.193 INFO:tasks.workunit.client.0.smithi031.stdout:9/156: read d2/d1b/d14/d1d/f23 [3593610,57460] 0 2024-04-24T23:13:32.194 INFO:tasks.workunit.client.0.smithi031.stdout:3/138: read d6/d14/d1d/f1f [171703,91445] 0 2024-04-24T23:13:32.194 INFO:tasks.workunit.client.0.smithi031.stdout:4/118: creat d3/f26 x:0 0 0 2024-04-24T23:13:32.195 INFO:tasks.workunit.client.0.smithi031.stdout:3/139: fdatasync d6/d14/d1d/f1e 0 2024-04-24T23:13:32.195 INFO:tasks.workunit.client.0.smithi031.stdout:6/159: rename d13/d1b/d3b to d13/d15/d1d/d23/d3e 0 2024-04-24T23:13:32.195 INFO:tasks.workunit.client.0.smithi031.stdout:3/140: stat d6/d15/d1a/f22 0 2024-04-24T23:13:32.196 INFO:tasks.workunit.client.0.smithi031.stdout:9/157: creat d2/d16/f2b x:0 0 0 2024-04-24T23:13:32.196 INFO:tasks.workunit.client.0.smithi031.stdout:6/160: chown d13/d15/l19 18209128 1 2024-04-24T23:13:32.200 INFO:tasks.workunit.client.0.smithi031.stdout:8/96: symlink d8/l20 0 2024-04-24T23:13:32.203 INFO:tasks.workunit.client.0.smithi031.stdout:4/119: mknod d3/d20/c27 0 2024-04-24T23:13:32.208 INFO:tasks.workunit.client.0.smithi031.stdout:9/158: symlink d2/d1b/d14/d18/l2c 0 2024-04-24T23:13:32.210 INFO:tasks.workunit.client.0.smithi031.stdout:4/120: dwrite d3/f1b [0,4194304] 0 2024-04-24T23:13:32.211 INFO:tasks.workunit.client.0.smithi031.stdout:8/97: creat d8/d1a/f21 x:0 0 0 2024-04-24T23:13:32.216 INFO:tasks.workunit.client.0.smithi031.stdout:4/121: dread d3/f7 [0,4194304] 0 2024-04-24T23:13:32.218 INFO:tasks.workunit.client.0.smithi031.stdout:8/98: dread d8/fc [0,4194304] 0 2024-04-24T23:13:32.218 INFO:tasks.workunit.client.0.smithi031.stdout:4/122: write d3/f14 [4773072,52355] 0 2024-04-24T23:13:32.219 INFO:tasks.workunit.client.0.smithi031.stdout:8/99: fdatasync d8/d1a/f21 0 2024-04-24T23:13:32.219 INFO:tasks.workunit.client.0.smithi031.stdout:3/141: creat d6/f2b x:0 0 0 2024-04-24T23:13:32.221 INFO:tasks.workunit.client.0.smithi031.stdout:6/161: creat d13/f3f x:0 0 0 2024-04-24T23:13:32.223 INFO:tasks.workunit.client.0.smithi031.stdout:9/159: fsync d2/d1b/d14/f19 0 2024-04-24T23:13:32.229 INFO:tasks.workunit.client.0.smithi031.stdout:4/123: dwrite d3/f7 [0,4194304] 0 2024-04-24T23:13:32.230 INFO:tasks.workunit.client.0.smithi031.stdout:3/142: dwrite d6/f8 [0,4194304] 0 2024-04-24T23:13:32.232 INFO:tasks.workunit.client.0.smithi031.stdout:9/160: fdatasync d2/d16/f20 0 2024-04-24T23:13:32.233 INFO:tasks.workunit.client.0.smithi031.stdout:9/161: chown d2 126666454 1 2024-04-24T23:13:32.235 INFO:tasks.workunit.client.0.smithi031.stdout:9/162: dread - d2/d16/f2b zero size 2024-04-24T23:13:32.236 INFO:tasks.workunit.client.0.smithi031.stdout:6/162: dread f12 [0,4194304] 0 2024-04-24T23:13:32.236 INFO:tasks.workunit.client.0.smithi031.stdout:9/163: chown d2/f12 1579364 1 2024-04-24T23:13:32.237 INFO:tasks.workunit.client.0.smithi031.stdout:9/164: chown d2/d16/l26 1140 1 2024-04-24T23:13:32.238 INFO:tasks.workunit.client.0.smithi031.stdout:3/143: rename d6/d15/d1a/l20 to d6/d15/l2c 0 2024-04-24T23:13:32.239 INFO:tasks.workunit.client.0.smithi031.stdout:4/124: fdatasync d3/fb 0 2024-04-24T23:13:32.240 INFO:tasks.workunit.client.0.smithi031.stdout:4/125: chown d3/l17 100 1 2024-04-24T23:13:32.241 INFO:tasks.workunit.client.0.smithi031.stdout:6/163: creat d13/d15/d1d/d23/d24/f40 x:0 0 0 2024-04-24T23:13:32.242 INFO:tasks.workunit.client.0.smithi031.stdout:3/144: rename d6/l11 to d6/df/d16/d2a/l2d 0 2024-04-24T23:13:32.244 INFO:tasks.workunit.client.0.smithi031.stdout:8/100: write d8/fe [6120460,83624] 0 2024-04-24T23:13:32.245 INFO:tasks.workunit.client.0.smithi031.stdout:8/101: stat l4 0 2024-04-24T23:13:32.246 INFO:tasks.workunit.client.0.smithi031.stdout:3/145: creat d6/df/d16/d2a/f2e x:0 0 0 2024-04-24T23:13:32.250 INFO:tasks.workunit.client.0.smithi031.stdout:3/146: dread d6/d14/d1d/f1f [0,4194304] 0 2024-04-24T23:13:32.254 INFO:tasks.workunit.client.0.smithi031.stdout:3/147: dread d6/d14/d1d/f1e [0,4194304] 0 2024-04-24T23:13:32.256 INFO:tasks.workunit.client.0.smithi031.stdout:3/148: dread d6/d14/d1d/f24 [0,4194304] 0 2024-04-24T23:13:32.262 INFO:tasks.workunit.client.0.smithi031.stdout:6/164: write fd [486210,77413] 0 2024-04-24T23:13:32.262 INFO:tasks.workunit.client.0.smithi031.stdout:6/165: readlink l6 0 2024-04-24T23:13:32.265 INFO:tasks.workunit.client.0.smithi031.stdout:4/126: truncate d3/dd/f13 69287 0 2024-04-24T23:13:32.267 INFO:tasks.workunit.client.0.smithi031.stdout:3/149: truncate f0 3557637 0 2024-04-24T23:13:32.268 INFO:tasks.workunit.client.0.smithi031.stdout:3/150: write d6/df/d16/d2a/f2e [852493,119971] 0 2024-04-24T23:13:32.269 INFO:tasks.workunit.client.0.smithi031.stdout:4/127: dread d3/fc [0,4194304] 0 2024-04-24T23:13:32.271 INFO:tasks.workunit.client.0.smithi031.stdout:6/166: rename d13/d15/f16 to d13/d15/d1d/d23/d2d/f41 0 2024-04-24T23:13:32.276 INFO:tasks.workunit.client.0.smithi031.stdout:3/151: creat d6/d14/f2f x:0 0 0 2024-04-24T23:13:32.277 INFO:tasks.workunit.client.0.smithi031.stdout:6/167: dwrite f11 [0,4194304] 0 2024-04-24T23:13:32.278 INFO:tasks.workunit.client.0.smithi031.stdout:2/30: sync 2024-04-24T23:13:32.279 INFO:tasks.workunit.client.0.smithi031.stdout:4/128: mknod d3/d23/c28 0 2024-04-24T23:13:32.287 INFO:tasks.workunit.client.0.smithi031.stdout:6/168: symlink d13/d15/l42 0 2024-04-24T23:13:32.291 INFO:tasks.workunit.client.0.smithi031.stdout:6/169: mkdir d13/d15/d1d/d23/d24/d43 0 2024-04-24T23:13:32.292 INFO:tasks.workunit.client.0.smithi031.stdout:6/170: dread - d13/f3f zero size 2024-04-24T23:13:32.293 INFO:tasks.workunit.client.0.smithi031.stdout:6/171: write d13/d15/d1d/d23/f35 [1425920,29980] 0 2024-04-24T23:13:32.299 INFO:tasks.workunit.client.0.smithi031.stdout:6/172: unlink ca 0 2024-04-24T23:13:32.302 INFO:tasks.workunit.client.0.smithi031.stdout:6/173: mkdir d13/d15/d1d/d23/d44 0 2024-04-24T23:13:32.304 INFO:tasks.workunit.client.0.smithi031.stdout:6/174: fsync fc 0 2024-04-24T23:13:32.308 INFO:tasks.workunit.client.0.smithi031.stdout:2/31: truncate f4 23865 0 2024-04-24T23:13:32.309 INFO:tasks.workunit.client.0.smithi031.stdout:6/175: unlink d13/f25 0 2024-04-24T23:13:32.328 INFO:tasks.workunit.client.0.smithi031.stdout:6/176: creat d13/d1b/d32/f45 x:0 0 0 2024-04-24T23:13:32.331 INFO:tasks.workunit.client.0.smithi031.stdout:6/177: creat d13/d1b/f46 x:0 0 0 2024-04-24T23:13:32.335 INFO:tasks.workunit.client.0.smithi031.stdout:6/178: mkdir d13/d15/d1d/d23/d44/d47 0 2024-04-24T23:13:32.339 INFO:tasks.workunit.client.0.smithi031.stdout:6/179: unlink fd 0 2024-04-24T23:13:32.345 INFO:tasks.workunit.client.0.smithi031.stdout:6/180: rename d13/d15/d1d/d23/c33 to d13/d15/d1d/d23/c48 0 2024-04-24T23:13:32.349 INFO:tasks.workunit.client.0.smithi031.stdout:6/181: creat d13/d1b/f49 x:0 0 0 2024-04-24T23:13:32.350 INFO:tasks.workunit.client.0.smithi031.stdout:6/182: readlink d13/d15/l3a 0 2024-04-24T23:13:32.401 INFO:tasks.workunit.client.0.smithi031.stdout:6/183: dread - d13/d1b/f46 zero size 2024-04-24T23:13:32.401 INFO:tasks.workunit.client.0.smithi031.stdout:2/32: truncate f4 200445 0 2024-04-24T23:13:32.401 INFO:tasks.workunit.client.0.smithi031.stdout:6/184: stat d13/d15/d1d/d23/d2d 0 2024-04-24T23:13:32.406 INFO:tasks.workunit.client.0.smithi031.stdout:6/185: creat d13/d15/d1d/d23/d44/d47/f4a x:0 0 0 2024-04-24T23:13:32.409 INFO:tasks.workunit.client.0.smithi031.stdout:6/186: fsync f12 0 2024-04-24T23:13:32.487 INFO:tasks.workunit.client.0.smithi031.stdout:6/187: truncate d13/d15/d1d/d23/d2d/f41 2498877 0 2024-04-24T23:13:32.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/188: chown d13/d15/d1d/d23 32809478 1 2024-04-24T23:13:32.493 INFO:tasks.workunit.client.0.smithi031.stdout:6/189: rmdir d13/d15/d1d/d23/d3e 0 2024-04-24T23:13:32.494 INFO:tasks.workunit.client.0.smithi031.stdout:6/190: dread - d13/f3f zero size 2024-04-24T23:13:32.497 INFO:tasks.workunit.client.0.smithi031.stdout:6/191: creat d13/d15/d1d/d23/d24/d43/f4b x:0 0 0 2024-04-24T23:13:32.500 INFO:tasks.workunit.client.0.smithi031.stdout:6/192: creat d13/d15/d1d/d23/d2d/f4c x:0 0 0 2024-04-24T23:13:32.502 INFO:tasks.workunit.client.0.smithi031.stdout:6/193: mkdir d13/d15/d1d/d23/d4d 0 2024-04-24T23:13:32.503 INFO:tasks.workunit.client.0.smithi031.stdout:6/194: write d13/d1b/f3c [928198,128340] 0 2024-04-24T23:13:32.504 INFO:tasks.workunit.client.0.smithi031.stdout:6/195: chown d13/d1b/f3c 29 1 2024-04-24T23:13:32.592 INFO:tasks.workunit.client.0.smithi031.stdout:6/196: write d13/d15/f1f [74546,110003] 0 2024-04-24T23:13:32.594 INFO:tasks.workunit.client.0.smithi031.stdout:6/197: rmdir d13/d15/d1d 39 2024-04-24T23:13:32.599 INFO:tasks.workunit.client.0.smithi031.stdout:6/198: link d13/d15/d1d/d23/d2d/f3d d13/d15/d1d/d23/d44/f4e 0 2024-04-24T23:13:32.601 INFO:tasks.workunit.client.0.smithi031.stdout:6/199: truncate d13/d15/d1d/d23/d24/f40 892186 0 2024-04-24T23:13:32.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/200: symlink d13/d15/d1d/d23/d2d/d36/l4f 0 2024-04-24T23:13:32.612 INFO:tasks.workunit.client.0.smithi031.stdout:6/201: dwrite d13/d15/d1d/d23/f35 [0,4194304] 0 2024-04-24T23:13:32.613 INFO:tasks.workunit.client.0.smithi031.stdout:6/202: chown d13/d15/d1d/l20 14556529 1 2024-04-24T23:13:32.615 INFO:tasks.workunit.client.0.smithi031.stdout:6/203: truncate d13/d1b/d32/f45 726201 0 2024-04-24T23:13:32.617 INFO:tasks.workunit.client.0.smithi031.stdout:6/204: creat d13/d15/d1d/d23/d24/f50 x:0 0 0 2024-04-24T23:13:32.620 INFO:tasks.workunit.client.0.smithi031.stdout:6/205: stat d13/d15/d1d/d23/c48 0 2024-04-24T23:13:32.620 INFO:tasks.workunit.client.0.smithi031.stdout:6/206: dread - d13/d1b/f49 zero size 2024-04-24T23:13:32.653 INFO:tasks.workunit.client.0.smithi031.stdout:1/94: sync 2024-04-24T23:13:32.653 INFO:tasks.workunit.client.0.smithi031.stdout:5/52: sync 2024-04-24T23:13:32.662 INFO:tasks.workunit.client.0.smithi031.stdout:1/95: dwrite d1/f17 [0,4194304] 0 2024-04-24T23:13:32.662 INFO:tasks.workunit.client.0.smithi031.stdout:5/53: dwrite d0/d2/f5 [0,4194304] 0 2024-04-24T23:13:32.665 INFO:tasks.workunit.client.0.smithi031.stdout:1/96: write d1/f3 [611830,13652] 0 2024-04-24T23:13:32.671 INFO:tasks.workunit.client.0.smithi031.stdout:6/207: dwrite fc [0,4194304] 0 2024-04-24T23:13:32.672 INFO:tasks.workunit.client.0.smithi031.stdout:5/54: dread d0/d2/f4 [0,4194304] 0 2024-04-24T23:13:32.673 INFO:tasks.workunit.client.0.smithi031.stdout:6/208: chown d13/d1b/f46 547693 1 2024-04-24T23:13:32.678 INFO:tasks.workunit.client.0.smithi031.stdout:5/55: unlink d0/d2/lf 0 2024-04-24T23:13:32.679 INFO:tasks.workunit.client.0.smithi031.stdout:5/56: readlink d0/d7/l3 0 2024-04-24T23:13:32.684 INFO:tasks.workunit.client.0.smithi031.stdout:6/209: symlink d13/d15/d1d/d23/d44/d47/l51 0 2024-04-24T23:13:32.686 INFO:tasks.workunit.client.0.smithi031.stdout:1/97: rename d1/c7 to d1/dc/d11/c22 0 2024-04-24T23:13:32.687 INFO:tasks.workunit.client.0.smithi031.stdout:5/57: dwrite d0/f9 [0,4194304] 0 2024-04-24T23:13:32.692 INFO:tasks.workunit.client.0.smithi031.stdout:7/83: sync 2024-04-24T23:13:32.693 INFO:tasks.workunit.client.0.smithi031.stdout:3/152: sync 2024-04-24T23:13:32.693 INFO:tasks.workunit.client.0.smithi031.stdout:8/102: sync 2024-04-24T23:13:32.693 INFO:tasks.workunit.client.0.smithi031.stdout:4/129: sync 2024-04-24T23:13:32.693 INFO:tasks.workunit.client.0.smithi031.stdout:2/33: sync 2024-04-24T23:13:32.693 INFO:tasks.workunit.client.0.smithi031.stdout:0/111: sync 2024-04-24T23:13:32.693 INFO:tasks.workunit.client.0.smithi031.stdout:9/165: sync 2024-04-24T23:13:32.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/112: chown da/fd 1942093993 1 2024-04-24T23:13:32.695 INFO:tasks.workunit.client.0.smithi031.stdout:3/153: dread - d6/d14/f2f zero size 2024-04-24T23:13:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:0/113: write da/ff [267259,23595] 0 2024-04-24T23:13:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:1/98: chown d1/d1d/l1f 63946 1 2024-04-24T23:13:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:6/210: symlink d13/d15/d1d/d23/d4d/l52 0 2024-04-24T23:13:32.709 INFO:tasks.workunit.client.0.smithi031.stdout:4/130: dwrite d3/f14 [4194304,4194304] 0 2024-04-24T23:13:32.710 INFO:tasks.workunit.client.0.smithi031.stdout:7/84: dread f0 [0,4194304] 0 2024-04-24T23:13:32.713 INFO:tasks.workunit.client.0.smithi031.stdout:9/166: dwrite d2/d1b/d14/d18/f27 [0,4194304] 0 2024-04-24T23:13:32.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/167: stat d2/d1b/f13 0 2024-04-24T23:13:32.715 INFO:tasks.workunit.client.0.smithi031.stdout:9/168: write d2/f8 [2291115,104248] 0 2024-04-24T23:13:32.716 INFO:tasks.workunit.client.0.smithi031.stdout:3/154: write d6/d15/d1a/f22 [268588,80605] 0 2024-04-24T23:13:32.718 INFO:tasks.workunit.client.0.smithi031.stdout:0/114: creat da/d11/f25 x:0 0 0 2024-04-24T23:13:32.719 INFO:tasks.workunit.client.0.smithi031.stdout:7/85: dread d1/d2/f7 [0,4194304] 0 2024-04-24T23:13:32.719 INFO:tasks.workunit.client.0.smithi031.stdout:3/155: read d6/d14/d1d/f1e [3580217,87589] 0 2024-04-24T23:13:32.721 INFO:tasks.workunit.client.0.smithi031.stdout:1/99: creat d1/da/d12/f23 x:0 0 0 2024-04-24T23:13:32.722 INFO:tasks.workunit.client.0.smithi031.stdout:1/100: readlink d1/d1d/l1f 0 2024-04-24T23:13:32.723 INFO:tasks.workunit.client.0.smithi031.stdout:7/86: dread f0 [0,4194304] 0 2024-04-24T23:13:32.724 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:32 smithi040 ceph-mon[33128]: pgmap v19: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 393 KiB/s rd, 17 MiB/s wr, 363 op/s 2024-04-24T23:13:32.725 INFO:tasks.workunit.client.0.smithi031.stdout:5/58: link d0/d6/ca d0/d2/c10 0 2024-04-24T23:13:32.726 INFO:tasks.workunit.client.0.smithi031.stdout:5/59: write d0/f9 [2972750,42373] 0 2024-04-24T23:13:32.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/211: mknod d13/d15/d1d/d23/d2d/d36/c53 0 2024-04-24T23:13:32.735 INFO:tasks.workunit.client.0.smithi031.stdout:3/156: chown d6/d15/l2c 220949 1 2024-04-24T23:13:32.735 INFO:tasks.workunit.client.0.smithi031.stdout:0/115: dread da/f13 [0,4194304] 0 2024-04-24T23:13:32.737 INFO:tasks.workunit.client.0.smithi031.stdout:8/103: creat d8/f22 x:0 0 0 2024-04-24T23:13:32.738 INFO:tasks.workunit.client.0.smithi031.stdout:1/101: creat d1/dc/d11/f24 x:0 0 0 2024-04-24T23:13:32.739 INFO:tasks.workunit.client.0.smithi031.stdout:7/87: creat d1/d2/f19 x:0 0 0 2024-04-24T23:13:32.743 INFO:tasks.workunit.client.0.smithi031.stdout:5/60: creat d0/d2/f11 x:0 0 0 2024-04-24T23:13:32.745 INFO:tasks.workunit.client.0.smithi031.stdout:6/212: creat d13/d1b/f54 x:0 0 0 2024-04-24T23:13:32.745 INFO:tasks.workunit.client.0.smithi031.stdout:8/104: dwrite d8/d15/f16 [4194304,4194304] 0 2024-04-24T23:13:32.746 INFO:tasks.workunit.client.0.smithi031.stdout:8/105: chown f0 20899 1 2024-04-24T23:13:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:32 smithi031 ceph-mon[26411]: pgmap v19: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 393 KiB/s rd, 17 MiB/s wr, 363 op/s 2024-04-24T23:13:32.781 INFO:tasks.workunit.client.0.smithi031.stdout:8/106: write d8/d15/f16 [1554647,53387] 0 2024-04-24T23:13:32.782 INFO:tasks.workunit.client.0.smithi031.stdout:4/131: creat d3/f29 x:0 0 0 2024-04-24T23:13:32.784 INFO:tasks.workunit.client.0.smithi031.stdout:4/132: fsync d3/f14 0 2024-04-24T23:13:32.785 INFO:tasks.workunit.client.0.smithi031.stdout:0/116: write f6 [208937,6258] 0 2024-04-24T23:13:32.789 INFO:tasks.workunit.client.0.smithi031.stdout:2/34: link f4 d5/d8/fa 0 2024-04-24T23:13:32.790 INFO:tasks.workunit.client.0.smithi031.stdout:3/157: truncate f0 540667 0 2024-04-24T23:13:32.790 INFO:tasks.workunit.client.0.smithi031.stdout:5/61: dwrite d0/d2/f5 [4194304,4194304] 0 2024-04-24T23:13:32.791 INFO:tasks.workunit.client.0.smithi031.stdout:1/102: symlink d1/l25 0 2024-04-24T23:13:32.794 INFO:tasks.workunit.client.0.smithi031.stdout:4/133: symlink d3/d23/l2a 0 2024-04-24T23:13:32.794 INFO:tasks.workunit.client.0.smithi031.stdout:9/169: dwrite d2/d16/f20 [0,4194304] 0 2024-04-24T23:13:32.796 INFO:tasks.workunit.client.0.smithi031.stdout:0/117: creat da/d18/d1b/f26 x:0 0 0 2024-04-24T23:13:32.799 INFO:tasks.workunit.client.0.smithi031.stdout:8/107: dread d8/d15/f16 [0,4194304] 0 2024-04-24T23:13:32.800 INFO:tasks.workunit.client.0.smithi031.stdout:2/35: mkdir d5/db 0 2024-04-24T23:13:32.800 INFO:tasks.workunit.client.0.smithi031.stdout:8/108: chown d8/fb 0 1 2024-04-24T23:13:32.802 INFO:tasks.workunit.client.0.smithi031.stdout:2/36: read f4 [105946,77711] 0 2024-04-24T23:13:32.803 INFO:tasks.workunit.client.0.smithi031.stdout:5/62: mkdir d0/d6/d12 0 2024-04-24T23:13:32.805 INFO:tasks.workunit.client.0.smithi031.stdout:0/118: dread da/ff [0,4194304] 0 2024-04-24T23:13:32.805 INFO:tasks.workunit.client.0.smithi031.stdout:5/63: write d0/d2/f5 [2350212,71923] 0 2024-04-24T23:13:32.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/158: mknod d6/d15/c30 0 2024-04-24T23:13:32.810 INFO:tasks.workunit.client.0.smithi031.stdout:3/159: dread - d6/d14/f2f zero size 2024-04-24T23:13:32.811 INFO:tasks.workunit.client.0.smithi031.stdout:4/134: unlink d3/dd/l12 0 2024-04-24T23:13:32.812 INFO:tasks.workunit.client.0.smithi031.stdout:5/64: dread d0/f9 [0,4194304] 0 2024-04-24T23:13:32.812 INFO:tasks.workunit.client.0.smithi031.stdout:4/135: write d3/f9 [1493561,92881] 0 2024-04-24T23:13:32.813 INFO:tasks.workunit.client.0.smithi031.stdout:5/65: write d0/f9 [526556,129389] 0 2024-04-24T23:13:32.816 INFO:tasks.workunit.client.0.smithi031.stdout:9/170: symlink d2/d16/l2d 0 2024-04-24T23:13:32.821 INFO:tasks.workunit.client.0.smithi031.stdout:1/103: symlink d1/dc/d11/d21/l26 0 2024-04-24T23:13:32.824 INFO:tasks.workunit.client.0.smithi031.stdout:4/136: dwrite d3/f11 [0,4194304] 0 2024-04-24T23:13:32.828 INFO:tasks.workunit.client.0.smithi031.stdout:9/171: dread d2/d1b/d14/d18/f27 [0,4194304] 0 2024-04-24T23:13:32.828 INFO:tasks.workunit.client.0.smithi031.stdout:6/213: dwrite d13/d15/d1d/d23/d2d/f41 [0,4194304] 0 2024-04-24T23:13:32.830 INFO:tasks.workunit.client.0.smithi031.stdout:9/172: truncate d2/f5 1362021 0 2024-04-24T23:13:32.831 INFO:tasks.workunit.client.0.smithi031.stdout:1/104: dread d1/f17 [0,4194304] 0 2024-04-24T23:13:32.836 INFO:tasks.workunit.client.0.smithi031.stdout:9/173: write d2/d1b/d14/f22 [1877361,79794] 0 2024-04-24T23:13:32.842 INFO:tasks.workunit.client.0.smithi031.stdout:8/109: mknod d8/c23 0 2024-04-24T23:13:32.842 INFO:tasks.workunit.client.0.smithi031.stdout:8/110: dread - d8/f22 zero size 2024-04-24T23:13:32.848 INFO:tasks.workunit.client.0.smithi031.stdout:2/37: write d5/d8/fa [950854,108745] 0 2024-04-24T23:13:32.850 INFO:tasks.workunit.client.0.smithi031.stdout:2/38: write f4 [261630,128837] 0 2024-04-24T23:13:32.851 INFO:tasks.workunit.client.0.smithi031.stdout:0/119: creat da/d22/f27 x:0 0 0 2024-04-24T23:13:32.853 INFO:tasks.workunit.client.0.smithi031.stdout:6/214: mknod d13/d15/d1d/d23/d44/c55 0 2024-04-24T23:13:32.857 INFO:tasks.workunit.client.0.smithi031.stdout:1/105: mknod d1/dc/d11/c27 0 2024-04-24T23:13:32.858 INFO:tasks.workunit.client.0.smithi031.stdout:3/160: dwrite d6/d14/d1d/f24 [0,4194304] 0 2024-04-24T23:13:32.859 INFO:tasks.workunit.client.0.smithi031.stdout:9/174: rename d2 to d2/d1b/d2e 22 2024-04-24T23:13:32.860 INFO:tasks.workunit.client.0.smithi031.stdout:3/161: write d6/d14/f2f [123597,56439] 0 2024-04-24T23:13:32.863 INFO:tasks.workunit.client.0.smithi031.stdout:9/175: dread d2/d1b/d14/f19 [0,4194304] 0 2024-04-24T23:13:32.863 INFO:tasks.workunit.client.0.smithi031.stdout:5/66: symlink d0/d6/d12/l13 0 2024-04-24T23:13:32.864 INFO:tasks.workunit.client.0.smithi031.stdout:9/176: chown d2/d1b 1822510826 1 2024-04-24T23:13:32.865 INFO:tasks.workunit.client.0.smithi031.stdout:9/177: readlink d2/d16/l2d 0 2024-04-24T23:13:32.867 INFO:tasks.workunit.client.0.smithi031.stdout:9/178: read d2/f5 [1266757,94236] 0 2024-04-24T23:13:32.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/120: symlink da/d11/l28 0 2024-04-24T23:13:32.871 INFO:tasks.workunit.client.0.smithi031.stdout:9/179: truncate d2/d1b/f10 5128021 0 2024-04-24T23:13:32.871 INFO:tasks.workunit.client.0.smithi031.stdout:4/137: fdatasync d3/dd/f13 0 2024-04-24T23:13:32.873 INFO:tasks.workunit.client.0.smithi031.stdout:9/180: write d2/f5 [2097755,87492] 0 2024-04-24T23:13:32.878 INFO:tasks.workunit.client.0.smithi031.stdout:9/181: fdatasync d2/d16/f20 0 2024-04-24T23:13:32.879 INFO:tasks.workunit.client.0.smithi031.stdout:8/111: dwrite d8/d15/f16 [4194304,4194304] 0 2024-04-24T23:13:32.879 INFO:tasks.workunit.client.0.smithi031.stdout:5/67: symlink d0/d2/l14 0 2024-04-24T23:13:32.885 INFO:tasks.workunit.client.0.smithi031.stdout:0/121: dwrite da/d18/d1b/f26 [0,4194304] 0 2024-04-24T23:13:32.886 INFO:tasks.workunit.client.0.smithi031.stdout:0/122: stat da/l21 0 2024-04-24T23:13:32.886 INFO:tasks.workunit.client.0.smithi031.stdout:3/162: dread d6/fe [0,4194304] 0 2024-04-24T23:13:32.888 INFO:tasks.workunit.client.0.smithi031.stdout:0/123: dread - da/d22/f27 zero size 2024-04-24T23:13:32.889 INFO:tasks.workunit.client.0.smithi031.stdout:0/124: chown da/fb 91 1 2024-04-24T23:13:32.889 INFO:tasks.workunit.client.0.smithi031.stdout:0/125: chown l7 23151 1 2024-04-24T23:13:32.902 INFO:tasks.workunit.client.0.smithi031.stdout:5/68: mknod d0/d2/c15 0 2024-04-24T23:13:32.904 INFO:tasks.workunit.client.0.smithi031.stdout:2/39: rmdir d5/db 0 2024-04-24T23:13:32.904 INFO:tasks.workunit.client.0.smithi031.stdout:2/40: chown d5 5675 1 2024-04-24T23:13:32.905 INFO:tasks.workunit.client.0.smithi031.stdout:3/163: creat d6/df/d16/d2a/f31 x:0 0 0 2024-04-24T23:13:32.905 INFO:tasks.workunit.client.0.smithi031.stdout:4/138: rmdir d3/d1c 39 2024-04-24T23:13:32.906 INFO:tasks.workunit.client.0.smithi031.stdout:8/112: dwrite f3 [4194304,4194304] 0 2024-04-24T23:13:32.907 INFO:tasks.workunit.client.0.smithi031.stdout:0/126: creat da/d18/d1a/f29 x:0 0 0 2024-04-24T23:13:32.914 INFO:tasks.workunit.client.0.smithi031.stdout:2/41: dwrite f4 [0,4194304] 0 2024-04-24T23:13:32.916 INFO:tasks.workunit.client.0.smithi031.stdout:4/139: dread d3/f9 [0,4194304] 0 2024-04-24T23:13:32.926 INFO:tasks.workunit.client.0.smithi031.stdout:0/127: mknod da/d22/c2a 0 2024-04-24T23:13:32.928 INFO:tasks.workunit.client.0.smithi031.stdout:6/215: link d13/d15/d1d/d23/d24/c2a d13/d15/d1d/d23/d44/d47/c56 0 2024-04-24T23:13:32.929 INFO:tasks.workunit.client.0.smithi031.stdout:5/69: dread d0/f9 [0,4194304] 0 2024-04-24T23:13:32.929 INFO:tasks.workunit.client.0.smithi031.stdout:0/128: readlink da/d11/l28 0 2024-04-24T23:13:32.929 INFO:tasks.workunit.client.0.smithi031.stdout:1/106: dwrite d1/f2 [0,4194304] 0 2024-04-24T23:13:32.932 INFO:tasks.workunit.client.0.smithi031.stdout:4/140: dwrite d3/fa [0,4194304] 0 2024-04-24T23:13:32.937 INFO:tasks.workunit.client.0.smithi031.stdout:3/164: symlink d6/d15/d29/l32 0 2024-04-24T23:13:32.942 INFO:tasks.workunit.client.0.smithi031.stdout:1/107: dwrite d1/dc/d11/f13 [0,4194304] 0 2024-04-24T23:13:32.944 INFO:tasks.workunit.client.0.smithi031.stdout:5/70: symlink d0/l16 0 2024-04-24T23:13:32.945 INFO:tasks.workunit.client.0.smithi031.stdout:5/71: stat d0/d6/d12/l13 0 2024-04-24T23:13:32.945 INFO:tasks.workunit.client.0.smithi031.stdout:9/182: truncate d2/f9 3296444 0 2024-04-24T23:13:32.946 INFO:tasks.workunit.client.0.smithi031.stdout:5/72: stat d0/d7/db/dc 0 2024-04-24T23:13:32.948 INFO:tasks.workunit.client.0.smithi031.stdout:5/73: fdatasync d0/d2/f11 0 2024-04-24T23:13:32.954 INFO:tasks.workunit.client.0.smithi031.stdout:8/113: rename d8/d15/f16 to d8/f24 0 2024-04-24T23:13:32.959 INFO:tasks.workunit.client.0.smithi031.stdout:1/108: unlink d1/f17 0 2024-04-24T23:13:32.964 INFO:tasks.workunit.client.0.smithi031.stdout:2/42: dwrite f4 [4194304,4194304] 0 2024-04-24T23:13:32.966 INFO:tasks.workunit.client.0.smithi031.stdout:5/74: fdatasync d0/f9 0 2024-04-24T23:13:32.972 INFO:tasks.workunit.client.0.smithi031.stdout:9/183: dread d2/d1b/d14/d18/f21 [0,4194304] 0 2024-04-24T23:13:32.978 INFO:tasks.workunit.client.0.smithi031.stdout:1/109: dwrite d1/da/d12/f19 [0,4194304] 0 2024-04-24T23:13:32.979 INFO:tasks.workunit.client.0.smithi031.stdout:2/43: dwrite d5/d8/fa [4194304,4194304] 0 2024-04-24T23:13:32.980 INFO:tasks.workunit.client.0.smithi031.stdout:5/75: dread d0/d2/f5 [4194304,4194304] 0 2024-04-24T23:13:32.982 INFO:tasks.workunit.client.0.smithi031.stdout:6/216: rename d13/d15/d1d/d23/c26 to d13/d15/d1d/d23/d2d/d36/c57 0 2024-04-24T23:13:32.982 INFO:tasks.workunit.client.0.smithi031.stdout:2/44: rename d5 to d5/dc 22 2024-04-24T23:13:32.995 INFO:tasks.workunit.client.0.smithi031.stdout:0/129: link da/d18/d1b/f23 da/f2b 0 2024-04-24T23:13:32.995 INFO:tasks.workunit.client.0.smithi031.stdout:0/130: read - da/f2b zero size 2024-04-24T23:13:32.996 INFO:tasks.workunit.client.0.smithi031.stdout:0/131: fdatasync da/f19 0 2024-04-24T23:13:32.997 INFO:tasks.workunit.client.0.smithi031.stdout:0/132: readlink l7 0 2024-04-24T23:13:32.998 INFO:tasks.workunit.client.0.smithi031.stdout:0/133: read da/d11/f12 [2368490,98948] 0 2024-04-24T23:13:33.000 INFO:tasks.workunit.client.0.smithi031.stdout:0/134: truncate da/f19 6199199 0 2024-04-24T23:13:33.001 INFO:tasks.workunit.client.0.smithi031.stdout:8/114: mknod d8/c25 0 2024-04-24T23:13:33.002 INFO:tasks.workunit.client.0.smithi031.stdout:0/135: dread da/d18/d1a/f1e [0,4194304] 0 2024-04-24T23:13:33.004 INFO:tasks.workunit.client.0.smithi031.stdout:5/76: creat d0/d2/f17 x:0 0 0 2024-04-24T23:13:33.006 INFO:tasks.workunit.client.0.smithi031.stdout:1/110: rename d1/f3 to d1/dc/d11/d21/f28 0 2024-04-24T23:13:33.008 INFO:tasks.workunit.client.0.smithi031.stdout:3/165: rmdir d6/d15/d1a/d25/d28 0 2024-04-24T23:13:33.009 INFO:tasks.workunit.client.0.smithi031.stdout:7/88: sync 2024-04-24T23:13:33.013 INFO:tasks.workunit.client.0.smithi031.stdout:1/111: dwrite d1/dc/fe [0,4194304] 0 2024-04-24T23:13:33.024 INFO:tasks.workunit.client.0.smithi031.stdout:4/141: link d3/f9 d3/f2b 0 2024-04-24T23:13:33.029 INFO:tasks.workunit.client.0.smithi031.stdout:8/115: symlink d8/d1a/d1d/l26 0 2024-04-24T23:13:33.032 INFO:tasks.workunit.client.0.smithi031.stdout:0/136: rmdir da/d20 39 2024-04-24T23:13:33.034 INFO:tasks.workunit.client.0.smithi031.stdout:2/45: mkdir d5/d8/dd 0 2024-04-24T23:13:33.034 INFO:tasks.workunit.client.0.smithi031.stdout:2/46: readlink d5/l9 0 2024-04-24T23:13:33.040 INFO:tasks.workunit.client.0.smithi031.stdout:7/89: rmdir d1 39 2024-04-24T23:13:33.040 INFO:tasks.workunit.client.0.smithi031.stdout:5/77: dread d0/f9 [0,4194304] 0 2024-04-24T23:13:33.046 INFO:tasks.workunit.client.0.smithi031.stdout:3/166: dwrite d6/fe [0,4194304] 0 2024-04-24T23:13:33.050 INFO:tasks.workunit.client.0.smithi031.stdout:9/184: creat d2/d1b/f2f x:0 0 0 2024-04-24T23:13:33.050 INFO:tasks.workunit.client.0.smithi031.stdout:9/185: chown d2/f5 31468210 1 2024-04-24T23:13:33.059 INFO:tasks.workunit.client.0.smithi031.stdout:0/137: dwrite da/f2b [0,4194304] 0 2024-04-24T23:13:33.061 INFO:tasks.workunit.client.0.smithi031.stdout:8/116: write d8/fc [3798698,93208] 0 2024-04-24T23:13:33.065 INFO:tasks.workunit.client.0.smithi031.stdout:3/167: rename d6/d14/f2f to d6/d14/f33 0 2024-04-24T23:13:33.067 INFO:tasks.workunit.client.0.smithi031.stdout:1/112: dwrite d1/dc/d11/f13 [0,4194304] 0 2024-04-24T23:13:33.070 INFO:tasks.workunit.client.0.smithi031.stdout:6/217: rename lf to d13/d15/l58 0 2024-04-24T23:13:33.075 INFO:tasks.workunit.client.0.smithi031.stdout:5/78: write d0/d2/f4 [3923738,21954] 0 2024-04-24T23:13:33.076 INFO:tasks.workunit.client.0.smithi031.stdout:7/90: mknod d1/d2/c1a 0 2024-04-24T23:13:33.080 INFO:tasks.workunit.client.0.smithi031.stdout:8/117: mknod d8/d1b/c27 0 2024-04-24T23:13:33.080 INFO:tasks.workunit.client.0.smithi031.stdout:6/218: dwrite d13/d1b/f3c [0,4194304] 0 2024-04-24T23:13:33.082 INFO:tasks.workunit.client.0.smithi031.stdout:0/138: symlink da/d18/l2c 0 2024-04-24T23:13:33.083 INFO:tasks.workunit.client.0.smithi031.stdout:6/219: dread - d13/d15/d1d/d23/d44/d47/f4a zero size 2024-04-24T23:13:33.083 INFO:tasks.workunit.client.0.smithi031.stdout:0/139: fsync da/d18/d1b/f23 0 2024-04-24T23:13:33.084 INFO:tasks.workunit.client.0.smithi031.stdout:9/186: mknod d2/d1b/c30 0 2024-04-24T23:13:33.085 INFO:tasks.workunit.client.0.smithi031.stdout:2/47: link d5/l9 d5/le 0 2024-04-24T23:13:33.091 INFO:tasks.workunit.client.0.smithi031.stdout:3/168: rename d6/df/f17 to d6/df/f34 0 2024-04-24T23:13:33.095 INFO:tasks.workunit.client.0.smithi031.stdout:4/142: rename d3/f14 to d3/d1c/f2c 0 2024-04-24T23:13:33.099 INFO:tasks.workunit.client.0.smithi031.stdout:7/91: rename d1/c18 to d1/c1b 0 2024-04-24T23:13:33.108 INFO:tasks.workunit.client.0.smithi031.stdout:9/187: rename d2/d1b/d14/d18/f27 to d2/d1b/d14/d18/f31 0 2024-04-24T23:13:33.109 INFO:tasks.workunit.client.0.smithi031.stdout:9/188: dread - d2/f12 zero size 2024-04-24T23:13:33.110 INFO:tasks.workunit.client.0.smithi031.stdout:9/189: chown d2/d16/l26 53857 1 2024-04-24T23:13:33.110 INFO:tasks.workunit.client.0.smithi031.stdout:2/48: unlink d5/le 0 2024-04-24T23:13:33.111 INFO:tasks.workunit.client.0.smithi031.stdout:9/190: write d2/d1b/d14/f22 [1761307,97058] 0 2024-04-24T23:13:33.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/169: truncate d6/d14/d1d/f1f 5006703 0 2024-04-24T23:13:33.119 INFO:tasks.workunit.client.0.smithi031.stdout:4/143: fdatasync d3/f2b 0 2024-04-24T23:13:33.123 INFO:tasks.workunit.client.0.smithi031.stdout:5/79: dwrite d0/f9 [0,4194304] 0 2024-04-24T23:13:33.125 INFO:tasks.workunit.client.0.smithi031.stdout:7/92: mkdir d1/d1c 0 2024-04-24T23:13:33.126 INFO:tasks.workunit.client.0.smithi031.stdout:7/93: fdatasync d1/ff 0 2024-04-24T23:13:33.133 INFO:tasks.workunit.client.0.smithi031.stdout:1/113: creat d1/f29 x:0 0 0 2024-04-24T23:13:33.135 INFO:tasks.workunit.client.0.smithi031.stdout:3/170: creat d6/df/d16/f35 x:0 0 0 2024-04-24T23:13:33.139 INFO:tasks.workunit.client.0.smithi031.stdout:4/144: mkdir d3/d20/d2d 0 2024-04-24T23:13:33.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/80: mkdir d0/d18 0 2024-04-24T23:13:33.141 INFO:tasks.workunit.client.0.smithi031.stdout:0/140: truncate f3 2899418 0 2024-04-24T23:13:33.143 INFO:tasks.workunit.client.0.smithi031.stdout:8/118: creat d8/f28 x:0 0 0 2024-04-24T23:13:33.143 INFO:tasks.workunit.client.0.smithi031.stdout:8/119: read - d8/f22 zero size 2024-04-24T23:13:33.144 INFO:tasks.workunit.client.0.smithi031.stdout:6/220: creat d13/f59 x:0 0 0 2024-04-24T23:13:33.145 INFO:tasks.workunit.client.0.smithi031.stdout:6/221: read d13/d1b/f22 [8167631,41300] 0 2024-04-24T23:13:33.153 INFO:tasks.workunit.client.0.smithi031.stdout:4/145: rmdir d3 39 2024-04-24T23:13:33.155 INFO:tasks.workunit.client.0.smithi031.stdout:5/81: symlink d0/d6/l19 0 2024-04-24T23:13:33.160 INFO:tasks.workunit.client.0.smithi031.stdout:7/94: write d1/ff [4546606,1178] 0 2024-04-24T23:13:33.161 INFO:tasks.workunit.client.0.smithi031.stdout:9/191: creat d2/f32 x:0 0 0 2024-04-24T23:13:33.165 INFO:tasks.workunit.client.0.smithi031.stdout:6/222: rename fc to d13/d15/d1d/d23/d24/d43/f5a 0 2024-04-24T23:13:33.166 INFO:tasks.workunit.client.0.smithi031.stdout:1/114: symlink d1/dc/d11/d21/l2a 0 2024-04-24T23:13:33.167 INFO:tasks.workunit.client.0.smithi031.stdout:1/115: fsync d1/da/d10/f1c 0 2024-04-24T23:13:33.170 INFO:tasks.workunit.client.0.smithi031.stdout:3/171: link d6/fc d6/d15/d29/f36 0 2024-04-24T23:13:33.173 INFO:tasks.workunit.client.0.smithi031.stdout:0/141: write da/ff [1240868,68466] 0 2024-04-24T23:13:33.174 INFO:tasks.workunit.client.0.smithi031.stdout:0/142: chown l4 0 1 2024-04-24T23:13:33.174 INFO:tasks.workunit.client.0.smithi031.stdout:5/82: creat d0/d6/f1a x:0 0 0 2024-04-24T23:13:33.177 INFO:tasks.workunit.client.0.smithi031.stdout:8/120: symlink d8/l29 0 2024-04-24T23:13:33.177 INFO:tasks.workunit.client.0.smithi031.stdout:4/146: dread d3/f7 [0,4194304] 0 2024-04-24T23:13:33.177 INFO:tasks.workunit.client.0.smithi031.stdout:8/121: dread - d8/f22 zero size 2024-04-24T23:13:33.181 INFO:tasks.workunit.client.0.smithi031.stdout:9/192: rename d2/d1b/f1f to d2/d16/f33 0 2024-04-24T23:13:33.186 INFO:tasks.workunit.client.0.smithi031.stdout:3/172: write d6/d15/d29/f36 [3703192,36886] 0 2024-04-24T23:13:33.187 INFO:tasks.workunit.client.0.smithi031.stdout:0/143: creat da/d22/f2d x:0 0 0 2024-04-24T23:13:33.190 INFO:tasks.workunit.client.0.smithi031.stdout:5/83: symlink d0/d7/l1b 0 2024-04-24T23:13:33.190 INFO:tasks.workunit.client.0.smithi031.stdout:9/193: dwrite d2/d1b/d14/f22 [0,4194304] 0 2024-04-24T23:13:33.191 INFO:tasks.workunit.client.0.smithi031.stdout:4/147: mkdir d3/d1c/d2e 0 2024-04-24T23:13:33.194 INFO:tasks.workunit.client.0.smithi031.stdout:8/122: creat d8/d15/f2a x:0 0 0 2024-04-24T23:13:33.196 INFO:tasks.workunit.client.0.smithi031.stdout:7/95: mknod d1/d1c/c1d 0 2024-04-24T23:13:33.199 INFO:tasks.workunit.client.0.smithi031.stdout:8/123: dread d8/fe [8388608,4194304] 0 2024-04-24T23:13:33.200 INFO:tasks.workunit.client.0.smithi031.stdout:8/124: dread - d8/d1a/f21 zero size 2024-04-24T23:13:33.203 INFO:tasks.workunit.client.0.smithi031.stdout:6/223: rename d13/f59 to d13/d15/d1d/d23/d44/f5b 0 2024-04-24T23:13:33.205 INFO:tasks.workunit.client.0.smithi031.stdout:6/224: write d13/d15/d1d/d23/d44/d47/f4a [701968,121003] 0 2024-04-24T23:13:33.207 INFO:tasks.workunit.client.0.smithi031.stdout:6/225: truncate d13/d15/d1d/d23/d44/f4e 669388 0 2024-04-24T23:13:33.211 INFO:tasks.workunit.client.0.smithi031.stdout:3/173: creat d6/d14/f37 x:0 0 0 2024-04-24T23:13:33.211 INFO:tasks.workunit.client.0.smithi031.stdout:0/144: mknod da/d22/c2e 0 2024-04-24T23:13:33.213 INFO:tasks.workunit.client.0.smithi031.stdout:5/84: mknod d0/d6/c1c 0 2024-04-24T23:13:33.213 INFO:tasks.workunit.client.0.smithi031.stdout:5/85: stat d0/d2/f17 0 2024-04-24T23:13:33.214 INFO:tasks.workunit.client.0.smithi031.stdout:5/86: fdatasync d0/d2/f4 0 2024-04-24T23:13:33.222 INFO:tasks.workunit.client.0.smithi031.stdout:5/87: dwrite d0/d2/f17 [0,4194304] 0 2024-04-24T23:13:33.229 INFO:tasks.workunit.client.0.smithi031.stdout:6/226: unlink d13/c2c 0 2024-04-24T23:13:33.230 INFO:tasks.workunit.client.0.smithi031.stdout:5/88: dwrite d0/d6/f1a [0,4194304] 0 2024-04-24T23:13:33.231 INFO:tasks.workunit.client.0.smithi031.stdout:0/145: rename da/d11/f17 to da/d18/f2f 0 2024-04-24T23:13:33.239 INFO:tasks.workunit.client.0.smithi031.stdout:4/148: write d3/f2b [1734119,91569] 0 2024-04-24T23:13:33.243 INFO:tasks.workunit.client.0.smithi031.stdout:8/125: mknod d8/c2b 0 2024-04-24T23:13:33.244 INFO:tasks.workunit.client.0.smithi031.stdout:1/116: getdents d1/da/d12 0 2024-04-24T23:13:33.244 INFO:tasks.workunit.client.0.smithi031.stdout:8/126: write f0 [3955769,64055] 0 2024-04-24T23:13:33.245 INFO:tasks.workunit.client.0.smithi031.stdout:8/127: fsync d8/fc 0 2024-04-24T23:13:33.246 INFO:tasks.workunit.client.0.smithi031.stdout:1/117: chown d1/da/d10/l15 1117 1 2024-04-24T23:13:33.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/149: fdatasync d3/f11 0 2024-04-24T23:13:33.248 INFO:tasks.workunit.client.0.smithi031.stdout:1/118: write d1/f8 [1137090,107209] 0 2024-04-24T23:13:33.248 INFO:tasks.workunit.client.0.smithi031.stdout:6/227: dread - d13/f14 zero size 2024-04-24T23:13:33.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/146: creat da/d22/f30 x:0 0 0 2024-04-24T23:13:33.250 INFO:tasks.workunit.client.0.smithi031.stdout:6/228: write d13/f3f [321865,110437] 0 2024-04-24T23:13:33.251 INFO:tasks.workunit.client.0.smithi031.stdout:0/147: chown da/d22/c2a 50287000 1 2024-04-24T23:13:33.252 INFO:tasks.workunit.client.0.smithi031.stdout:0/148: dread - da/d22/f2d zero size 2024-04-24T23:13:33.255 INFO:tasks.workunit.client.0.smithi031.stdout:5/89: dread d0/d2/f5 [0,4194304] 0 2024-04-24T23:13:33.260 INFO:tasks.workunit.client.0.smithi031.stdout:0/149: dwrite da/f19 [4194304,4194304] 0 2024-04-24T23:13:33.260 INFO:tasks.workunit.client.0.smithi031.stdout:0/150: chown da/d11/f25 1296258 1 2024-04-24T23:13:33.263 INFO:tasks.workunit.client.0.smithi031.stdout:4/150: mknod d3/d1c/c2f 0 2024-04-24T23:13:33.263 INFO:tasks.workunit.client.0.smithi031.stdout:5/90: write d0/d6/f1a [5119327,31803] 0 2024-04-24T23:13:33.265 INFO:tasks.workunit.client.0.smithi031.stdout:1/119: fsync d1/dc/d11/f24 0 2024-04-24T23:13:33.267 INFO:tasks.workunit.client.0.smithi031.stdout:6/229: symlink d13/d1b/d32/l5c 0 2024-04-24T23:13:33.268 INFO:tasks.workunit.client.0.smithi031.stdout:6/230: chown d13/d1b/f1c 38635 1 2024-04-24T23:13:33.272 INFO:tasks.workunit.client.0.smithi031.stdout:7/96: getdents d1/d2/dd 0 2024-04-24T23:13:33.273 INFO:tasks.workunit.client.0.smithi031.stdout:7/97: dread - d1/d2/f19 zero size 2024-04-24T23:13:33.273 INFO:tasks.workunit.client.0.smithi031.stdout:7/98: fsync d1/d2/f12 0 2024-04-24T23:13:33.277 INFO:tasks.workunit.client.0.smithi031.stdout:0/151: mkdir da/d18/d1b/d31 0 2024-04-24T23:13:33.277 INFO:tasks.workunit.client.0.smithi031.stdout:0/152: read - da/fe zero size 2024-04-24T23:13:33.280 INFO:tasks.workunit.client.0.smithi031.stdout:5/91: rmdir d0 39 2024-04-24T23:13:33.286 INFO:tasks.workunit.client.0.smithi031.stdout:9/194: link d2/f9 d2/d1b/f34 0 2024-04-24T23:13:33.286 INFO:tasks.workunit.client.0.smithi031.stdout:9/195: readlink d2/d1b/d14/l29 0 2024-04-24T23:13:33.288 INFO:tasks.workunit.client.0.smithi031.stdout:9/196: write d2/d1b/f13 [3734028,127441] 0 2024-04-24T23:13:33.288 INFO:tasks.workunit.client.0.smithi031.stdout:9/197: read d2/d16/f20 [1582594,113273] 0 2024-04-24T23:13:33.292 INFO:tasks.workunit.client.0.smithi031.stdout:7/99: write d1/f9 [1524270,35920] 0 2024-04-24T23:13:33.296 INFO:tasks.workunit.client.0.smithi031.stdout:1/120: write d1/ff [987748,18129] 0 2024-04-24T23:13:33.302 INFO:tasks.workunit.client.0.smithi031.stdout:0/153: write da/d11/f12 [2691470,107059] 0 2024-04-24T23:13:33.306 INFO:tasks.workunit.client.0.smithi031.stdout:4/151: truncate d3/fa 1557393 0 2024-04-24T23:13:33.306 INFO:tasks.workunit.client.0.smithi031.stdout:4/152: fdatasync d3/f11 0 2024-04-24T23:13:33.311 INFO:tasks.workunit.client.0.smithi031.stdout:7/100: rmdir d1/d2/dd 39 2024-04-24T23:13:33.312 INFO:tasks.workunit.client.0.smithi031.stdout:7/101: truncate d1/f10 1764608 0 2024-04-24T23:13:33.315 INFO:tasks.workunit.client.0.smithi031.stdout:9/198: dwrite d2/d1b/d14/d18/f21 [0,4194304] 0 2024-04-24T23:13:33.318 INFO:tasks.workunit.client.0.smithi031.stdout:8/128: getdents d8/d1a/d1d 0 2024-04-24T23:13:33.318 INFO:tasks.workunit.client.0.smithi031.stdout:9/199: read - d2/d16/f2b zero size 2024-04-24T23:13:33.319 INFO:tasks.workunit.client.0.smithi031.stdout:6/231: truncate f12 4059112 0 2024-04-24T23:13:33.319 INFO:tasks.workunit.client.0.smithi031.stdout:9/200: fsync d2/d16/f20 0 2024-04-24T23:13:33.319 INFO:tasks.workunit.client.0.smithi031.stdout:6/232: chown d13/d15/l3a 6214 1 2024-04-24T23:13:33.320 INFO:tasks.workunit.client.0.smithi031.stdout:1/121: creat d1/dc/f2b x:0 0 0 2024-04-24T23:13:33.321 INFO:tasks.workunit.client.0.smithi031.stdout:1/122: dread - d1/da/d12/f23 zero size 2024-04-24T23:13:33.322 INFO:tasks.workunit.client.0.smithi031.stdout:1/123: chown d1/dc/d11/f13 13374333 1 2024-04-24T23:13:33.322 INFO:tasks.workunit.client.0.smithi031.stdout:5/92: mkdir d0/d7/db/d1d 0 2024-04-24T23:13:33.323 INFO:tasks.workunit.client.0.smithi031.stdout:5/93: chown d0/d7/l3 461 1 2024-04-24T23:13:33.325 INFO:tasks.workunit.client.0.smithi031.stdout:0/154: mknod da/d18/d1b/c32 0 2024-04-24T23:13:33.330 INFO:tasks.workunit.client.0.smithi031.stdout:8/129: rmdir d8/d1a 39 2024-04-24T23:13:33.331 INFO:tasks.workunit.client.0.smithi031.stdout:8/130: chown d8/l29 13 1 2024-04-24T23:13:33.333 INFO:tasks.workunit.client.0.smithi031.stdout:0/155: dwrite da/d11/f12 [0,4194304] 0 2024-04-24T23:13:33.334 INFO:tasks.workunit.client.0.smithi031.stdout:6/233: mknod d13/d15/d1d/d23/d2d/c5d 0 2024-04-24T23:13:33.337 INFO:tasks.workunit.client.0.smithi031.stdout:6/234: write d13/d15/d1d/d23/d24/d43/f4b [344209,4685] 0 2024-04-24T23:13:33.339 INFO:tasks.workunit.client.0.smithi031.stdout:5/94: creat d0/d6/f1e x:0 0 0 2024-04-24T23:13:33.339 INFO:tasks.workunit.client.0.smithi031.stdout:5/95: dread - d0/d6/f1e zero size 2024-04-24T23:13:33.340 INFO:tasks.workunit.client.0.smithi031.stdout:5/96: readlink d0/d7/l1b 0 2024-04-24T23:13:33.345 INFO:tasks.workunit.client.0.smithi031.stdout:5/97: dread d0/d6/f1a [0,4194304] 0 2024-04-24T23:13:33.349 INFO:tasks.workunit.client.0.smithi031.stdout:4/153: rename d3/fa to d3/dd/f30 0 2024-04-24T23:13:33.351 INFO:tasks.workunit.client.0.smithi031.stdout:7/102: mkdir d1/d2/d1e 0 2024-04-24T23:13:33.353 INFO:tasks.workunit.client.0.smithi031.stdout:8/131: chown d8/lf 82265130 1 2024-04-24T23:13:33.355 INFO:tasks.workunit.client.0.smithi031.stdout:9/201: write d2/f9 [1163086,39585] 0 2024-04-24T23:13:33.357 INFO:tasks.workunit.client.0.smithi031.stdout:7/103: dwrite d1/ff [0,4194304] 0 2024-04-24T23:13:33.359 INFO:tasks.workunit.client.0.smithi031.stdout:7/104: write d1/d2/fa [445642,102586] 0 2024-04-24T23:13:33.408 INFO:tasks.workunit.client.0.smithi031.stdout:7/105: dwrite d1/d2/f12 [0,4194304] 0 2024-04-24T23:13:33.410 INFO:tasks.workunit.client.0.smithi031.stdout:0/156: creat da/d11/f33 x:0 0 0 2024-04-24T23:13:33.410 INFO:tasks.workunit.client.0.smithi031.stdout:7/106: write d1/ff [5348929,20404] 0 2024-04-24T23:13:33.418 INFO:tasks.workunit.client.0.smithi031.stdout:6/235: creat d13/d15/d1d/f5e x:0 0 0 2024-04-24T23:13:33.419 INFO:tasks.workunit.client.0.smithi031.stdout:6/236: read f9 [1389080,14346] 0 2024-04-24T23:13:33.422 INFO:tasks.workunit.client.0.smithi031.stdout:6/237: dread d13/d15/d1d/d23/d24/f40 [0,4194304] 0 2024-04-24T23:13:33.423 INFO:tasks.workunit.client.0.smithi031.stdout:5/98: truncate d0/d2/f5 7371830 0 2024-04-24T23:13:33.426 INFO:tasks.workunit.client.0.smithi031.stdout:4/154: creat d3/dd/f31 x:0 0 0 2024-04-24T23:13:33.433 INFO:tasks.workunit.client.0.smithi031.stdout:9/202: creat d2/d1b/d14/d1d/f35 x:0 0 0 2024-04-24T23:13:33.436 INFO:tasks.workunit.client.0.smithi031.stdout:1/124: truncate d1/f2 4367286 0 2024-04-24T23:13:33.440 INFO:tasks.workunit.client.0.smithi031.stdout:9/203: dwrite d2/d1b/f34 [0,4194304] 0 2024-04-24T23:13:33.441 INFO:tasks.workunit.client.0.smithi031.stdout:0/157: mknod da/d18/d1a/c34 0 2024-04-24T23:13:33.443 INFO:tasks.workunit.client.0.smithi031.stdout:9/204: write d2/d1b/d14/d1d/f24 [401150,82598] 0 2024-04-24T23:13:33.444 INFO:tasks.workunit.client.0.smithi031.stdout:7/107: unlink d1/f10 0 2024-04-24T23:13:33.446 INFO:tasks.workunit.client.0.smithi031.stdout:0/158: truncate da/d11/f33 648840 0 2024-04-24T23:13:33.446 INFO:tasks.workunit.client.0.smithi031.stdout:1/125: dwrite d1/f18 [0,4194304] 0 2024-04-24T23:13:33.461 INFO:tasks.workunit.client.0.smithi031.stdout:6/238: creat d13/d15/d1d/d23/d4d/f5f x:0 0 0 2024-04-24T23:13:33.462 INFO:tasks.workunit.client.0.smithi031.stdout:6/239: fdatasync d13/d15/d1d/d23/d24/f50 0 2024-04-24T23:13:33.477 INFO:tasks.workunit.client.0.smithi031.stdout:1/126: creat d1/da/f2c x:0 0 0 2024-04-24T23:13:33.484 INFO:tasks.workunit.client.0.smithi031.stdout:1/127: dwrite d1/dc/d11/f13 [4194304,4194304] 0 2024-04-24T23:13:33.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/240: unlink d13/d15/d1d/c21 0 2024-04-24T23:13:33.485 INFO:tasks.workunit.client.0.smithi031.stdout:1/128: truncate d1/da/d12/f23 552588 0 2024-04-24T23:13:33.487 INFO:tasks.workunit.client.0.smithi031.stdout:1/129: fdatasync d1/dc/d11/f24 0 2024-04-24T23:13:33.487 INFO:tasks.workunit.client.0.smithi031.stdout:6/241: chown d13/d15/d1d/d23/d2d/f3d 4118 1 2024-04-24T23:13:33.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/242: chown l0 239189892 1 2024-04-24T23:13:33.488 INFO:tasks.workunit.client.0.smithi031.stdout:1/130: chown d1/da/d10/f1c 2695444 1 2024-04-24T23:13:33.490 INFO:tasks.workunit.client.0.smithi031.stdout:6/243: write d13/d15/d1d/d23/d4d/f5f [962284,109905] 0 2024-04-24T23:13:33.491 INFO:tasks.workunit.client.0.smithi031.stdout:1/131: chown d1/dc/d11/d21/l2a 3015 1 2024-04-24T23:13:33.492 INFO:tasks.workunit.client.0.smithi031.stdout:6/244: dread - d13/d1b/f54 zero size 2024-04-24T23:13:33.492 INFO:tasks.workunit.client.0.smithi031.stdout:7/108: mknod d1/d2/d1e/c1f 0 2024-04-24T23:13:33.493 INFO:tasks.workunit.client.0.smithi031.stdout:0/159: mknod da/d20/c35 0 2024-04-24T23:13:33.495 INFO:tasks.workunit.client.0.smithi031.stdout:0/160: write da/d11/f12 [297740,77043] 0 2024-04-24T23:13:33.496 INFO:tasks.workunit.client.0.smithi031.stdout:0/161: chown da/l21 1902456 1 2024-04-24T23:13:33.497 INFO:tasks.workunit.client.0.smithi031.stdout:0/162: dread - da/d18/d1a/f29 zero size 2024-04-24T23:13:33.503 INFO:tasks.workunit.client.0.smithi031.stdout:4/155: rmdir d3/d1c/d2e 0 2024-04-24T23:13:33.504 INFO:tasks.workunit.client.0.smithi031.stdout:0/163: dwrite da/ff [0,4194304] 0 2024-04-24T23:13:33.505 INFO:tasks.workunit.client.0.smithi031.stdout:0/164: fdatasync da/d22/f2d 0 2024-04-24T23:13:33.506 INFO:tasks.workunit.client.0.smithi031.stdout:8/132: link d8/c2b d8/c2c 0 2024-04-24T23:13:33.510 INFO:tasks.workunit.client.0.smithi031.stdout:1/132: chown d1/dc/d11/c27 1515612620 1 2024-04-24T23:13:33.511 INFO:tasks.workunit.client.0.smithi031.stdout:1/133: truncate d1/ff 1226643 0 2024-04-24T23:13:33.512 INFO:tasks.workunit.client.0.smithi031.stdout:6/245: mknod d13/d1b/d32/c60 0 2024-04-24T23:13:33.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/109: creat d1/d1c/f20 x:0 0 0 2024-04-24T23:13:33.543 INFO:tasks.workunit.client.0.smithi031.stdout:2/49: sync 2024-04-24T23:13:33.544 INFO:tasks.workunit.client.0.smithi031.stdout:3/174: sync 2024-04-24T23:13:33.548 INFO:tasks.workunit.client.0.smithi031.stdout:8/133: rmdir d8/d15 39 2024-04-24T23:13:33.548 INFO:tasks.workunit.client.0.smithi031.stdout:1/134: dread d1/f8 [0,4194304] 0 2024-04-24T23:13:33.548 INFO:tasks.workunit.client.0.smithi031.stdout:2/50: chown d5 31509033 1 2024-04-24T23:13:33.549 INFO:tasks.workunit.client.0.smithi031.stdout:9/205: getdents d2/d1b 0 2024-04-24T23:13:33.553 INFO:tasks.workunit.client.0.smithi031.stdout:6/246: rename d13/d1b/f1c to d13/d15/d1d/d23/d2d/f61 0 2024-04-24T23:13:33.555 INFO:tasks.workunit.client.0.smithi031.stdout:2/51: dread f4 [4194304,4194304] 0 2024-04-24T23:13:33.555 INFO:tasks.workunit.client.0.smithi031.stdout:6/247: readlink d13/d1b/d32/l5c 0 2024-04-24T23:13:33.558 INFO:tasks.workunit.client.0.smithi031.stdout:9/206: dwrite d2/f8 [0,4194304] 0 2024-04-24T23:13:33.559 INFO:tasks.workunit.client.0.smithi031.stdout:6/248: write d13/d15/d1d/d23/f2b [2093217,98337] 0 2024-04-24T23:13:33.561 INFO:tasks.workunit.client.0.smithi031.stdout:6/249: stat d13/d1b/f22 0 2024-04-24T23:13:33.562 INFO:tasks.workunit.client.0.smithi031.stdout:0/165: dread f3 [0,4194304] 0 2024-04-24T23:13:33.563 INFO:tasks.workunit.client.0.smithi031.stdout:0/166: readlink da/d11/l16 0 2024-04-24T23:13:33.566 INFO:tasks.workunit.client.0.smithi031.stdout:8/134: creat d8/d1b/f2d x:0 0 0 2024-04-24T23:13:33.571 INFO:tasks.workunit.client.0.smithi031.stdout:0/167: dwrite da/d22/f27 [0,4194304] 0 2024-04-24T23:13:33.572 INFO:tasks.workunit.client.0.smithi031.stdout:0/168: dread - da/d18/d1a/f1f zero size 2024-04-24T23:13:33.576 INFO:tasks.workunit.client.0.smithi031.stdout:0/169: dread da/f10 [0,4194304] 0 2024-04-24T23:13:33.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/135: mkdir d1/d2d 0 2024-04-24T23:13:33.587 INFO:tasks.workunit.client.0.smithi031.stdout:1/136: fsync d1/dc/f2b 0 2024-04-24T23:13:33.590 INFO:tasks.workunit.client.0.smithi031.stdout:6/250: rename d13/d15/d1d/f5e to d13/d15/d1d/d23/d2d/f62 0 2024-04-24T23:13:33.594 INFO:tasks.workunit.client.0.smithi031.stdout:5/99: truncate d0/d2/f5 3011647 0 2024-04-24T23:13:33.595 INFO:tasks.workunit.client.0.smithi031.stdout:5/100: write d0/d2/f17 [2559139,94363] 0 2024-04-24T23:13:33.597 INFO:tasks.workunit.client.0.smithi031.stdout:8/135: write d8/d15/f2a [289081,122488] 0 2024-04-24T23:13:33.602 INFO:tasks.workunit.client.0.smithi031.stdout:2/52: mkdir d5/d6/df 0 2024-04-24T23:13:33.605 INFO:tasks.workunit.client.0.smithi031.stdout:7/110: link d1/d2/f4 d1/f21 0 2024-04-24T23:13:33.608 INFO:tasks.workunit.client.0.smithi031.stdout:0/170: symlink da/d11/l36 0 2024-04-24T23:13:33.614 INFO:tasks.workunit.client.0.smithi031.stdout:1/137: symlink d1/da/d10/l2e 0 2024-04-24T23:13:33.617 INFO:tasks.workunit.client.0.smithi031.stdout:3/175: dwrite f0 [0,4194304] 0 2024-04-24T23:13:33.619 INFO:tasks.workunit.client.0.smithi031.stdout:6/251: mkdir d13/d15/d1d/d23/d24/d63 0 2024-04-24T23:13:33.619 INFO:tasks.workunit.client.0.smithi031.stdout:6/252: chown d13/d15/d1d/d23/d4d/l52 6697755 1 2024-04-24T23:13:33.624 INFO:tasks.workunit.client.0.smithi031.stdout:5/101: mknod d0/d7/c1f 0 2024-04-24T23:13:33.627 INFO:tasks.workunit.client.0.smithi031.stdout:8/136: creat d8/d1b/f2e x:0 0 0 2024-04-24T23:13:33.627 INFO:tasks.workunit.client.0.smithi031.stdout:8/137: fsync d8/f22 0 2024-04-24T23:13:33.631 INFO:tasks.workunit.client.0.smithi031.stdout:8/138: read d8/fc [1099074,1374] 0 2024-04-24T23:13:33.631 INFO:tasks.workunit.client.0.smithi031.stdout:7/111: write d1/f21 [183597,90925] 0 2024-04-24T23:13:33.633 INFO:tasks.workunit.client.0.smithi031.stdout:7/112: chown d1/d2/d1e/c1f 125974027 1 2024-04-24T23:13:33.635 INFO:tasks.workunit.client.0.smithi031.stdout:0/171: rename da/d18/d1b to da/d18/d1a/d37 0 2024-04-24T23:13:33.639 INFO:tasks.workunit.client.0.smithi031.stdout:8/139: dwrite d8/d15/f2a [0,4194304] 0 2024-04-24T23:13:33.641 INFO:tasks.workunit.client.0.smithi031.stdout:1/138: creat d1/d1d/f2f x:0 0 0 2024-04-24T23:13:33.646 INFO:tasks.workunit.client.0.smithi031.stdout:3/176: symlink d6/df/d16/l38 0 2024-04-24T23:13:33.656 INFO:tasks.workunit.client.0.smithi031.stdout:0/172: fsync f3 0 2024-04-24T23:13:33.657 INFO:tasks.workunit.client.0.smithi031.stdout:0/173: read - da/fe zero size 2024-04-24T23:13:33.658 INFO:tasks.workunit.client.0.smithi031.stdout:7/113: dwrite d1/d2/dd/f15 [0,4194304] 0 2024-04-24T23:13:33.658 INFO:tasks.workunit.client.0.smithi031.stdout:9/207: link d2/d1b/c11 d2/c36 0 2024-04-24T23:13:33.667 INFO:tasks.workunit.client.0.smithi031.stdout:6/253: symlink d13/d15/l64 0 2024-04-24T23:13:33.670 INFO:tasks.workunit.client.0.smithi031.stdout:5/102: rename d0/d2/c10 to d0/d7/db/dc/c20 0 2024-04-24T23:13:33.671 INFO:tasks.workunit.client.0.smithi031.stdout:5/103: chown d0/l16 2651 1 2024-04-24T23:13:33.678 INFO:tasks.workunit.client.0.smithi031.stdout:7/114: mknod d1/d1c/c22 0 2024-04-24T23:13:33.680 INFO:tasks.workunit.client.0.smithi031.stdout:8/140: creat d8/d1a/f2f x:0 0 0 2024-04-24T23:13:33.682 INFO:tasks.workunit.client.0.smithi031.stdout:8/141: read d8/fc [2241337,31785] 0 2024-04-24T23:13:33.682 INFO:tasks.workunit.client.0.smithi031.stdout:3/177: truncate d6/d14/d1d/f1f 1270772 0 2024-04-24T23:13:33.684 INFO:tasks.workunit.client.0.smithi031.stdout:6/254: mknod d13/d1b/d32/c65 0 2024-04-24T23:13:33.688 INFO:tasks.workunit.client.0.smithi031.stdout:8/142: dwrite d8/f18 [0,4194304] 0 2024-04-24T23:13:33.689 INFO:tasks.workunit.client.0.smithi031.stdout:0/174: rename da/d11/c15 to da/d18/d1a/c38 0 2024-04-24T23:13:33.690 INFO:tasks.workunit.client.0.smithi031.stdout:0/175: truncate da/fe 208675 0 2024-04-24T23:13:33.695 INFO:tasks.workunit.client.0.smithi031.stdout:9/208: symlink d2/l37 0 2024-04-24T23:13:33.697 INFO:tasks.workunit.client.0.smithi031.stdout:7/115: creat d1/d2/dd/f23 x:0 0 0 2024-04-24T23:13:33.697 INFO:tasks.workunit.client.0.smithi031.stdout:7/116: fsync d1/d2/dd/f17 0 2024-04-24T23:13:33.698 INFO:tasks.workunit.client.0.smithi031.stdout:3/178: fdatasync d6/d14/f33 0 2024-04-24T23:13:33.699 INFO:tasks.workunit.client.0.smithi031.stdout:3/179: read f0 [3411418,29171] 0 2024-04-24T23:13:33.700 INFO:tasks.workunit.client.0.smithi031.stdout:3/180: read d6/df/d16/d2a/f2e [645965,60012] 0 2024-04-24T23:13:33.701 INFO:tasks.workunit.client.0.smithi031.stdout:6/255: mknod d13/d1b/d32/c66 0 2024-04-24T23:13:33.706 INFO:tasks.workunit.client.0.smithi031.stdout:8/143: write d8/f24 [2853774,37714] 0 2024-04-24T23:13:33.710 INFO:tasks.workunit.client.0.smithi031.stdout:0/176: read da/d18/d1a/f1e [134636,12273] 0 2024-04-24T23:13:33.710 INFO:tasks.workunit.client.0.smithi031.stdout:3/181: dwrite d6/d14/f33 [0,4194304] 0 2024-04-24T23:13:33.710 INFO:tasks.workunit.client.0.smithi031.stdout:0/177: write da/d22/f27 [685933,1096] 0 2024-04-24T23:13:33.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/209: symlink d2/d16/l38 0 2024-04-24T23:13:33.716 INFO:tasks.workunit.client.0.smithi031.stdout:7/117: creat d1/d2/d1e/f24 x:0 0 0 2024-04-24T23:13:33.725 INFO:tasks.workunit.client.0.smithi031.stdout:5/104: write d0/d6/f1a [3051513,30452] 0 2024-04-24T23:13:33.726 INFO:tasks.workunit.client.0.smithi031.stdout:5/105: dread - d0/d6/f1e zero size 2024-04-24T23:13:33.733 INFO:tasks.workunit.client.0.smithi031.stdout:3/182: stat d6/df/d16/d2a/l2d 0 2024-04-24T23:13:33.736 INFO:tasks.workunit.client.0.smithi031.stdout:0/178: rmdir da/d11 39 2024-04-24T23:13:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:9/210: creat d2/d1b/d14/f39 x:0 0 0 2024-04-24T23:13:33.745 INFO:tasks.workunit.client.0.smithi031.stdout:9/211: dwrite d2/f32 [0,4194304] 0 2024-04-24T23:13:33.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/106: creat d0/d6/d12/f21 x:0 0 0 2024-04-24T23:13:33.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/107: chown d0/d2/l14 2 1 2024-04-24T23:13:33.752 INFO:tasks.workunit.client.0.smithi031.stdout:8/144: symlink d8/l30 0 2024-04-24T23:13:33.755 INFO:tasks.workunit.client.0.smithi031.stdout:0/179: rename c0 to da/d20/c39 0 2024-04-24T23:13:33.755 INFO:tasks.workunit.client.0.smithi031.stdout:0/180: readlink l4 0 2024-04-24T23:13:33.756 INFO:tasks.workunit.client.0.smithi031.stdout:0/181: dread da/d18/d1a/f1e [0,4194304] 0 2024-04-24T23:13:33.757 INFO:tasks.workunit.client.0.smithi031.stdout:0/182: fsync da/d22/f27 0 2024-04-24T23:13:33.757 INFO:tasks.workunit.client.0.smithi031.stdout:7/118: link d1/d2/f12 d1/d1c/f25 0 2024-04-24T23:13:33.760 INFO:tasks.workunit.client.0.smithi031.stdout:9/212: mknod d2/d1b/d14/d1d/c3a 0 2024-04-24T23:13:33.761 INFO:tasks.workunit.client.0.smithi031.stdout:9/213: dread - d2/d1b/d14/d1d/f28 zero size 2024-04-24T23:13:33.761 INFO:tasks.workunit.client.0.smithi031.stdout:9/214: dread - d2/d1b/f2f zero size 2024-04-24T23:13:33.770 INFO:tasks.workunit.client.0.smithi031.stdout:5/108: creat d0/d6/f22 x:0 0 0 2024-04-24T23:13:33.770 INFO:tasks.workunit.client.0.smithi031.stdout:9/215: fdatasync d2/f32 0 2024-04-24T23:13:33.770 INFO:tasks.workunit.client.0.smithi031.stdout:7/119: symlink d1/l26 0 2024-04-24T23:13:33.771 INFO:tasks.workunit.client.0.smithi031.stdout:5/109: dread - d0/d2/f11 zero size 2024-04-24T23:13:33.772 INFO:tasks.workunit.client.0.smithi031.stdout:5/110: fsync d0/d2/f11 0 2024-04-24T23:13:33.773 INFO:tasks.workunit.client.0.smithi031.stdout:9/216: dread - d2/d1b/d14/f39 zero size 2024-04-24T23:13:33.774 INFO:tasks.workunit.client.0.smithi031.stdout:5/111: read - d0/d6/f22 zero size 2024-04-24T23:13:33.774 INFO:tasks.workunit.client.0.smithi031.stdout:9/217: chown d2/d16/f20 7 1 2024-04-24T23:13:33.777 INFO:tasks.workunit.client.0.smithi031.stdout:7/120: dread d1/ff [0,4194304] 0 2024-04-24T23:13:33.779 INFO:tasks.workunit.client.0.smithi031.stdout:8/145: mkdir d8/d31 0 2024-04-24T23:13:33.781 INFO:tasks.workunit.client.0.smithi031.stdout:6/256: write d13/d15/d1d/d23/d24/d43/f5a [288607,117658] 0 2024-04-24T23:13:33.781 INFO:tasks.workunit.client.0.smithi031.stdout:5/112: dwrite d0/d6/f1a [0,4194304] 0 2024-04-24T23:13:33.784 INFO:tasks.workunit.client.0.smithi031.stdout:0/183: rename da/d18/d1a/l1c to da/d11/l3a 0 2024-04-24T23:13:33.790 INFO:tasks.workunit.client.0.smithi031.stdout:9/218: unlink d2/f9 0 2024-04-24T23:13:33.793 INFO:tasks.workunit.client.0.smithi031.stdout:7/121: chown d1/c1b 1 1 2024-04-24T23:13:33.803 INFO:tasks.workunit.client.0.smithi031.stdout:0/184: mkdir da/d18/d3b 0 2024-04-24T23:13:33.812 INFO:tasks.workunit.client.0.smithi031.stdout:5/113: getdents d0/d18 0 2024-04-24T23:13:33.814 INFO:tasks.workunit.client.0.smithi031.stdout:6/257: creat d13/d15/d1d/d23/d24/d63/f67 x:0 0 0 2024-04-24T23:13:33.822 INFO:tasks.workunit.client.0.smithi031.stdout:8/146: link d8/d1b/l1e d8/d1a/l32 0 2024-04-24T23:13:33.824 INFO:tasks.workunit.client.0.smithi031.stdout:5/114: creat d0/d7/db/dc/f23 x:0 0 0 2024-04-24T23:13:33.826 INFO:tasks.workunit.client.0.smithi031.stdout:6/258: mkdir d13/d15/d1d/d23/d24/d43/d68 0 2024-04-24T23:13:33.830 INFO:tasks.workunit.client.0.smithi031.stdout:9/219: link d2/d1b/d14/f19 d2/d1b/d14/f3b 0 2024-04-24T23:13:33.833 INFO:tasks.workunit.client.0.smithi031.stdout:6/259: dwrite d13/d15/d1d/d23/d2d/d36/f39 [0,4194304] 0 2024-04-24T23:13:33.833 INFO:tasks.workunit.client.0.smithi031.stdout:8/147: unlink d8/c23 0 2024-04-24T23:13:33.833 INFO:tasks.workunit.client.0.smithi031.stdout:7/122: write d1/d2/f12 [3094281,51669] 0 2024-04-24T23:13:33.838 INFO:tasks.workunit.client.0.smithi031.stdout:7/123: write d1/d1c/f25 [433599,21894] 0 2024-04-24T23:13:33.843 INFO:tasks.workunit.client.0.smithi031.stdout:5/115: rename d0/d6/f1e to d0/d7/db/dc/f24 0 2024-04-24T23:13:33.846 INFO:tasks.workunit.client.0.smithi031.stdout:0/185: write da/d18/d1a/f1e [1311432,87461] 0 2024-04-24T23:13:33.847 INFO:tasks.workunit.client.0.smithi031.stdout:0/186: write da/d11/f12 [3614873,97318] 0 2024-04-24T23:13:33.855 INFO:tasks.workunit.client.0.smithi031.stdout:0/187: dwrite da/d22/f2d [0,4194304] 0 2024-04-24T23:13:33.856 INFO:tasks.workunit.client.0.smithi031.stdout:0/188: read f6 [22704,95035] 0 2024-04-24T23:13:33.857 INFO:tasks.workunit.client.0.smithi031.stdout:7/124: symlink d1/d1c/l27 0 2024-04-24T23:13:33.864 INFO:tasks.workunit.client.0.smithi031.stdout:8/148: rename d8/fe to d8/d15/f33 0 2024-04-24T23:13:33.866 INFO:tasks.workunit.client.0.smithi031.stdout:7/125: dwrite d1/d2/f4 [0,4194304] 0 2024-04-24T23:13:33.869 INFO:tasks.workunit.client.0.smithi031.stdout:5/116: mkdir d0/d7/db/dc/d25 0 2024-04-24T23:13:33.877 INFO:tasks.workunit.client.0.smithi031.stdout:0/189: rmdir da/d22 39 2024-04-24T23:13:33.880 INFO:tasks.workunit.client.0.smithi031.stdout:0/190: dread da/d11/f33 [0,4194304] 0 2024-04-24T23:13:33.887 INFO:tasks.workunit.client.0.smithi031.stdout:8/149: fsync d8/fb 0 2024-04-24T23:13:33.890 INFO:tasks.workunit.client.0.smithi031.stdout:7/126: creat d1/f28 x:0 0 0 2024-04-24T23:13:33.891 INFO:tasks.workunit.client.0.smithi031.stdout:7/127: write d1/d2/f12 [1988210,78490] 0 2024-04-24T23:13:33.892 INFO:tasks.workunit.client.0.smithi031.stdout:7/128: write d1/d2/d1e/f24 [170407,84137] 0 2024-04-24T23:13:33.892 INFO:tasks.workunit.client.0.smithi031.stdout:5/117: chown d0/d2/f5 220 1 2024-04-24T23:13:33.893 INFO:tasks.workunit.client.0.smithi031.stdout:5/118: readlink d0/d7/l3 0 2024-04-24T23:13:33.901 INFO:tasks.workunit.client.0.smithi031.stdout:0/191: read da/f13 [199573,26095] 0 2024-04-24T23:13:33.908 INFO:tasks.workunit.client.0.smithi031.stdout:5/119: creat d0/d7/db/dc/f26 x:0 0 0 2024-04-24T23:13:33.909 INFO:tasks.workunit.client.0.smithi031.stdout:6/260: getdents d13/d15/d1d/d23/d24/d63 0 2024-04-24T23:13:33.912 INFO:tasks.workunit.client.0.smithi031.stdout:0/192: creat da/d18/d1a/d37/f3c x:0 0 0 2024-04-24T23:13:33.913 INFO:tasks.workunit.client.0.smithi031.stdout:9/220: rename d2/d1b/d14/f3b to d2/d1b/d14/d1d/f3c 0 2024-04-24T23:13:33.917 INFO:tasks.workunit.client.0.smithi031.stdout:7/129: link d1/d2/d1e/f24 d1/d1c/f29 0 2024-04-24T23:13:33.918 INFO:tasks.workunit.client.0.smithi031.stdout:6/261: mkdir d13/d15/d1d/d23/d44/d47/d69 0 2024-04-24T23:13:33.921 INFO:tasks.workunit.client.0.smithi031.stdout:5/120: rename d0/d2 to d0/d18/d27 0 2024-04-24T23:13:33.921 INFO:tasks.workunit.client.0.smithi031.stdout:4/156: sync 2024-04-24T23:13:33.922 INFO:tasks.workunit.client.0.smithi031.stdout:5/121: readlink d0/d18/d27/l14 0 2024-04-24T23:13:33.922 INFO:tasks.workunit.client.0.smithi031.stdout:9/221: symlink d2/d1b/d14/d1d/l3d 0 2024-04-24T23:13:33.929 INFO:tasks.workunit.client.0.smithi031.stdout:6/262: rmdir d13/d15/d1d/d23 39 2024-04-24T23:13:33.930 INFO:tasks.workunit.client.0.smithi031.stdout:6/263: chown d13/d1b/d32/c65 384 1 2024-04-24T23:13:33.933 INFO:tasks.workunit.client.0.smithi031.stdout:5/122: unlink d0/d6/ce 0 2024-04-24T23:13:33.933 INFO:tasks.workunit.client.0.smithi031.stdout:5/123: chown d0/d7/db/dc 24 1 2024-04-24T23:13:33.935 INFO:tasks.workunit.client.0.smithi031.stdout:9/222: symlink d2/d1b/d14/d1d/l3e 0 2024-04-24T23:13:33.937 INFO:tasks.workunit.client.0.smithi031.stdout:5/124: dread d0/d6/f1a [0,4194304] 0 2024-04-24T23:13:33.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/150: getdents d8/d1b 0 2024-04-24T23:13:33.939 INFO:tasks.workunit.client.0.smithi031.stdout:8/151: read - d8/f22 zero size 2024-04-24T23:13:33.939 INFO:tasks.workunit.client.0.smithi031.stdout:5/125: dread - d0/d6/f22 zero size 2024-04-24T23:13:33.943 INFO:tasks.workunit.client.0.smithi031.stdout:6/264: fdatasync d13/d15/d1d/d23/d24/d43/f4b 0 2024-04-24T23:13:33.944 INFO:tasks.workunit.client.0.smithi031.stdout:4/157: unlink d3/c10 0 2024-04-24T23:13:33.944 INFO:tasks.workunit.client.0.smithi031.stdout:9/223: dwrite d2/d1b/d14/f39 [0,4194304] 0 2024-04-24T23:13:33.945 INFO:tasks.workunit.client.0.smithi031.stdout:4/158: chown d3/dd/f13 1585 1 2024-04-24T23:13:33.946 INFO:tasks.workunit.client.0.smithi031.stdout:9/224: fsync d2/d1b/d14/d1d/f24 0 2024-04-24T23:13:33.949 INFO:tasks.workunit.client.0.smithi031.stdout:7/130: creat d1/d2/f2a x:0 0 0 2024-04-24T23:13:33.953 INFO:tasks.workunit.client.0.smithi031.stdout:8/152: creat d8/d1b/f34 x:0 0 0 2024-04-24T23:13:33.956 INFO:tasks.workunit.client.0.smithi031.stdout:5/126: mkdir d0/d18/d28 0 2024-04-24T23:13:33.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/265: creat d13/d15/d1d/d23/d4d/f6a x:0 0 0 2024-04-24T23:13:33.962 INFO:tasks.workunit.client.0.smithi031.stdout:8/153: dwrite d8/d1a/f21 [0,4194304] 0 2024-04-24T23:13:33.964 INFO:tasks.workunit.client.0.smithi031.stdout:8/154: read d8/f24 [2348317,127529] 0 2024-04-24T23:13:33.965 INFO:tasks.workunit.client.0.smithi031.stdout:9/225: symlink d2/d16/l3f 0 2024-04-24T23:13:33.966 INFO:tasks.workunit.client.0.smithi031.stdout:8/155: chown d8/fd 902367730 1 2024-04-24T23:13:33.966 INFO:tasks.workunit.client.0.smithi031.stdout:4/159: dread d3/f7 [0,4194304] 0 2024-04-24T23:13:33.969 INFO:tasks.workunit.client.0.smithi031.stdout:7/131: creat d1/f2b x:0 0 0 2024-04-24T23:13:33.974 INFO:tasks.workunit.client.0.smithi031.stdout:6/266: creat d13/d1b/f6b x:0 0 0 2024-04-24T23:13:33.979 INFO:tasks.workunit.client.0.smithi031.stdout:8/156: mkdir d8/d1a/d1d/d35 0 2024-04-24T23:13:33.981 INFO:tasks.workunit.client.0.smithi031.stdout:4/160: rmdir d3/d23 39 2024-04-24T23:13:33.986 INFO:tasks.workunit.client.0.smithi031.stdout:8/157: dwrite d8/d1a/f2f [0,4194304] 0 2024-04-24T23:13:33.987 INFO:tasks.workunit.client.0.smithi031.stdout:4/161: dread d3/f16 [0,4194304] 0 2024-04-24T23:13:34.000 INFO:tasks.workunit.client.0.smithi031.stdout:7/132: rmdir d1 39 2024-04-24T23:13:34.004 INFO:tasks.workunit.client.0.smithi031.stdout:5/127: getdents d0/d6/d12 0 2024-04-24T23:13:34.006 INFO:tasks.workunit.client.0.smithi031.stdout:4/162: mknod d3/c32 0 2024-04-24T23:13:34.008 INFO:tasks.workunit.client.0.smithi031.stdout:8/158: link d8/f18 d8/d1a/f36 0 2024-04-24T23:13:34.008 INFO:tasks.workunit.client.0.smithi031.stdout:5/128: dread d0/d18/d27/f5 [0,4194304] 0 2024-04-24T23:13:34.010 INFO:tasks.workunit.client.0.smithi031.stdout:5/129: truncate d0/d6/f22 25884 0 2024-04-24T23:13:34.010 INFO:tasks.workunit.client.0.smithi031.stdout:6/267: creat d13/f6c x:0 0 0 2024-04-24T23:13:34.012 INFO:tasks.workunit.client.0.smithi031.stdout:9/226: getdents d2/d1b/d14 0 2024-04-24T23:13:34.013 INFO:tasks.workunit.client.0.smithi031.stdout:9/227: write d2/fe [1194019,120891] 0 2024-04-24T23:13:34.017 INFO:tasks.workunit.client.0.smithi031.stdout:8/159: rename d8/d1b to d8/d15/d37 0 2024-04-24T23:13:34.018 INFO:tasks.workunit.client.0.smithi031.stdout:7/133: rmdir d1/d1c 39 2024-04-24T23:13:34.019 INFO:tasks.workunit.client.0.smithi031.stdout:8/160: write d8/f14 [4035054,61159] 0 2024-04-24T23:13:34.019 INFO:tasks.workunit.client.0.smithi031.stdout:7/134: fsync d1/d2/f2a 0 2024-04-24T23:13:34.021 INFO:tasks.workunit.client.0.smithi031.stdout:7/135: write d1/f13 [5068090,50124] 0 2024-04-24T23:13:34.028 INFO:tasks.workunit.client.0.smithi031.stdout:5/130: rename d0/d18/d27/f5 to d0/d7/db/dc/d25/f29 0 2024-04-24T23:13:34.029 INFO:tasks.workunit.client.0.smithi031.stdout:8/161: unlink d8/l29 0 2024-04-24T23:13:34.032 INFO:tasks.workunit.client.0.smithi031.stdout:6/268: link d13/f6c d13/d1b/d32/f6d 0 2024-04-24T23:13:34.034 INFO:tasks.workunit.client.0.smithi031.stdout:5/131: dwrite d0/d18/d27/f4 [0,4194304] 0 2024-04-24T23:13:34.036 INFO:tasks.workunit.client.0.smithi031.stdout:5/132: dread - d0/d7/db/dc/f23 zero size 2024-04-24T23:13:34.037 INFO:tasks.workunit.client.0.smithi031.stdout:7/136: mkdir d1/d1c/d2c 0 2024-04-24T23:13:34.038 INFO:tasks.workunit.client.0.smithi031.stdout:7/137: chown d1/d2/f7 15919 1 2024-04-24T23:13:34.040 INFO:tasks.workunit.client.0.smithi031.stdout:4/163: link d3/dd/f30 d3/dd/f33 0 2024-04-24T23:13:34.044 INFO:tasks.workunit.client.0.smithi031.stdout:9/228: getdents d2 0 2024-04-24T23:13:34.048 INFO:tasks.workunit.client.0.smithi031.stdout:5/133: mknod d0/d7/db/d1d/c2a 0 2024-04-24T23:13:34.050 INFO:tasks.workunit.client.0.smithi031.stdout:4/164: symlink d3/l34 0 2024-04-24T23:13:34.059 INFO:tasks.workunit.client.0.smithi031.stdout:7/138: link d1/d2/f2a d1/d2/f2d 0 2024-04-24T23:13:34.062 INFO:tasks.workunit.client.0.smithi031.stdout:8/162: link d8/d15/f33 d8/f38 0 2024-04-24T23:13:34.062 INFO:tasks.workunit.client.0.smithi031.stdout:7/139: dread d1/d2/d1e/f24 [0,4194304] 0 2024-04-24T23:13:34.063 INFO:tasks.workunit.client.0.smithi031.stdout:7/140: fdatasync d1/d2/f19 0 2024-04-24T23:13:34.064 INFO:tasks.workunit.client.0.smithi031.stdout:5/134: mknod d0/d18/d28/c2b 0 2024-04-24T23:13:34.066 INFO:tasks.workunit.client.0.smithi031.stdout:4/165: mknod d3/d23/c35 0 2024-04-24T23:13:34.067 INFO:tasks.workunit.client.0.smithi031.stdout:4/166: write d3/f2b [3564465,118840] 0 2024-04-24T23:13:34.070 INFO:tasks.workunit.client.0.smithi031.stdout:9/229: creat d2/d1b/f40 x:0 0 0 2024-04-24T23:13:34.072 INFO:tasks.workunit.client.0.smithi031.stdout:8/163: mknod d8/d1a/d1d/d35/c39 0 2024-04-24T23:13:34.075 INFO:tasks.workunit.client.0.smithi031.stdout:4/167: read d3/f16 [457501,29440] 0 2024-04-24T23:13:34.076 INFO:tasks.workunit.client.0.smithi031.stdout:9/230: chown d2/d1b/c11 31 1 2024-04-24T23:13:34.084 INFO:tasks.workunit.client.0.smithi031.stdout:9/231: dread - d2/d1b/f2f zero size 2024-04-24T23:13:34.085 INFO:tasks.workunit.client.0.smithi031.stdout:8/164: symlink d8/d31/l3a 0 2024-04-24T23:13:34.131 INFO:tasks.workunit.client.0.smithi031.stdout:2/53: sync 2024-04-24T23:13:34.132 INFO:tasks.workunit.client.0.smithi031.stdout:7/141: truncate d1/d2/f12 1272525 0 2024-04-24T23:13:34.132 INFO:tasks.workunit.client.0.smithi031.stdout:8/165: mknod d8/d1a/d1d/d35/c3b 0 2024-04-24T23:13:34.135 INFO:tasks.workunit.client.0.smithi031.stdout:2/54: creat d5/d8/f10 x:0 0 0 2024-04-24T23:13:34.137 INFO:tasks.workunit.client.0.smithi031.stdout:8/166: symlink d8/d15/l3c 0 2024-04-24T23:13:34.138 INFO:tasks.workunit.client.0.smithi031.stdout:2/55: rename d5 to d5/d8/dd/d11 22 2024-04-24T23:13:34.139 INFO:tasks.workunit.client.0.smithi031.stdout:7/142: symlink d1/l2e 0 2024-04-24T23:13:34.139 INFO:tasks.workunit.client.0.smithi031.stdout:4/168: dwrite d3/fb [0,4194304] 0 2024-04-24T23:13:34.140 INFO:tasks.workunit.client.0.smithi031.stdout:2/56: dread - d5/d8/f10 zero size 2024-04-24T23:13:34.141 INFO:tasks.workunit.client.0.smithi031.stdout:7/143: fdatasync d1/d2/dd/f15 0 2024-04-24T23:13:34.141 INFO:tasks.workunit.client.0.smithi031.stdout:9/232: rename d2/d1b/f34 to d2/d1b/f41 0 2024-04-24T23:13:34.143 INFO:tasks.workunit.client.0.smithi031.stdout:8/167: creat d8/d15/f3d x:0 0 0 2024-04-24T23:13:34.146 INFO:tasks.workunit.client.0.smithi031.stdout:4/169: fdatasync d3/f7 0 2024-04-24T23:13:34.150 INFO:tasks.workunit.client.0.smithi031.stdout:9/233: dwrite d2/d1b/d14/f39 [0,4194304] 0 2024-04-24T23:13:34.156 INFO:tasks.workunit.client.0.smithi031.stdout:7/144: unlink d1/d1c/f20 0 2024-04-24T23:13:34.161 INFO:tasks.workunit.client.0.smithi031.stdout:8/168: dread d8/fb [0,4194304] 0 2024-04-24T23:13:34.162 INFO:tasks.workunit.client.0.smithi031.stdout:4/170: rmdir d3 39 2024-04-24T23:13:34.163 INFO:tasks.workunit.client.0.smithi031.stdout:8/169: write d8/d1a/f2f [3743024,82615] 0 2024-04-24T23:13:34.164 INFO:tasks.workunit.client.0.smithi031.stdout:2/57: creat d5/d8/dd/f12 x:0 0 0 2024-04-24T23:13:34.165 INFO:tasks.workunit.client.0.smithi031.stdout:2/58: write d5/d8/f10 [261113,55231] 0 2024-04-24T23:13:34.167 INFO:tasks.workunit.client.0.smithi031.stdout:9/234: creat d2/f42 x:0 0 0 2024-04-24T23:13:34.174 INFO:tasks.workunit.client.0.smithi031.stdout:8/170: rmdir d8 39 2024-04-24T23:13:34.178 INFO:tasks.workunit.client.0.smithi031.stdout:2/59: rename d5/d8/fa to d5/f13 0 2024-04-24T23:13:34.193 INFO:tasks.workunit.client.0.smithi031.stdout:7/145: truncate d1/d2/fa 512063 0 2024-04-24T23:13:34.196 INFO:tasks.workunit.client.0.smithi031.stdout:8/171: creat d8/d15/f3e x:0 0 0 2024-04-24T23:13:34.199 INFO:tasks.workunit.client.0.smithi031.stdout:2/60: rename d5/l9 to d5/d8/dd/l14 0 2024-04-24T23:13:34.199 INFO:tasks.workunit.client.0.smithi031.stdout:7/146: dwrite f0 [0,4194304] 0 2024-04-24T23:13:34.199 INFO:tasks.workunit.client.0.smithi031.stdout:2/61: chown d5/d8/dd 26709269 1 2024-04-24T23:13:34.200 INFO:tasks.workunit.client.0.smithi031.stdout:7/147: chown d1/d2/f19 122363919 1 2024-04-24T23:13:34.201 INFO:tasks.workunit.client.0.smithi031.stdout:4/171: truncate d3/f9 2419944 0 2024-04-24T23:13:34.202 INFO:tasks.workunit.client.0.smithi031.stdout:2/62: truncate d5/d8/dd/f12 196508 0 2024-04-24T23:13:34.203 INFO:tasks.workunit.client.0.smithi031.stdout:9/235: mkdir d2/d43 0 2024-04-24T23:13:34.204 INFO:tasks.workunit.client.0.smithi031.stdout:7/148: write d1/d2/dd/f15 [3752119,53287] 0 2024-04-24T23:13:34.204 INFO:tasks.workunit.client.0.smithi031.stdout:2/63: write d5/d8/dd/f12 [759914,74110] 0 2024-04-24T23:13:34.209 INFO:tasks.workunit.client.0.smithi031.stdout:4/172: dread d3/dd/f30 [0,4194304] 0 2024-04-24T23:13:34.211 INFO:tasks.workunit.client.0.smithi031.stdout:7/149: unlink d1/d2/d1e/f24 0 2024-04-24T23:13:34.213 INFO:tasks.workunit.client.0.smithi031.stdout:2/64: mkdir d5/d8/d15 0 2024-04-24T23:13:34.216 INFO:tasks.workunit.client.0.smithi031.stdout:9/236: mknod d2/c44 0 2024-04-24T23:13:34.222 INFO:tasks.workunit.client.0.smithi031.stdout:4/173: symlink d3/dd/l36 0 2024-04-24T23:13:34.225 INFO:tasks.workunit.client.0.smithi031.stdout:2/65: read d5/f13 [3649032,124256] 0 2024-04-24T23:13:34.226 INFO:tasks.workunit.client.0.smithi031.stdout:9/237: rmdir d2 39 2024-04-24T23:13:34.229 INFO:tasks.workunit.client.0.smithi031.stdout:8/172: rename d8/d15/d37/l1e to d8/l3f 0 2024-04-24T23:13:34.233 INFO:tasks.workunit.client.0.smithi031.stdout:4/174: fsync d3/f7 0 2024-04-24T23:13:34.242 INFO:tasks.workunit.client.0.smithi031.stdout:8/173: unlink d8/d15/d37/c27 0 2024-04-24T23:13:34.248 INFO:tasks.workunit.client.0.smithi031.stdout:3/183: sync 2024-04-24T23:13:34.248 INFO:tasks.workunit.client.0.smithi031.stdout:1/139: sync 2024-04-24T23:13:34.249 INFO:tasks.workunit.client.0.smithi031.stdout:2/66: write d5/f13 [5316941,33978] 0 2024-04-24T23:13:34.250 INFO:tasks.workunit.client.0.smithi031.stdout:7/150: link d1/d2/d1e/c1f d1/d1c/c2f 0 2024-04-24T23:13:34.251 INFO:tasks.workunit.client.0.smithi031.stdout:1/140: chown d1/da/d12/f23 2 1 2024-04-24T23:13:34.256 INFO:tasks.workunit.client.0.smithi031.stdout:8/174: creat d8/d15/f40 x:0 0 0 2024-04-24T23:13:34.258 INFO:tasks.workunit.client.0.smithi031.stdout:9/238: dwrite d2/f32 [0,4194304] 0 2024-04-24T23:13:34.260 INFO:tasks.workunit.client.0.smithi031.stdout:3/184: rename d6/df/d16/d2a/l2d to d6/df/d16/d2a/l39 0 2024-04-24T23:13:34.262 INFO:tasks.workunit.client.0.smithi031.stdout:7/151: symlink d1/d2/d1e/l30 0 2024-04-24T23:13:34.265 INFO:tasks.workunit.client.0.smithi031.stdout:4/175: truncate d3/f7 5072282 0 2024-04-24T23:13:34.266 INFO:tasks.workunit.client.0.smithi031.stdout:1/141: truncate d1/f8 3672140 0 2024-04-24T23:13:34.267 INFO:tasks.workunit.client.0.smithi031.stdout:1/142: write d1/d1d/f2f [26673,31697] 0 2024-04-24T23:13:34.271 INFO:tasks.workunit.client.0.smithi031.stdout:2/67: link d5/d8/dd/f12 d5/d6/f16 0 2024-04-24T23:13:34.272 INFO:tasks.workunit.client.0.smithi031.stdout:4/176: mkdir d3/d23/d37 0 2024-04-24T23:13:34.272 INFO:tasks.workunit.client.0.smithi031.stdout:4/177: stat d3/dd/lf 0 2024-04-24T23:13:34.273 INFO:tasks.workunit.client.0.smithi031.stdout:4/178: chown d3/dd/le 17 1 2024-04-24T23:13:34.274 INFO:tasks.workunit.client.0.smithi031.stdout:8/175: rename d8/d15/f33 to d8/d1a/d1d/d35/f41 0 2024-04-24T23:13:34.275 INFO:tasks.workunit.client.0.smithi031.stdout:9/239: mkdir d2/d1b/d14/d1d/d45 0 2024-04-24T23:13:34.277 INFO:tasks.workunit.client.0.smithi031.stdout:1/143: mknod d1/da/d10/c30 0 2024-04-24T23:13:34.277 INFO:tasks.workunit.client.0.smithi031.stdout:0/193: sync 2024-04-24T23:13:34.287 INFO:tasks.workunit.client.0.smithi031.stdout:2/68: creat d5/d6/df/f17 x:0 0 0 2024-04-24T23:13:34.291 INFO:tasks.workunit.client.0.smithi031.stdout:7/152: symlink d1/d1c/l31 0 2024-04-24T23:13:34.295 INFO:tasks.workunit.client.0.smithi031.stdout:0/194: creat da/d18/d1a/d37/f3d x:0 0 0 2024-04-24T23:13:34.298 INFO:tasks.workunit.client.0.smithi031.stdout:7/153: dwrite d1/d2/f4 [0,4194304] 0 2024-04-24T23:13:34.299 INFO:tasks.workunit.client.0.smithi031.stdout:7/154: readlink d1/l6 0 2024-04-24T23:13:34.301 INFO:tasks.workunit.client.0.smithi031.stdout:3/185: getdents d6/d14/d1d 0 2024-04-24T23:13:34.305 INFO:tasks.workunit.client.0.smithi031.stdout:9/240: fsync d2/d1b/f41 0 2024-04-24T23:13:34.307 INFO:tasks.workunit.client.0.smithi031.stdout:9/241: chown d2/d1b/d14/f39 12542 1 2024-04-24T23:13:34.307 INFO:tasks.workunit.client.0.smithi031.stdout:8/176: write d8/fd [4588196,32134] 0 2024-04-24T23:13:34.309 INFO:tasks.workunit.client.0.smithi031.stdout:1/144: mkdir d1/d31 0 2024-04-24T23:13:34.318 INFO:tasks.workunit.client.0.smithi031.stdout:4/179: link d3/d1c/c1e d3/d1c/c38 0 2024-04-24T23:13:34.322 INFO:tasks.workunit.client.0.smithi031.stdout:3/186: symlink d6/df/d16/l3a 0 2024-04-24T23:13:34.325 INFO:tasks.workunit.client.0.smithi031.stdout:8/177: stat d8/f38 0 2024-04-24T23:13:34.327 INFO:tasks.workunit.client.0.smithi031.stdout:9/242: creat d2/d1b/d14/f46 x:0 0 0 2024-04-24T23:13:34.328 INFO:tasks.workunit.client.0.smithi031.stdout:1/145: symlink d1/da/l32 0 2024-04-24T23:13:34.332 INFO:tasks.workunit.client.0.smithi031.stdout:1/146: dread d1/f18 [0,4194304] 0 2024-04-24T23:13:34.333 INFO:tasks.workunit.client.0.smithi031.stdout:1/147: chown d1/da/d10/f1c 82950619 1 2024-04-24T23:13:34.336 INFO:tasks.workunit.client.0.smithi031.stdout:3/187: dread d6/df/f34 [0,4194304] 0 2024-04-24T23:13:34.337 INFO:tasks.workunit.client.0.smithi031.stdout:4/180: dread d3/f11 [0,4194304] 0 2024-04-24T23:13:34.338 INFO:tasks.workunit.client.0.smithi031.stdout:3/188: write d6/df/d16/d2a/f31 [898622,93974] 0 2024-04-24T23:13:34.339 INFO:tasks.workunit.client.0.smithi031.stdout:9/243: chown d2/d16/l38 1264174 1 2024-04-24T23:13:34.340 INFO:tasks.workunit.client.0.smithi031.stdout:9/244: write d2/d1b/f2f [88435,62647] 0 2024-04-24T23:13:34.342 INFO:tasks.workunit.client.0.smithi031.stdout:0/195: rmdir da/d18/d1a/d37/d31 0 2024-04-24T23:13:34.346 INFO:tasks.workunit.client.0.smithi031.stdout:7/155: mknod d1/d1c/c32 0 2024-04-24T23:13:34.348 INFO:tasks.workunit.client.0.smithi031.stdout:0/196: dwrite da/f2b [0,4194304] 0 2024-04-24T23:13:34.350 INFO:tasks.workunit.client.0.smithi031.stdout:8/178: mknod d8/c42 0 2024-04-24T23:13:34.355 INFO:tasks.workunit.client.0.smithi031.stdout:3/189: mknod d6/df/d16/c3b 0 2024-04-24T23:13:34.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/156: mkdir d1/d2/dd/d33 0 2024-04-24T23:13:34.369 INFO:tasks.workunit.client.0.smithi031.stdout:7/157: write d1/d2/f4 [4829186,87608] 0 2024-04-24T23:13:34.371 INFO:tasks.workunit.client.0.smithi031.stdout:0/197: truncate da/d22/f30 899777 0 2024-04-24T23:13:34.372 INFO:tasks.workunit.client.0.smithi031.stdout:8/179: fdatasync d8/f38 0 2024-04-24T23:13:34.375 INFO:tasks.workunit.client.0.smithi031.stdout:4/181: creat d3/d20/d2d/f39 x:0 0 0 2024-04-24T23:13:34.377 INFO:tasks.workunit.client.0.smithi031.stdout:3/190: mknod d6/df/d16/c3c 0 2024-04-24T23:13:34.385 INFO:tasks.workunit.client.0.smithi031.stdout:0/198: mknod da/d22/c3e 0 2024-04-24T23:13:34.388 INFO:tasks.workunit.client.0.smithi031.stdout:8/180: mkdir d8/d15/d37/d43 0 2024-04-24T23:13:34.390 INFO:tasks.workunit.client.0.smithi031.stdout:3/191: creat d6/d15/d1a/d25/f3d x:0 0 0 2024-04-24T23:13:34.391 INFO:tasks.workunit.client.0.smithi031.stdout:7/158: mknod d1/d2/dd/d33/c34 0 2024-04-24T23:13:34.394 INFO:tasks.workunit.client.0.smithi031.stdout:0/199: unlink f5 0 2024-04-24T23:13:34.395 INFO:tasks.workunit.client.0.smithi031.stdout:9/245: write d2/d1b/d14/f22 [5077370,73010] 0 2024-04-24T23:13:34.397 INFO:tasks.workunit.client.0.smithi031.stdout:9/246: fsync d2/f32 0 2024-04-24T23:13:34.398 INFO:tasks.workunit.client.0.smithi031.stdout:3/192: unlink d6/fe 0 2024-04-24T23:13:34.398 INFO:tasks.workunit.client.0.smithi031.stdout:9/247: fdatasync d2/f42 0 2024-04-24T23:13:34.399 INFO:tasks.workunit.client.0.smithi031.stdout:7/159: dwrite d1/d2/f7 [0,4194304] 0 2024-04-24T23:13:34.400 INFO:tasks.workunit.client.0.smithi031.stdout:9/248: fdatasync d2/d1b/d14/d18/f21 0 2024-04-24T23:13:34.401 INFO:tasks.workunit.client.0.smithi031.stdout:7/160: dread - d1/f28 zero size 2024-04-24T23:13:34.401 INFO:tasks.workunit.client.0.smithi031.stdout:7/161: stat d1/c1b 0 2024-04-24T23:13:34.406 INFO:tasks.workunit.client.0.smithi031.stdout:1/148: dwrite d1/f2 [4194304,4194304] 0 2024-04-24T23:13:34.406 INFO:tasks.workunit.client.0.smithi031.stdout:1/149: stat d1/da 0 2024-04-24T23:13:34.408 INFO:tasks.workunit.client.0.smithi031.stdout:1/150: write d1/f2 [9339758,23904] 0 2024-04-24T23:13:34.410 INFO:tasks.workunit.client.0.smithi031.stdout:9/249: readlink d2/d1b/l25 0 2024-04-24T23:13:34.424 INFO:tasks.workunit.client.0.smithi031.stdout:9/250: dwrite d2/f8 [0,4194304] 0 2024-04-24T23:13:34.425 INFO:tasks.workunit.client.0.smithi031.stdout:0/200: rename da/d22/f2d to da/f3f 0 2024-04-24T23:13:34.435 INFO:tasks.workunit.client.0.smithi031.stdout:3/193: rename c5 to d6/d14/c3e 0 2024-04-24T23:13:34.438 INFO:tasks.workunit.client.0.smithi031.stdout:0/201: mkdir da/d18/d1a/d37/d40 0 2024-04-24T23:13:34.443 INFO:tasks.workunit.client.0.smithi031.stdout:6/269: sync 2024-04-24T23:13:34.443 INFO:tasks.workunit.client.0.smithi031.stdout:5/135: sync 2024-04-24T23:13:34.444 INFO:tasks.workunit.client.0.smithi031.stdout:6/270: write d13/d15/d1d/d23/d24/d43/f4b [1000813,6637] 0 2024-04-24T23:13:34.448 INFO:tasks.workunit.client.0.smithi031.stdout:5/136: mknod d0/d7/db/c2c 0 2024-04-24T23:13:34.450 INFO:tasks.workunit.client.0.smithi031.stdout:6/271: fsync d13/f14 0 2024-04-24T23:13:34.454 INFO:tasks.workunit.client.0.smithi031.stdout:3/194: dread d6/d14/d1d/f24 [0,4194304] 0 2024-04-24T23:13:34.457 INFO:tasks.workunit.client.0.smithi031.stdout:6/272: creat d13/d1b/f6e x:0 0 0 2024-04-24T23:13:34.462 INFO:tasks.workunit.client.0.smithi031.stdout:3/195: rename d6/l12 to d6/d15/d1a/l3f 0 2024-04-24T23:13:34.464 INFO:tasks.workunit.client.0.smithi031.stdout:6/273: dwrite d13/d15/d1d/d23/d2d/f4c [0,4194304] 0 2024-04-24T23:13:34.470 INFO:tasks.workunit.client.0.smithi031.stdout:6/274: dwrite d13/d1b/f54 [0,4194304] 0 2024-04-24T23:13:34.471 INFO:tasks.workunit.client.0.smithi031.stdout:3/196: rename d6/cd to d6/df/d16/d2a/c40 0 2024-04-24T23:13:34.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/275: chown d13/d15/d1d/d23/d24/d43/d68 647760622 1 2024-04-24T23:13:34.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/276: dread - d13/f14 zero size 2024-04-24T23:13:34.473 INFO:tasks.workunit.client.0.smithi031.stdout:7/162: write d1/d1c/f25 [1048338,50776] 0 2024-04-24T23:13:34.475 INFO:tasks.workunit.client.0.smithi031.stdout:6/277: fdatasync d13/d15/d1d/d23/d44/d47/f4a 0 2024-04-24T23:13:34.486 INFO:tasks.workunit.client.0.smithi031.stdout:6/278: unlink d13/d15/d1d/d23/d44/d47/l51 0 2024-04-24T23:13:34.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/279: write d13/d15/d1d/d23/d24/d43/f5a [3424273,128165] 0 2024-04-24T23:13:34.493 INFO:tasks.workunit.client.0.smithi031.stdout:3/197: dread d6/f8 [0,4194304] 0 2024-04-24T23:13:34.493 INFO:tasks.workunit.client.0.smithi031.stdout:3/198: stat d6/d15/d29/f36 0 2024-04-24T23:13:34.494 INFO:tasks.workunit.client.0.smithi031.stdout:3/199: write d6/d14/f33 [3632892,59170] 0 2024-04-24T23:13:34.496 INFO:tasks.workunit.client.0.smithi031.stdout:6/280: creat d13/d15/d1d/d23/f6f x:0 0 0 2024-04-24T23:13:34.496 INFO:tasks.workunit.client.0.smithi031.stdout:5/137: write d0/d6/f1a [5344583,107610] 0 2024-04-24T23:13:34.498 INFO:tasks.workunit.client.0.smithi031.stdout:7/163: creat d1/d1c/f35 x:0 0 0 2024-04-24T23:13:34.499 INFO:tasks.workunit.client.0.smithi031.stdout:7/164: fsync d1/d2/f19 0 2024-04-24T23:13:34.499 INFO:tasks.workunit.client.0.smithi031.stdout:7/165: chown d1/d1c 20 1 2024-04-24T23:13:34.502 INFO:tasks.workunit.client.0.smithi031.stdout:3/200: mknod d6/d14/c41 0 2024-04-24T23:13:34.504 INFO:tasks.workunit.client.0.smithi031.stdout:5/138: rmdir d0/d7/db/d1d 39 2024-04-24T23:13:34.506 INFO:tasks.workunit.client.0.smithi031.stdout:6/281: symlink d13/d15/d1d/d23/d4d/l70 0 2024-04-24T23:13:34.507 INFO:tasks.workunit.client.0.smithi031.stdout:6/282: chown d13/d15/d1d/d23/d2d/f3d 0 1 2024-04-24T23:13:34.508 INFO:tasks.workunit.client.0.smithi031.stdout:5/139: dread d0/d6/f1a [0,4194304] 0 2024-04-24T23:13:34.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/140: stat d0/d7/db/dc/f26 0 2024-04-24T23:13:34.509 INFO:tasks.workunit.client.0.smithi031.stdout:7/166: unlink d1/d1c/f25 0 2024-04-24T23:13:34.512 INFO:tasks.workunit.client.0.smithi031.stdout:3/201: symlink d6/df/d16/l42 0 2024-04-24T23:13:34.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/167: dwrite d1/d2/f4 [0,4194304] 0 2024-04-24T23:13:34.520 INFO:tasks.workunit.client.0.smithi031.stdout:5/141: mknod d0/d6/c2d 0 2024-04-24T23:13:34.521 INFO:tasks.workunit.client.0.smithi031.stdout:5/142: dread - d0/d18/d27/f11 zero size 2024-04-24T23:13:34.524 INFO:tasks.workunit.client.0.smithi031.stdout:3/202: creat d6/d14/f43 x:0 0 0 2024-04-24T23:13:34.530 INFO:tasks.workunit.client.0.smithi031.stdout:6/283: getdents d13/d15/d1d/d23/d44/d47/d69 0 2024-04-24T23:13:34.533 INFO:tasks.workunit.client.0.smithi031.stdout:3/203: symlink d6/df/d16/d2a/l44 0 2024-04-24T23:13:34.537 INFO:tasks.workunit.client.0.smithi031.stdout:6/284: rename d13/d15/d1d/d23/f6f to d13/d15/d1d/d23/d24/d43/f71 0 2024-04-24T23:13:34.539 INFO:tasks.workunit.client.0.smithi031.stdout:3/204: write d6/df/f34 [830675,82916] 0 2024-04-24T23:13:34.545 INFO:tasks.workunit.client.0.smithi031.stdout:5/143: rename d0/d18 to d0/d7/db/d1d/d2e 0 2024-04-24T23:13:34.546 INFO:tasks.workunit.client.0.smithi031.stdout:5/144: write d0/d7/db/d1d/d2e/d27/f4 [4858484,5386] 0 2024-04-24T23:13:34.552 INFO:tasks.workunit.client.0.smithi031.stdout:5/145: mknod d0/d7/db/d1d/c2f 0 2024-04-24T23:13:34.553 INFO:tasks.workunit.client.0.smithi031.stdout:6/285: getdents d13/d15/d1d/d23/d44/d47/d69 0 2024-04-24T23:13:34.556 INFO:tasks.workunit.client.0.smithi031.stdout:5/146: truncate d0/d6/f1a 5089369 0 2024-04-24T23:13:34.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/147: creat d0/d7/db/d1d/d2e/d28/f30 x:0 0 0 2024-04-24T23:13:34.561 INFO:tasks.workunit.client.0.smithi031.stdout:5/148: dread - d0/d7/db/dc/f23 zero size 2024-04-24T23:13:34.562 INFO:tasks.workunit.client.0.smithi031.stdout:5/149: chown d0/d7/db/d1d/c2a 217596738 1 2024-04-24T23:13:34.563 INFO:tasks.workunit.client.0.smithi031.stdout:6/286: link d13/d15/d1d/d23/f2b d13/d15/d1d/d23/d24/d43/d68/f72 0 2024-04-24T23:13:34.565 INFO:tasks.workunit.client.0.smithi031.stdout:5/150: creat d0/d6/d12/f31 x:0 0 0 2024-04-24T23:13:34.568 INFO:tasks.workunit.client.0.smithi031.stdout:7/168: dwrite d1/d1c/f29 [0,4194304] 0 2024-04-24T23:13:34.568 INFO:tasks.workunit.client.0.smithi031.stdout:6/287: creat d13/d15/d1d/d23/d24/f73 x:0 0 0 2024-04-24T23:13:34.571 INFO:tasks.workunit.client.0.smithi031.stdout:6/288: dread d13/d15/d1d/d23/d44/f4e [0,4194304] 0 2024-04-24T23:13:34.580 INFO:tasks.workunit.client.0.smithi031.stdout:3/205: dwrite f0 [0,4194304] 0 2024-04-24T23:13:34.587 INFO:tasks.workunit.client.0.smithi031.stdout:7/169: fdatasync d1/d2/f12 0 2024-04-24T23:13:34.588 INFO:tasks.workunit.client.0.smithi031.stdout:3/206: mknod d6/df/c45 0 2024-04-24T23:13:34.593 INFO:tasks.workunit.client.0.smithi031.stdout:7/170: creat d1/d2/d1e/f36 x:0 0 0 2024-04-24T23:13:34.594 INFO:tasks.workunit.client.0.smithi031.stdout:7/171: write d1/d2/f7 [4440369,74897] 0 2024-04-24T23:13:34.614 INFO:tasks.workunit.client.0.smithi031.stdout:2/69: sync 2024-04-24T23:13:34.614 INFO:tasks.workunit.client.0.smithi031.stdout:7/172: symlink d1/d1c/l37 0 2024-04-24T23:13:34.615 INFO:tasks.workunit.client.0.smithi031.stdout:2/70: stat d5/d8/dd 0 2024-04-24T23:13:34.616 INFO:tasks.workunit.client.0.smithi031.stdout:7/173: write d1/d2/dd/f15 [4544741,104275] 0 2024-04-24T23:13:34.616 INFO:tasks.workunit.client.0.smithi031.stdout:2/71: write f4 [9197242,5551] 0 2024-04-24T23:13:34.618 INFO:tasks.workunit.client.0.smithi031.stdout:2/72: dread - d5/d6/df/f17 zero size 2024-04-24T23:13:34.625 INFO:tasks.workunit.client.0.smithi031.stdout:7/174: dread d1/d2/fa [0,4194304] 0 2024-04-24T23:13:34.626 INFO:tasks.workunit.client.0.smithi031.stdout:2/73: fsync d5/d6/f16 0 2024-04-24T23:13:34.626 INFO:tasks.workunit.client.0.smithi031.stdout:7/175: write d1/d2/f11 [1208111,107041] 0 2024-04-24T23:13:34.627 INFO:tasks.workunit.client.0.smithi031.stdout:2/74: write f4 [3848046,9427] 0 2024-04-24T23:13:34.628 INFO:tasks.workunit.client.0.smithi031.stdout:2/75: fsync d5/d6/df/f17 0 2024-04-24T23:13:34.631 INFO:tasks.workunit.client.0.smithi031.stdout:6/289: write d13/d1b/f3c [4076201,127557] 0 2024-04-24T23:13:34.633 INFO:tasks.workunit.client.0.smithi031.stdout:6/290: truncate d13/d15/d1d/d23/d2d/f62 54997 0 2024-04-24T23:13:34.639 INFO:tasks.workunit.client.0.smithi031.stdout:5/151: dwrite d0/d6/f1a [0,4194304] 0 2024-04-24T23:13:34.644 INFO:tasks.workunit.client.0.smithi031.stdout:2/76: mkdir d5/d8/d15/d18 0 2024-04-24T23:13:34.650 INFO:tasks.workunit.client.0.smithi031.stdout:7/176: mkdir d1/d1c/d2c/d38 0 2024-04-24T23:13:34.651 INFO:tasks.workunit.client.0.smithi031.stdout:7/177: dread - d1/f2b zero size 2024-04-24T23:13:34.651 INFO:tasks.workunit.client.0.smithi031.stdout:3/207: dwrite d6/df/d16/d2a/f2e [0,4194304] 0 2024-04-24T23:13:34.653 INFO:tasks.workunit.client.0.smithi031.stdout:7/178: truncate d1/f2b 383930 0 2024-04-24T23:13:34.653 INFO:tasks.workunit.client.0.smithi031.stdout:3/208: write d6/d14/f43 [479561,127067] 0 2024-04-24T23:13:34.654 INFO:tasks.workunit.client.0.smithi031.stdout:7/179: fsync d1/d1c/f29 0 2024-04-24T23:13:34.656 INFO:tasks.workunit.client.0.smithi031.stdout:3/209: chown d6/d15/d1a/d25/f3d 22806186 1 2024-04-24T23:13:34.656 INFO:tasks.workunit.client.0.smithi031.stdout:6/291: creat d13/d15/d1d/f74 x:0 0 0 2024-04-24T23:13:34.658 INFO:tasks.workunit.client.0.smithi031.stdout:6/292: write d13/d1b/f46 [808078,64] 0 2024-04-24T23:13:34.660 INFO:tasks.workunit.client.0.smithi031.stdout:5/152: unlink d0/d6/ca 0 2024-04-24T23:13:34.660 INFO:tasks.workunit.client.0.smithi031.stdout:5/153: read - d0/d6/d12/f21 zero size 2024-04-24T23:13:34.670 INFO:tasks.workunit.client.0.smithi031.stdout:6/293: truncate d13/f14 830213 0 2024-04-24T23:13:34.673 INFO:tasks.workunit.client.0.smithi031.stdout:5/154: rename d0/d7/db/d1d/d2e/d27 to d0/d6/d32 0 2024-04-24T23:13:34.677 INFO:tasks.workunit.client.0.smithi031.stdout:7/180: mknod d1/d1c/c39 0 2024-04-24T23:13:34.678 INFO:tasks.workunit.client.0.smithi031.stdout:7/181: stat d1/d1c/l31 0 2024-04-24T23:13:34.680 INFO:tasks.workunit.client.0.smithi031.stdout:5/155: dwrite d0/d7/db/dc/f26 [0,4194304] 0 2024-04-24T23:13:34.682 INFO:tasks.workunit.client.0.smithi031.stdout:6/294: write d13/d15/d1d/d23/d24/d43/f71 [136025,77924] 0 2024-04-24T23:13:34.685 INFO:tasks.workunit.client.0.smithi031.stdout:2/77: getdents d5/d6/df 0 2024-04-24T23:13:34.687 INFO:tasks.workunit.client.0.smithi031.stdout:7/182: dwrite d1/d2/f7 [0,4194304] 0 2024-04-24T23:13:34.690 INFO:tasks.workunit.client.0.smithi031.stdout:5/156: mknod d0/d6/d12/c33 0 2024-04-24T23:13:34.692 INFO:tasks.workunit.client.0.smithi031.stdout:6/295: rmdir d13/d1b 39 2024-04-24T23:13:34.693 INFO:tasks.workunit.client.0.smithi031.stdout:6/296: write d13/d15/d1d/d23/d4d/f6a [986235,55323] 0 2024-04-24T23:13:34.694 INFO:tasks.workunit.client.0.smithi031.stdout:2/78: creat d5/d8/d15/f19 x:0 0 0 2024-04-24T23:13:34.695 INFO:tasks.workunit.client.0.smithi031.stdout:6/297: chown d13/f3f 1116867 1 2024-04-24T23:13:34.697 INFO:tasks.workunit.client.0.smithi031.stdout:7/183: read d1/ff [1836267,113679] 0 2024-04-24T23:13:34.698 INFO:tasks.workunit.client.0.smithi031.stdout:7/184: dread - d1/d2/f2a zero size 2024-04-24T23:13:34.698 INFO:tasks.workunit.client.0.smithi031.stdout:5/157: symlink d0/d6/l34 0 2024-04-24T23:13:34.699 INFO:tasks.workunit.client.0.smithi031.stdout:7/185: chown d1/l2e 978 1 2024-04-24T23:13:34.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/158: fdatasync d0/d6/d12/f31 0 2024-04-24T23:13:34.700 INFO:tasks.workunit.client.0.smithi031.stdout:7/186: stat d1/f21 0 2024-04-24T23:13:34.701 INFO:tasks.workunit.client.0.smithi031.stdout:2/79: dread d5/d8/dd/f12 [0,4194304] 0 2024-04-24T23:13:34.701 INFO:tasks.workunit.client.0.smithi031.stdout:6/298: unlink d13/d15/d1d/d23/c48 0 2024-04-24T23:13:34.701 INFO:tasks.workunit.client.0.smithi031.stdout:6/299: chown f12 8110 1 2024-04-24T23:13:34.702 INFO:tasks.workunit.client.0.smithi031.stdout:7/187: write d1/d2/f19 [361149,78471] 0 2024-04-24T23:13:34.709 INFO:tasks.workunit.client.0.smithi031.stdout:5/159: dwrite d0/d7/db/dc/f23 [0,4194304] 0 2024-04-24T23:13:34.709 INFO:tasks.workunit.client.0.smithi031.stdout:6/300: fdatasync f10 0 2024-04-24T23:13:34.711 INFO:tasks.workunit.client.0.smithi031.stdout:6/301: chown d13/f6c 1437774 1 2024-04-24T23:13:34.711 INFO:tasks.workunit.client.0.smithi031.stdout:5/160: write d0/d6/d32/f4 [4518236,56785] 0 2024-04-24T23:13:34.718 INFO:tasks.workunit.client.0.smithi031.stdout:6/302: dread - d13/d1b/f6e zero size 2024-04-24T23:13:34.723 INFO:tasks.workunit.client.0.smithi031.stdout:5/161: mknod d0/d7/db/d1d/c35 0 2024-04-24T23:13:34.723 INFO:tasks.workunit.client.0.smithi031.stdout:3/210: write d6/d14/d1d/f1f [1077667,117971] 0 2024-04-24T23:13:34.734 INFO:tasks.workunit.client.0.smithi031.stdout:3/211: symlink d6/d14/l46 0 2024-04-24T23:13:34.734 INFO:tasks.workunit.client.0.smithi031.stdout:4/182: sync 2024-04-24T23:13:34.734 INFO:tasks.workunit.client.0.smithi031.stdout:3/212: fsync d6/d14/d1d/f1f 0 2024-04-24T23:13:34.737 INFO:tasks.workunit.client.0.smithi031.stdout:4/183: rmdir d3/dd 39 2024-04-24T23:13:34.739 INFO:tasks.workunit.client.0.smithi031.stdout:3/213: dwrite d6/df/f34 [0,4194304] 0 2024-04-24T23:13:34.742 INFO:tasks.workunit.client.0.smithi031.stdout:6/303: rename d13/d15/d1d/d23/d2d/d36 to d13/d75 0 2024-04-24T23:13:34.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:34 smithi031 ceph-mon[26411]: pgmap v20: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 393 KiB/s rd, 15 MiB/s wr, 197 op/s 2024-04-24T23:13:34.752 INFO:tasks.workunit.client.0.smithi031.stdout:3/214: dwrite d6/d14/d1d/f24 [0,4194304] 0 2024-04-24T23:13:34.752 INFO:tasks.workunit.client.0.smithi031.stdout:6/304: creat d13/d15/f76 x:0 0 0 2024-04-24T23:13:34.755 INFO:tasks.workunit.client.0.smithi031.stdout:4/184: getdents d3/d20 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:2/80: write d5/d8/dd/f12 [1782604,54209] 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:4/185: write d3/f1b [4150233,69548] 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:2/81: write d5/f13 [119121,118859] 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:6/305: rename d13/d15/d1d/d23/f35 to d13/d15/d1d/d23/d24/f77 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:4/186: creat d3/d23/f3a x:0 0 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/188: truncate d1/f9 718147 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:4/187: chown d3/f9 22 1 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:2/82: creat d5/d6/f1a x:0 0 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:6/306: dwrite d13/d15/d1d/d23/d24/f73 [0,4194304] 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/189: mknod d1/d1c/d2c/c3a 0 2024-04-24T23:13:34.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/190: write d1/f21 [4476406,117455] 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/83: rmdir d5/d8/d15 39 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/84: chown d5/d6/f16 8 1 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/215: dread d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/216: read d6/d15/d29/f36 [1129216,48879] 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/217: fsync d6/df/d16/d2a/f31 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:6/307: rmdir d13/d15/d1d 39 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/218: write d6/d15/d1a/d25/f3d [579715,123022] 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/85: mknod d5/d8/d15/c1b 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/86: chown d5/d6/df/f17 1 1 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/219: dread d6/f8 [0,4194304] 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/87: dread d5/d8/dd/f12 [0,4194304] 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:6/308: stat d13/d15/d1d/d23/d2d 0 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:6/309: chown d13/d1b/f46 122194375 1 2024-04-24T23:13:34.809 INFO:tasks.workunit.client.0.smithi031.stdout:7/191: rename f0 to d1/d2/f3b 0 2024-04-24T23:13:34.810 INFO:tasks.workunit.client.0.smithi031.stdout:2/88: dwrite d5/d6/f1a [0,4194304] 0 2024-04-24T23:13:34.810 INFO:tasks.workunit.client.0.smithi031.stdout:2/89: stat d5/d6/df/f17 0 2024-04-24T23:13:34.813 INFO:tasks.workunit.client.0.smithi031.stdout:7/192: chown d1/d1c/l27 56483216 1 2024-04-24T23:13:34.815 INFO:tasks.workunit.client.0.smithi031.stdout:6/310: dwrite d13/d15/d1d/d23/d2d/f62 [0,4194304] 0 2024-04-24T23:13:34.827 INFO:tasks.workunit.client.0.smithi031.stdout:6/311: mknod d13/d15/d1d/c78 0 2024-04-24T23:13:34.832 INFO:tasks.workunit.client.0.smithi031.stdout:6/312: mkdir d13/d75/d79 0 2024-04-24T23:13:34.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/90: getdents d5/d8/dd 0 2024-04-24T23:13:34.837 INFO:tasks.workunit.client.0.smithi031.stdout:2/91: creat d5/d6/f1c x:0 0 0 2024-04-24T23:13:34.840 INFO:tasks.workunit.client.0.smithi031.stdout:3/220: truncate d6/d14/f33 3389803 0 2024-04-24T23:13:34.843 INFO:tasks.workunit.client.0.smithi031.stdout:6/313: dwrite d13/d1b/f6b [0,4194304] 0 2024-04-24T23:13:34.845 INFO:tasks.workunit.client.0.smithi031.stdout:6/314: chown c5 210087639 1 2024-04-24T23:13:34.850 INFO:tasks.workunit.client.0.smithi031.stdout:6/315: read d13/d15/d1d/d23/d2d/f4c [1438056,102785] 0 2024-04-24T23:13:34.850 INFO:tasks.workunit.client.0.smithi031.stdout:2/92: creat d5/d8/f1d x:0 0 0 2024-04-24T23:13:34.852 INFO:tasks.workunit.client.0.smithi031.stdout:7/193: dwrite d1/d2/fa [0,4194304] 0 2024-04-24T23:13:34.856 INFO:tasks.workunit.client.0.smithi031.stdout:3/221: dwrite d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:34.859 INFO:tasks.workunit.client.0.smithi031.stdout:2/93: creat d5/d6/df/f1e x:0 0 0 2024-04-24T23:13:34.863 INFO:tasks.workunit.client.0.smithi031.stdout:3/222: mkdir d6/df/d16/d2a/d47 0 2024-04-24T23:13:34.864 INFO:tasks.workunit.client.0.smithi031.stdout:6/316: dwrite d13/d15/d1d/d23/d4d/f5f [0,4194304] 0 2024-04-24T23:13:34.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/94: rename d5/d6/f1c to d5/d6/f1f 0 2024-04-24T23:13:34.871 INFO:tasks.workunit.client.0.smithi031.stdout:7/194: dwrite d1/d2/f7 [0,4194304] 0 2024-04-24T23:13:34.873 INFO:tasks.workunit.client.0.smithi031.stdout:2/95: symlink d5/d6/df/l20 0 2024-04-24T23:13:34.874 INFO:tasks.workunit.client.0.smithi031.stdout:6/317: symlink d13/d15/d1d/l7a 0 2024-04-24T23:13:34.874 INFO:tasks.workunit.client.0.smithi031.stdout:3/223: dread d6/df/f34 [0,4194304] 0 2024-04-24T23:13:34.876 INFO:tasks.workunit.client.0.smithi031.stdout:6/318: truncate d13/d1b/f6e 192870 0 2024-04-24T23:13:34.876 INFO:tasks.workunit.client.0.smithi031.stdout:3/224: stat d6/df/d16/f1c 0 2024-04-24T23:13:34.879 INFO:tasks.workunit.client.0.smithi031.stdout:7/195: link d1/d2/dd/f17 d1/d1c/f3c 0 2024-04-24T23:13:34.881 INFO:tasks.workunit.client.0.smithi031.stdout:7/196: chown d1/d2/dd/d33/c34 728020085 1 2024-04-24T23:13:34.883 INFO:tasks.workunit.client.0.smithi031.stdout:3/225: creat d6/f48 x:0 0 0 2024-04-24T23:13:34.885 INFO:tasks.workunit.client.0.smithi031.stdout:7/197: rename d1/l6 to d1/d2/dd/d33/l3d 0 2024-04-24T23:13:34.895 INFO:tasks.workunit.client.0.smithi031.stdout:3/226: truncate d6/d14/f33 4104888 0 2024-04-24T23:13:34.898 INFO:tasks.workunit.client.0.smithi031.stdout:3/227: rmdir d6/df/d16/d2a 39 2024-04-24T23:13:34.901 INFO:tasks.workunit.client.0.smithi031.stdout:3/228: rmdir d6/df/d16 39 2024-04-24T23:13:34.904 INFO:tasks.workunit.client.0.smithi031.stdout:3/229: symlink d6/d14/d1d/l49 0 2024-04-24T23:13:34.906 INFO:tasks.workunit.client.0.smithi031.stdout:8/181: sync 2024-04-24T23:13:34.906 INFO:tasks.workunit.client.0.smithi031.stdout:1/151: sync 2024-04-24T23:13:34.907 INFO:tasks.workunit.client.0.smithi031.stdout:1/152: chown d1/dc/d11/d21 13 1 2024-04-24T23:13:34.910 INFO:tasks.workunit.client.0.smithi031.stdout:8/182: truncate d8/d15/d37/f2e 956399 0 2024-04-24T23:13:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:34 smithi040 ceph-mon[33128]: pgmap v20: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 393 KiB/s rd, 15 MiB/s wr, 197 op/s 2024-04-24T23:13:34.912 INFO:tasks.workunit.client.0.smithi031.stdout:1/153: mkdir d1/d33 0 2024-04-24T23:13:34.912 INFO:tasks.workunit.client.0.smithi031.stdout:8/183: fdatasync f3 0 2024-04-24T23:13:34.916 INFO:tasks.workunit.client.0.smithi031.stdout:2/96: dwrite d5/d6/f16 [0,4194304] 0 2024-04-24T23:13:34.917 INFO:tasks.workunit.client.0.smithi031.stdout:8/184: chown d8/l30 405197074 1 2024-04-24T23:13:34.918 INFO:tasks.workunit.client.0.smithi031.stdout:7/198: write d1/d2/f2d [848665,107261] 0 2024-04-24T23:13:34.924 INFO:tasks.workunit.client.0.smithi031.stdout:3/230: dwrite d6/f8 [0,4194304] 0 2024-04-24T23:13:34.926 INFO:tasks.workunit.client.0.smithi031.stdout:6/319: dwrite d13/d15/d1d/d23/d44/f4e [0,4194304] 0 2024-04-24T23:13:34.929 INFO:tasks.workunit.client.0.smithi031.stdout:7/199: write d1/d2/dd/f23 [411002,42019] 0 2024-04-24T23:13:34.930 INFO:tasks.workunit.client.0.smithi031.stdout:3/231: write d6/f8 [751212,2478] 0 2024-04-24T23:13:34.932 INFO:tasks.workunit.client.0.smithi031.stdout:8/185: fsync d8/d1a/d1d/d35/f41 0 2024-04-24T23:13:34.933 INFO:tasks.workunit.client.0.smithi031.stdout:1/154: dwrite d1/f2 [4194304,4194304] 0 2024-04-24T23:13:34.937 INFO:tasks.workunit.client.0.smithi031.stdout:7/200: chown d1/ff 29759360 1 2024-04-24T23:13:34.937 INFO:tasks.workunit.client.0.smithi031.stdout:2/97: rename d5/d8/f1d to d5/d8/dd/f21 0 2024-04-24T23:13:34.939 INFO:tasks.workunit.client.0.smithi031.stdout:6/320: mkdir d13/d15/d1d/d23/d4d/d7b 0 2024-04-24T23:13:34.941 INFO:tasks.workunit.client.0.smithi031.stdout:8/186: dread d8/d15/d37/f2e [0,4194304] 0 2024-04-24T23:13:34.941 INFO:tasks.workunit.client.0.smithi031.stdout:8/187: readlink l4 0 2024-04-24T23:13:34.945 INFO:tasks.workunit.client.0.smithi031.stdout:1/155: dread d1/f2 [4194304,4194304] 0 2024-04-24T23:13:34.946 INFO:tasks.workunit.client.0.smithi031.stdout:1/156: chown d1 8012 1 2024-04-24T23:13:34.947 INFO:tasks.workunit.client.0.smithi031.stdout:1/157: chown d1/da/l1a 1 1 2024-04-24T23:13:34.962 INFO:tasks.workunit.client.0.smithi031.stdout:6/321: rename d13/d15/d1d/d23/d24/f73 to d13/f7c 0 2024-04-24T23:13:34.965 INFO:tasks.workunit.client.0.smithi031.stdout:2/98: getdents d5/d6 0 2024-04-24T23:13:34.965 INFO:tasks.workunit.client.0.smithi031.stdout:2/99: read - d5/d8/dd/f21 zero size 2024-04-24T23:13:34.967 INFO:tasks.workunit.client.0.smithi031.stdout:2/100: dread - d5/d6/df/f1e zero size 2024-04-24T23:13:34.967 INFO:tasks.workunit.client.0.smithi031.stdout:8/188: getdents d8/d15/d37 0 2024-04-24T23:13:34.970 INFO:tasks.workunit.client.0.smithi031.stdout:6/322: rmdir d13/d15/d1d/d23/d2d 39 2024-04-24T23:13:34.971 INFO:tasks.workunit.client.0.smithi031.stdout:8/189: dread d8/d15/d37/f2e [0,4194304] 0 2024-04-24T23:13:34.973 INFO:tasks.workunit.client.0.smithi031.stdout:7/201: truncate d1/d2/dd/f23 74547 0 2024-04-24T23:13:34.975 INFO:tasks.workunit.client.0.smithi031.stdout:2/101: dwrite d5/d6/df/f1e [0,4194304] 0 2024-04-24T23:13:34.977 INFO:tasks.workunit.client.0.smithi031.stdout:1/158: truncate d1/dc/d11/d21/f28 730705 0 2024-04-24T23:13:34.979 INFO:tasks.workunit.client.0.smithi031.stdout:8/190: creat d8/d31/f44 x:0 0 0 2024-04-24T23:13:34.980 INFO:tasks.workunit.client.0.smithi031.stdout:6/323: dwrite f10 [0,4194304] 0 2024-04-24T23:13:34.982 INFO:tasks.workunit.client.0.smithi031.stdout:8/191: truncate d8/f22 470254 0 2024-04-24T23:13:34.986 INFO:tasks.workunit.client.0.smithi031.stdout:6/324: dread - d13/d15/d1d/f74 zero size 2024-04-24T23:13:34.986 INFO:tasks.workunit.client.0.smithi031.stdout:7/202: mknod d1/d1c/c3e 0 2024-04-24T23:13:34.987 INFO:tasks.workunit.client.0.smithi031.stdout:8/192: dread - d8/d15/f40 zero size 2024-04-24T23:13:34.989 INFO:tasks.workunit.client.0.smithi031.stdout:1/159: dwrite d1/d1d/f2f [0,4194304] 0 2024-04-24T23:13:34.992 INFO:tasks.workunit.client.0.smithi031.stdout:8/193: symlink d8/d31/l45 0 2024-04-24T23:13:34.996 INFO:tasks.workunit.client.0.smithi031.stdout:8/194: symlink d8/d15/l46 0 2024-04-24T23:13:34.999 INFO:tasks.workunit.client.0.smithi031.stdout:2/102: getdents d5 0 2024-04-24T23:13:35.001 INFO:tasks.workunit.client.0.smithi031.stdout:2/103: dread - d5/d8/dd/f21 zero size 2024-04-24T23:13:35.002 INFO:tasks.workunit.client.0.smithi031.stdout:2/104: chown d5/d6 40 1 2024-04-24T23:13:35.005 INFO:tasks.workunit.client.0.smithi031.stdout:8/195: dwrite d8/d15/d37/f34 [0,4194304] 0 2024-04-24T23:13:35.005 INFO:tasks.workunit.client.0.smithi031.stdout:1/160: rename d1/f29 to d1/f34 0 2024-04-24T23:13:35.006 INFO:tasks.workunit.client.0.smithi031.stdout:2/105: read d5/d8/f10 [50997,59377] 0 2024-04-24T23:13:35.007 INFO:tasks.workunit.client.0.smithi031.stdout:2/106: chown d5/d8/dd/f12 1701655 1 2024-04-24T23:13:35.010 INFO:tasks.workunit.client.0.smithi031.stdout:6/325: link d13/d1b/d32/c66 d13/d15/d1d/d23/c7d 0 2024-04-24T23:13:35.016 INFO:tasks.workunit.client.0.smithi031.stdout:2/107: dwrite d5/d6/df/f17 [0,4194304] 0 2024-04-24T23:13:35.026 INFO:tasks.workunit.client.0.smithi031.stdout:2/108: creat d5/d8/d15/f22 x:0 0 0 2024-04-24T23:13:35.028 INFO:tasks.workunit.client.0.smithi031.stdout:1/161: rmdir d1/d2d 0 2024-04-24T23:13:35.029 INFO:tasks.workunit.client.0.smithi031.stdout:1/162: write d1/dc/fe [1487512,85152] 0 2024-04-24T23:13:35.031 INFO:tasks.workunit.client.0.smithi031.stdout:2/109: rename d5/d8/dd to d5/d8/d15/d23 0 2024-04-24T23:13:35.035 INFO:tasks.workunit.client.0.smithi031.stdout:8/196: getdents d8/d1a/d1d 0 2024-04-24T23:13:35.036 INFO:tasks.workunit.client.0.smithi031.stdout:1/163: write d1/f34 [160334,127171] 0 2024-04-24T23:13:35.037 INFO:tasks.workunit.client.0.smithi031.stdout:2/110: creat d5/f24 x:0 0 0 2024-04-24T23:13:35.037 INFO:tasks.workunit.client.0.smithi031.stdout:2/111: chown d5/d6 64328489 1 2024-04-24T23:13:35.038 INFO:tasks.workunit.client.0.smithi031.stdout:8/197: mkdir d8/d31/d47 0 2024-04-24T23:13:35.039 INFO:tasks.workunit.client.0.smithi031.stdout:8/198: dread - d8/d15/f3d zero size 2024-04-24T23:13:35.042 INFO:tasks.workunit.client.0.smithi031.stdout:2/112: creat d5/f25 x:0 0 0 2024-04-24T23:13:35.043 INFO:tasks.workunit.client.0.smithi031.stdout:2/113: stat d5/d6/f1a 0 2024-04-24T23:13:35.045 INFO:tasks.workunit.client.0.smithi031.stdout:6/326: dwrite d13/d15/f27 [0,4194304] 0 2024-04-24T23:13:35.045 INFO:tasks.workunit.client.0.smithi031.stdout:8/199: rename d8/c19 to d8/d15/d37/c48 0 2024-04-24T23:13:35.052 INFO:tasks.workunit.client.0.smithi031.stdout:1/164: dwrite d1/dc/d11/f20 [4194304,4194304] 0 2024-04-24T23:13:35.054 INFO:tasks.workunit.client.0.smithi031.stdout:6/327: mknod d13/d15/d1d/d23/d2d/c7e 0 2024-04-24T23:13:35.055 INFO:tasks.workunit.client.0.smithi031.stdout:8/200: creat d8/d31/d47/f49 x:0 0 0 2024-04-24T23:13:35.059 INFO:tasks.workunit.client.0.smithi031.stdout:1/165: creat d1/d1d/f35 x:0 0 0 2024-04-24T23:13:35.061 INFO:tasks.workunit.client.0.smithi031.stdout:1/166: chown d1/dc/d11/d21/l26 933792 1 2024-04-24T23:13:35.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/201: creat d8/d1a/d1d/d35/f4a x:0 0 0 2024-04-24T23:13:35.065 INFO:tasks.workunit.client.0.smithi031.stdout:8/202: dread - d8/f28 zero size 2024-04-24T23:13:35.066 INFO:tasks.workunit.client.0.smithi031.stdout:6/328: creat d13/d15/d1d/d23/d44/d47/d69/f7f x:0 0 0 2024-04-24T23:13:35.067 INFO:tasks.workunit.client.0.smithi031.stdout:6/329: write d13/d15/d1d/d23/d2d/f3d [3426585,123177] 0 2024-04-24T23:13:35.069 INFO:tasks.workunit.client.0.smithi031.stdout:1/167: rmdir d1/dc 39 2024-04-24T23:13:35.070 INFO:tasks.workunit.client.0.smithi031.stdout:8/203: rmdir d8 39 2024-04-24T23:13:35.073 INFO:tasks.workunit.client.0.smithi031.stdout:6/330: rmdir d13/d75 39 2024-04-24T23:13:35.076 INFO:tasks.workunit.client.0.smithi031.stdout:8/204: chown d8/d1a/d1d/l26 12303050 1 2024-04-24T23:13:35.077 INFO:tasks.workunit.client.0.smithi031.stdout:8/205: chown d8/d15/l46 18 1 2024-04-24T23:13:35.079 INFO:tasks.workunit.client.0.smithi031.stdout:1/168: mkdir d1/d33/d36 0 2024-04-24T23:13:35.082 INFO:tasks.workunit.client.0.smithi031.stdout:1/169: chown d1/dc 56023255 1 2024-04-24T23:13:35.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/206: creat d8/d15/d37/d43/f4b x:0 0 0 2024-04-24T23:13:35.091 INFO:tasks.workunit.client.0.smithi031.stdout:6/331: dread d13/d15/d1d/d23/d24/d43/d68/f72 [0,4194304] 0 2024-04-24T23:13:35.091 INFO:tasks.workunit.client.0.smithi031.stdout:1/170: dwrite d1/f2 [0,4194304] 0 2024-04-24T23:13:35.093 INFO:tasks.workunit.client.0.smithi031.stdout:1/171: chown d1/da/d12/f19 6975 1 2024-04-24T23:13:35.098 INFO:tasks.workunit.client.0.smithi031.stdout:1/172: mknod d1/dc/d11/d21/c37 0 2024-04-24T23:13:35.098 INFO:tasks.workunit.client.0.smithi031.stdout:8/207: dwrite d8/d1a/f36 [0,4194304] 0 2024-04-24T23:13:35.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/173: chown d1/f34 1108598567 1 2024-04-24T23:13:35.101 INFO:tasks.workunit.client.0.smithi031.stdout:8/208: fdatasync d8/d15/d37/f2d 0 2024-04-24T23:13:35.105 INFO:tasks.workunit.client.0.smithi031.stdout:6/332: dwrite d13/d15/d1d/d23/d2d/f61 [0,4194304] 0 2024-04-24T23:13:35.106 INFO:tasks.workunit.client.0.smithi031.stdout:8/209: creat d8/d31/f4c x:0 0 0 2024-04-24T23:13:35.107 INFO:tasks.workunit.client.0.smithi031.stdout:9/251: sync 2024-04-24T23:13:35.107 INFO:tasks.workunit.client.0.smithi031.stdout:0/202: sync 2024-04-24T23:13:35.107 INFO:tasks.workunit.client.0.smithi031.stdout:5/162: sync 2024-04-24T23:13:35.107 INFO:tasks.workunit.client.0.smithi031.stdout:4/188: sync 2024-04-24T23:13:35.108 INFO:tasks.workunit.client.0.smithi031.stdout:0/203: stat da/d22/c2e 0 2024-04-24T23:13:35.108 INFO:tasks.workunit.client.0.smithi031.stdout:5/163: rmdir d0 39 2024-04-24T23:13:35.108 INFO:tasks.workunit.client.0.smithi031.stdout:8/210: stat d8/d15/c17 0 2024-04-24T23:13:35.109 INFO:tasks.workunit.client.0.smithi031.stdout:4/189: write d3/d23/f3a [1001664,60465] 0 2024-04-24T23:13:35.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/333: creat d13/d15/d1d/d23/d44/f80 x:0 0 0 2024-04-24T23:13:35.111 INFO:tasks.workunit.client.0.smithi031.stdout:9/252: creat d2/d1b/d14/d1d/f47 x:0 0 0 2024-04-24T23:13:35.112 INFO:tasks.workunit.client.0.smithi031.stdout:0/204: write da/f3f [1193867,74066] 0 2024-04-24T23:13:35.145 INFO:tasks.workunit.client.0.smithi031.stdout:8/211: mknod d8/d15/c4d 0 2024-04-24T23:13:35.145 INFO:tasks.workunit.client.0.smithi031.stdout:9/253: truncate d2/d16/f2a 480805 0 2024-04-24T23:13:35.148 INFO:tasks.workunit.client.0.smithi031.stdout:0/205: dread da/d22/f30 [0,4194304] 0 2024-04-24T23:13:35.150 INFO:tasks.workunit.client.0.smithi031.stdout:5/164: dread d0/d6/d32/f4 [0,4194304] 0 2024-04-24T23:13:35.151 INFO:tasks.workunit.client.0.smithi031.stdout:9/254: truncate d2/fe 1767418 0 2024-04-24T23:13:35.153 INFO:tasks.workunit.client.0.smithi031.stdout:8/212: rename d8/d15/d37/f34 to d8/d15/d37/f4e 0 2024-04-24T23:13:35.153 INFO:tasks.workunit.client.0.smithi031.stdout:6/334: getdents d13/d75/d79 0 2024-04-24T23:13:35.154 INFO:tasks.workunit.client.0.smithi031.stdout:6/335: dread - d13/d15/d1d/d23/d44/f80 zero size 2024-04-24T23:13:35.156 INFO:tasks.workunit.client.0.smithi031.stdout:0/206: dread da/d11/f33 [0,4194304] 0 2024-04-24T23:13:35.156 INFO:tasks.workunit.client.0.smithi031.stdout:5/165: creat d0/d6/d12/f36 x:0 0 0 2024-04-24T23:13:35.159 INFO:tasks.workunit.client.0.smithi031.stdout:8/213: rename f0 to d8/d31/d47/f4f 0 2024-04-24T23:13:35.159 INFO:tasks.workunit.client.0.smithi031.stdout:4/190: dread d3/d1c/f2c [0,4194304] 0 2024-04-24T23:13:35.160 INFO:tasks.workunit.client.0.smithi031.stdout:4/191: chown d3/d23/f3a 4724638 1 2024-04-24T23:13:35.163 INFO:tasks.workunit.client.0.smithi031.stdout:9/255: fsync d2/fc 0 2024-04-24T23:13:35.165 INFO:tasks.workunit.client.0.smithi031.stdout:0/207: mkdir da/d18/d1a/d37/d41 0 2024-04-24T23:13:35.168 INFO:tasks.workunit.client.0.smithi031.stdout:5/166: symlink d0/d7/db/d1d/l37 0 2024-04-24T23:13:35.169 INFO:tasks.workunit.client.0.smithi031.stdout:8/214: dwrite d8/f24 [8388608,4194304] 0 2024-04-24T23:13:35.170 INFO:tasks.workunit.client.0.smithi031.stdout:4/192: dwrite d3/fb [0,4194304] 0 2024-04-24T23:13:35.171 INFO:tasks.workunit.client.0.smithi031.stdout:5/167: dread - d0/d6/d12/f21 zero size 2024-04-24T23:13:35.172 INFO:tasks.workunit.client.0.smithi031.stdout:5/168: read - d0/d6/d32/f11 zero size 2024-04-24T23:13:35.173 INFO:tasks.workunit.client.0.smithi031.stdout:6/336: rename c5 to d13/d15/d1d/d23/d44/c81 0 2024-04-24T23:13:35.178 INFO:tasks.workunit.client.0.smithi031.stdout:6/337: dread d13/d15/d1d/d23/d24/f77 [0,4194304] 0 2024-04-24T23:13:35.179 INFO:tasks.workunit.client.0.smithi031.stdout:8/215: mkdir d8/d1a/d1d/d50 0 2024-04-24T23:13:35.183 INFO:tasks.workunit.client.0.smithi031.stdout:4/193: fsync d3/f2b 0 2024-04-24T23:13:35.185 INFO:tasks.workunit.client.0.smithi031.stdout:5/169: rename d0/d6/l19 to d0/d7/db/d1d/d2e/d28/l38 0 2024-04-24T23:13:35.187 INFO:tasks.workunit.client.0.smithi031.stdout:6/338: dwrite d13/d1b/f3c [0,4194304] 0 2024-04-24T23:13:35.190 INFO:tasks.workunit.client.0.smithi031.stdout:5/170: dread d0/d6/f22 [0,4194304] 0 2024-04-24T23:13:35.192 INFO:tasks.workunit.client.0.smithi031.stdout:8/216: symlink d8/d15/d37/l51 0 2024-04-24T23:13:35.197 INFO:tasks.workunit.client.0.smithi031.stdout:4/194: mkdir d3/d23/d3b 0 2024-04-24T23:13:35.200 INFO:tasks.workunit.client.0.smithi031.stdout:9/256: link d2/d1b/c11 d2/d1b/d14/d1d/c48 0 2024-04-24T23:13:35.202 INFO:tasks.workunit.client.0.smithi031.stdout:5/171: unlink d0/d7/db/d1d/d2e/d28/c2b 0 2024-04-24T23:13:35.203 INFO:tasks.workunit.client.0.smithi031.stdout:5/172: chown d0/d7 133732716 1 2024-04-24T23:13:35.204 INFO:tasks.workunit.client.0.smithi031.stdout:5/173: chown d0/d6/d32/f4 286354 1 2024-04-24T23:13:35.211 INFO:tasks.workunit.client.0.smithi031.stdout:6/339: creat d13/d15/d1d/d23/d4d/d7b/f82 x:0 0 0 2024-04-24T23:13:35.219 INFO:tasks.workunit.client.0.smithi031.stdout:9/257: mkdir d2/d1b/d49 0 2024-04-24T23:13:35.222 INFO:tasks.workunit.client.0.smithi031.stdout:6/340: unlink d13/d15/d1d/d23/d4d/f5f 0 2024-04-24T23:13:35.227 INFO:tasks.workunit.client.0.smithi031.stdout:5/174: dread d0/d7/db/dc/f26 [0,4194304] 0 2024-04-24T23:13:35.229 INFO:tasks.workunit.client.0.smithi031.stdout:6/341: creat d13/d1b/d32/f83 x:0 0 0 2024-04-24T23:13:35.229 INFO:tasks.workunit.client.0.smithi031.stdout:8/217: write d8/d15/d37/f2e [768688,26536] 0 2024-04-24T23:13:35.233 INFO:tasks.workunit.client.0.smithi031.stdout:9/258: mkdir d2/d1b/d49/d4a 0 2024-04-24T23:13:35.234 INFO:tasks.workunit.client.0.smithi031.stdout:9/259: chown d2/d1b/f2f 21490745 1 2024-04-24T23:13:35.234 INFO:tasks.workunit.client.0.smithi031.stdout:5/175: read d0/d6/f22 [11491,93870] 0 2024-04-24T23:13:35.236 INFO:tasks.workunit.client.0.smithi031.stdout:4/195: truncate d3/f1b 3363425 0 2024-04-24T23:13:35.237 INFO:tasks.workunit.client.0.smithi031.stdout:8/218: dwrite d8/d15/d37/f2d [0,4194304] 0 2024-04-24T23:13:35.238 INFO:tasks.workunit.client.0.smithi031.stdout:8/219: write d8/d31/f44 [509956,22543] 0 2024-04-24T23:13:35.246 INFO:tasks.workunit.client.0.smithi031.stdout:6/342: creat d13/d75/d79/f84 x:0 0 0 2024-04-24T23:13:35.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/196: chown d3/dd 103 1 2024-04-24T23:13:35.248 INFO:tasks.workunit.client.0.smithi031.stdout:8/220: rmdir d8/d1a/d1d/d35 39 2024-04-24T23:13:35.253 INFO:tasks.workunit.client.0.smithi031.stdout:9/260: mkdir d2/d1b/d49/d4a/d4b 0 2024-04-24T23:13:35.253 INFO:tasks.workunit.client.0.smithi031.stdout:9/261: fdatasync d2/d1b/d14/d1d/f35 0 2024-04-24T23:13:35.256 INFO:tasks.workunit.client.0.smithi031.stdout:8/221: chown d8/l3f 0 1 2024-04-24T23:13:35.257 INFO:tasks.workunit.client.0.smithi031.stdout:8/222: fsync d8/d15/f40 0 2024-04-24T23:13:35.257 INFO:tasks.workunit.client.0.smithi031.stdout:5/176: truncate d0/d7/db/dc/d25/f29 1263469 0 2024-04-24T23:13:35.259 INFO:tasks.workunit.client.0.smithi031.stdout:5/177: readlink d0/d7/l1b 0 2024-04-24T23:13:35.259 INFO:tasks.workunit.client.0.smithi031.stdout:8/223: write d8/d15/d37/d43/f4b [370708,97986] 0 2024-04-24T23:13:35.262 INFO:tasks.workunit.client.0.smithi031.stdout:4/197: dwrite d3/d1c/f2c [4194304,4194304] 0 2024-04-24T23:13:35.263 INFO:tasks.workunit.client.0.smithi031.stdout:4/198: truncate d3/f29 53856 0 2024-04-24T23:13:35.264 INFO:tasks.workunit.client.0.smithi031.stdout:4/199: write d3/d1c/f2c [6540664,105544] 0 2024-04-24T23:13:35.266 INFO:tasks.workunit.client.0.smithi031.stdout:8/224: mknod d8/d15/d37/c52 0 2024-04-24T23:13:35.271 INFO:tasks.workunit.client.0.smithi031.stdout:8/225: creat d8/d31/d47/f53 x:0 0 0 2024-04-24T23:13:35.271 INFO:tasks.workunit.client.0.smithi031.stdout:5/178: getdents d0 0 2024-04-24T23:13:35.273 INFO:tasks.workunit.client.0.smithi031.stdout:4/200: dwrite d3/f29 [0,4194304] 0 2024-04-24T23:13:35.274 INFO:tasks.workunit.client.0.smithi031.stdout:5/179: creat d0/d6/d32/f39 x:0 0 0 2024-04-24T23:13:35.274 INFO:tasks.workunit.client.0.smithi031.stdout:8/226: mknod d8/d15/d37/d43/c54 0 2024-04-24T23:13:35.276 INFO:tasks.workunit.client.0.smithi031.stdout:8/227: write d8/d15/f40 [204723,101644] 0 2024-04-24T23:13:35.277 INFO:tasks.workunit.client.0.smithi031.stdout:8/228: chown d8/f18 14 1 2024-04-24T23:13:35.282 INFO:tasks.workunit.client.0.smithi031.stdout:5/180: dwrite d0/d7/db/d1d/d2e/d28/f30 [0,4194304] 0 2024-04-24T23:13:35.307 INFO:tasks.workunit.client.0.smithi031.stdout:8/229: getdents d8/d1a 0 2024-04-24T23:13:35.311 INFO:tasks.workunit.client.0.smithi031.stdout:9/262: dwrite d2/f5 [0,4194304] 0 2024-04-24T23:13:35.316 INFO:tasks.workunit.client.0.smithi031.stdout:9/263: mknod d2/d1b/d14/d1d/c4c 0 2024-04-24T23:13:35.316 INFO:tasks.workunit.client.0.smithi031.stdout:8/230: rmdir d8/d15 39 2024-04-24T23:13:35.320 INFO:tasks.workunit.client.0.smithi031.stdout:8/231: read d8/f14 [3203034,21052] 0 2024-04-24T23:13:35.332 INFO:tasks.workunit.client.0.smithi031.stdout:9/264: creat d2/d1b/f4d x:0 0 0 2024-04-24T23:13:35.333 INFO:tasks.workunit.client.0.smithi031.stdout:8/232: dwrite d8/d31/d47/f4f [0,4194304] 0 2024-04-24T23:13:35.337 INFO:tasks.workunit.client.0.smithi031.stdout:9/265: mknod d2/d1b/d14/d18/c4e 0 2024-04-24T23:13:35.340 INFO:tasks.workunit.client.0.smithi031.stdout:8/233: rename d8/d31/l3a to d8/d31/d47/l55 0 2024-04-24T23:13:35.352 INFO:tasks.workunit.client.0.smithi031.stdout:8/234: dwrite d8/d15/d37/f2d [0,4194304] 0 2024-04-24T23:13:35.357 INFO:tasks.workunit.client.0.smithi031.stdout:9/266: dread d2/d1b/d14/f22 [0,4194304] 0 2024-04-24T23:13:35.358 INFO:tasks.workunit.client.0.smithi031.stdout:8/235: symlink d8/d1a/l56 0 2024-04-24T23:13:35.359 INFO:tasks.workunit.client.0.smithi031.stdout:9/267: chown d2/d1b/d14/d1d/c3a 489515491 1 2024-04-24T23:13:35.360 INFO:tasks.workunit.client.0.smithi031.stdout:8/236: read d8/d15/f2a [1356339,106627] 0 2024-04-24T23:13:35.361 INFO:tasks.workunit.client.0.smithi031.stdout:8/237: dread - d8/f28 zero size 2024-04-24T23:13:35.363 INFO:tasks.workunit.client.0.smithi031.stdout:8/238: dread d8/fb [0,4194304] 0 2024-04-24T23:13:35.365 INFO:tasks.workunit.client.0.smithi031.stdout:9/268: dread - d2/f1c zero size 2024-04-24T23:13:35.370 INFO:tasks.workunit.client.0.smithi031.stdout:8/239: creat d8/d15/f57 x:0 0 0 2024-04-24T23:13:35.376 INFO:tasks.workunit.client.0.smithi031.stdout:8/240: dwrite d8/d31/f44 [0,4194304] 0 2024-04-24T23:13:35.378 INFO:tasks.workunit.client.0.smithi031.stdout:9/269: creat d2/d1b/f4f x:0 0 0 2024-04-24T23:13:35.378 INFO:tasks.workunit.client.0.smithi031.stdout:9/270: chown d2/d16/l3f 28 1 2024-04-24T23:13:35.379 INFO:tasks.workunit.client.0.smithi031.stdout:9/271: write d2/f42 [228746,106423] 0 2024-04-24T23:13:35.381 INFO:tasks.workunit.client.0.smithi031.stdout:9/272: rename d2/d1b/d14/d18 to d2/d1b/d14/d18/d50 22 2024-04-24T23:13:35.390 INFO:tasks.workunit.client.0.smithi031.stdout:8/241: getdents d8/d15/d37/d43 0 2024-04-24T23:13:35.399 INFO:tasks.workunit.client.0.smithi031.stdout:8/242: rename d8/d31 to d8/d1a/d1d/d35/d58 0 2024-04-24T23:13:35.403 INFO:tasks.workunit.client.0.smithi031.stdout:9/273: dread d2/d16/f20 [0,4194304] 0 2024-04-24T23:13:35.403 INFO:tasks.workunit.client.0.smithi031.stdout:9/274: dread - d2/f1c zero size 2024-04-24T23:13:35.405 INFO:tasks.workunit.client.0.smithi031.stdout:8/243: fsync d8/d15/f57 0 2024-04-24T23:13:35.415 INFO:tasks.workunit.client.0.smithi031.stdout:9/275: link d2/lb d2/d1b/d14/l51 0 2024-04-24T23:13:35.419 INFO:tasks.workunit.client.0.smithi031.stdout:8/244: dread d8/d1a/d1d/d35/f41 [4194304,4194304] 0 2024-04-24T23:13:35.420 INFO:tasks.workunit.client.0.smithi031.stdout:8/245: fdatasync d8/d15/f3d 0 2024-04-24T23:13:35.420 INFO:tasks.workunit.client.0.smithi031.stdout:9/276: rename d2/d16/f2b to d2/d1b/d49/d4a/d4b/f52 0 2024-04-24T23:13:35.426 INFO:tasks.workunit.client.0.smithi031.stdout:8/246: dwrite d8/f24 [8388608,4194304] 0 2024-04-24T23:13:35.426 INFO:tasks.workunit.client.0.smithi031.stdout:8/247: readlink l4 0 2024-04-24T23:13:35.429 INFO:tasks.workunit.client.0.smithi031.stdout:9/277: mknod d2/d1b/d14/c53 0 2024-04-24T23:13:35.434 INFO:tasks.workunit.client.0.smithi031.stdout:9/278: chown d2/d1b/c11 1389 1 2024-04-24T23:13:35.434 INFO:tasks.workunit.client.0.smithi031.stdout:9/279: fdatasync d2/fc 0 2024-04-24T23:13:35.435 INFO:tasks.workunit.client.0.smithi031.stdout:3/232: sync 2024-04-24T23:13:35.443 INFO:tasks.workunit.client.0.smithi031.stdout:9/280: dwrite d2/d1b/f10 [0,4194304] 0 2024-04-24T23:13:35.445 INFO:tasks.workunit.client.0.smithi031.stdout:8/248: link d8/d15/l3c d8/d1a/d1d/l59 0 2024-04-24T23:13:35.453 INFO:tasks.workunit.client.0.smithi031.stdout:8/249: dwrite d8/d1a/d1d/d35/d58/d47/f49 [0,4194304] 0 2024-04-24T23:13:35.460 INFO:tasks.workunit.client.0.smithi031.stdout:8/250: unlink d8/d1a/d1d/d35/f41 0 2024-04-24T23:13:35.463 INFO:tasks.workunit.client.0.smithi031.stdout:9/281: getdents d2/d1b/d49 0 2024-04-24T23:13:35.465 INFO:tasks.workunit.client.0.smithi031.stdout:8/251: unlink d8/l11 0 2024-04-24T23:13:35.466 INFO:tasks.workunit.client.0.smithi031.stdout:8/252: fdatasync d8/d15/f40 0 2024-04-24T23:13:35.467 INFO:tasks.workunit.client.0.smithi031.stdout:3/233: link d6/df/d16/d2a/c40 d6/d14/c4a 0 2024-04-24T23:13:35.468 INFO:tasks.workunit.client.0.smithi031.stdout:3/234: write d6/d14/d1d/f1e [4860121,67430] 0 2024-04-24T23:13:35.469 INFO:tasks.workunit.client.0.smithi031.stdout:9/282: rmdir d2 39 2024-04-24T23:13:35.474 INFO:tasks.workunit.client.0.smithi031.stdout:3/235: unlink d6/f2b 0 2024-04-24T23:13:35.478 INFO:tasks.workunit.client.0.smithi031.stdout:8/253: mknod d8/c5a 0 2024-04-24T23:13:35.480 INFO:tasks.workunit.client.0.smithi031.stdout:9/283: dread d2/d1b/d14/f22 [0,4194304] 0 2024-04-24T23:13:35.481 INFO:tasks.workunit.client.0.smithi031.stdout:8/254: dread d8/d15/f40 [0,4194304] 0 2024-04-24T23:13:35.485 INFO:tasks.workunit.client.0.smithi031.stdout:8/255: mkdir d8/d1a/d1d/d35/d58/d5b 0 2024-04-24T23:13:35.485 INFO:tasks.workunit.client.0.smithi031.stdout:8/256: fdatasync d8/f18 0 2024-04-24T23:13:35.487 INFO:tasks.workunit.client.0.smithi031.stdout:3/236: rmdir d6/df/d16/d2a/d47 0 2024-04-24T23:13:35.488 INFO:tasks.workunit.client.0.smithi031.stdout:3/237: chown d6/d15/d29 28369 1 2024-04-24T23:13:35.490 INFO:tasks.workunit.client.0.smithi031.stdout:9/284: link d2/d1b/f10 d2/f54 0 2024-04-24T23:13:35.491 INFO:tasks.workunit.client.0.smithi031.stdout:8/257: mkdir d8/d15/d37/d43/d5c 0 2024-04-24T23:13:35.492 INFO:tasks.workunit.client.0.smithi031.stdout:8/258: stat d8/f22 0 2024-04-24T23:13:35.493 INFO:tasks.workunit.client.0.smithi031.stdout:3/238: mknod d6/d15/c4b 0 2024-04-24T23:13:35.493 INFO:tasks.workunit.client.0.smithi031.stdout:3/239: fsync d6/d14/d1d/f24 0 2024-04-24T23:13:35.495 INFO:tasks.workunit.client.0.smithi031.stdout:8/259: creat d8/d15/d37/f5d x:0 0 0 2024-04-24T23:13:35.498 INFO:tasks.workunit.client.0.smithi031.stdout:3/240: write d6/d15/d29/f36 [2849524,37348] 0 2024-04-24T23:13:35.500 INFO:tasks.workunit.client.0.smithi031.stdout:8/260: creat d8/d15/d37/f5e x:0 0 0 2024-04-24T23:13:35.502 INFO:tasks.workunit.client.0.smithi031.stdout:3/241: creat d6/d14/d1d/f4c x:0 0 0 2024-04-24T23:13:35.504 INFO:tasks.workunit.client.0.smithi031.stdout:8/261: symlink d8/d15/l5f 0 2024-04-24T23:13:35.505 INFO:tasks.workunit.client.0.smithi031.stdout:9/285: rename d2/c36 to d2/c55 0 2024-04-24T23:13:35.516 INFO:tasks.workunit.client.0.smithi031.stdout:9/286: dwrite d2/d1b/d49/d4a/d4b/f52 [0,4194304] 0 2024-04-24T23:13:35.523 INFO:tasks.workunit.client.0.smithi031.stdout:8/262: dread d8/d15/f2a [0,4194304] 0 2024-04-24T23:13:35.529 INFO:tasks.workunit.client.0.smithi031.stdout:9/287: mkdir d2/d1b/d14/d1d/d45/d56 0 2024-04-24T23:13:35.533 INFO:tasks.workunit.client.0.smithi031.stdout:9/288: readlink d2/l37 0 2024-04-24T23:13:35.539 INFO:tasks.workunit.client.0.smithi031.stdout:9/289: dwrite d2/fe [0,4194304] 0 2024-04-24T23:13:35.546 INFO:tasks.workunit.client.0.smithi031.stdout:9/290: creat d2/d1b/d49/d4a/d4b/f57 x:0 0 0 2024-04-24T23:13:35.546 INFO:tasks.workunit.client.0.smithi031.stdout:9/291: readlink d2/l37 0 2024-04-24T23:13:35.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/292: rename d2/d1b/d14/f22 to d2/d43/f58 0 2024-04-24T23:13:35.550 INFO:tasks.workunit.client.0.smithi031.stdout:9/293: chown d2/fe 6366263 1 2024-04-24T23:13:35.555 INFO:tasks.workunit.client.0.smithi031.stdout:9/294: dread d2/d1b/d49/d4a/d4b/f52 [0,4194304] 0 2024-04-24T23:13:35.559 INFO:tasks.workunit.client.0.smithi031.stdout:9/295: mknod d2/d1b/d49/d4a/c59 0 2024-04-24T23:13:35.559 INFO:tasks.workunit.client.0.smithi031.stdout:9/296: readlink d2/d16/l1e 0 2024-04-24T23:13:35.574 INFO:tasks.workunit.client.0.smithi031.stdout:3/242: truncate d6/fc 317631 0 2024-04-24T23:13:35.580 INFO:tasks.workunit.client.0.smithi031.stdout:3/243: dwrite d6/d14/d1d/f1f [0,4194304] 0 2024-04-24T23:13:35.585 INFO:tasks.workunit.client.0.smithi031.stdout:3/244: stat d6/d14/f33 0 2024-04-24T23:13:35.588 INFO:tasks.workunit.client.0.smithi031.stdout:3/245: mknod d6/d14/c4d 0 2024-04-24T23:13:35.593 INFO:tasks.workunit.client.0.smithi031.stdout:3/246: symlink d6/d15/d1a/l4e 0 2024-04-24T23:13:35.600 INFO:tasks.workunit.client.0.smithi031.stdout:9/297: rmdir d2/d1b/d49/d4a/d4b 39 2024-04-24T23:13:35.612 INFO:tasks.workunit.client.0.smithi031.stdout:8/263: dread d8/fc [0,4194304] 0 2024-04-24T23:13:35.616 INFO:tasks.workunit.client.0.smithi031.stdout:9/298: dread d2/d1b/d14/d18/f21 [0,4194304] 0 2024-04-24T23:13:35.617 INFO:tasks.workunit.client.0.smithi031.stdout:9/299: fdatasync d2/f42 0 2024-04-24T23:13:35.617 INFO:tasks.workunit.client.0.smithi031.stdout:9/300: readlink d2/d16/l2d 0 2024-04-24T23:13:35.621 INFO:tasks.workunit.client.0.smithi031.stdout:8/264: rmdir d8/d1a/d1d/d35 39 2024-04-24T23:13:35.626 INFO:tasks.workunit.client.0.smithi031.stdout:8/265: write d8/d1a/d1d/d35/d58/d47/f53 [591728,15827] 0 2024-04-24T23:13:35.628 INFO:tasks.workunit.client.0.smithi031.stdout:9/301: symlink d2/d1b/d49/d4a/d4b/l5a 0 2024-04-24T23:13:35.629 INFO:tasks.workunit.client.0.smithi031.stdout:9/302: write d2/f5 [1362642,91507] 0 2024-04-24T23:13:35.632 INFO:tasks.workunit.client.0.smithi031.stdout:8/266: mknod d8/d1a/d1d/d35/d58/d5b/c60 0 2024-04-24T23:13:35.691 INFO:tasks.workunit.client.0.smithi031.stdout:8/267: fsync d8/d15/d37/d43/f4b 0 2024-04-24T23:13:35.698 INFO:tasks.workunit.client.0.smithi031.stdout:8/268: dwrite d8/d15/f57 [0,4194304] 0 2024-04-24T23:13:35.699 INFO:tasks.workunit.client.0.smithi031.stdout:8/269: read - d8/d15/f3e zero size 2024-04-24T23:13:35.701 INFO:tasks.workunit.client.0.smithi031.stdout:9/303: getdents d2/d1b/d14/d1d 0 2024-04-24T23:13:35.702 INFO:tasks.workunit.client.0.smithi031.stdout:9/304: chown d2/d16/f33 2 1 2024-04-24T23:13:35.708 INFO:tasks.workunit.client.0.smithi031.stdout:8/270: dwrite d8/d1a/d1d/d35/f4a [0,4194304] 0 2024-04-24T23:13:35.708 INFO:tasks.workunit.client.0.smithi031.stdout:9/305: mknod d2/d1b/d14/c5b 0 2024-04-24T23:13:35.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/306: chown d2/d16/f2a 51574 1 2024-04-24T23:13:35.711 INFO:tasks.workunit.client.0.smithi031.stdout:9/307: truncate d2/d1b/d14/d1d/f23 4723378 0 2024-04-24T23:13:35.715 INFO:tasks.workunit.client.0.smithi031.stdout:9/308: write d2/d1b/f17 [2274087,117551] 0 2024-04-24T23:13:35.717 INFO:tasks.workunit.client.0.smithi031.stdout:8/271: dread d8/f22 [0,4194304] 0 2024-04-24T23:13:35.722 INFO:tasks.workunit.client.0.smithi031.stdout:9/309: symlink d2/d1b/d49/l5c 0 2024-04-24T23:13:35.729 INFO:tasks.workunit.client.0.smithi031.stdout:9/310: creat d2/d43/f5d x:0 0 0 2024-04-24T23:13:35.730 INFO:tasks.workunit.client.0.smithi031.stdout:8/272: dwrite d8/f18 [4194304,4194304] 0 2024-04-24T23:13:35.736 INFO:tasks.workunit.client.0.smithi031.stdout:9/311: creat d2/d1b/d14/d1d/d45/f5e x:0 0 0 2024-04-24T23:13:35.737 INFO:tasks.workunit.client.0.smithi031.stdout:9/312: fdatasync d2/f32 0 2024-04-24T23:13:35.739 INFO:tasks.workunit.client.0.smithi031.stdout:8/273: rename d8/d1a/c1f to d8/d15/d37/d43/c61 0 2024-04-24T23:13:35.740 INFO:tasks.workunit.client.0.smithi031.stdout:9/313: stat d2/d1b/d14/d1d/c48 0 2024-04-24T23:13:35.747 INFO:tasks.workunit.client.0.smithi031.stdout:9/314: dwrite d2/f42 [0,4194304] 0 2024-04-24T23:13:35.777 INFO:tasks.workunit.client.0.smithi031.stdout:8/274: dwrite d8/fb [0,4194304] 0 2024-04-24T23:13:35.780 INFO:tasks.workunit.client.0.smithi031.stdout:8/275: fdatasync d8/fd 0 2024-04-24T23:13:35.781 INFO:tasks.workunit.client.0.smithi031.stdout:8/276: chown d8/d15/c4d 1222193917 1 2024-04-24T23:13:35.788 INFO:tasks.workunit.client.0.smithi031.stdout:8/277: rmdir d8/d15 39 2024-04-24T23:13:35.817 INFO:tasks.workunit.client.0.smithi031.stdout:8/278: write d8/f22 [1345136,10665] 0 2024-04-24T23:13:35.818 INFO:tasks.workunit.client.0.smithi031.stdout:8/279: chown d8/d1a/d1d 5469487 1 2024-04-24T23:13:35.822 INFO:tasks.workunit.client.0.smithi031.stdout:8/280: mkdir d8/d15/d37/d62 0 2024-04-24T23:13:35.828 INFO:tasks.workunit.client.0.smithi031.stdout:8/281: link d8/d15/l3c d8/d1a/d1d/l63 0 2024-04-24T23:13:35.830 INFO:tasks.workunit.client.0.smithi031.stdout:8/282: creat d8/d1a/d1d/d35/d58/f64 x:0 0 0 2024-04-24T23:13:35.832 INFO:tasks.workunit.client.0.smithi031.stdout:8/283: mknod d8/d15/d37/d62/c65 0 2024-04-24T23:13:35.833 INFO:tasks.workunit.client.0.smithi031.stdout:8/284: write d8/fd [2220117,115706] 0 2024-04-24T23:13:35.834 INFO:tasks.workunit.client.0.smithi031.stdout:8/285: stat d8/d1a/d1d/d35/c39 0 2024-04-24T23:13:35.836 INFO:tasks.workunit.client.0.smithi031.stdout:8/286: rmdir d8/d1a/d1d/d35/d58/d47 39 2024-04-24T23:13:35.840 INFO:tasks.workunit.client.0.smithi031.stdout:8/287: link d8/d1a/l32 d8/d15/d37/d43/d5c/l66 0 2024-04-24T23:13:35.844 INFO:tasks.workunit.client.0.smithi031.stdout:8/288: dread d8/d15/d37/f2e [0,4194304] 0 2024-04-24T23:13:35.846 INFO:tasks.workunit.client.0.smithi031.stdout:7/203: sync 2024-04-24T23:13:35.847 INFO:tasks.workunit.client.0.smithi031.stdout:8/289: dread d8/d15/d37/d43/f4b [0,4194304] 0 2024-04-24T23:13:35.847 INFO:tasks.workunit.client.0.smithi031.stdout:7/204: fsync d1/d2/dd/f15 0 2024-04-24T23:13:35.848 INFO:tasks.workunit.client.0.smithi031.stdout:8/290: stat d8/d1a/d1d/d35/d58/f4c 0 2024-04-24T23:13:35.849 INFO:tasks.workunit.client.0.smithi031.stdout:8/291: symlink d8/d1a/d1d/d35/d58/d5b/l67 0 2024-04-24T23:13:35.852 INFO:tasks.workunit.client.0.smithi031.stdout:8/292: rename d8/d15/d37/d43/c54 to d8/d15/c68 0 2024-04-24T23:13:35.860 INFO:tasks.workunit.client.0.smithi031.stdout:8/293: dwrite d8/d1a/d1d/d35/d58/d47/f53 [0,4194304] 0 2024-04-24T23:13:35.862 INFO:tasks.workunit.client.0.smithi031.stdout:8/294: readlink d8/d1a/d1d/l63 0 2024-04-24T23:13:35.968 INFO:tasks.workunit.client.0.smithi031.stdout:2/114: sync 2024-04-24T23:13:35.969 INFO:tasks.workunit.client.0.smithi031.stdout:2/115: chown d5/d6/df/f17 23 1 2024-04-24T23:13:35.970 INFO:tasks.workunit.client.0.smithi031.stdout:2/116: write d5/f24 [898674,129537] 0 2024-04-24T23:13:35.972 INFO:tasks.workunit.client.0.smithi031.stdout:2/117: mkdir d5/d8/d15/d23/d26 0 2024-04-24T23:13:35.974 INFO:tasks.workunit.client.0.smithi031.stdout:2/118: truncate d5/d8/d15/f19 69556 0 2024-04-24T23:13:35.975 INFO:tasks.workunit.client.0.smithi031.stdout:2/119: creat d5/f27 x:0 0 0 2024-04-24T23:13:35.976 INFO:tasks.workunit.client.0.smithi031.stdout:2/120: write d5/d6/f1a [2684143,115364] 0 2024-04-24T23:13:35.979 INFO:tasks.workunit.client.0.smithi031.stdout:2/121: mkdir d5/d28 0 2024-04-24T23:13:36.026 INFO:tasks.workunit.client.0.smithi031.stdout:2/122: rename d5/d8/d15/d23 to d5/d6/d29 0 2024-04-24T23:13:36.026 INFO:tasks.workunit.client.0.smithi031.stdout:2/123: fsync d5/d6/df/f17 0 2024-04-24T23:13:36.027 INFO:tasks.workunit.client.0.smithi031.stdout:2/124: readlink d5/d6/df/l20 0 2024-04-24T23:13:36.029 INFO:tasks.workunit.client.0.smithi031.stdout:2/125: truncate d5/d8/f10 536097 0 2024-04-24T23:13:36.030 INFO:tasks.workunit.client.0.smithi031.stdout:2/126: chown d5/d6/df/f17 93961138 1 2024-04-24T23:13:36.035 INFO:tasks.workunit.client.0.smithi031.stdout:2/127: link d5/d6/df/f17 d5/d6/f2a 0 2024-04-24T23:13:36.036 INFO:tasks.workunit.client.0.smithi031.stdout:2/128: chown d5/d8/d15 0 1 2024-04-24T23:13:36.039 INFO:tasks.workunit.client.0.smithi031.stdout:2/129: rename d5/d6/f2a to d5/d8/d15/f2b 0 2024-04-24T23:13:36.040 INFO:tasks.workunit.client.0.smithi031.stdout:2/130: write f4 [2379054,24369] 0 2024-04-24T23:13:36.043 INFO:tasks.workunit.client.0.smithi031.stdout:2/131: mknod d5/d8/d15/d18/c2c 0 2024-04-24T23:13:36.056 INFO:tasks.workunit.client.0.smithi031.stdout:2/132: rmdir d5 39 2024-04-24T23:13:36.059 INFO:tasks.workunit.client.0.smithi031.stdout:2/133: symlink d5/d28/l2d 0 2024-04-24T23:13:36.067 INFO:tasks.workunit.client.0.smithi031.stdout:2/134: truncate d5/d6/df/f1e 2484784 0 2024-04-24T23:13:36.069 INFO:tasks.workunit.client.0.smithi031.stdout:2/135: fdatasync d5/d6/d29/f21 0 2024-04-24T23:13:36.071 INFO:tasks.workunit.client.0.smithi031.stdout:2/136: symlink d5/d6/d29/l2e 0 2024-04-24T23:13:36.072 INFO:tasks.workunit.client.0.smithi031.stdout:2/137: write f4 [6398471,23529] 0 2024-04-24T23:13:36.080 INFO:tasks.workunit.client.0.smithi031.stdout:2/138: getdents d5/d6/d29 0 2024-04-24T23:13:36.082 INFO:tasks.workunit.client.0.smithi031.stdout:2/139: rmdir d5/d6/df 39 2024-04-24T23:13:36.085 INFO:tasks.workunit.client.0.smithi031.stdout:2/140: mkdir d5/d8/d15/d18/d2f 0 2024-04-24T23:13:36.086 INFO:tasks.workunit.client.0.smithi031.stdout:2/141: write d5/d6/d29/f12 [3859246,2538] 0 2024-04-24T23:13:36.087 INFO:tasks.workunit.client.0.smithi031.stdout:2/142: write d5/f25 [446993,30345] 0 2024-04-24T23:13:36.088 INFO:tasks.workunit.client.0.smithi031.stdout:2/143: chown d5 24352 1 2024-04-24T23:13:36.089 INFO:tasks.workunit.client.0.smithi031.stdout:2/144: truncate d5/d6/d29/f21 826925 0 2024-04-24T23:13:36.094 INFO:tasks.workunit.client.0.smithi031.stdout:2/145: creat d5/d6/f30 x:0 0 0 2024-04-24T23:13:36.095 INFO:tasks.workunit.client.0.smithi031.stdout:2/146: dread d5/d8/d15/f19 [0,4194304] 0 2024-04-24T23:13:36.096 INFO:tasks.workunit.client.0.smithi031.stdout:2/147: dread - d5/d8/d15/f22 zero size 2024-04-24T23:13:36.097 INFO:tasks.workunit.client.0.smithi031.stdout:2/148: write d5/f25 [1128807,70075] 0 2024-04-24T23:13:36.098 INFO:tasks.workunit.client.0.smithi031.stdout:2/149: write d5/d6/f30 [262061,24961] 0 2024-04-24T23:13:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:8/295: sync 2024-04-24T23:13:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/174: sync 2024-04-24T23:13:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:7/205: sync 2024-04-24T23:13:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:3/247: sync 2024-04-24T23:13:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:0/208: sync 2024-04-24T23:13:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:5/181: sync 2024-04-24T23:13:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:6/343: sync 2024-04-24T23:13:36.100 INFO:tasks.workunit.client.0.smithi031.stdout:4/201: sync 2024-04-24T23:13:36.101 INFO:tasks.workunit.client.0.smithi031.stdout:9/315: sync 2024-04-24T23:13:36.101 INFO:tasks.workunit.client.0.smithi031.stdout:4/202: chown d3/d20 13155 1 2024-04-24T23:13:36.102 INFO:tasks.workunit.client.0.smithi031.stdout:2/150: chown d5/d6/df 860566417 1 2024-04-24T23:13:36.104 INFO:tasks.workunit.client.0.smithi031.stdout:9/316: write d2/d1b/d14/d1d/f28 [623082,52837] 0 2024-04-24T23:13:36.105 INFO:tasks.workunit.client.0.smithi031.stdout:3/248: chown d6/d15/d29/f36 7 1 2024-04-24T23:13:36.108 INFO:tasks.workunit.client.0.smithi031.stdout:1/175: unlink d1/da/d10/l15 0 2024-04-24T23:13:36.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/344: creat d13/d75/d79/f85 x:0 0 0 2024-04-24T23:13:36.110 INFO:tasks.workunit.client.0.smithi031.stdout:1/176: read d1/dc/d11/f20 [5907936,687] 0 2024-04-24T23:13:36.111 INFO:tasks.workunit.client.0.smithi031.stdout:6/345: chown d13/d15/d1d/f74 4 1 2024-04-24T23:13:36.113 INFO:tasks.workunit.client.0.smithi031.stdout:3/249: dwrite d6/d14/d1d/f4c [0,4194304] 0 2024-04-24T23:13:36.113 INFO:tasks.workunit.client.0.smithi031.stdout:4/203: unlink d3/d1c/l21 0 2024-04-24T23:13:36.115 INFO:tasks.workunit.client.0.smithi031.stdout:7/206: mknod d1/d1c/c3f 0 2024-04-24T23:13:36.115 INFO:tasks.workunit.client.0.smithi031.stdout:6/346: read d13/d15/d1d/d23/d24/d43/f71 [12139,34032] 0 2024-04-24T23:13:36.117 INFO:tasks.workunit.client.0.smithi031.stdout:6/347: chown d13/d15/d1d/d23/d4d 7033078 1 2024-04-24T23:13:36.118 INFO:tasks.workunit.client.0.smithi031.stdout:4/204: write d3/f26 [84357,35994] 0 2024-04-24T23:13:36.118 INFO:tasks.workunit.client.0.smithi031.stdout:6/348: stat d13/d75 0 2024-04-24T23:13:36.119 INFO:tasks.workunit.client.0.smithi031.stdout:8/296: rename d8/d1a/f21 to d8/d15/f69 0 2024-04-24T23:13:36.120 INFO:tasks.workunit.client.0.smithi031.stdout:4/205: fdatasync d3/dd/f31 0 2024-04-24T23:13:36.121 INFO:tasks.workunit.client.0.smithi031.stdout:5/182: creat d0/d7/f3a x:0 0 0 2024-04-24T23:13:36.124 INFO:tasks.workunit.client.0.smithi031.stdout:3/250: dread d6/d14/d1d/f4c [0,4194304] 0 2024-04-24T23:13:36.127 INFO:tasks.workunit.client.0.smithi031.stdout:3/251: write d6/df/d16/f1c [1557752,13489] 0 2024-04-24T23:13:36.130 INFO:tasks.workunit.client.0.smithi031.stdout:8/297: dwrite d8/d1a/d1d/d35/d58/d47/f49 [0,4194304] 0 2024-04-24T23:13:36.133 INFO:tasks.workunit.client.0.smithi031.stdout:9/317: link d2/d1b/d14/d1d/f35 d2/d1b/d14/f5f 0 2024-04-24T23:13:36.135 INFO:tasks.workunit.client.0.smithi031.stdout:4/206: symlink d3/d23/l3c 0 2024-04-24T23:13:36.138 INFO:tasks.workunit.client.0.smithi031.stdout:5/183: mkdir d0/d6/d3b 0 2024-04-24T23:13:36.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/184: readlink d0/d6/d32/l14 0 2024-04-24T23:13:36.140 INFO:tasks.workunit.client.0.smithi031.stdout:9/318: dwrite d2/d16/f2a [0,4194304] 0 2024-04-24T23:13:36.141 INFO:tasks.workunit.client.0.smithi031.stdout:5/185: write d0/f9 [1045066,66795] 0 2024-04-24T23:13:36.145 INFO:tasks.workunit.client.0.smithi031.stdout:7/207: dread d1/ff [0,4194304] 0 2024-04-24T23:13:36.150 INFO:tasks.workunit.client.0.smithi031.stdout:9/319: dwrite d2/d1b/d14/d1d/d45/f5e [0,4194304] 0 2024-04-24T23:13:36.153 INFO:tasks.workunit.client.0.smithi031.stdout:3/252: fdatasync d6/df/f34 0 2024-04-24T23:13:36.160 INFO:tasks.workunit.client.0.smithi031.stdout:6/349: mknod d13/d15/d1d/d23/d24/d43/d68/c86 0 2024-04-24T23:13:36.162 INFO:tasks.workunit.client.0.smithi031.stdout:5/186: symlink d0/d6/l3c 0 2024-04-24T23:13:36.166 INFO:tasks.workunit.client.0.smithi031.stdout:1/177: creat d1/dc/d11/f38 x:0 0 0 2024-04-24T23:13:36.167 INFO:tasks.workunit.client.0.smithi031.stdout:9/320: stat d2/f12 0 2024-04-24T23:13:36.168 INFO:tasks.workunit.client.0.smithi031.stdout:9/321: chown d2/d1b/f4d 11408507 1 2024-04-24T23:13:36.169 INFO:tasks.workunit.client.0.smithi031.stdout:9/322: write d2/f42 [2291717,49101] 0 2024-04-24T23:13:36.173 INFO:tasks.workunit.client.0.smithi031.stdout:2/151: rename d5/d8/d15/d18 to d5/d8/d31 0 2024-04-24T23:13:36.178 INFO:tasks.workunit.client.0.smithi031.stdout:9/323: dwrite d2/d1b/f4d [0,4194304] 0 2024-04-24T23:13:36.179 INFO:tasks.workunit.client.0.smithi031.stdout:5/187: readlink d0/d7/db/d1d/d2e/d28/l38 0 2024-04-24T23:13:36.181 INFO:tasks.workunit.client.0.smithi031.stdout:8/298: dread d8/f38 [0,4194304] 0 2024-04-24T23:13:36.181 INFO:tasks.workunit.client.0.smithi031.stdout:2/152: dread d5/f25 [0,4194304] 0 2024-04-24T23:13:36.182 INFO:tasks.workunit.client.0.smithi031.stdout:8/299: fdatasync d8/d1a/f36 0 2024-04-24T23:13:36.186 INFO:tasks.workunit.client.0.smithi031.stdout:3/253: link d6/d14/d1d/f1f d6/df/d16/d2a/f4f 0 2024-04-24T23:13:36.188 INFO:tasks.workunit.client.0.smithi031.stdout:3/254: chown d6/d15/d29/f36 6846457 1 2024-04-24T23:13:36.190 INFO:tasks.workunit.client.0.smithi031.stdout:9/324: dwrite d2/d16/f20 [0,4194304] 0 2024-04-24T23:13:36.193 INFO:tasks.workunit.client.0.smithi031.stdout:4/207: rename d3/f16 to d3/d23/f3d 0 2024-04-24T23:13:36.198 INFO:tasks.workunit.client.0.smithi031.stdout:9/325: dwrite d2/d1b/d14/d1d/f47 [0,4194304] 0 2024-04-24T23:13:36.205 INFO:tasks.workunit.client.0.smithi031.stdout:4/208: dwrite d3/d1c/f2c [0,4194304] 0 2024-04-24T23:13:36.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/178: link d1/d1d/f35 d1/d31/f39 0 2024-04-24T23:13:36.211 INFO:tasks.workunit.client.0.smithi031.stdout:2/153: unlink d5/d8/d15/f19 0 2024-04-24T23:13:36.212 INFO:tasks.workunit.client.0.smithi031.stdout:2/154: write d5/d6/f16 [4068550,44403] 0 2024-04-24T23:13:36.213 INFO:tasks.workunit.client.0.smithi031.stdout:8/300: mknod d8/d1a/d1d/d35/d58/d47/c6a 0 2024-04-24T23:13:36.216 INFO:tasks.workunit.client.0.smithi031.stdout:3/255: unlink d6/d15/d29/f36 0 2024-04-24T23:13:36.216 INFO:tasks.workunit.client.0.smithi031.stdout:5/188: dread d0/d6/d32/f17 [0,4194304] 0 2024-04-24T23:13:36.218 INFO:tasks.workunit.client.0.smithi031.stdout:6/350: creat d13/f87 x:0 0 0 2024-04-24T23:13:36.222 INFO:tasks.workunit.client.0.smithi031.stdout:7/208: getdents d1 0 2024-04-24T23:13:36.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/209: fsync d1/d2/f11 0 2024-04-24T23:13:36.226 INFO:tasks.workunit.client.0.smithi031.stdout:2/155: dwrite d5/d6/f1a [4194304,4194304] 0 2024-04-24T23:13:36.226 INFO:tasks.workunit.client.0.smithi031.stdout:2/156: dread - d5/d8/d15/f22 zero size 2024-04-24T23:13:36.226 INFO:tasks.workunit.client.0.smithi031.stdout:4/209: truncate d3/f11 1375731 0 2024-04-24T23:13:36.232 INFO:tasks.workunit.client.0.smithi031.stdout:3/256: symlink d6/d15/d29/l50 0 2024-04-24T23:13:36.235 INFO:tasks.workunit.client.0.smithi031.stdout:6/351: creat d13/d15/d1d/d23/d4d/f88 x:0 0 0 2024-04-24T23:13:36.242 INFO:tasks.workunit.client.0.smithi031.stdout:1/179: link d1/da/f2c d1/dc/d11/d21/f3a 0 2024-04-24T23:13:36.245 INFO:tasks.workunit.client.0.smithi031.stdout:9/326: link d2/d1b/d14/d1d/f35 d2/d1b/d49/f60 0 2024-04-24T23:13:36.246 INFO:tasks.workunit.client.0.smithi031.stdout:9/327: write d2/d43/f5d [833624,127714] 0 2024-04-24T23:13:36.246 INFO:tasks.workunit.client.0.smithi031.stdout:4/210: mknod d3/d20/d25/c3e 0 2024-04-24T23:13:36.248 INFO:tasks.workunit.client.0.smithi031.stdout:4/211: dread d3/f26 [0,4194304] 0 2024-04-24T23:13:36.257 INFO:tasks.workunit.client.0.smithi031.stdout:4/212: mkdir d3/dd/d3f 0 2024-04-24T23:13:36.260 INFO:tasks.workunit.client.0.smithi031.stdout:1/180: creat d1/d33/d36/f3b x:0 0 0 2024-04-24T23:13:36.261 INFO:tasks.workunit.client.0.smithi031.stdout:5/189: link d0/d7/db/dc/c20 d0/d7/db/d1d/c3d 0 2024-04-24T23:13:36.261 INFO:tasks.workunit.client.0.smithi031.stdout:7/210: dwrite d1/ff [4194304,4194304] 0 2024-04-24T23:13:36.261 INFO:tasks.workunit.client.0.smithi031.stdout:1/181: chown d1/l25 472097221 1 2024-04-24T23:13:36.263 INFO:tasks.workunit.client.0.smithi031.stdout:5/190: truncate d0/d6/d32/f39 777749 0 2024-04-24T23:13:36.264 INFO:tasks.workunit.client.0.smithi031.stdout:5/191: fdatasync d0/d6/d32/f39 0 2024-04-24T23:13:36.265 INFO:tasks.workunit.client.0.smithi031.stdout:5/192: truncate d0/d6/d12/f36 708732 0 2024-04-24T23:13:36.266 INFO:tasks.workunit.client.0.smithi031.stdout:5/193: chown d0/d7/db 508258294 1 2024-04-24T23:13:36.269 INFO:tasks.workunit.client.0.smithi031.stdout:6/352: truncate f12 3106183 0 2024-04-24T23:13:36.274 INFO:tasks.workunit.client.0.smithi031.stdout:5/194: unlink d0/d7/l1b 0 2024-04-24T23:13:36.277 INFO:tasks.workunit.client.0.smithi031.stdout:1/182: mknod d1/d31/c3c 0 2024-04-24T23:13:36.277 INFO:tasks.workunit.client.0.smithi031.stdout:1/183: read - d1/da/f2c zero size 2024-04-24T23:13:36.278 INFO:tasks.workunit.client.0.smithi031.stdout:1/184: chown d1/da/d12/c14 0 1 2024-04-24T23:13:36.279 INFO:tasks.workunit.client.0.smithi031.stdout:9/328: link d2/d1b/d14/l51 d2/d1b/d14/d1d/d45/l61 0 2024-04-24T23:13:36.279 INFO:tasks.workunit.client.0.smithi031.stdout:1/185: chown d1/dc/d11/d21/c37 1979307 1 2024-04-24T23:13:36.280 INFO:tasks.workunit.client.0.smithi031.stdout:1/186: fdatasync d1/da/d12/f19 0 2024-04-24T23:13:36.281 INFO:tasks.workunit.client.0.smithi031.stdout:1/187: chown d1/dc/d11/f13 360 1 2024-04-24T23:13:36.287 INFO:tasks.workunit.client.0.smithi031.stdout:9/329: creat d2/d1b/d14/f62 x:0 0 0 2024-04-24T23:13:36.291 INFO:tasks.workunit.client.0.smithi031.stdout:9/330: dread d2/d16/f2a [0,4194304] 0 2024-04-24T23:13:36.292 INFO:tasks.workunit.client.0.smithi031.stdout:1/188: creat d1/da/d12/f3d x:0 0 0 2024-04-24T23:13:36.294 INFO:tasks.workunit.client.0.smithi031.stdout:4/213: write d3/d23/f3d [1363275,20094] 0 2024-04-24T23:13:36.295 INFO:tasks.workunit.client.0.smithi031.stdout:7/211: rename d1/d1c/c1d to d1/d2/c40 0 2024-04-24T23:13:36.297 INFO:tasks.workunit.client.0.smithi031.stdout:6/353: write d13/d1b/d32/f6d [184550,17277] 0 2024-04-24T23:13:36.297 INFO:tasks.workunit.client.0.smithi031.stdout:6/354: readlink d13/d15/d1d/l7a 0 2024-04-24T23:13:36.299 INFO:tasks.workunit.client.0.smithi031.stdout:9/331: mknod d2/d1b/d14/d1d/c63 0 2024-04-24T23:13:36.300 INFO:tasks.workunit.client.0.smithi031.stdout:9/332: chown d2/l37 716685 1 2024-04-24T23:13:36.302 INFO:tasks.workunit.client.0.smithi031.stdout:4/214: creat d3/d20/d2d/f40 x:0 0 0 2024-04-24T23:13:36.303 INFO:tasks.workunit.client.0.smithi031.stdout:7/212: mkdir d1/d1c/d2c/d41 0 2024-04-24T23:13:36.304 INFO:tasks.workunit.client.0.smithi031.stdout:7/213: chown d1/l16 431381499 1 2024-04-24T23:13:36.306 INFO:tasks.workunit.client.0.smithi031.stdout:6/355: creat d13/d15/d1d/d23/d44/d47/d69/f89 x:0 0 0 2024-04-24T23:13:36.306 INFO:tasks.workunit.client.0.smithi031.stdout:6/356: stat l0 0 2024-04-24T23:13:36.352 INFO:tasks.workunit.client.0.smithi031.stdout:5/195: truncate d0/d6/d32/f4 2092042 0 2024-04-24T23:13:36.354 INFO:tasks.workunit.client.0.smithi031.stdout:6/357: creat d13/d1b/d32/f8a x:0 0 0 2024-04-24T23:13:36.356 INFO:tasks.workunit.client.0.smithi031.stdout:7/214: dwrite d1/d2/f2d [0,4194304] 0 2024-04-24T23:13:36.357 INFO:tasks.workunit.client.0.smithi031.stdout:4/215: creat d3/d23/d3b/f41 x:0 0 0 2024-04-24T23:13:36.360 INFO:tasks.workunit.client.0.smithi031.stdout:5/196: mknod d0/d6/d32/c3e 0 2024-04-24T23:13:36.361 INFO:tasks.workunit.client.0.smithi031.stdout:1/189: dwrite d1/da/f2c [0,4194304] 0 2024-04-24T23:13:36.363 INFO:tasks.workunit.client.0.smithi031.stdout:9/333: creat d2/f64 x:0 0 0 2024-04-24T23:13:36.363 INFO:tasks.workunit.client.0.smithi031.stdout:1/190: write d1/dc/fe [443628,122994] 0 2024-04-24T23:13:36.363 INFO:tasks.workunit.client.0.smithi031.stdout:5/197: read d0/d6/d32/f39 [629068,127483] 0 2024-04-24T23:13:36.372 INFO:tasks.workunit.client.0.smithi031.stdout:7/215: rename d1/d2/fa to d1/d2/dd/d33/f42 0 2024-04-24T23:13:36.372 INFO:tasks.workunit.client.0.smithi031.stdout:6/358: dwrite d13/f7c [0,4194304] 0 2024-04-24T23:13:36.372 INFO:tasks.workunit.client.0.smithi031.stdout:5/198: dread d0/d7/db/d1d/d2e/d28/f30 [0,4194304] 0 2024-04-24T23:13:36.373 INFO:tasks.workunit.client.0.smithi031.stdout:6/359: stat d13/f6c 0 2024-04-24T23:13:36.375 INFO:tasks.workunit.client.0.smithi031.stdout:5/199: chown d0/d7/l3 173304 1 2024-04-24T23:13:36.375 INFO:tasks.workunit.client.0.smithi031.stdout:7/216: truncate d1/d1c/f29 5038550 0 2024-04-24T23:13:36.384 INFO:tasks.workunit.client.0.smithi031.stdout:7/217: symlink d1/d2/dd/l43 0 2024-04-24T23:13:36.385 INFO:tasks.workunit.client.0.smithi031.stdout:5/200: mknod d0/d7/db/d1d/d2e/d28/c3f 0 2024-04-24T23:13:36.390 INFO:tasks.workunit.client.0.smithi031.stdout:4/216: dread d3/f1b [0,4194304] 0 2024-04-24T23:13:36.394 INFO:tasks.workunit.client.0.smithi031.stdout:7/218: unlink d1/ff 0 2024-04-24T23:13:36.398 INFO:tasks.workunit.client.0.smithi031.stdout:4/217: mknod d3/d20/d2d/c42 0 2024-04-24T23:13:36.401 INFO:tasks.workunit.client.0.smithi031.stdout:5/201: dwrite d0/d7/db/d1d/d2e/d28/f30 [0,4194304] 0 2024-04-24T23:13:36.401 INFO:tasks.workunit.client.0.smithi031.stdout:1/191: getdents d1/dc/d11/d21 0 2024-04-24T23:13:36.403 INFO:tasks.workunit.client.0.smithi031.stdout:4/218: fdatasync d3/dd/f33 0 2024-04-24T23:13:36.404 INFO:tasks.workunit.client.0.smithi031.stdout:5/202: write d0/d7/db/d1d/d2e/d28/f30 [1283455,129845] 0 2024-04-24T23:13:36.404 INFO:tasks.workunit.client.0.smithi031.stdout:4/219: stat d3/d20/d2d/f39 0 2024-04-24T23:13:36.412 INFO:tasks.workunit.client.0.smithi031.stdout:5/203: dwrite d0/d6/d12/f21 [0,4194304] 0 2024-04-24T23:13:36.423 INFO:tasks.workunit.client.0.smithi031.stdout:4/220: link d3/d1c/f2c d3/d23/d3b/f43 0 2024-04-24T23:13:36.427 INFO:tasks.workunit.client.0.smithi031.stdout:1/192: getdents d1/d33 0 2024-04-24T23:13:36.429 INFO:tasks.workunit.client.0.smithi031.stdout:4/221: dread d3/f26 [0,4194304] 0 2024-04-24T23:13:36.439 INFO:tasks.workunit.client.0.smithi031.stdout:1/193: mkdir d1/d3e 0 2024-04-24T23:13:36.452 INFO:tasks.workunit.client.0.smithi031.stdout:1/194: dread d1/dc/fe [0,4194304] 0 2024-04-24T23:13:36.464 INFO:tasks.workunit.client.0.smithi031.stdout:1/195: dread d1/f8 [0,4194304] 0 2024-04-24T23:13:36.465 INFO:tasks.workunit.client.0.smithi031.stdout:1/196: chown d1/da/d12/f3d 2 1 2024-04-24T23:13:36.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/197: dread d1/f2 [4194304,4194304] 0 2024-04-24T23:13:36.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/198: chown d1/dc/fe 89255980 1 2024-04-24T23:13:36.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/209: sync 2024-04-24T23:13:36.521 INFO:tasks.workunit.client.0.smithi031.stdout:1/199: symlink d1/da/l3f 0 2024-04-24T23:13:36.522 INFO:tasks.workunit.client.0.smithi031.stdout:1/200: chown d1/f2 0 1 2024-04-24T23:13:36.524 INFO:tasks.workunit.client.0.smithi031.stdout:0/210: mknod da/d18/d1a/c42 0 2024-04-24T23:13:36.527 INFO:tasks.workunit.client.0.smithi031.stdout:1/201: symlink d1/dc/d11/d21/l40 0 2024-04-24T23:13:36.529 INFO:tasks.workunit.client.0.smithi031.stdout:0/211: rmdir da/d18 39 2024-04-24T23:13:36.531 INFO:tasks.workunit.client.0.smithi031.stdout:1/202: mkdir d1/d33/d41 0 2024-04-24T23:13:36.532 INFO:tasks.workunit.client.0.smithi031.stdout:1/203: truncate d1/d33/d36/f3b 173254 0 2024-04-24T23:13:36.533 INFO:tasks.workunit.client.0.smithi031.stdout:1/204: fdatasync d1/da/d12/f19 0 2024-04-24T23:13:36.540 INFO:tasks.workunit.client.0.smithi031.stdout:0/212: rmdir da/d18/d1a/d37/d40 0 2024-04-24T23:13:36.568 INFO:tasks.workunit.client.0.smithi031.stdout:9/334: dwrite d2/d1b/d14/f5f [0,4194304] 0 2024-04-24T23:13:36.568 INFO:tasks.workunit.client.0.smithi031.stdout:6/360: truncate d13/d1b/f6b 4020565 0 2024-04-24T23:13:36.569 INFO:tasks.workunit.client.0.smithi031.stdout:6/361: dread - d13/d1b/d32/f8a zero size 2024-04-24T23:13:36.570 INFO:tasks.workunit.client.0.smithi031.stdout:9/335: truncate d2/d1b/f40 163364 0 2024-04-24T23:13:36.574 INFO:tasks.workunit.client.0.smithi031.stdout:6/362: dread d13/f3f [0,4194304] 0 2024-04-24T23:13:36.578 INFO:tasks.workunit.client.0.smithi031.stdout:6/363: mkdir d13/d15/d1d/d23/d24/d8b 0 2024-04-24T23:13:36.581 INFO:tasks.workunit.client.0.smithi031.stdout:6/364: write d13/d15/d1d/d23/f2b [5077231,78344] 0 2024-04-24T23:13:36.582 INFO:tasks.workunit.client.0.smithi031.stdout:9/336: getdents d2/d16 0 2024-04-24T23:13:36.587 INFO:tasks.workunit.client.0.smithi031.stdout:9/337: unlink d2/d1b/f4d 0 2024-04-24T23:13:36.592 INFO:tasks.workunit.client.0.smithi031.stdout:9/338: mkdir d2/d1b/d14/d1d/d45/d65 0 2024-04-24T23:13:36.594 INFO:tasks.workunit.client.0.smithi031.stdout:6/365: getdents d13/d75/d79 0 2024-04-24T23:13:36.596 INFO:tasks.workunit.client.0.smithi031.stdout:6/366: write d13/d15/d1d/d23/d2d/f61 [4490349,29432] 0 2024-04-24T23:13:36.600 INFO:tasks.workunit.client.0.smithi031.stdout:9/339: truncate d2/d1b/d14/d1d/f3c 1980755 0 2024-04-24T23:13:36.602 INFO:tasks.workunit.client.0.smithi031.stdout:4/222: rename d3/d1c to d3/d20/d2d/d44 0 2024-04-24T23:13:36.608 INFO:tasks.workunit.client.0.smithi031.stdout:5/204: dwrite d0/d7/db/dc/d25/f29 [0,4194304] 0 2024-04-24T23:13:36.610 INFO:tasks.workunit.client.0.smithi031.stdout:6/367: creat d13/d15/d1d/f8c x:0 0 0 2024-04-24T23:13:36.612 INFO:tasks.workunit.client.0.smithi031.stdout:6/368: dread - d13/d15/d1d/f8c zero size 2024-04-24T23:13:36.616 INFO:tasks.workunit.client.0.smithi031.stdout:5/205: rmdir d0/d7 39 2024-04-24T23:13:36.618 INFO:tasks.workunit.client.0.smithi031.stdout:5/206: truncate d0/d6/d32/f39 1421536 0 2024-04-24T23:13:36.621 INFO:tasks.workunit.client.0.smithi031.stdout:6/369: creat d13/d15/d1d/d23/d24/d8b/f8d x:0 0 0 2024-04-24T23:13:36.624 INFO:tasks.workunit.client.0.smithi031.stdout:5/207: rename d0/d6/d32/f11 to d0/d6/f40 0 2024-04-24T23:13:36.629 INFO:tasks.workunit.client.0.smithi031.stdout:6/370: link d13/d15/d1d/d23/d4d/d7b/f82 d13/d75/f8e 0 2024-04-24T23:13:36.635 INFO:tasks.workunit.client.0.smithi031.stdout:6/371: getdents d13/d15/d1d/d23/d2d 0 2024-04-24T23:13:36.637 INFO:tasks.workunit.client.0.smithi031.stdout:6/372: getdents d13/d15/d1d/d23/d2d 0 2024-04-24T23:13:36.638 INFO:tasks.workunit.client.0.smithi031.stdout:6/373: write d13/d1b/d32/f6d [705708,41970] 0 2024-04-24T23:13:36.644 INFO:tasks.workunit.client.0.smithi031.stdout:6/374: dwrite d13/d15/d1d/d23/f2b [0,4194304] 0 2024-04-24T23:13:36.645 INFO:tasks.workunit.client.0.smithi031.stdout:6/375: dread - d13/d15/d1d/d23/d24/d8b/f8d zero size 2024-04-24T23:13:36.677 INFO:tasks.workunit.client.0.smithi031.stdout:9/340: truncate d2/d16/f33 193514 0 2024-04-24T23:13:36.681 INFO:tasks.workunit.client.0.smithi031.stdout:4/223: dwrite d3/dd/f33 [0,4194304] 0 2024-04-24T23:13:36.685 INFO:tasks.workunit.client.0.smithi031.stdout:9/341: creat d2/d1b/d14/d1d/d45/d56/f66 x:0 0 0 2024-04-24T23:13:36.688 INFO:tasks.workunit.client.0.smithi031.stdout:4/224: dwrite d3/dd/f33 [0,4194304] 0 2024-04-24T23:13:36.689 INFO:tasks.workunit.client.0.smithi031.stdout:4/225: dread - d3/dd/f31 zero size 2024-04-24T23:13:36.690 INFO:tasks.workunit.client.0.smithi031.stdout:4/226: fdatasync d3/dd/f31 0 2024-04-24T23:13:36.691 INFO:tasks.workunit.client.0.smithi031.stdout:9/342: unlink d2/d1b/d14/c5b 0 2024-04-24T23:13:36.694 INFO:tasks.workunit.client.0.smithi031.stdout:4/227: mknod d3/d23/c45 0 2024-04-24T23:13:36.695 INFO:tasks.workunit.client.0.smithi031.stdout:5/208: write d0/d7/db/dc/f26 [2478513,130677] 0 2024-04-24T23:13:36.696 INFO:tasks.workunit.client.0.smithi031.stdout:9/343: fsync d2/d43/f58 0 2024-04-24T23:13:36.702 INFO:tasks.workunit.client.0.smithi031.stdout:9/344: rename d2/d1b/d49 to d2/d1b/d14/d1d/d45/d56/d67 0 2024-04-24T23:13:36.703 INFO:tasks.workunit.client.0.smithi031.stdout:9/345: write d2/d43/f5d [333965,18725] 0 2024-04-24T23:13:36.705 INFO:tasks.workunit.client.0.smithi031.stdout:9/346: write d2/d1b/f4f [812947,130716] 0 2024-04-24T23:13:36.708 INFO:tasks.workunit.client.0.smithi031.stdout:4/228: creat d3/f46 x:0 0 0 2024-04-24T23:13:36.709 INFO:tasks.workunit.client.0.smithi031.stdout:4/229: chown d3/d20/d2d/c42 13770549 1 2024-04-24T23:13:36.710 INFO:tasks.workunit.client.0.smithi031.stdout:9/347: symlink d2/d43/l68 0 2024-04-24T23:13:36.711 INFO:tasks.workunit.client.0.smithi031.stdout:9/348: fdatasync d2/d1b/d14/d1d/d45/d56/f66 0 2024-04-24T23:13:36.714 INFO:tasks.workunit.client.0.smithi031.stdout:4/230: symlink d3/d23/l47 0 2024-04-24T23:13:36.714 INFO:tasks.workunit.client.0.smithi031.stdout:4/231: dread - d3/f46 zero size 2024-04-24T23:13:36.716 INFO:tasks.workunit.client.0.smithi031.stdout:9/349: dread d2/d43/f5d [0,4194304] 0 2024-04-24T23:13:36.716 INFO:tasks.workunit.client.0.smithi031.stdout:9/350: stat d2/d1b/d14/d1d/d45/d56 0 2024-04-24T23:13:36.721 INFO:tasks.workunit.client.0.smithi031.stdout:4/232: unlink d3/dd/le 0 2024-04-24T23:13:36.722 INFO:tasks.workunit.client.0.smithi031.stdout:9/351: creat d2/d1b/d14/f69 x:0 0 0 2024-04-24T23:13:36.724 INFO:tasks.workunit.client.0.smithi031.stdout:4/233: creat d3/d20/d2d/d44/f48 x:0 0 0 2024-04-24T23:13:36.728 INFO:tasks.workunit.client.0.smithi031.stdout:4/234: unlink d3/l18 0 2024-04-24T23:13:36.728 INFO:tasks.workunit.client.0.smithi031.stdout:4/235: chown d3/d23/c35 119643 1 2024-04-24T23:13:36.729 INFO:tasks.workunit.client.0.smithi031.stdout:9/352: mknod d2/d1b/d14/d1d/d45/d56/d67/d4a/c6a 0 2024-04-24T23:13:36.732 INFO:tasks.workunit.client.0.smithi031.stdout:9/353: mkdir d2/d1b/d14/d1d/d6b 0 2024-04-24T23:13:36.736 INFO:tasks.workunit.client.0.smithi031.stdout:9/354: dread d2/d1b/f4f [0,4194304] 0 2024-04-24T23:13:36.736 INFO:tasks.workunit.client.0.smithi031.stdout:5/209: write d0/d6/d32/f4 [2317429,96422] 0 2024-04-24T23:13:36.738 INFO:tasks.workunit.client.0.smithi031.stdout:5/210: write d0/d6/f1a [3660782,68180] 0 2024-04-24T23:13:36.742 INFO:tasks.workunit.client.0.smithi031.stdout:4/236: link d3/d20/d2d/d44/c38 d3/d23/d37/c49 0 2024-04-24T23:13:36.743 INFO:tasks.workunit.client.0.smithi031.stdout:5/211: dread d0/d7/db/dc/d25/f29 [0,4194304] 0 2024-04-24T23:13:36.743 INFO:tasks.workunit.client.0.smithi031.stdout:9/355: mknod d2/d1b/d14/d1d/d45/d56/d67/c6c 0 2024-04-24T23:13:36.744 INFO:tasks.workunit.client.0.smithi031.stdout:9/356: write d2/d43/f58 [2879128,94632] 0 2024-04-24T23:13:36.747 INFO:tasks.workunit.client.0.smithi031.stdout:5/212: dread d0/d6/d32/f17 [0,4194304] 0 2024-04-24T23:13:36.748 INFO:tasks.workunit.client.0.smithi031.stdout:5/213: write d0/f9 [2229764,10877] 0 2024-04-24T23:13:36.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:36 smithi031 ceph-mon[26411]: pgmap v21: 65 pgs: 65 active+clean; 390 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 708 KiB/s rd, 29 MiB/s wr, 237 op/s 2024-04-24T23:13:36.750 INFO:tasks.workunit.client.0.smithi031.stdout:9/357: mknod d2/d1b/d14/d1d/d45/d56/c6d 0 2024-04-24T23:13:36.824 INFO:tasks.workunit.client.0.smithi031.stdout:4/237: creat d3/f4a x:0 0 0 2024-04-24T23:13:36.826 INFO:tasks.workunit.client.0.smithi031.stdout:4/238: write d3/d20/d2d/f40 [1028637,121046] 0 2024-04-24T23:13:36.829 INFO:tasks.workunit.client.0.smithi031.stdout:4/239: symlink d3/d20/d2d/l4b 0 2024-04-24T23:13:36.829 INFO:tasks.workunit.client.0.smithi031.stdout:5/214: dwrite d0/d7/db/dc/d25/f29 [0,4194304] 0 2024-04-24T23:13:36.831 INFO:tasks.workunit.client.0.smithi031.stdout:4/240: creat d3/d20/d2d/d44/f4c x:0 0 0 2024-04-24T23:13:36.903 INFO:tasks.workunit.client.0.smithi031.stdout:5/215: fsync d0/d7/db/dc/f24 0 2024-04-24T23:13:36.904 INFO:tasks.workunit.client.0.smithi031.stdout:4/241: truncate d3/dd/f30 5001997 0 2024-04-24T23:13:36.904 INFO:tasks.workunit.client.0.smithi031.stdout:4/242: stat d3/f46 0 2024-04-24T23:13:36.907 INFO:tasks.workunit.client.0.smithi031.stdout:9/358: dwrite d2/d1b/d14/d18/f31 [0,4194304] 0 2024-04-24T23:13:36.909 INFO:tasks.workunit.client.0.smithi031.stdout:5/216: rename d0/d7/l3 to d0/d6/d12/l41 0 2024-04-24T23:13:36.909 INFO:tasks.workunit.client.0.smithi031.stdout:5/217: chown d0/d7/db/d1d/d2e 8143455 1 2024-04-24T23:13:36.910 INFO:tasks.workunit.client.0.smithi031.stdout:5/218: fdatasync d0/f9 0 2024-04-24T23:13:36.911 INFO:tasks.workunit.client.0.smithi031.stdout:9/359: fdatasync d2/d43/f5d 0 2024-04-24T23:13:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:36 smithi040 ceph-mon[33128]: pgmap v21: 65 pgs: 65 active+clean; 390 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 708 KiB/s rd, 29 MiB/s wr, 237 op/s 2024-04-24T23:13:36.915 INFO:tasks.workunit.client.0.smithi031.stdout:9/360: mkdir d2/d43/d6e 0 2024-04-24T23:13:36.915 INFO:tasks.workunit.client.0.smithi031.stdout:5/219: dread d0/d7/db/d1d/d2e/d28/f30 [0,4194304] 0 2024-04-24T23:13:36.918 INFO:tasks.workunit.client.0.smithi031.stdout:9/361: read d2/d1b/f2f [71644,62894] 0 2024-04-24T23:13:36.919 INFO:tasks.workunit.client.0.smithi031.stdout:9/362: chown d2/d1b/d14/d1d/c4c 5 1 2024-04-24T23:13:36.925 INFO:tasks.workunit.client.0.smithi031.stdout:5/220: getdents d0/d6 0 2024-04-24T23:13:36.926 INFO:tasks.workunit.client.0.smithi031.stdout:5/221: fdatasync d0/d7/f3a 0 2024-04-24T23:13:36.927 INFO:tasks.workunit.client.0.smithi031.stdout:5/222: readlink d0/l16 0 2024-04-24T23:13:36.927 INFO:tasks.workunit.client.0.smithi031.stdout:5/223: fsync d0/f9 0 2024-04-24T23:13:36.928 INFO:tasks.workunit.client.0.smithi031.stdout:5/224: readlink d0/d6/d32/l14 0 2024-04-24T23:13:36.933 INFO:tasks.workunit.client.0.smithi031.stdout:9/363: dread d2/d1b/d14/d1d/f3c [0,4194304] 0 2024-04-24T23:13:36.940 INFO:tasks.workunit.client.0.smithi031.stdout:9/364: rename d2/d1b/c11 to d2/d43/d6e/c6f 0 2024-04-24T23:13:36.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/225: link d0/d7/db/d1d/c3d d0/d6/d3b/c42 0 2024-04-24T23:13:36.946 INFO:tasks.workunit.client.0.smithi031.stdout:5/226: mknod d0/d6/d12/c43 0 2024-04-24T23:13:36.948 INFO:tasks.workunit.client.0.smithi031.stdout:5/227: truncate d0/d7/db/dc/f24 759504 0 2024-04-24T23:13:36.955 INFO:tasks.workunit.client.0.smithi031.stdout:5/228: dwrite d0/d7/db/dc/d25/f29 [0,4194304] 0 2024-04-24T23:13:36.966 INFO:tasks.workunit.client.0.smithi031.stdout:5/229: link d0/d7/db/d1d/d2e/d28/f30 d0/d7/db/dc/f44 0 2024-04-24T23:13:36.969 INFO:tasks.workunit.client.0.smithi031.stdout:5/230: rename d0/d6/c2d to d0/d6/c45 0 2024-04-24T23:13:36.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/231: truncate d0/d6/f1a 1882473 0 2024-04-24T23:13:36.987 INFO:tasks.workunit.client.0.smithi031.stdout:5/232: write d0/d6/d12/f21 [2961937,70669] 0 2024-04-24T23:13:36.991 INFO:tasks.workunit.client.0.smithi031.stdout:5/233: truncate d0/d7/db/dc/f44 2264224 0 2024-04-24T23:13:36.992 INFO:tasks.workunit.client.0.smithi031.stdout:5/234: write d0/d7/db/dc/f24 [646018,32132] 0 2024-04-24T23:13:36.995 INFO:tasks.workunit.client.0.smithi031.stdout:5/235: creat d0/d6/d32/f46 x:0 0 0 2024-04-24T23:13:37.001 INFO:tasks.workunit.client.0.smithi031.stdout:5/236: creat d0/d7/db/dc/f47 x:0 0 0 2024-04-24T23:13:37.051 INFO:tasks.workunit.client.0.smithi031.stdout:8/301: sync 2024-04-24T23:13:37.051 INFO:tasks.workunit.client.0.smithi031.stdout:2/157: sync 2024-04-24T23:13:37.053 INFO:tasks.workunit.client.0.smithi031.stdout:8/302: mknod d8/d15/c6b 0 2024-04-24T23:13:37.054 INFO:tasks.workunit.client.0.smithi031.stdout:2/158: mknod d5/d6/df/c32 0 2024-04-24T23:13:37.056 INFO:tasks.workunit.client.0.smithi031.stdout:2/159: fsync d5/d6/f1f 0 2024-04-24T23:13:37.057 INFO:tasks.workunit.client.0.smithi031.stdout:2/160: chown d5/d6/f1a 135 1 2024-04-24T23:13:37.058 INFO:tasks.workunit.client.0.smithi031.stdout:2/161: mkdir d5/d8/d15/d33 0 2024-04-24T23:13:37.060 INFO:tasks.workunit.client.0.smithi031.stdout:2/162: read - d5/d6/f1f zero size 2024-04-24T23:13:37.060 INFO:tasks.workunit.client.0.smithi031.stdout:8/303: getdents d8/d1a/d1d/d35/d58/d47 0 2024-04-24T23:13:37.061 INFO:tasks.workunit.client.0.smithi031.stdout:2/163: write d5/f27 [929084,21006] 0 2024-04-24T23:13:37.061 INFO:tasks.workunit.client.0.smithi031.stdout:2/164: chown d5 1762421 1 2024-04-24T23:13:37.064 INFO:tasks.workunit.client.0.smithi031.stdout:2/165: creat d5/d8/d15/f34 x:0 0 0 2024-04-24T23:13:37.065 INFO:tasks.workunit.client.0.smithi031.stdout:8/304: dread d8/d1a/d1d/d35/d58/f44 [0,4194304] 0 2024-04-24T23:13:37.067 INFO:tasks.workunit.client.0.smithi031.stdout:8/305: fdatasync d8/f24 0 2024-04-24T23:13:37.072 INFO:tasks.workunit.client.0.smithi031.stdout:8/306: symlink d8/d1a/d1d/l6c 0 2024-04-24T23:13:37.075 INFO:tasks.workunit.client.0.smithi031.stdout:8/307: creat d8/d1a/d1d/d35/f6d x:0 0 0 2024-04-24T23:13:37.079 INFO:tasks.workunit.client.0.smithi031.stdout:2/166: dread d5/d6/d29/f12 [0,4194304] 0 2024-04-24T23:13:37.082 INFO:tasks.workunit.client.0.smithi031.stdout:8/308: getdents d8/d15/d37/d43 0 2024-04-24T23:13:37.085 INFO:tasks.workunit.client.0.smithi031.stdout:8/309: mknod d8/d1a/c6e 0 2024-04-24T23:13:37.086 INFO:tasks.workunit.client.0.smithi031.stdout:2/167: creat d5/d6/f35 x:0 0 0 2024-04-24T23:13:37.087 INFO:tasks.workunit.client.0.smithi031.stdout:8/310: fsync d8/d15/f69 0 2024-04-24T23:13:37.089 INFO:tasks.workunit.client.0.smithi031.stdout:2/168: rename f4 to d5/f36 0 2024-04-24T23:13:37.093 INFO:tasks.workunit.client.0.smithi031.stdout:8/311: creat d8/d15/d37/d43/f6f x:0 0 0 2024-04-24T23:13:37.094 INFO:tasks.workunit.client.0.smithi031.stdout:2/169: creat d5/d28/f37 x:0 0 0 2024-04-24T23:13:37.100 INFO:tasks.workunit.client.0.smithi031.stdout:2/170: dread d5/d8/f10 [0,4194304] 0 2024-04-24T23:13:37.103 INFO:tasks.workunit.client.0.smithi031.stdout:8/312: rename d8/d1a/f36 to d8/f70 0 2024-04-24T23:13:37.105 INFO:tasks.workunit.client.0.smithi031.stdout:2/171: creat d5/d6/d29/d26/f38 x:0 0 0 2024-04-24T23:13:37.107 INFO:tasks.workunit.client.0.smithi031.stdout:2/172: write d5/f36 [7957703,99435] 0 2024-04-24T23:13:37.110 INFO:tasks.workunit.client.0.smithi031.stdout:2/173: mknod d5/d8/d15/c39 0 2024-04-24T23:13:37.115 INFO:tasks.workunit.client.0.smithi031.stdout:2/174: dread d5/f13 [8388608,4194304] 0 2024-04-24T23:13:37.115 INFO:tasks.workunit.client.0.smithi031.stdout:2/175: readlink d5/d28/l2d 0 2024-04-24T23:13:37.121 INFO:tasks.workunit.client.0.smithi031.stdout:8/313: dread d8/f24 [4194304,4194304] 0 2024-04-24T23:13:37.122 INFO:tasks.workunit.client.0.smithi031.stdout:2/176: dwrite d5/f24 [0,4194304] 0 2024-04-24T23:13:37.126 INFO:tasks.workunit.client.0.smithi031.stdout:8/314: mknod d8/d1a/d1d/c71 0 2024-04-24T23:13:37.129 INFO:tasks.workunit.client.0.smithi031.stdout:8/315: write d8/f70 [6965916,88000] 0 2024-04-24T23:13:37.133 INFO:tasks.workunit.client.0.smithi031.stdout:8/316: dread d8/f70 [4194304,4194304] 0 2024-04-24T23:13:37.136 INFO:tasks.workunit.client.0.smithi031.stdout:8/317: creat d8/d1a/d1d/d35/d58/d47/f72 x:0 0 0 2024-04-24T23:13:37.152 INFO:tasks.workunit.client.0.smithi031.stdout:8/318: dread d8/d15/d37/f4e [0,4194304] 0 2024-04-24T23:13:37.153 INFO:tasks.workunit.client.0.smithi031.stdout:8/319: dread - d8/d1a/d1d/d35/d58/d47/f72 zero size 2024-04-24T23:13:37.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/320: mkdir d8/d1a/d73 0 2024-04-24T23:13:37.156 INFO:tasks.workunit.client.0.smithi031.stdout:8/321: write d8/d1a/d1d/d35/f6d [725684,90927] 0 2024-04-24T23:13:37.163 INFO:tasks.workunit.client.0.smithi031.stdout:2/177: dwrite d5/f25 [0,4194304] 0 2024-04-24T23:13:37.165 INFO:tasks.workunit.client.0.smithi031.stdout:8/322: dwrite f3 [4194304,4194304] 0 2024-04-24T23:13:37.169 INFO:tasks.workunit.client.0.smithi031.stdout:8/323: readlink d8/d1a/d1d/l63 0 2024-04-24T23:13:37.172 INFO:tasks.workunit.client.0.smithi031.stdout:2/178: dwrite d5/f24 [0,4194304] 0 2024-04-24T23:13:37.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/324: unlink d8/d1a/d1d/d35/d58/d47/f4f 0 2024-04-24T23:13:37.174 INFO:tasks.workunit.client.0.smithi031.stdout:2/179: write d5/d28/f37 [274347,18700] 0 2024-04-24T23:13:37.178 INFO:tasks.workunit.client.0.smithi031.stdout:8/325: truncate d8/d1a/d1d/d35/d58/f4c 823577 0 2024-04-24T23:13:37.179 INFO:tasks.workunit.client.0.smithi031.stdout:2/180: creat d5/d28/f3a x:0 0 0 2024-04-24T23:13:37.180 INFO:tasks.workunit.client.0.smithi031.stdout:8/326: mkdir d8/d1a/d74 0 2024-04-24T23:13:37.185 INFO:tasks.workunit.client.0.smithi031.stdout:2/181: write d5/f24 [4252812,31805] 0 2024-04-24T23:13:37.186 INFO:tasks.workunit.client.0.smithi031.stdout:8/327: mknod d8/d15/d37/d43/c75 0 2024-04-24T23:13:37.187 INFO:tasks.workunit.client.0.smithi031.stdout:8/328: chown d8/d1a/d1d/d50 2 1 2024-04-24T23:13:37.189 INFO:tasks.workunit.client.0.smithi031.stdout:2/182: link d5/f24 d5/d8/d15/f3b 0 2024-04-24T23:13:37.190 INFO:tasks.workunit.client.0.smithi031.stdout:2/183: write d5/d6/f1a [3190464,30843] 0 2024-04-24T23:13:37.190 INFO:tasks.workunit.client.0.smithi031.stdout:8/329: mkdir d8/d15/d37/d62/d76 0 2024-04-24T23:13:37.194 INFO:tasks.workunit.client.0.smithi031.stdout:2/184: write d5/d6/f16 [3661025,20154] 0 2024-04-24T23:13:37.194 INFO:tasks.workunit.client.0.smithi031.stdout:8/330: truncate d8/d15/f2a 394979 0 2024-04-24T23:13:37.196 INFO:tasks.workunit.client.0.smithi031.stdout:8/331: creat d8/d1a/d1d/d35/d58/d47/f77 x:0 0 0 2024-04-24T23:13:37.198 INFO:tasks.workunit.client.0.smithi031.stdout:8/332: fsync d8/f28 0 2024-04-24T23:13:37.199 INFO:tasks.workunit.client.0.smithi031.stdout:8/333: readlink d8/d1a/d1d/d35/d58/d47/l55 0 2024-04-24T23:13:37.200 INFO:tasks.workunit.client.0.smithi031.stdout:8/334: chown l5 0 1 2024-04-24T23:13:37.201 INFO:tasks.workunit.client.0.smithi031.stdout:2/185: creat d5/d6/f3c x:0 0 0 2024-04-24T23:13:37.315 INFO:tasks.workunit.client.0.smithi031.stdout:3/257: sync 2024-04-24T23:13:37.322 INFO:tasks.workunit.client.0.smithi031.stdout:3/258: dwrite d6/d14/d1d/f4c [0,4194304] 0 2024-04-24T23:13:37.338 INFO:tasks.workunit.client.0.smithi031.stdout:3/259: dread d6/d14/f33 [0,4194304] 0 2024-04-24T23:13:37.343 INFO:tasks.workunit.client.0.smithi031.stdout:3/260: rename d6/df/d16/d2a/c40 to d6/d15/c51 0 2024-04-24T23:13:37.348 INFO:tasks.workunit.client.0.smithi031.stdout:3/261: rename d6/d15/c4b to d6/df/c52 0 2024-04-24T23:13:37.348 INFO:tasks.workunit.client.0.smithi031.stdout:3/262: chown d6/df/d16/c3b 300 1 2024-04-24T23:13:37.349 INFO:tasks.workunit.client.0.smithi031.stdout:3/263: truncate d6/d15/d1a/d25/f3d 1284515 0 2024-04-24T23:13:37.350 INFO:tasks.workunit.client.0.smithi031.stdout:3/264: fsync d6/d14/d1d/f24 0 2024-04-24T23:13:37.414 INFO:tasks.workunit.client.0.smithi031.stdout:1/205: sync 2024-04-24T23:13:37.414 INFO:tasks.workunit.client.0.smithi031.stdout:7/219: sync 2024-04-24T23:13:37.418 INFO:tasks.workunit.client.0.smithi031.stdout:1/206: mkdir d1/da/d42 0 2024-04-24T23:13:37.422 INFO:tasks.workunit.client.0.smithi031.stdout:1/207: dread d1/dc/fe [0,4194304] 0 2024-04-24T23:13:37.429 INFO:tasks.workunit.client.0.smithi031.stdout:1/208: symlink d1/dc/l43 0 2024-04-24T23:13:37.433 INFO:tasks.workunit.client.0.smithi031.stdout:7/220: rmdir d1/d1c/d2c/d41 0 2024-04-24T23:13:37.436 INFO:tasks.workunit.client.0.smithi031.stdout:1/209: unlink d1/d1d/f2f 0 2024-04-24T23:13:37.437 INFO:tasks.workunit.client.0.smithi031.stdout:7/221: rmdir d1/d1c/d2c 39 2024-04-24T23:13:37.441 INFO:tasks.workunit.client.0.smithi031.stdout:1/210: fdatasync d1/f8 0 2024-04-24T23:13:37.450 INFO:tasks.workunit.client.0.smithi031.stdout:7/222: symlink d1/d2/l44 0 2024-04-24T23:13:37.452 INFO:tasks.workunit.client.0.smithi031.stdout:7/223: write d1/d1c/f35 [898181,126147] 0 2024-04-24T23:13:37.453 INFO:tasks.workunit.client.0.smithi031.stdout:3/265: dwrite d6/fc [0,4194304] 0 2024-04-24T23:13:37.458 INFO:tasks.workunit.client.0.smithi031.stdout:7/224: fdatasync d1/d2/f12 0 2024-04-24T23:13:37.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/266: dwrite d6/d15/d1a/d25/f3d [0,4194304] 0 2024-04-24T23:13:37.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/267: chown d6/d15 99553 1 2024-04-24T23:13:37.465 INFO:tasks.workunit.client.0.smithi031.stdout:7/225: write d1/f13 [5042311,110604] 0 2024-04-24T23:13:37.470 INFO:tasks.workunit.client.0.smithi031.stdout:3/268: chown d6/l10 67 1 2024-04-24T23:13:37.474 INFO:tasks.workunit.client.0.smithi031.stdout:7/226: dread d1/d2/dd/f23 [0,4194304] 0 2024-04-24T23:13:37.480 INFO:tasks.workunit.client.0.smithi031.stdout:7/227: dwrite d1/d1c/f29 [0,4194304] 0 2024-04-24T23:13:37.480 INFO:tasks.workunit.client.0.smithi031.stdout:7/228: dread - d1/d1c/f3c zero size 2024-04-24T23:13:37.487 INFO:tasks.workunit.client.0.smithi031.stdout:7/229: readlink d1/lb 0 2024-04-24T23:13:37.493 INFO:tasks.workunit.client.0.smithi031.stdout:7/230: mkdir d1/d45 0 2024-04-24T23:13:37.502 INFO:tasks.workunit.client.0.smithi031.stdout:7/231: dwrite d1/d2/d1e/f36 [0,4194304] 0 2024-04-24T23:13:37.510 INFO:tasks.workunit.client.0.smithi031.stdout:7/232: rename d1/d2/f19 to d1/d2/f46 0 2024-04-24T23:13:37.517 INFO:tasks.workunit.client.0.smithi031.stdout:7/233: creat d1/d45/f47 x:0 0 0 2024-04-24T23:13:37.520 INFO:tasks.workunit.client.0.smithi031.stdout:9/365: sync 2024-04-24T23:13:37.520 INFO:tasks.workunit.client.0.smithi031.stdout:0/213: sync 2024-04-24T23:13:37.525 INFO:tasks.workunit.client.0.smithi031.stdout:0/214: mknod da/d22/c43 0 2024-04-24T23:13:37.526 INFO:tasks.workunit.client.0.smithi031.stdout:9/366: dwrite d2/d16/f20 [0,4194304] 0 2024-04-24T23:13:37.527 INFO:tasks.workunit.client.0.smithi031.stdout:9/367: write d2/d1b/d14/f5f [1298418,121720] 0 2024-04-24T23:13:37.532 INFO:tasks.workunit.client.0.smithi031.stdout:0/215: truncate f3 3136985 0 2024-04-24T23:13:37.538 INFO:tasks.workunit.client.0.smithi031.stdout:0/216: creat da/d20/f44 x:0 0 0 2024-04-24T23:13:37.540 INFO:tasks.workunit.client.0.smithi031.stdout:9/368: unlink d2/d1b/d14/d1d/c48 0 2024-04-24T23:13:37.541 INFO:tasks.workunit.client.0.smithi031.stdout:9/369: readlink d2/d16/l2d 0 2024-04-24T23:13:37.541 INFO:tasks.workunit.client.0.smithi031.stdout:9/370: truncate d2/fc 424927 0 2024-04-24T23:13:37.548 INFO:tasks.workunit.client.0.smithi031.stdout:0/217: mknod da/d18/d1a/d37/d41/c45 0 2024-04-24T23:13:37.554 INFO:tasks.workunit.client.0.smithi031.stdout:1/211: write d1/d1d/f35 [18873,7034] 0 2024-04-24T23:13:37.561 INFO:tasks.workunit.client.0.smithi031.stdout:5/237: sync 2024-04-24T23:13:37.561 INFO:tasks.workunit.client.0.smithi031.stdout:6/376: sync 2024-04-24T23:13:37.561 INFO:tasks.workunit.client.0.smithi031.stdout:8/335: sync 2024-04-24T23:13:37.561 INFO:tasks.workunit.client.0.smithi031.stdout:2/186: sync 2024-04-24T23:13:37.562 INFO:tasks.workunit.client.0.smithi031.stdout:4/243: sync 2024-04-24T23:13:37.564 INFO:tasks.workunit.client.0.smithi031.stdout:0/218: dread da/ff [0,4194304] 0 2024-04-24T23:13:37.564 INFO:tasks.workunit.client.0.smithi031.stdout:0/219: chown da/d11/l16 51664064 1 2024-04-24T23:13:37.565 INFO:tasks.workunit.client.0.smithi031.stdout:2/187: mkdir d5/d3d 0 2024-04-24T23:13:37.568 INFO:tasks.workunit.client.0.smithi031.stdout:5/238: unlink d0/f9 0 2024-04-24T23:13:37.568 INFO:tasks.workunit.client.0.smithi031.stdout:5/239: write d0/d7/f3a [993763,122161] 0 2024-04-24T23:13:37.570 INFO:tasks.workunit.client.0.smithi031.stdout:6/377: creat d13/d1b/f8f x:0 0 0 2024-04-24T23:13:37.572 INFO:tasks.workunit.client.0.smithi031.stdout:6/378: write d13/d75/d79/f84 [502090,89544] 0 2024-04-24T23:13:37.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/240: dread d0/d6/d32/f17 [0,4194304] 0 2024-04-24T23:13:37.574 INFO:tasks.workunit.client.0.smithi031.stdout:5/241: chown d0/d6/d32 2 1 2024-04-24T23:13:37.576 INFO:tasks.workunit.client.0.smithi031.stdout:0/220: unlink da/d22/c2a 0 2024-04-24T23:13:37.577 INFO:tasks.workunit.client.0.smithi031.stdout:0/221: dread - da/d18/d1a/d37/f3d zero size 2024-04-24T23:13:37.578 INFO:tasks.workunit.client.0.smithi031.stdout:0/222: write da/fd [149208,35843] 0 2024-04-24T23:13:37.579 INFO:tasks.workunit.client.0.smithi031.stdout:5/242: dread d0/d7/f3a [0,4194304] 0 2024-04-24T23:13:37.580 INFO:tasks.workunit.client.0.smithi031.stdout:3/269: write d6/d15/d1a/d25/f3d [4801273,2492] 0 2024-04-24T23:13:37.581 INFO:tasks.workunit.client.0.smithi031.stdout:5/243: readlink d0/d7/db/d1d/d2e/d28/l38 0 2024-04-24T23:13:37.587 INFO:tasks.workunit.client.0.smithi031.stdout:3/270: dwrite d6/d15/d1a/f22 [0,4194304] 0 2024-04-24T23:13:37.592 INFO:tasks.workunit.client.0.smithi031.stdout:8/336: rename d8/f24 to d8/d1a/d1d/f78 0 2024-04-24T23:13:37.595 INFO:tasks.workunit.client.0.smithi031.stdout:0/223: symlink da/d11/l46 0 2024-04-24T23:13:37.595 INFO:tasks.workunit.client.0.smithi031.stdout:0/224: chown da/f2b 136 1 2024-04-24T23:13:37.598 INFO:tasks.workunit.client.0.smithi031.stdout:6/379: link d13/d75/f39 d13/d15/d1d/d23/d2d/f90 0 2024-04-24T23:13:37.600 INFO:tasks.workunit.client.0.smithi031.stdout:6/380: write d13/d15/d1d/d23/d2d/f3d [1551996,125685] 0 2024-04-24T23:13:37.602 INFO:tasks.workunit.client.0.smithi031.stdout:4/244: link d3/fc d3/d20/f4d 0 2024-04-24T23:13:37.602 INFO:tasks.workunit.client.0.smithi031.stdout:4/245: dread - d3/dd/f31 zero size 2024-04-24T23:13:37.608 INFO:tasks.workunit.client.0.smithi031.stdout:2/188: rename d5/d6/df/l20 to d5/d8/d15/l3e 0 2024-04-24T23:13:37.610 INFO:tasks.workunit.client.0.smithi031.stdout:0/225: creat da/d18/d1a/d37/d41/f47 x:0 0 0 2024-04-24T23:13:37.612 INFO:tasks.workunit.client.0.smithi031.stdout:5/244: symlink d0/d7/db/dc/l48 0 2024-04-24T23:13:37.614 INFO:tasks.workunit.client.0.smithi031.stdout:6/381: symlink d13/d15/d1d/d23/d44/d47/l91 0 2024-04-24T23:13:37.617 INFO:tasks.workunit.client.0.smithi031.stdout:3/271: mkdir d6/df/d16/d53 0 2024-04-24T23:13:37.620 INFO:tasks.workunit.client.0.smithi031.stdout:8/337: rename d8/d1a/d1d/d35/f4a to d8/d1a/d1d/d35/d58/d5b/f79 0 2024-04-24T23:13:37.621 INFO:tasks.workunit.client.0.smithi031.stdout:2/189: creat d5/d6/d29/d26/f3f x:0 0 0 2024-04-24T23:13:37.622 INFO:tasks.workunit.client.0.smithi031.stdout:4/246: dread d3/f9 [0,4194304] 0 2024-04-24T23:13:37.622 INFO:tasks.workunit.client.0.smithi031.stdout:2/190: write d5/f27 [529336,63103] 0 2024-04-24T23:13:37.626 INFO:tasks.workunit.client.0.smithi031.stdout:6/382: symlink d13/d15/d1d/d23/d24/d8b/l92 0 2024-04-24T23:13:37.627 INFO:tasks.workunit.client.0.smithi031.stdout:7/234: truncate d1/d2/d1e/f36 1812341 0 2024-04-24T23:13:37.628 INFO:tasks.workunit.client.0.smithi031.stdout:6/383: write d13/d1b/f54 [1227082,45170] 0 2024-04-24T23:13:37.629 INFO:tasks.workunit.client.0.smithi031.stdout:3/272: creat d6/d15/d1a/d25/f54 x:0 0 0 2024-04-24T23:13:37.631 INFO:tasks.workunit.client.0.smithi031.stdout:2/191: dwrite d5/d6/f35 [0,4194304] 0 2024-04-24T23:13:37.635 INFO:tasks.workunit.client.0.smithi031.stdout:4/247: mknod d3/d20/d25/c4e 0 2024-04-24T23:13:37.642 INFO:tasks.workunit.client.0.smithi031.stdout:7/235: fsync d1/f2b 0 2024-04-24T23:13:37.646 INFO:tasks.workunit.client.0.smithi031.stdout:2/192: creat d5/d28/f40 x:0 0 0 2024-04-24T23:13:37.650 INFO:tasks.workunit.client.0.smithi031.stdout:9/371: dwrite d2/d1b/f41 [0,4194304] 0 2024-04-24T23:13:37.651 INFO:tasks.workunit.client.0.smithi031.stdout:5/245: creat d0/f49 x:0 0 0 2024-04-24T23:13:37.655 INFO:tasks.workunit.client.0.smithi031.stdout:7/236: fsync d1/f28 0 2024-04-24T23:13:37.656 INFO:tasks.workunit.client.0.smithi031.stdout:5/246: dread d0/d7/db/dc/d25/f29 [0,4194304] 0 2024-04-24T23:13:37.657 INFO:tasks.workunit.client.0.smithi031.stdout:5/247: readlink d0/l16 0 2024-04-24T23:13:37.658 INFO:tasks.workunit.client.0.smithi031.stdout:5/248: chown d0/d6/f22 351285595 1 2024-04-24T23:13:37.661 INFO:tasks.workunit.client.0.smithi031.stdout:9/372: creat d2/d43/f70 x:0 0 0 2024-04-24T23:13:37.663 INFO:tasks.workunit.client.0.smithi031.stdout:7/237: dwrite d1/d2/f11 [0,4194304] 0 2024-04-24T23:13:37.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/238: read d1/d2/dd/f23 [51070,10992] 0 2024-04-24T23:13:37.666 INFO:tasks.workunit.client.0.smithi031.stdout:8/338: link d8/c9 d8/d1a/d74/c7a 0 2024-04-24T23:13:37.668 INFO:tasks.workunit.client.0.smithi031.stdout:1/212: dwrite d1/dc/d11/f20 [4194304,4194304] 0 2024-04-24T23:13:37.675 INFO:tasks.workunit.client.0.smithi031.stdout:9/373: dwrite d2/f64 [0,4194304] 0 2024-04-24T23:13:37.676 INFO:tasks.workunit.client.0.smithi031.stdout:3/273: getdents d6/d14/d1d 0 2024-04-24T23:13:37.679 INFO:tasks.workunit.client.0.smithi031.stdout:3/274: fdatasync d6/df/d16/d2a/f2e 0 2024-04-24T23:13:37.680 INFO:tasks.workunit.client.0.smithi031.stdout:5/249: dwrite d0/d6/d32/f17 [0,4194304] 0 2024-04-24T23:13:37.682 INFO:tasks.workunit.client.0.smithi031.stdout:5/250: fsync d0/d6/d12/f36 0 2024-04-24T23:13:37.684 INFO:tasks.workunit.client.0.smithi031.stdout:1/213: creat d1/d33/d36/f44 x:0 0 0 2024-04-24T23:13:37.687 INFO:tasks.workunit.client.0.smithi031.stdout:8/339: mkdir d8/d15/d37/d43/d7b 0 2024-04-24T23:13:37.688 INFO:tasks.workunit.client.0.smithi031.stdout:8/340: write d8/d1a/f2f [842538,108450] 0 2024-04-24T23:13:37.695 INFO:tasks.workunit.client.0.smithi031.stdout:5/251: creat d0/d6/d32/f4a x:0 0 0 2024-04-24T23:13:37.697 INFO:tasks.workunit.client.0.smithi031.stdout:1/214: creat d1/dc/d11/d21/f45 x:0 0 0 2024-04-24T23:13:37.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/341: creat d8/d1a/d1d/d35/f7c x:0 0 0 2024-04-24T23:13:37.710 INFO:tasks.workunit.client.0.smithi031.stdout:7/239: rename d1/d1c/l31 to d1/l48 0 2024-04-24T23:13:37.710 INFO:tasks.workunit.client.0.smithi031.stdout:7/240: chown d1/d2/d1e 5 1 2024-04-24T23:13:37.711 INFO:tasks.workunit.client.0.smithi031.stdout:8/342: write d8/f70 [7915548,23037] 0 2024-04-24T23:13:37.713 INFO:tasks.workunit.client.0.smithi031.stdout:7/241: rename d1/d2/d1e to d1/d2/d1e/d49 22 2024-04-24T23:13:37.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/215: mkdir d1/d3e/d46 0 2024-04-24T23:13:37.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/216: dread - d1/da/d10/f1c zero size 2024-04-24T23:13:37.717 INFO:tasks.workunit.client.0.smithi031.stdout:1/217: chown d1/da/d12 242553 1 2024-04-24T23:13:37.718 INFO:tasks.workunit.client.0.smithi031.stdout:3/275: getdents d6/df 0 2024-04-24T23:13:37.719 INFO:tasks.workunit.client.0.smithi031.stdout:3/276: stat d6/d14/c18 0 2024-04-24T23:13:37.723 INFO:tasks.workunit.client.0.smithi031.stdout:1/218: fdatasync d1/dc/fe 0 2024-04-24T23:13:37.725 INFO:tasks.workunit.client.0.smithi031.stdout:3/277: chown d6/la 3734025 1 2024-04-24T23:13:37.726 INFO:tasks.workunit.client.0.smithi031.stdout:3/278: write d6/df/f34 [2367548,4941] 0 2024-04-24T23:13:37.729 INFO:tasks.workunit.client.0.smithi031.stdout:3/279: fdatasync d6/df/d16/f1c 0 2024-04-24T23:13:37.730 INFO:tasks.workunit.client.0.smithi031.stdout:7/242: mknod d1/c4a 0 2024-04-24T23:13:37.732 INFO:tasks.workunit.client.0.smithi031.stdout:1/219: dread d1/dc/d11/d21/f3a [0,4194304] 0 2024-04-24T23:13:37.734 INFO:tasks.workunit.client.0.smithi031.stdout:0/226: dwrite da/f13 [0,4194304] 0 2024-04-24T23:13:37.743 INFO:tasks.workunit.client.0.smithi031.stdout:1/220: creat d1/d33/f47 x:0 0 0 2024-04-24T23:13:37.745 INFO:tasks.workunit.client.0.smithi031.stdout:1/221: write d1/d1d/f35 [785605,8979] 0 2024-04-24T23:13:37.754 INFO:tasks.workunit.client.0.smithi031.stdout:0/227: creat da/d18/d3b/f48 x:0 0 0 2024-04-24T23:13:37.755 INFO:tasks.workunit.client.0.smithi031.stdout:7/243: getdents d1/d1c/d2c/d38 0 2024-04-24T23:13:37.755 INFO:tasks.workunit.client.0.smithi031.stdout:6/384: dwrite d13/d1b/f6b [0,4194304] 0 2024-04-24T23:13:37.757 INFO:tasks.workunit.client.0.smithi031.stdout:3/280: getdents d6/d15/d29 0 2024-04-24T23:13:37.761 INFO:tasks.workunit.client.0.smithi031.stdout:6/385: unlink d13/d1b/f3c 0 2024-04-24T23:13:37.763 INFO:tasks.workunit.client.0.smithi031.stdout:0/228: dwrite da/d18/d1a/d37/f3d [0,4194304] 0 2024-04-24T23:13:37.764 INFO:tasks.workunit.client.0.smithi031.stdout:0/229: stat da/d18/d1a/d37/f26 0 2024-04-24T23:13:37.765 INFO:tasks.workunit.client.0.smithi031.stdout:0/230: chown da/f10 148 1 2024-04-24T23:13:37.770 INFO:tasks.workunit.client.0.smithi031.stdout:3/281: mkdir d6/d55 0 2024-04-24T23:13:37.772 INFO:tasks.workunit.client.0.smithi031.stdout:7/244: creat d1/f4b x:0 0 0 2024-04-24T23:13:37.773 INFO:tasks.workunit.client.0.smithi031.stdout:7/245: readlink d1/d2/d1e/l30 0 2024-04-24T23:13:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:0/231: mknod da/d11/c49 0 2024-04-24T23:13:37.774 INFO:tasks.workunit.client.0.smithi031.stdout:7/246: write d1/f28 [408354,14075] 0 2024-04-24T23:13:37.780 INFO:tasks.workunit.client.0.smithi031.stdout:3/282: rmdir d6/d15/d1a/d25 39 2024-04-24T23:13:37.781 INFO:tasks.workunit.client.0.smithi031.stdout:3/283: chown d6/df/d16/d53 23 1 2024-04-24T23:13:37.782 INFO:tasks.workunit.client.0.smithi031.stdout:9/374: dwrite d2/d16/f33 [0,4194304] 0 2024-04-24T23:13:37.789 INFO:tasks.workunit.client.0.smithi031.stdout:6/386: rename d13/d15/d1d/d23/c7d to d13/c93 0 2024-04-24T23:13:37.793 INFO:tasks.workunit.client.0.smithi031.stdout:5/252: dwrite d0/d6/f1a [0,4194304] 0 2024-04-24T23:13:37.798 INFO:tasks.workunit.client.0.smithi031.stdout:0/232: rename da/d11/f12 to da/d18/d1a/d37/f4a 0 2024-04-24T23:13:37.799 INFO:tasks.workunit.client.0.smithi031.stdout:8/343: write d8/d15/f2a [8047,82345] 0 2024-04-24T23:13:37.801 INFO:tasks.workunit.client.0.smithi031.stdout:0/233: write da/f2b [2208644,9375] 0 2024-04-24T23:13:37.802 INFO:tasks.workunit.client.0.smithi031.stdout:7/247: dread d1/d2/f7 [0,4194304] 0 2024-04-24T23:13:37.809 INFO:tasks.workunit.client.0.smithi031.stdout:0/234: dwrite da/d18/d1a/f29 [0,4194304] 0 2024-04-24T23:13:37.811 INFO:tasks.workunit.client.0.smithi031.stdout:7/248: dread d1/d2/f4 [0,4194304] 0 2024-04-24T23:13:37.817 INFO:tasks.workunit.client.0.smithi031.stdout:7/249: dread d1/d2/f2d [0,4194304] 0 2024-04-24T23:13:37.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/253: rename d0/d7/db to d0/d6/d3b/d4b 0 2024-04-24T23:13:37.821 INFO:tasks.workunit.client.0.smithi031.stdout:5/254: write d0/d6/d32/f4 [2217428,741] 0 2024-04-24T23:13:37.822 INFO:tasks.workunit.client.0.smithi031.stdout:1/222: dwrite d1/f8 [0,4194304] 0 2024-04-24T23:13:37.824 INFO:tasks.workunit.client.0.smithi031.stdout:1/223: chown d1/dc/d11/f24 1 1 2024-04-24T23:13:37.825 INFO:tasks.workunit.client.0.smithi031.stdout:9/375: creat d2/d1b/d14/d1d/d45/d56/d67/d4a/f71 x:0 0 0 2024-04-24T23:13:37.825 INFO:tasks.workunit.client.0.smithi031.stdout:1/224: stat d1/da/d12/c14 0 2024-04-24T23:13:37.826 INFO:tasks.workunit.client.0.smithi031.stdout:9/376: write d2/d1b/d14/d18/f31 [4091663,5079] 0 2024-04-24T23:13:37.828 INFO:tasks.workunit.client.0.smithi031.stdout:0/235: creat da/d22/f4b x:0 0 0 2024-04-24T23:13:37.829 INFO:tasks.workunit.client.0.smithi031.stdout:0/236: readlink l4 0 2024-04-24T23:13:37.833 INFO:tasks.workunit.client.0.smithi031.stdout:7/250: readlink d1/lb 0 2024-04-24T23:13:37.836 INFO:tasks.workunit.client.0.smithi031.stdout:5/255: symlink d0/d6/d12/l4c 0 2024-04-24T23:13:37.838 INFO:tasks.workunit.client.0.smithi031.stdout:3/284: write d6/df/d16/d2a/f4f [2501855,121483] 0 2024-04-24T23:13:37.839 INFO:tasks.workunit.client.0.smithi031.stdout:1/225: creat d1/d33/f48 x:0 0 0 2024-04-24T23:13:37.843 INFO:tasks.workunit.client.0.smithi031.stdout:0/237: mknod da/d20/c4c 0 2024-04-24T23:13:37.843 INFO:tasks.workunit.client.0.smithi031.stdout:6/387: write d13/d15/d1d/d23/d4d/d7b/f82 [474046,100390] 0 2024-04-24T23:13:37.847 INFO:tasks.workunit.client.0.smithi031.stdout:8/344: link d8/d1a/d1d/d35/f6d d8/d1a/d1d/f7d 0 2024-04-24T23:13:37.848 INFO:tasks.workunit.client.0.smithi031.stdout:7/251: creat d1/d1c/d2c/f4c x:0 0 0 2024-04-24T23:13:37.850 INFO:tasks.workunit.client.0.smithi031.stdout:5/256: creat d0/d6/d3b/f4d x:0 0 0 2024-04-24T23:13:37.855 INFO:tasks.workunit.client.0.smithi031.stdout:3/285: dread d6/d14/d1d/f1e [0,4194304] 0 2024-04-24T23:13:37.860 INFO:tasks.workunit.client.0.smithi031.stdout:6/388: symlink d13/d15/d1d/d23/d44/d47/l94 0 2024-04-24T23:13:37.861 INFO:tasks.workunit.client.0.smithi031.stdout:8/345: creat d8/d1a/f7e x:0 0 0 2024-04-24T23:13:37.865 INFO:tasks.workunit.client.0.smithi031.stdout:1/226: mkdir d1/d49 0 2024-04-24T23:13:37.869 INFO:tasks.workunit.client.0.smithi031.stdout:6/389: unlink d13/d1b/d32/l5c 0 2024-04-24T23:13:37.871 INFO:tasks.workunit.client.0.smithi031.stdout:8/346: creat d8/d15/d37/d62/f7f x:0 0 0 2024-04-24T23:13:37.874 INFO:tasks.workunit.client.0.smithi031.stdout:5/257: mkdir d0/d6/d3b/d4b/d4e 0 2024-04-24T23:13:37.875 INFO:tasks.workunit.client.0.smithi031.stdout:1/227: mknod d1/dc/c4a 0 2024-04-24T23:13:37.876 INFO:tasks.workunit.client.0.smithi031.stdout:9/377: truncate d2/f42 3182851 0 2024-04-24T23:13:37.877 INFO:tasks.workunit.client.0.smithi031.stdout:9/378: chown d2/d1b/d14/d1d/d45/d56/d67/d4a/d4b/f52 1529137 1 2024-04-24T23:13:37.881 INFO:tasks.workunit.client.0.smithi031.stdout:7/252: link d1/c4a d1/d2/d1e/c4d 0 2024-04-24T23:13:37.882 INFO:tasks.workunit.client.0.smithi031.stdout:7/253: read d1/d2/f7 [4598843,13482] 0 2024-04-24T23:13:37.883 INFO:tasks.workunit.client.0.smithi031.stdout:0/238: write f3 [1958959,53910] 0 2024-04-24T23:13:37.883 INFO:tasks.workunit.client.0.smithi031.stdout:5/258: chown d0/d6/d3b/d4b/d1d/c3d 139288 1 2024-04-24T23:13:37.885 INFO:tasks.workunit.client.0.smithi031.stdout:9/379: dwrite d2/d1b/d14/d1d/d45/d56/d67/d4a/f71 [0,4194304] 0 2024-04-24T23:13:37.886 INFO:tasks.workunit.client.0.smithi031.stdout:0/239: dread - da/d18/d1a/d37/d41/f47 zero size 2024-04-24T23:13:37.887 INFO:tasks.workunit.client.0.smithi031.stdout:1/228: rename d1/dc/d11/d21/f45 to d1/dc/d11/d21/f4b 0 2024-04-24T23:13:37.887 INFO:tasks.workunit.client.0.smithi031.stdout:3/286: link d6/d15/l2c d6/d55/l56 0 2024-04-24T23:13:37.888 INFO:tasks.workunit.client.0.smithi031.stdout:6/390: mknod d13/d15/d1d/c95 0 2024-04-24T23:13:37.888 INFO:tasks.workunit.client.0.smithi031.stdout:9/380: read d2/d1b/f17 [3596873,33288] 0 2024-04-24T23:13:37.889 INFO:tasks.workunit.client.0.smithi031.stdout:8/347: creat d8/d1a/d73/f80 x:0 0 0 2024-04-24T23:13:37.893 INFO:tasks.workunit.client.0.smithi031.stdout:7/254: creat d1/d2/dd/d33/f4e x:0 0 0 2024-04-24T23:13:37.895 INFO:tasks.workunit.client.0.smithi031.stdout:3/287: dwrite d6/d14/d1d/f4c [0,4194304] 0 2024-04-24T23:13:37.895 INFO:tasks.workunit.client.0.smithi031.stdout:5/259: mkdir d0/d6/d3b/d4f 0 2024-04-24T23:13:37.900 INFO:tasks.workunit.client.0.smithi031.stdout:0/240: mkdir da/d20/d4d 0 2024-04-24T23:13:37.905 INFO:tasks.workunit.client.0.smithi031.stdout:1/229: mkdir d1/dc/d4c 0 2024-04-24T23:13:37.906 INFO:tasks.workunit.client.0.smithi031.stdout:1/230: chown d1/d33/d36/f44 22169 1 2024-04-24T23:13:37.907 INFO:tasks.workunit.client.0.smithi031.stdout:6/391: unlink d13/d15/f1f 0 2024-04-24T23:13:37.911 INFO:tasks.workunit.client.0.smithi031.stdout:1/231: dread d1/f8 [0,4194304] 0 2024-04-24T23:13:37.928 INFO:tasks.workunit.client.0.smithi031.stdout:0/241: rename da/d18/d3b to da/d20/d4e 0 2024-04-24T23:13:37.929 INFO:tasks.workunit.client.0.smithi031.stdout:0/242: chown da/d20/d4e/f48 910957 1 2024-04-24T23:13:37.929 INFO:tasks.workunit.client.0.smithi031.stdout:0/243: fsync da/d18/d1a/f1f 0 2024-04-24T23:13:37.931 INFO:tasks.workunit.client.0.smithi031.stdout:6/392: creat d13/d15/d1d/d23/d24/d63/f96 x:0 0 0 2024-04-24T23:13:37.934 INFO:tasks.workunit.client.0.smithi031.stdout:9/381: fdatasync d2/d1b/f13 0 2024-04-24T23:13:37.938 INFO:tasks.workunit.client.0.smithi031.stdout:6/393: dwrite d13/d15/d1d/d23/f2b [0,4194304] 0 2024-04-24T23:13:37.944 INFO:tasks.workunit.client.0.smithi031.stdout:3/288: mkdir d6/df/d16/d57 0 2024-04-24T23:13:37.945 INFO:tasks.workunit.client.0.smithi031.stdout:3/289: write d6/d14/f37 [487049,9353] 0 2024-04-24T23:13:37.954 INFO:tasks.workunit.client.0.smithi031.stdout:0/244: dwrite da/d18/d1a/d37/f4a [0,4194304] 0 2024-04-24T23:13:37.955 INFO:tasks.workunit.client.0.smithi031.stdout:1/232: link d1/dc/d11/f20 d1/da/d10/f4d 0 2024-04-24T23:13:37.957 INFO:tasks.workunit.client.0.smithi031.stdout:1/233: truncate d1/d33/d36/f44 846364 0 2024-04-24T23:13:37.958 INFO:tasks.workunit.client.0.smithi031.stdout:8/348: write d8/f14 [5191711,11552] 0 2024-04-24T23:13:37.959 INFO:tasks.workunit.client.0.smithi031.stdout:9/382: write d2/d1b/d14/d1d/d45/d56/d67/d4a/d4b/f52 [3435609,76485] 0 2024-04-24T23:13:37.962 INFO:tasks.workunit.client.0.smithi031.stdout:6/394: rmdir d13/d15/d1d/d23/d24/d8b 39 2024-04-24T23:13:37.963 INFO:tasks.workunit.client.0.smithi031.stdout:5/260: rmdir d0/d6/d3b/d4b/d4e 0 2024-04-24T23:13:37.965 INFO:tasks.workunit.client.0.smithi031.stdout:2/193: sync 2024-04-24T23:13:37.965 INFO:tasks.workunit.client.0.smithi031.stdout:4/248: sync 2024-04-24T23:13:37.966 INFO:tasks.workunit.client.0.smithi031.stdout:3/290: creat d6/d14/f58 x:0 0 0 2024-04-24T23:13:37.968 INFO:tasks.workunit.client.0.smithi031.stdout:4/249: read - d3/d23/d3b/f41 zero size 2024-04-24T23:13:37.971 INFO:tasks.workunit.client.0.smithi031.stdout:8/349: creat d8/d15/d37/d62/f81 x:0 0 0 2024-04-24T23:13:37.971 INFO:tasks.workunit.client.0.smithi031.stdout:6/395: dwrite d13/d75/d79/f85 [0,4194304] 0 2024-04-24T23:13:37.973 INFO:tasks.workunit.client.0.smithi031.stdout:9/383: rename d2/d1b/f40 to d2/d1b/d14/d1d/d45/f72 0 2024-04-24T23:13:37.976 INFO:tasks.workunit.client.0.smithi031.stdout:7/255: getdents d1/d2/dd/d33 0 2024-04-24T23:13:37.976 INFO:tasks.workunit.client.0.smithi031.stdout:7/256: stat d1/d2/f2d 0 2024-04-24T23:13:37.977 INFO:tasks.workunit.client.0.smithi031.stdout:5/261: creat d0/d6/d3b/d4b/d1d/d2e/f50 x:0 0 0 2024-04-24T23:13:37.978 INFO:tasks.workunit.client.0.smithi031.stdout:0/245: symlink da/d20/d4d/l4f 0 2024-04-24T23:13:37.979 INFO:tasks.workunit.client.0.smithi031.stdout:0/246: stat da/d18/d1a/d37/f4a 0 2024-04-24T23:13:37.981 INFO:tasks.workunit.client.0.smithi031.stdout:2/194: rmdir d5/d8/d15 39 2024-04-24T23:13:37.983 INFO:tasks.workunit.client.0.smithi031.stdout:2/195: dread - d5/d28/f40 zero size 2024-04-24T23:13:37.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/262: dwrite d0/d6/d32/f4 [0,4194304] 0 2024-04-24T23:13:37.993 INFO:tasks.workunit.client.0.smithi031.stdout:4/250: rename d3/d20/l22 to d3/dd/d3f/l4f 0 2024-04-24T23:13:37.993 INFO:tasks.workunit.client.0.smithi031.stdout:4/251: dread - d3/d20/d2d/d44/f48 zero size 2024-04-24T23:13:37.994 INFO:tasks.workunit.client.0.smithi031.stdout:4/252: chown d3/f9 3373866 1 2024-04-24T23:13:37.995 INFO:tasks.workunit.client.0.smithi031.stdout:9/384: creat d2/d1b/d14/d1d/d45/f73 x:0 0 0 2024-04-24T23:13:37.998 INFO:tasks.workunit.client.0.smithi031.stdout:7/257: mkdir d1/d2/d1e/d4f 0 2024-04-24T23:13:38.000 INFO:tasks.workunit.client.0.smithi031.stdout:0/247: mknod da/d18/c50 0 2024-04-24T23:13:38.002 INFO:tasks.workunit.client.0.smithi031.stdout:2/196: creat d5/f41 x:0 0 0 2024-04-24T23:13:38.004 INFO:tasks.workunit.client.0.smithi031.stdout:2/197: dread - d5/d6/d29/d26/f3f zero size 2024-04-24T23:13:38.008 INFO:tasks.workunit.client.0.smithi031.stdout:4/253: creat d3/d20/d2d/f50 x:0 0 0 2024-04-24T23:13:38.010 INFO:tasks.workunit.client.0.smithi031.stdout:4/254: write d3/f46 [25799,51672] 0 2024-04-24T23:13:38.016 INFO:tasks.workunit.client.0.smithi031.stdout:1/234: getdents d1/dc 0 2024-04-24T23:13:38.017 INFO:tasks.workunit.client.0.smithi031.stdout:3/291: truncate d6/df/f34 613406 0 2024-04-24T23:13:38.018 INFO:tasks.workunit.client.0.smithi031.stdout:0/248: dread da/fd [0,4194304] 0 2024-04-24T23:13:38.021 INFO:tasks.workunit.client.0.smithi031.stdout:9/385: symlink d2/d1b/d14/d1d/d45/d56/d67/d4a/l74 0 2024-04-24T23:13:38.022 INFO:tasks.workunit.client.0.smithi031.stdout:9/386: dread - d2/d1b/d14/f46 zero size 2024-04-24T23:13:38.025 INFO:tasks.workunit.client.0.smithi031.stdout:5/263: creat d0/d6/d3b/d4b/f51 x:0 0 0 2024-04-24T23:13:38.026 INFO:tasks.workunit.client.0.smithi031.stdout:0/249: creat da/d20/d4e/f51 x:0 0 0 2024-04-24T23:13:38.028 INFO:tasks.workunit.client.0.smithi031.stdout:8/350: link d8/d1a/d74/c7a d8/d1a/c82 0 2024-04-24T23:13:38.029 INFO:tasks.workunit.client.0.smithi031.stdout:6/396: getdents d13/d15/d1d/d23/d2d 0 2024-04-24T23:13:38.030 INFO:tasks.workunit.client.0.smithi031.stdout:4/255: truncate d3/d20/f4d 1174432 0 2024-04-24T23:13:38.031 INFO:tasks.workunit.client.0.smithi031.stdout:4/256: write d3/d20/d2d/f40 [1052904,16326] 0 2024-04-24T23:13:38.032 INFO:tasks.workunit.client.0.smithi031.stdout:1/235: mkdir d1/dc/d4c/d4e 0 2024-04-24T23:13:38.034 INFO:tasks.workunit.client.0.smithi031.stdout:2/198: unlink d5/d8/d15/l3e 0 2024-04-24T23:13:38.036 INFO:tasks.workunit.client.0.smithi031.stdout:5/264: creat d0/d6/d3b/d4b/d1d/d2e/f52 x:0 0 0 2024-04-24T23:13:38.037 INFO:tasks.workunit.client.0.smithi031.stdout:0/250: creat da/d20/f52 x:0 0 0 2024-04-24T23:13:38.037 INFO:tasks.workunit.client.0.smithi031.stdout:5/265: dread - d0/d6/d12/f31 zero size 2024-04-24T23:13:38.040 INFO:tasks.workunit.client.0.smithi031.stdout:7/258: rename d1/l16 to d1/d1c/l50 0 2024-04-24T23:13:38.041 INFO:tasks.workunit.client.0.smithi031.stdout:7/259: dread - d1/d2/dd/f17 zero size 2024-04-24T23:13:38.042 INFO:tasks.workunit.client.0.smithi031.stdout:7/260: read d1/d1c/f35 [615590,77087] 0 2024-04-24T23:13:38.044 INFO:tasks.workunit.client.0.smithi031.stdout:7/261: write d1/d2/f11 [2948289,103090] 0 2024-04-24T23:13:38.044 INFO:tasks.workunit.client.0.smithi031.stdout:1/236: fsync d1/f2 0 2024-04-24T23:13:38.044 INFO:tasks.workunit.client.0.smithi031.stdout:7/262: stat d1/d2/f46 0 2024-04-24T23:13:38.045 INFO:tasks.workunit.client.0.smithi031.stdout:1/237: dread - d1/dc/f2b zero size 2024-04-24T23:13:38.046 INFO:tasks.workunit.client.0.smithi031.stdout:2/199: creat d5/d8/d15/f42 x:0 0 0 2024-04-24T23:13:38.047 INFO:tasks.workunit.client.0.smithi031.stdout:1/238: dread - d1/d33/f47 zero size 2024-04-24T23:13:38.048 INFO:tasks.workunit.client.0.smithi031.stdout:1/239: chown d1/d49 28 1 2024-04-24T23:13:38.050 INFO:tasks.workunit.client.0.smithi031.stdout:5/266: rmdir d0/d6/d3b/d4b/d1d 39 2024-04-24T23:13:38.051 INFO:tasks.workunit.client.0.smithi031.stdout:2/200: dread d5/d6/f16 [0,4194304] 0 2024-04-24T23:13:38.052 INFO:tasks.workunit.client.0.smithi031.stdout:8/351: mknod d8/c83 0 2024-04-24T23:13:38.052 INFO:tasks.workunit.client.0.smithi031.stdout:8/352: chown l4 81 1 2024-04-24T23:13:38.054 INFO:tasks.workunit.client.0.smithi031.stdout:8/353: truncate d8/d15/f2a 679349 0 2024-04-24T23:13:38.055 INFO:tasks.workunit.client.0.smithi031.stdout:9/387: creat d2/f75 x:0 0 0 2024-04-24T23:13:38.057 INFO:tasks.workunit.client.0.smithi031.stdout:6/397: rename d13/c1a to d13/d75/d79/c97 0 2024-04-24T23:13:38.063 INFO:tasks.workunit.client.0.smithi031.stdout:7/263: unlink d1/d2/dd/d33/f42 0 2024-04-24T23:13:38.068 INFO:tasks.workunit.client.0.smithi031.stdout:1/240: creat d1/d3e/f4f x:0 0 0 2024-04-24T23:13:38.070 INFO:tasks.workunit.client.0.smithi031.stdout:3/292: truncate d6/d14/d1d/f1e 2394101 0 2024-04-24T23:13:38.079 INFO:tasks.workunit.client.0.smithi031.stdout:6/398: symlink d13/d15/d1d/d23/d24/d43/d68/l98 0 2024-04-24T23:13:38.080 INFO:tasks.workunit.client.0.smithi031.stdout:2/201: rmdir d5 39 2024-04-24T23:13:38.085 INFO:tasks.workunit.client.0.smithi031.stdout:0/251: creat da/f53 x:0 0 0 2024-04-24T23:13:38.088 INFO:tasks.workunit.client.0.smithi031.stdout:4/257: write d3/d20/f4d [2069969,49716] 0 2024-04-24T23:13:38.092 INFO:tasks.workunit.client.0.smithi031.stdout:8/354: mknod d8/d1a/d1d/d50/c84 0 2024-04-24T23:13:38.098 INFO:tasks.workunit.client.0.smithi031.stdout:0/252: creat da/d22/f54 x:0 0 0 2024-04-24T23:13:38.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/241: symlink d1/l50 0 2024-04-24T23:13:38.100 INFO:tasks.workunit.client.0.smithi031.stdout:9/388: write d2/d1b/d14/f19 [247460,125631] 0 2024-04-24T23:13:38.101 INFO:tasks.workunit.client.0.smithi031.stdout:1/242: write d1/d3e/f4f [814393,21160] 0 2024-04-24T23:13:38.103 INFO:tasks.workunit.client.0.smithi031.stdout:1/243: truncate d1/d33/f48 164206 0 2024-04-24T23:13:38.106 INFO:tasks.workunit.client.0.smithi031.stdout:8/355: creat d8/d15/d37/f85 x:0 0 0 2024-04-24T23:13:38.108 INFO:tasks.workunit.client.0.smithi031.stdout:2/202: mkdir d5/d28/d43 0 2024-04-24T23:13:38.110 INFO:tasks.workunit.client.0.smithi031.stdout:5/267: truncate d0/d6/d32/f39 1028496 0 2024-04-24T23:13:38.111 INFO:tasks.workunit.client.0.smithi031.stdout:6/399: mkdir d13/d15/d1d/d23/d99 0 2024-04-24T23:13:38.113 INFO:tasks.workunit.client.0.smithi031.stdout:7/264: rename d1/d2/l44 to d1/d1c/l51 0 2024-04-24T23:13:38.115 INFO:tasks.workunit.client.0.smithi031.stdout:0/253: mknod da/d22/c55 0 2024-04-24T23:13:38.116 INFO:tasks.workunit.client.0.smithi031.stdout:9/389: write d2/d1b/f2f [484043,23203] 0 2024-04-24T23:13:38.117 INFO:tasks.workunit.client.0.smithi031.stdout:1/244: creat d1/da/d10/f51 x:0 0 0 2024-04-24T23:13:38.120 INFO:tasks.workunit.client.0.smithi031.stdout:3/293: truncate d6/d14/d1d/f4c 2483214 0 2024-04-24T23:13:38.121 INFO:tasks.workunit.client.0.smithi031.stdout:3/294: fdatasync d6/df/d16/d2a/f31 0 2024-04-24T23:13:38.121 INFO:tasks.workunit.client.0.smithi031.stdout:2/203: unlink d5/d6/df/f17 0 2024-04-24T23:13:38.122 INFO:tasks.workunit.client.0.smithi031.stdout:2/204: chown d5/d6/f35 174 1 2024-04-24T23:13:38.124 INFO:tasks.workunit.client.0.smithi031.stdout:5/268: mkdir d0/d6/d3b/d4b/dc/d25/d53 0 2024-04-24T23:13:38.125 INFO:tasks.workunit.client.0.smithi031.stdout:6/400: write d13/d1b/f22 [1811284,80689] 0 2024-04-24T23:13:38.129 INFO:tasks.workunit.client.0.smithi031.stdout:6/401: dread - d13/d15/d1d/d23/d4d/f88 zero size 2024-04-24T23:13:38.129 INFO:tasks.workunit.client.0.smithi031.stdout:1/245: mkdir d1/da/d10/d52 0 2024-04-24T23:13:38.130 INFO:tasks.workunit.client.0.smithi031.stdout:4/258: dwrite d3/f11 [0,4194304] 0 2024-04-24T23:13:38.132 INFO:tasks.workunit.client.0.smithi031.stdout:8/356: mkdir d8/d15/d37/d43/d7b/d86 0 2024-04-24T23:13:38.132 INFO:tasks.workunit.client.0.smithi031.stdout:4/259: write d3/d23/f3a [1959384,86279] 0 2024-04-24T23:13:38.134 INFO:tasks.workunit.client.0.smithi031.stdout:3/295: mkdir d6/d14/d1d/d59 0 2024-04-24T23:13:38.135 INFO:tasks.workunit.client.0.smithi031.stdout:2/205: creat d5/d6/df/f44 x:0 0 0 2024-04-24T23:13:38.137 INFO:tasks.workunit.client.0.smithi031.stdout:7/265: unlink d1/d1c/l50 0 2024-04-24T23:13:38.143 INFO:tasks.workunit.client.0.smithi031.stdout:4/260: fdatasync d3/f26 0 2024-04-24T23:13:38.145 INFO:tasks.workunit.client.0.smithi031.stdout:8/357: mkdir d8/d15/d37/d43/d7b/d87 0 2024-04-24T23:13:38.146 INFO:tasks.workunit.client.0.smithi031.stdout:3/296: creat d6/df/d16/d2a/f5a x:0 0 0 2024-04-24T23:13:38.147 INFO:tasks.workunit.client.0.smithi031.stdout:7/266: fdatasync d1/d1c/f3c 0 2024-04-24T23:13:38.148 INFO:tasks.workunit.client.0.smithi031.stdout:5/269: creat d0/d6/d3b/d4b/d1d/d2e/f54 x:0 0 0 2024-04-24T23:13:38.149 INFO:tasks.workunit.client.0.smithi031.stdout:5/270: stat d0/d6/d32/f17 0 2024-04-24T23:13:38.149 INFO:tasks.workunit.client.0.smithi031.stdout:9/390: creat d2/d1b/d14/d1d/d45/d56/d67/d4a/f76 x:0 0 0 2024-04-24T23:13:38.151 INFO:tasks.workunit.client.0.smithi031.stdout:2/206: dread d5/f36 [0,4194304] 0 2024-04-24T23:13:38.154 INFO:tasks.workunit.client.0.smithi031.stdout:1/246: truncate d1/dc/fe 3049945 0 2024-04-24T23:13:38.155 INFO:tasks.workunit.client.0.smithi031.stdout:1/247: stat d1/da/d12/f19 0 2024-04-24T23:13:38.155 INFO:tasks.workunit.client.0.smithi031.stdout:1/248: fdatasync d1/d1d/f35 0 2024-04-24T23:13:38.157 INFO:tasks.workunit.client.0.smithi031.stdout:1/249: truncate d1/da/d10/f51 926414 0 2024-04-24T23:13:38.158 INFO:tasks.workunit.client.0.smithi031.stdout:1/250: write d1/ff [532159,55396] 0 2024-04-24T23:13:38.159 INFO:tasks.workunit.client.0.smithi031.stdout:4/261: creat d3/d20/f51 x:0 0 0 2024-04-24T23:13:38.159 INFO:tasks.workunit.client.0.smithi031.stdout:4/262: chown d3/dd/f33 407990 1 2024-04-24T23:13:38.166 INFO:tasks.workunit.client.0.smithi031.stdout:7/267: mknod d1/d2/dd/d33/c52 0 2024-04-24T23:13:38.168 INFO:tasks.workunit.client.0.smithi031.stdout:0/254: getdents da/d11 0 2024-04-24T23:13:38.168 INFO:tasks.workunit.client.0.smithi031.stdout:0/255: fdatasync da/d18/d1a/f29 0 2024-04-24T23:13:38.171 INFO:tasks.workunit.client.0.smithi031.stdout:9/391: creat d2/d16/f77 x:0 0 0 2024-04-24T23:13:38.174 INFO:tasks.workunit.client.0.smithi031.stdout:0/256: dread da/d18/d1a/d37/f4a [0,4194304] 0 2024-04-24T23:13:38.175 INFO:tasks.workunit.client.0.smithi031.stdout:9/392: dread d2/fc [0,4194304] 0 2024-04-24T23:13:38.181 INFO:tasks.workunit.client.0.smithi031.stdout:4/263: creat d3/d20/d2d/f52 x:0 0 0 2024-04-24T23:13:38.183 INFO:tasks.workunit.client.0.smithi031.stdout:0/257: dwrite da/d18/d1a/d37/d41/f47 [0,4194304] 0 2024-04-24T23:13:38.183 INFO:tasks.workunit.client.0.smithi031.stdout:4/264: chown d3/l5 320 1 2024-04-24T23:13:38.183 INFO:tasks.workunit.client.0.smithi031.stdout:0/258: stat da/d20/c35 0 2024-04-24T23:13:38.186 INFO:tasks.workunit.client.0.smithi031.stdout:8/358: link d8/d15/d37/d43/f6f d8/d1a/d1d/d50/f88 0 2024-04-24T23:13:38.187 INFO:tasks.workunit.client.0.smithi031.stdout:8/359: fsync d8/d15/d37/f5d 0 2024-04-24T23:13:38.198 INFO:tasks.workunit.client.0.smithi031.stdout:9/393: mkdir d2/d1b/d14/d78 0 2024-04-24T23:13:38.202 INFO:tasks.workunit.client.0.smithi031.stdout:4/265: fdatasync d3/f1b 0 2024-04-24T23:13:38.204 INFO:tasks.workunit.client.0.smithi031.stdout:9/394: dread d2/d1b/d14/d1d/f28 [0,4194304] 0 2024-04-24T23:13:38.206 INFO:tasks.workunit.client.0.smithi031.stdout:8/360: mkdir d8/d15/d37/d43/d7b/d89 0 2024-04-24T23:13:38.213 INFO:tasks.workunit.client.0.smithi031.stdout:1/251: write d1/da/d10/f4d [9431091,27683] 0 2024-04-24T23:13:38.216 INFO:tasks.workunit.client.0.smithi031.stdout:3/297: write d6/df/f34 [367284,29066] 0 2024-04-24T23:13:38.218 INFO:tasks.workunit.client.0.smithi031.stdout:4/266: fsync d3/f1b 0 2024-04-24T23:13:38.219 INFO:tasks.workunit.client.0.smithi031.stdout:0/259: mknod da/c56 0 2024-04-24T23:13:38.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/260: dwrite da/f3f [4194304,4194304] 0 2024-04-24T23:13:38.227 INFO:tasks.workunit.client.0.smithi031.stdout:8/361: mknod d8/d15/d37/d43/c8a 0 2024-04-24T23:13:38.228 INFO:tasks.workunit.client.0.smithi031.stdout:8/362: readlink l4 0 2024-04-24T23:13:38.228 INFO:tasks.workunit.client.0.smithi031.stdout:5/271: getdents d0/d6/d3b/d4b 0 2024-04-24T23:13:38.230 INFO:tasks.workunit.client.0.smithi031.stdout:2/207: write d5/d6/df/f1e [17744,121289] 0 2024-04-24T23:13:38.234 INFO:tasks.workunit.client.0.smithi031.stdout:1/252: unlink d1/da/d12/f3d 0 2024-04-24T23:13:38.239 INFO:tasks.workunit.client.0.smithi031.stdout:4/267: mkdir d3/d20/d53 0 2024-04-24T23:13:38.241 INFO:tasks.workunit.client.0.smithi031.stdout:9/395: mkdir d2/d1b/d79 0 2024-04-24T23:13:38.242 INFO:tasks.workunit.client.0.smithi031.stdout:9/396: readlink d2/d16/l38 0 2024-04-24T23:13:38.245 INFO:tasks.workunit.client.0.smithi031.stdout:7/268: getdents d1 0 2024-04-24T23:13:38.245 INFO:tasks.workunit.client.0.smithi031.stdout:0/261: creat da/d20/d4e/f57 x:0 0 0 2024-04-24T23:13:38.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/268: dwrite d3/f11 [0,4194304] 0 2024-04-24T23:13:38.249 INFO:tasks.workunit.client.0.smithi031.stdout:8/363: creat d8/d1a/d1d/d35/d58/d5b/f8b x:0 0 0 2024-04-24T23:13:38.251 INFO:tasks.workunit.client.0.smithi031.stdout:5/272: mkdir d0/d6/d3b/d4b/dc/d25/d55 0 2024-04-24T23:13:38.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/253: symlink d1/d33/l53 0 2024-04-24T23:13:38.257 INFO:tasks.workunit.client.0.smithi031.stdout:9/397: creat d2/d1b/d14/d1d/d45/d56/d67/f7a x:0 0 0 2024-04-24T23:13:38.260 INFO:tasks.workunit.client.0.smithi031.stdout:7/269: creat d1/f53 x:0 0 0 2024-04-24T23:13:38.262 INFO:tasks.workunit.client.0.smithi031.stdout:8/364: chown d8/d15/d37/d43/d5c/l66 41182028 1 2024-04-24T23:13:38.263 INFO:tasks.workunit.client.0.smithi031.stdout:5/273: mkdir d0/d7/d56 0 2024-04-24T23:13:38.263 INFO:tasks.workunit.client.0.smithi031.stdout:8/365: dread - d8/d15/d37/d43/f6f zero size 2024-04-24T23:13:38.264 INFO:tasks.workunit.client.0.smithi031.stdout:2/208: mkdir d5/d8/d45 0 2024-04-24T23:13:38.266 INFO:tasks.workunit.client.0.smithi031.stdout:1/254: write d1/dc/d11/d21/f4b [42537,30578] 0 2024-04-24T23:13:38.266 INFO:tasks.workunit.client.0.smithi031.stdout:3/298: link d6/d14/l46 d6/df/l5b 0 2024-04-24T23:13:38.268 INFO:tasks.workunit.client.0.smithi031.stdout:3/299: write d6/d15/d1a/f22 [74870,52904] 0 2024-04-24T23:13:38.269 INFO:tasks.workunit.client.0.smithi031.stdout:9/398: unlink d2/d1b/d14/d1d/f23 0 2024-04-24T23:13:38.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/262: mknod da/c58 0 2024-04-24T23:13:38.274 INFO:tasks.workunit.client.0.smithi031.stdout:0/263: write da/d18/d1a/d37/f3d [3296699,95100] 0 2024-04-24T23:13:38.274 INFO:tasks.workunit.client.0.smithi031.stdout:4/269: mknod d3/d23/d37/c54 0 2024-04-24T23:13:38.275 INFO:tasks.workunit.client.0.smithi031.stdout:4/270: dread - d3/d20/d2d/f39 zero size 2024-04-24T23:13:38.281 INFO:tasks.workunit.client.0.smithi031.stdout:8/366: creat d8/d15/d37/d43/d7b/f8c x:0 0 0 2024-04-24T23:13:38.282 INFO:tasks.workunit.client.0.smithi031.stdout:2/209: readlink d5/d6/d29/l14 0 2024-04-24T23:13:38.282 INFO:tasks.workunit.client.0.smithi031.stdout:8/367: dread - d8/d1a/d1d/d50/f88 zero size 2024-04-24T23:13:38.283 INFO:tasks.workunit.client.0.smithi031.stdout:2/210: write d5/d6/f1a [7675000,126519] 0 2024-04-24T23:13:38.288 INFO:tasks.workunit.client.0.smithi031.stdout:3/300: rename d6/df/d16/c23 to d6/df/d16/d57/c5c 0 2024-04-24T23:13:38.297 INFO:tasks.workunit.client.0.smithi031.stdout:0/264: creat da/d11/f59 x:0 0 0 2024-04-24T23:13:38.298 INFO:tasks.workunit.client.0.smithi031.stdout:0/265: chown da/d11/f25 135423429 1 2024-04-24T23:13:38.299 INFO:tasks.workunit.client.0.smithi031.stdout:4/271: symlink d3/d20/d2d/l55 0 2024-04-24T23:13:38.304 INFO:tasks.workunit.client.0.smithi031.stdout:9/399: rmdir d2/d1b 39 2024-04-24T23:13:38.305 INFO:tasks.workunit.client.0.smithi031.stdout:9/400: write d2/d43/f70 [485215,50556] 0 2024-04-24T23:13:38.308 INFO:tasks.workunit.client.0.smithi031.stdout:8/368: mknod d8/d15/d37/d43/d7b/c8d 0 2024-04-24T23:13:38.315 INFO:tasks.workunit.client.0.smithi031.stdout:4/272: creat d3/d23/d3b/f56 x:0 0 0 2024-04-24T23:13:38.318 INFO:tasks.workunit.client.0.smithi031.stdout:1/255: creat d1/f54 x:0 0 0 2024-04-24T23:13:38.319 INFO:tasks.workunit.client.0.smithi031.stdout:1/256: fsync d1/dc/d11/f38 0 2024-04-24T23:13:38.323 INFO:tasks.workunit.client.0.smithi031.stdout:8/369: mknod d8/d1a/d1d/d35/d58/d47/c8e 0 2024-04-24T23:13:38.329 INFO:tasks.workunit.client.0.smithi031.stdout:0/266: write da/d11/f25 [45524,28845] 0 2024-04-24T23:13:38.333 INFO:tasks.workunit.client.0.smithi031.stdout:4/273: creat d3/dd/d3f/f57 x:0 0 0 2024-04-24T23:13:38.335 INFO:tasks.workunit.client.0.smithi031.stdout:7/270: dread d1/f13 [0,4194304] 0 2024-04-24T23:13:38.336 INFO:tasks.workunit.client.0.smithi031.stdout:8/370: dwrite d8/f28 [0,4194304] 0 2024-04-24T23:13:38.337 INFO:tasks.workunit.client.0.smithi031.stdout:5/274: getdents d0/d6/d3b/d4b/d1d 0 2024-04-24T23:13:38.338 INFO:tasks.workunit.client.0.smithi031.stdout:2/211: dwrite d5/d6/d29/f12 [0,4194304] 0 2024-04-24T23:13:38.339 INFO:tasks.workunit.client.0.smithi031.stdout:2/212: fsync d5/d6/d29/f21 0 2024-04-24T23:13:38.341 INFO:tasks.workunit.client.0.smithi031.stdout:2/213: write d5/d28/f37 [1336014,46095] 0 2024-04-24T23:13:38.341 INFO:tasks.workunit.client.0.smithi031.stdout:9/401: dread d2/d1b/d14/d18/f31 [0,4194304] 0 2024-04-24T23:13:38.343 INFO:tasks.workunit.client.0.smithi031.stdout:9/402: stat d2/d1b/d14/f62 0 2024-04-24T23:13:38.351 INFO:tasks.workunit.client.0.smithi031.stdout:4/274: symlink d3/dd/d3f/l58 0 2024-04-24T23:13:38.352 INFO:tasks.workunit.client.0.smithi031.stdout:4/275: readlink d3/dd/lf 0 2024-04-24T23:13:38.353 INFO:tasks.workunit.client.0.smithi031.stdout:9/403: dwrite d2/d1b/d14/d1d/d45/d56/f66 [0,4194304] 0 2024-04-24T23:13:38.355 INFO:tasks.workunit.client.0.smithi031.stdout:8/371: readlink d8/l30 0 2024-04-24T23:13:38.357 INFO:tasks.workunit.client.0.smithi031.stdout:0/267: dwrite da/ff [0,4194304] 0 2024-04-24T23:13:38.366 INFO:tasks.workunit.client.0.smithi031.stdout:2/214: creat d5/f46 x:0 0 0 2024-04-24T23:13:38.369 INFO:tasks.workunit.client.0.smithi031.stdout:3/301: rename d6/d14/l46 to d6/d15/d1a/l5d 0 2024-04-24T23:13:38.373 INFO:tasks.workunit.client.0.smithi031.stdout:7/271: creat d1/d2/d1e/f54 x:0 0 0 2024-04-24T23:13:38.374 INFO:tasks.workunit.client.0.smithi031.stdout:4/276: creat d3/dd/d3f/f59 x:0 0 0 2024-04-24T23:13:38.376 INFO:tasks.workunit.client.0.smithi031.stdout:9/404: creat d2/d1b/d14/d1d/f7b x:0 0 0 2024-04-24T23:13:38.377 INFO:tasks.workunit.client.0.smithi031.stdout:8/372: creat d8/d1a/d1d/d35/f8f x:0 0 0 2024-04-24T23:13:38.378 INFO:tasks.workunit.client.0.smithi031.stdout:3/302: dwrite d6/d14/f43 [0,4194304] 0 2024-04-24T23:13:38.379 INFO:tasks.workunit.client.0.smithi031.stdout:0/268: truncate da/d18/d1a/d37/f4a 4792227 0 2024-04-24T23:13:38.381 INFO:tasks.workunit.client.0.smithi031.stdout:8/373: read d8/d1a/d1d/f7d [226198,15498] 0 2024-04-24T23:13:38.381 INFO:tasks.workunit.client.0.smithi031.stdout:5/275: creat d0/d6/d3b/d4b/dc/d25/d53/f57 x:0 0 0 2024-04-24T23:13:38.383 INFO:tasks.workunit.client.0.smithi031.stdout:1/257: creat d1/dc/d11/f55 x:0 0 0 2024-04-24T23:13:38.386 INFO:tasks.workunit.client.0.smithi031.stdout:4/277: dwrite d3/d20/d2d/d44/f48 [0,4194304] 0 2024-04-24T23:13:38.387 INFO:tasks.workunit.client.0.smithi031.stdout:2/215: rename d5/d28/l2d to d5/d8/d31/l47 0 2024-04-24T23:13:38.397 INFO:tasks.workunit.client.0.smithi031.stdout:0/269: mkdir da/d18/d5a 0 2024-04-24T23:13:38.397 INFO:tasks.workunit.client.0.smithi031.stdout:0/270: chown da 4291 1 2024-04-24T23:13:38.404 INFO:tasks.workunit.client.0.smithi031.stdout:8/374: write d8/d15/d37/f4e [800670,95785] 0 2024-04-24T23:13:38.406 INFO:tasks.workunit.client.0.smithi031.stdout:1/258: dread d1/dc/fe [0,4194304] 0 2024-04-24T23:13:38.407 INFO:tasks.workunit.client.0.smithi031.stdout:1/259: write d1/da/d10/f1c [400990,9872] 0 2024-04-24T23:13:38.408 INFO:tasks.workunit.client.0.smithi031.stdout:4/278: mknod d3/d23/d3b/c5a 0 2024-04-24T23:13:38.409 INFO:tasks.workunit.client.0.smithi031.stdout:1/260: write d1/d33/d36/f44 [330162,35179] 0 2024-04-24T23:13:38.410 INFO:tasks.workunit.client.0.smithi031.stdout:4/279: read d3/f26 [3427,22837] 0 2024-04-24T23:13:38.410 INFO:tasks.workunit.client.0.smithi031.stdout:1/261: write d1/dc/d11/f13 [187993,55054] 0 2024-04-24T23:13:38.413 INFO:tasks.workunit.client.0.smithi031.stdout:9/405: mknod d2/d1b/d14/d1d/d6b/c7c 0 2024-04-24T23:13:38.414 INFO:tasks.workunit.client.0.smithi031.stdout:9/406: chown d2/d43/l68 877065 1 2024-04-24T23:13:38.415 INFO:tasks.workunit.client.0.smithi031.stdout:3/303: creat d6/d14/d1d/d59/f5e x:0 0 0 2024-04-24T23:13:38.420 INFO:tasks.workunit.client.0.smithi031.stdout:8/375: write d8/d1a/d1d/d35/d58/d5b/f79 [1846923,38341] 0 2024-04-24T23:13:38.428 INFO:tasks.workunit.client.0.smithi031.stdout:8/376: dwrite d8/d15/d37/d43/d7b/f8c [0,4194304] 0 2024-04-24T23:13:38.431 INFO:tasks.workunit.client.0.smithi031.stdout:2/216: mknod d5/d8/d31/d2f/c48 0 2024-04-24T23:13:38.431 INFO:tasks.workunit.client.0.smithi031.stdout:2/217: chown d5/d6/f1a 23273 1 2024-04-24T23:13:38.432 INFO:tasks.workunit.client.0.smithi031.stdout:4/280: mkdir d3/d20/d2d/d44/d5b 0 2024-04-24T23:13:38.440 INFO:tasks.workunit.client.0.smithi031.stdout:5/276: dwrite d0/d7/f3a [0,4194304] 0 2024-04-24T23:13:38.443 INFO:tasks.workunit.client.0.smithi031.stdout:7/272: rename d1/d2/dd/l14 to d1/l55 0 2024-04-24T23:13:38.446 INFO:tasks.workunit.client.0.smithi031.stdout:3/304: dwrite d6/d15/d1a/d25/f3d [4194304,4194304] 0 2024-04-24T23:13:38.446 INFO:tasks.workunit.client.0.smithi031.stdout:8/377: creat d8/d15/d37/d62/f90 x:0 0 0 2024-04-24T23:13:38.448 INFO:tasks.workunit.client.0.smithi031.stdout:8/378: truncate d8/d15/d37/f4e 4595812 0 2024-04-24T23:13:38.450 INFO:tasks.workunit.client.0.smithi031.stdout:6/402: sync 2024-04-24T23:13:38.454 INFO:tasks.workunit.client.0.smithi031.stdout:4/281: unlink d3/d23/l2a 0 2024-04-24T23:13:38.456 INFO:tasks.workunit.client.0.smithi031.stdout:3/305: dwrite f0 [4194304,4194304] 0 2024-04-24T23:13:38.459 INFO:tasks.workunit.client.0.smithi031.stdout:3/306: dread - d6/d15/d1a/d25/f54 zero size 2024-04-24T23:13:38.460 INFO:tasks.workunit.client.0.smithi031.stdout:8/379: dwrite d8/d15/d37/f4e [0,4194304] 0 2024-04-24T23:13:38.462 INFO:tasks.workunit.client.0.smithi031.stdout:0/271: write da/f10 [556688,234] 0 2024-04-24T23:13:38.463 INFO:tasks.workunit.client.0.smithi031.stdout:0/272: chown da/d18/d1a/f29 62602678 1 2024-04-24T23:13:38.464 INFO:tasks.workunit.client.0.smithi031.stdout:0/273: fsync da/d11/f59 0 2024-04-24T23:13:38.470 INFO:tasks.workunit.client.0.smithi031.stdout:5/277: creat d0/d7/f58 x:0 0 0 2024-04-24T23:13:38.470 INFO:tasks.workunit.client.0.smithi031.stdout:5/278: chown d0/d6/d12 184 1 2024-04-24T23:13:38.480 INFO:tasks.workunit.client.0.smithi031.stdout:6/403: mknod d13/d15/d1d/d23/d44/c9a 0 2024-04-24T23:13:38.483 INFO:tasks.workunit.client.0.smithi031.stdout:9/407: dwrite d2/d1b/f13 [0,4194304] 0 2024-04-24T23:13:38.486 INFO:tasks.workunit.client.0.smithi031.stdout:9/408: write d2/d1b/d14/d1d/d45/d56/f66 [928856,24343] 0 2024-04-24T23:13:38.487 INFO:tasks.workunit.client.0.smithi031.stdout:8/380: write d8/d1a/d1d/f78 [2392617,10314] 0 2024-04-24T23:13:38.488 INFO:tasks.workunit.client.0.smithi031.stdout:9/409: truncate d2/d1b/d14/d1d/f7b 547508 0 2024-04-24T23:13:38.491 INFO:tasks.workunit.client.0.smithi031.stdout:0/274: rmdir da/d20 39 2024-04-24T23:13:38.491 INFO:tasks.workunit.client.0.smithi031.stdout:3/307: dread d6/d14/d1d/f4c [0,4194304] 0 2024-04-24T23:13:38.496 INFO:tasks.workunit.client.0.smithi031.stdout:8/381: dwrite d8/d15/f69 [0,4194304] 0 2024-04-24T23:13:38.498 INFO:tasks.workunit.client.0.smithi031.stdout:3/308: dread f0 [4194304,4194304] 0 2024-04-24T23:13:38.498 INFO:tasks.workunit.client.0.smithi031.stdout:1/262: truncate d1/da/d10/f4d 3480812 0 2024-04-24T23:13:38.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/410: stat d2/lb 0 2024-04-24T23:13:38.513 INFO:tasks.workunit.client.0.smithi031.stdout:2/218: truncate d5/d28/f37 872669 0 2024-04-24T23:13:38.515 INFO:tasks.workunit.client.0.smithi031.stdout:8/382: unlink d8/d15/c17 0 2024-04-24T23:13:38.516 INFO:tasks.workunit.client.0.smithi031.stdout:7/273: mknod d1/d1c/d2c/d38/c56 0 2024-04-24T23:13:38.518 INFO:tasks.workunit.client.0.smithi031.stdout:3/309: stat d6/d15/d1a/l5d 0 2024-04-24T23:13:38.520 INFO:tasks.workunit.client.0.smithi031.stdout:1/263: unlink d1/da/d12/f19 0 2024-04-24T23:13:38.521 INFO:tasks.workunit.client.0.smithi031.stdout:1/264: write d1/d1d/f35 [1128170,68413] 0 2024-04-24T23:13:38.522 INFO:tasks.workunit.client.0.smithi031.stdout:1/265: chown d1/dc/d11/d21/f28 342 1 2024-04-24T23:13:38.523 INFO:tasks.workunit.client.0.smithi031.stdout:1/266: truncate d1/d1d/f35 2096787 0 2024-04-24T23:13:38.525 INFO:tasks.workunit.client.0.smithi031.stdout:4/282: rename d3/d20/d2d/d44/f48 to d3/f5c 0 2024-04-24T23:13:38.529 INFO:tasks.workunit.client.0.smithi031.stdout:5/279: link d0/d6/d3b/c42 d0/d6/d3b/d4b/d1d/d2e/d28/c59 0 2024-04-24T23:13:38.533 INFO:tasks.workunit.client.0.smithi031.stdout:8/383: creat d8/d1a/d1d/d35/d58/d5b/f91 x:0 0 0 2024-04-24T23:13:38.534 INFO:tasks.workunit.client.0.smithi031.stdout:6/404: dwrite d13/d15/d1d/d23/d2d/f4c [0,4194304] 0 2024-04-24T23:13:38.538 INFO:tasks.workunit.client.0.smithi031.stdout:3/310: creat d6/d15/d1a/d25/f5f x:0 0 0 2024-04-24T23:13:38.542 INFO:tasks.workunit.client.0.smithi031.stdout:8/384: dwrite d8/fb [0,4194304] 0 2024-04-24T23:13:38.543 INFO:tasks.workunit.client.0.smithi031.stdout:6/405: dwrite d13/d15/d1d/f8c [0,4194304] 0 2024-04-24T23:13:38.545 INFO:tasks.workunit.client.0.smithi031.stdout:6/406: read - d13/d15/d1d/d23/d44/f80 zero size 2024-04-24T23:13:38.547 INFO:tasks.workunit.client.0.smithi031.stdout:1/267: mknod d1/d1d/c56 0 2024-04-24T23:13:38.550 INFO:tasks.workunit.client.0.smithi031.stdout:8/385: dread d8/d15/f40 [0,4194304] 0 2024-04-24T23:13:38.552 INFO:tasks.workunit.client.0.smithi031.stdout:8/386: write d8/d15/d37/d43/d7b/f8c [206471,27624] 0 2024-04-24T23:13:38.557 INFO:tasks.workunit.client.0.smithi031.stdout:5/280: creat d0/d6/d3b/d4b/dc/d25/d53/f5a x:0 0 0 2024-04-24T23:13:38.562 INFO:tasks.workunit.client.0.smithi031.stdout:3/311: mkdir d6/d15/d60 0 2024-04-24T23:13:38.566 INFO:tasks.workunit.client.0.smithi031.stdout:6/407: rmdir d13/d1b 39 2024-04-24T23:13:38.569 INFO:tasks.workunit.client.0.smithi031.stdout:4/283: truncate d3/d20/d2d/d44/f2c 4558761 0 2024-04-24T23:13:38.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/387: rename d8/d1a/d1d/d35/f7c to d8/d15/d37/d43/d7b/d89/f92 0 2024-04-24T23:13:38.576 INFO:tasks.workunit.client.0.smithi031.stdout:5/281: rmdir d0/d6/d3b/d4b/d1d/d2e 39 2024-04-24T23:13:38.581 INFO:tasks.workunit.client.0.smithi031.stdout:8/388: dwrite d8/d15/d37/f5e [0,4194304] 0 2024-04-24T23:13:38.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/268: symlink d1/da/d42/l57 0 2024-04-24T23:13:38.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/269: chown d1/f34 11763681 1 2024-04-24T23:13:38.588 INFO:tasks.workunit.client.0.smithi031.stdout:4/284: chown d3/d20/d2d/d44/c38 0 1 2024-04-24T23:13:38.594 INFO:tasks.workunit.client.0.smithi031.stdout:2/219: rename d5/d8/d31 to d5/d8/d15/d33/d49 0 2024-04-24T23:13:38.597 INFO:tasks.workunit.client.0.smithi031.stdout:5/282: chown d0/d6/d3b/d4b/d1d/d2e/f50 11 1 2024-04-24T23:13:38.598 INFO:tasks.workunit.client.0.smithi031.stdout:5/283: write d0/d6/d32/f46 [525778,31426] 0 2024-04-24T23:13:38.603 INFO:tasks.workunit.client.0.smithi031.stdout:1/270: creat d1/d33/f58 x:0 0 0 2024-04-24T23:13:38.603 INFO:tasks.workunit.client.0.smithi031.stdout:1/271: fdatasync d1/dc/f2b 0 2024-04-24T23:13:38.609 INFO:tasks.workunit.client.0.smithi031.stdout:8/389: rename d8/d15/d37/d43/d5c/l66 to d8/d1a/d74/l93 0 2024-04-24T23:13:38.611 INFO:tasks.workunit.client.0.smithi031.stdout:2/220: write d5/d8/d15/f22 [624146,22466] 0 2024-04-24T23:13:38.613 INFO:tasks.workunit.client.0.smithi031.stdout:5/284: write d0/d6/d3b/d4b/dc/f23 [3605601,62667] 0 2024-04-24T23:13:38.618 INFO:tasks.workunit.client.0.smithi031.stdout:4/285: mkdir d3/d20/d2d/d44/d5b/d5d 0 2024-04-24T23:13:38.625 INFO:tasks.workunit.client.0.smithi031.stdout:8/390: mknod d8/d1a/d1d/d35/d58/d47/c94 0 2024-04-24T23:13:38.626 INFO:tasks.workunit.client.0.smithi031.stdout:8/391: write d8/d15/f40 [361762,60383] 0 2024-04-24T23:13:38.628 INFO:tasks.workunit.client.0.smithi031.stdout:8/392: truncate d8/d1a/d1d/d35/f8f 364377 0 2024-04-24T23:13:38.628 INFO:tasks.workunit.client.0.smithi031.stdout:3/312: getdents d6/d15/d1a/d25 0 2024-04-24T23:13:38.629 INFO:tasks.workunit.client.0.smithi031.stdout:8/393: write d8/d15/f3e [820397,35711] 0 2024-04-24T23:13:38.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/285: mknod d0/d6/d3b/d4b/dc/d25/c5b 0 2024-04-24T23:13:38.630 INFO:tasks.workunit.client.0.smithi031.stdout:6/408: getdents d13/d15/d1d/d23/d24/d63 0 2024-04-24T23:13:38.631 INFO:tasks.workunit.client.0.smithi031.stdout:6/409: chown d13/d15/d1d/c95 718211 1 2024-04-24T23:13:38.632 INFO:tasks.workunit.client.0.smithi031.stdout:1/272: creat d1/d3e/d46/f59 x:0 0 0 2024-04-24T23:13:38.632 INFO:tasks.workunit.client.0.smithi031.stdout:1/273: stat d1/dc/fe 0 2024-04-24T23:13:38.636 INFO:tasks.workunit.client.0.smithi031.stdout:1/274: dread d1/f34 [0,4194304] 0 2024-04-24T23:13:38.636 INFO:tasks.workunit.client.0.smithi031.stdout:2/221: mknod d5/d6/c4a 0 2024-04-24T23:13:38.731 INFO:tasks.workunit.client.0.smithi031.stdout:3/313: creat d6/df/f61 x:0 0 0 2024-04-24T23:13:38.731 INFO:tasks.workunit.client.0.smithi031.stdout:8/394: creat d8/d15/d37/f95 x:0 0 0 2024-04-24T23:13:38.732 INFO:tasks.workunit.client.0.smithi031.stdout:5/286: symlink d0/d6/d3b/d4b/dc/d25/l5c 0 2024-04-24T23:13:38.735 INFO:tasks.workunit.client.0.smithi031.stdout:5/287: read d0/d6/f1a [2282844,65934] 0 2024-04-24T23:13:38.735 INFO:tasks.workunit.client.0.smithi031.stdout:1/275: read d1/da/f2c [3918724,511] 0 2024-04-24T23:13:38.735 INFO:tasks.workunit.client.0.smithi031.stdout:3/314: write d6/df/d16/d2a/f31 [961234,83870] 0 2024-04-24T23:13:38.738 INFO:tasks.workunit.client.0.smithi031.stdout:4/286: dwrite d3/f1b [0,4194304] 0 2024-04-24T23:13:38.743 INFO:tasks.workunit.client.0.smithi031.stdout:1/276: mkdir d1/dc/d11/d21/d5a 0 2024-04-24T23:13:38.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:38 smithi031 ceph-mon[26411]: pgmap v22: 65 pgs: 65 active+clean; 689 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.5 MiB/s rd, 72 MiB/s wr, 298 op/s 2024-04-24T23:13:38.752 INFO:tasks.workunit.client.0.smithi031.stdout:6/410: link d13/d1b/f8f d13/d15/d1d/d23/d24/d8b/f9b 0 2024-04-24T23:13:38.753 INFO:tasks.workunit.client.0.smithi031.stdout:3/315: creat d6/d15/d1a/f62 x:0 0 0 2024-04-24T23:13:38.755 INFO:tasks.workunit.client.0.smithi031.stdout:4/287: rename d3/d20/d53 to d3/d5e 0 2024-04-24T23:13:38.756 INFO:tasks.workunit.client.0.smithi031.stdout:4/288: readlink d3/d20/d2d/l4b 0 2024-04-24T23:13:38.756 INFO:tasks.workunit.client.0.smithi031.stdout:6/411: rmdir d13/d15/d1d/d23/d2d 39 2024-04-24T23:13:38.759 INFO:tasks.workunit.client.0.smithi031.stdout:3/316: fsync f0 0 2024-04-24T23:13:38.761 INFO:tasks.workunit.client.0.smithi031.stdout:1/277: rename d1/dc/d11/d21/l2a to d1/da/d10/l5b 0 2024-04-24T23:13:38.762 INFO:tasks.workunit.client.0.smithi031.stdout:3/317: write d6/df/f61 [884164,45566] 0 2024-04-24T23:13:38.764 INFO:tasks.workunit.client.0.smithi031.stdout:3/318: chown d6/d55 337850668 1 2024-04-24T23:13:38.765 INFO:tasks.workunit.client.0.smithi031.stdout:4/289: dwrite d3/f46 [0,4194304] 0 2024-04-24T23:13:38.767 INFO:tasks.workunit.client.0.smithi031.stdout:1/278: unlink d1/dc/d11/d21/f3a 0 2024-04-24T23:13:38.768 INFO:tasks.workunit.client.0.smithi031.stdout:6/412: dwrite d13/d15/d1d/d23/f2b [0,4194304] 0 2024-04-24T23:13:38.770 INFO:tasks.workunit.client.0.smithi031.stdout:6/413: chown d13/d15/d1d/d23/d44/d47 5805 1 2024-04-24T23:13:38.773 INFO:tasks.workunit.client.0.smithi031.stdout:6/414: write d13/d15/d1d/d23/d4d/f88 [887226,32624] 0 2024-04-24T23:13:38.777 INFO:tasks.workunit.client.0.smithi031.stdout:1/279: dwrite d1/f54 [0,4194304] 0 2024-04-24T23:13:38.778 INFO:tasks.workunit.client.0.smithi031.stdout:1/280: chown d1/dc/d11/d21 561 1 2024-04-24T23:13:38.789 INFO:tasks.workunit.client.0.smithi031.stdout:3/319: dread d6/f8 [0,4194304] 0 2024-04-24T23:13:38.792 INFO:tasks.workunit.client.0.smithi031.stdout:6/415: getdents d13/d15/d1d/d23/d4d/d7b 0 2024-04-24T23:13:38.793 INFO:tasks.workunit.client.0.smithi031.stdout:6/416: chown d13/d15/d1d/d23/d44/c9a 156918 1 2024-04-24T23:13:38.796 INFO:tasks.workunit.client.0.smithi031.stdout:6/417: fsync d13/f14 0 2024-04-24T23:13:38.801 INFO:tasks.workunit.client.0.smithi031.stdout:6/418: dread d13/d75/d79/f85 [0,4194304] 0 2024-04-24T23:13:38.802 INFO:tasks.workunit.client.0.smithi031.stdout:6/419: write d13/d15/d1d/d23/d24/f77 [1627715,22270] 0 2024-04-24T23:13:38.804 INFO:tasks.workunit.client.0.smithi031.stdout:6/420: dread - d13/d15/d1d/d23/d44/d47/d69/f7f zero size 2024-04-24T23:13:38.805 INFO:tasks.workunit.client.0.smithi031.stdout:6/421: fsync d13/f6c 0 2024-04-24T23:13:38.806 INFO:tasks.workunit.client.0.smithi031.stdout:6/422: chown d13/f6c 602679 1 2024-04-24T23:13:38.807 INFO:tasks.workunit.client.0.smithi031.stdout:6/423: chown d13/d15/f76 240561967 1 2024-04-24T23:13:38.807 INFO:tasks.workunit.client.0.smithi031.stdout:2/222: write d5/d28/f37 [1222348,18061] 0 2024-04-24T23:13:38.808 INFO:tasks.workunit.client.0.smithi031.stdout:8/395: truncate f3 5864976 0 2024-04-24T23:13:38.814 INFO:tasks.workunit.client.0.smithi031.stdout:3/320: dread d6/df/d16/d2a/f2e [0,4194304] 0 2024-04-24T23:13:38.816 INFO:tasks.workunit.client.0.smithi031.stdout:2/223: creat d5/d8/d45/f4b x:0 0 0 2024-04-24T23:13:38.817 INFO:tasks.workunit.client.0.smithi031.stdout:8/396: getdents d8/d15/d37/d43/d7b/d87 0 2024-04-24T23:13:38.817 INFO:tasks.workunit.client.0.smithi031.stdout:6/424: dwrite f10 [4194304,4194304] 0 2024-04-24T23:13:38.821 INFO:tasks.workunit.client.0.smithi031.stdout:5/288: dwrite d0/d6/d3b/d4b/d1d/d2e/d28/f30 [0,4194304] 0 2024-04-24T23:13:38.821 INFO:tasks.workunit.client.0.smithi031.stdout:2/224: truncate d5/f27 1955020 0 2024-04-24T23:13:38.829 INFO:tasks.workunit.client.0.smithi031.stdout:3/321: rename c4 to d6/d15/d60/c63 0 2024-04-24T23:13:38.830 INFO:tasks.workunit.client.0.smithi031.stdout:3/322: fdatasync d6/df/d16/d2a/f4f 0 2024-04-24T23:13:38.830 INFO:tasks.workunit.client.0.smithi031.stdout:2/225: creat d5/d6/df/f4c x:0 0 0 2024-04-24T23:13:38.831 INFO:tasks.workunit.client.0.smithi031.stdout:6/425: dwrite d13/d15/f27 [0,4194304] 0 2024-04-24T23:13:38.832 INFO:tasks.workunit.client.0.smithi031.stdout:2/226: stat d5/d8/d15/f22 0 2024-04-24T23:13:38.833 INFO:tasks.workunit.client.0.smithi031.stdout:3/323: write d6/df/f61 [320610,58636] 0 2024-04-24T23:13:38.835 INFO:tasks.workunit.client.0.smithi031.stdout:4/290: truncate d3/fb 2803087 0 2024-04-24T23:13:38.836 INFO:tasks.workunit.client.0.smithi031.stdout:4/291: stat d3/d23/d3b/c5a 0 2024-04-24T23:13:38.837 INFO:tasks.workunit.client.0.smithi031.stdout:5/289: rename d0/d6/f22 to d0/d6/d12/f5d 0 2024-04-24T23:13:38.838 INFO:tasks.workunit.client.0.smithi031.stdout:4/292: chown d3/dd/d3f 1 1 2024-04-24T23:13:38.844 INFO:tasks.workunit.client.0.smithi031.stdout:3/324: dwrite d6/d14/f58 [0,4194304] 0 2024-04-24T23:13:38.844 INFO:tasks.workunit.client.0.smithi031.stdout:6/426: write d13/f3f [1087263,1945] 0 2024-04-24T23:13:38.847 INFO:tasks.workunit.client.0.smithi031.stdout:3/325: chown d6/d14/f33 10417 1 2024-04-24T23:13:38.848 INFO:tasks.workunit.client.0.smithi031.stdout:1/281: dwrite d1/f2 [4194304,4194304] 0 2024-04-24T23:13:38.849 INFO:tasks.workunit.client.0.smithi031.stdout:8/397: truncate d8/d15/d37/d43/f4b 433370 0 2024-04-24T23:13:38.851 INFO:tasks.workunit.client.0.smithi031.stdout:4/293: dwrite d3/f2b [0,4194304] 0 2024-04-24T23:13:38.854 INFO:tasks.workunit.client.0.smithi031.stdout:6/427: truncate f9 1802462 0 2024-04-24T23:13:38.855 INFO:tasks.workunit.client.0.smithi031.stdout:6/428: fdatasync d13/d15/d1d/d23/d44/f80 0 2024-04-24T23:13:38.858 INFO:tasks.workunit.client.0.smithi031.stdout:3/326: rename d6/d14/d1d/f1f to d6/d15/d60/f64 0 2024-04-24T23:13:38.861 INFO:tasks.workunit.client.0.smithi031.stdout:8/398: symlink d8/d15/d37/d43/d7b/l96 0 2024-04-24T23:13:38.862 INFO:tasks.workunit.client.0.smithi031.stdout:8/399: chown d8/d15/d37/d43/c8a 318 1 2024-04-24T23:13:38.864 INFO:tasks.workunit.client.0.smithi031.stdout:8/400: write d8/d15/d37/f85 [590828,70029] 0 2024-04-24T23:13:38.870 INFO:tasks.workunit.client.0.smithi031.stdout:3/327: creat d6/df/d16/d57/f65 x:0 0 0 2024-04-24T23:13:38.873 INFO:tasks.workunit.client.0.smithi031.stdout:3/328: read d6/d15/d1a/f22 [4128475,115475] 0 2024-04-24T23:13:38.873 INFO:tasks.workunit.client.0.smithi031.stdout:8/401: mknod d8/d15/d37/c97 0 2024-04-24T23:13:38.881 INFO:tasks.workunit.client.0.smithi031.stdout:3/329: mknod d6/df/d16/d2a/c66 0 2024-04-24T23:13:38.885 INFO:tasks.workunit.client.0.smithi031.stdout:8/402: rename d8/c2c to d8/d15/c98 0 2024-04-24T23:13:38.889 INFO:tasks.workunit.client.0.smithi031.stdout:6/429: getdents d13/d15/d1d/d23/d44/d47/d69 0 2024-04-24T23:13:38.889 INFO:tasks.workunit.client.0.smithi031.stdout:6/430: readlink l3 0 2024-04-24T23:13:38.892 INFO:tasks.workunit.client.0.smithi031.stdout:3/330: dwrite d6/f8 [4194304,4194304] 0 2024-04-24T23:13:38.896 INFO:tasks.workunit.client.0.smithi031.stdout:2/227: truncate d5/d6/df/f1e 1696061 0 2024-04-24T23:13:38.898 INFO:tasks.workunit.client.0.smithi031.stdout:3/331: write d6/d15/d60/f64 [688641,90133] 0 2024-04-24T23:13:38.910 INFO:tasks.workunit.client.0.smithi031.stdout:2/228: rename d5/d6/f30 to d5/d6/f4d 0 2024-04-24T23:13:38.911 INFO:tasks.workunit.client.0.smithi031.stdout:3/332: creat d6/df/d16/d53/f67 x:0 0 0 2024-04-24T23:13:38.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:38 smithi040 ceph-mon[33128]: pgmap v22: 65 pgs: 65 active+clean; 689 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.5 MiB/s rd, 72 MiB/s wr, 298 op/s 2024-04-24T23:13:38.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/294: dwrite d3/f26 [0,4194304] 0 2024-04-24T23:13:38.913 INFO:tasks.workunit.client.0.smithi031.stdout:2/229: read d5/d28/f37 [76149,37819] 0 2024-04-24T23:13:38.913 INFO:tasks.workunit.client.0.smithi031.stdout:3/333: chown d6/df/d16/l38 14 1 2024-04-24T23:13:38.915 INFO:tasks.workunit.client.0.smithi031.stdout:2/230: chown d5/d6/d29/d26/f38 974 1 2024-04-24T23:13:38.916 INFO:tasks.workunit.client.0.smithi031.stdout:2/231: dread - d5/d28/f3a zero size 2024-04-24T23:13:38.925 INFO:tasks.workunit.client.0.smithi031.stdout:6/431: write d13/d15/d1d/d23/d24/f40 [1530162,124144] 0 2024-04-24T23:13:38.934 INFO:tasks.workunit.client.0.smithi031.stdout:3/334: creat d6/df/d16/d2a/f68 x:0 0 0 2024-04-24T23:13:38.935 INFO:tasks.workunit.client.0.smithi031.stdout:2/232: mkdir d5/d28/d4e 0 2024-04-24T23:13:38.939 INFO:tasks.workunit.client.0.smithi031.stdout:6/432: mkdir d13/d1b/d32/d9c 0 2024-04-24T23:13:38.939 INFO:tasks.workunit.client.0.smithi031.stdout:2/233: dread d5/d8/f10 [0,4194304] 0 2024-04-24T23:13:38.940 INFO:tasks.workunit.client.0.smithi031.stdout:2/234: truncate d5/f13 9973009 0 2024-04-24T23:13:38.941 INFO:tasks.workunit.client.0.smithi031.stdout:3/335: rmdir d6/df/d16 39 2024-04-24T23:13:38.944 INFO:tasks.workunit.client.0.smithi031.stdout:6/433: creat d13/d15/d1d/d23/d24/d8b/f9d x:0 0 0 2024-04-24T23:13:38.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/235: mkdir d5/d6/d29/d4f 0 2024-04-24T23:13:38.947 INFO:tasks.workunit.client.0.smithi031.stdout:3/336: mkdir d6/d14/d1d/d69 0 2024-04-24T23:13:38.948 INFO:tasks.workunit.client.0.smithi031.stdout:3/337: read d6/d14/f43 [3337552,101708] 0 2024-04-24T23:13:38.948 INFO:tasks.workunit.client.0.smithi031.stdout:6/434: read d13/d15/d1d/d23/d24/d43/f71 [186231,57588] 0 2024-04-24T23:13:38.951 INFO:tasks.workunit.client.0.smithi031.stdout:3/338: symlink d6/d14/l6a 0 2024-04-24T23:13:38.953 INFO:tasks.workunit.client.0.smithi031.stdout:4/295: write d3/dd/f13 [472581,17535] 0 2024-04-24T23:13:38.958 INFO:tasks.workunit.client.0.smithi031.stdout:2/236: creat d5/d8/d15/f50 x:0 0 0 2024-04-24T23:13:38.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/435: mkdir d13/d15/d1d/d23/d2d/d9e 0 2024-04-24T23:13:38.966 INFO:tasks.workunit.client.0.smithi031.stdout:2/237: write d5/f13 [4060135,33185] 0 2024-04-24T23:13:38.967 INFO:tasks.workunit.client.0.smithi031.stdout:6/436: dwrite d13/d1b/f46 [0,4194304] 0 2024-04-24T23:13:38.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/296: unlink d3/dd/d3f/l4f 0 2024-04-24T23:13:38.970 INFO:tasks.workunit.client.0.smithi031.stdout:4/297: fdatasync d3/f26 0 2024-04-24T23:13:38.971 INFO:tasks.workunit.client.0.smithi031.stdout:4/298: fsync d3/d20/d2d/f52 0 2024-04-24T23:13:38.972 INFO:tasks.workunit.client.0.smithi031.stdout:3/339: creat d6/f6b x:0 0 0 2024-04-24T23:13:38.972 INFO:tasks.workunit.client.0.smithi031.stdout:4/299: write d3/dd/d3f/f57 [739296,77344] 0 2024-04-24T23:13:38.975 INFO:tasks.workunit.client.0.smithi031.stdout:2/238: link d5/d6/f16 d5/d28/d43/f51 0 2024-04-24T23:13:38.976 INFO:tasks.workunit.client.0.smithi031.stdout:2/239: chown d5/d28/f3a 0 1 2024-04-24T23:13:38.979 INFO:tasks.workunit.client.0.smithi031.stdout:6/437: creat d13/d15/d1d/d23/d24/f9f x:0 0 0 2024-04-24T23:13:38.980 INFO:tasks.workunit.client.0.smithi031.stdout:4/300: dwrite d3/dd/f30 [0,4194304] 0 2024-04-24T23:13:38.981 INFO:tasks.workunit.client.0.smithi031.stdout:6/438: fdatasync d13/d15/d1d/d23/d2d/f3d 0 2024-04-24T23:13:38.982 INFO:tasks.workunit.client.0.smithi031.stdout:4/301: write d3/dd/f31 [832429,39200] 0 2024-04-24T23:13:38.982 INFO:tasks.workunit.client.0.smithi031.stdout:6/439: stat d13/d1b/f54 0 2024-04-24T23:13:38.983 INFO:tasks.workunit.client.0.smithi031.stdout:2/240: mknod d5/d28/c52 0 2024-04-24T23:13:38.984 INFO:tasks.workunit.client.0.smithi031.stdout:4/302: write d3/d20/d2d/f50 [942074,20421] 0 2024-04-24T23:13:38.985 INFO:tasks.workunit.client.0.smithi031.stdout:2/241: chown d5/d6/df/c32 601 1 2024-04-24T23:13:38.990 INFO:tasks.workunit.client.0.smithi031.stdout:4/303: symlink d3/d23/d37/l5f 0 2024-04-24T23:13:38.991 INFO:tasks.workunit.client.0.smithi031.stdout:6/440: dwrite d13/d15/d1d/d23/d4d/f6a [0,4194304] 0 2024-04-24T23:13:38.997 INFO:tasks.workunit.client.0.smithi031.stdout:6/441: mkdir d13/d15/d1d/d23/d44/da0 0 2024-04-24T23:13:39.007 INFO:tasks.workunit.client.0.smithi031.stdout:6/442: dread d13/d75/f8e [0,4194304] 0 2024-04-24T23:13:39.013 INFO:tasks.workunit.client.0.smithi031.stdout:6/443: dread d13/d75/d79/f84 [0,4194304] 0 2024-04-24T23:13:39.014 INFO:tasks.workunit.client.0.smithi031.stdout:2/242: dwrite d5/d8/f10 [0,4194304] 0 2024-04-24T23:13:39.015 INFO:tasks.workunit.client.0.smithi031.stdout:2/243: readlink d5/d6/d29/l2e 0 2024-04-24T23:13:39.015 INFO:tasks.workunit.client.0.smithi031.stdout:4/304: truncate d3/d20/f4d 378772 0 2024-04-24T23:13:39.016 INFO:tasks.workunit.client.0.smithi031.stdout:6/444: write d13/d1b/f6e [883435,90656] 0 2024-04-24T23:13:39.022 INFO:tasks.workunit.client.0.smithi031.stdout:4/305: mkdir d3/d20/d25/d60 0 2024-04-24T23:13:39.024 INFO:tasks.workunit.client.0.smithi031.stdout:6/445: dwrite d13/f3f [0,4194304] 0 2024-04-24T23:13:39.026 INFO:tasks.workunit.client.0.smithi031.stdout:2/244: creat d5/d28/d43/f53 x:0 0 0 2024-04-24T23:13:39.032 INFO:tasks.workunit.client.0.smithi031.stdout:2/245: creat d5/d28/f54 x:0 0 0 2024-04-24T23:13:39.033 INFO:tasks.workunit.client.0.smithi031.stdout:6/446: mknod d13/d1b/d32/d9c/ca1 0 2024-04-24T23:13:39.033 INFO:tasks.workunit.client.0.smithi031.stdout:2/246: write d5/d28/f54 [83916,12644] 0 2024-04-24T23:13:39.035 INFO:tasks.workunit.client.0.smithi031.stdout:2/247: creat d5/d28/d43/f55 x:0 0 0 2024-04-24T23:13:39.039 INFO:tasks.workunit.client.0.smithi031.stdout:4/306: rename d3/d20/f4d to d3/dd/d3f/f61 0 2024-04-24T23:13:39.043 INFO:tasks.workunit.client.0.smithi031.stdout:2/248: dwrite d5/d28/d43/f53 [0,4194304] 0 2024-04-24T23:13:39.045 INFO:tasks.workunit.client.0.smithi031.stdout:4/307: creat d3/f62 x:0 0 0 2024-04-24T23:13:39.047 INFO:tasks.workunit.client.0.smithi031.stdout:4/308: fdatasync d3/f4a 0 2024-04-24T23:13:39.051 INFO:tasks.workunit.client.0.smithi031.stdout:4/309: dread - d3/d20/d2d/d44/f4c zero size 2024-04-24T23:13:39.052 INFO:tasks.workunit.client.0.smithi031.stdout:4/310: readlink d3/d20/d2d/l4b 0 2024-04-24T23:13:39.054 INFO:tasks.workunit.client.0.smithi031.stdout:2/249: dwrite d5/d8/d45/f4b [0,4194304] 0 2024-04-24T23:13:39.060 INFO:tasks.workunit.client.0.smithi031.stdout:6/447: dread d13/f7c [0,4194304] 0 2024-04-24T23:13:39.063 INFO:tasks.workunit.client.0.smithi031.stdout:6/448: chown d13/d1b/f6e 737787 1 2024-04-24T23:13:39.064 INFO:tasks.workunit.client.0.smithi031.stdout:2/250: write d5/d8/d45/f4b [5116602,56053] 0 2024-04-24T23:13:39.064 INFO:tasks.workunit.client.0.smithi031.stdout:4/311: dwrite d3/d20/d2d/d44/f4c [0,4194304] 0 2024-04-24T23:13:39.066 INFO:tasks.workunit.client.0.smithi031.stdout:4/312: dread - d3/f4a zero size 2024-04-24T23:13:39.067 INFO:tasks.workunit.client.0.smithi031.stdout:2/251: write d5/d8/d15/f50 [979160,97277] 0 2024-04-24T23:13:39.077 INFO:tasks.workunit.client.0.smithi031.stdout:2/252: mkdir d5/d8/d15/d56 0 2024-04-24T23:13:39.078 INFO:tasks.workunit.client.0.smithi031.stdout:6/449: link d13/d15/d1d/d23/d2d/c7e d13/d1b/d32/ca2 0 2024-04-24T23:13:39.084 INFO:tasks.workunit.client.0.smithi031.stdout:6/450: dwrite d13/d15/d1d/f8c [0,4194304] 0 2024-04-24T23:13:39.085 INFO:tasks.workunit.client.0.smithi031.stdout:4/313: truncate d3/dd/f30 4833891 0 2024-04-24T23:13:39.090 INFO:tasks.workunit.client.0.smithi031.stdout:0/275: sync 2024-04-24T23:13:39.090 INFO:tasks.workunit.client.0.smithi031.stdout:9/411: sync 2024-04-24T23:13:39.090 INFO:tasks.workunit.client.0.smithi031.stdout:7/274: sync 2024-04-24T23:13:39.091 INFO:tasks.workunit.client.0.smithi031.stdout:6/451: creat d13/d15/d1d/d23/d24/d43/d68/fa3 x:0 0 0 2024-04-24T23:13:39.091 INFO:tasks.workunit.client.0.smithi031.stdout:0/276: dread - da/f53 zero size 2024-04-24T23:13:39.091 INFO:tasks.workunit.client.0.smithi031.stdout:6/452: chown f11 2292 1 2024-04-24T23:13:39.092 INFO:tasks.workunit.client.0.smithi031.stdout:4/314: mkdir d3/d20/d25/d60/d63 0 2024-04-24T23:13:39.092 INFO:tasks.workunit.client.0.smithi031.stdout:2/253: write d5/d6/d29/f12 [1611330,76010] 0 2024-04-24T23:13:39.093 INFO:tasks.workunit.client.0.smithi031.stdout:9/412: write d2/d1b/d14/d1d/d45/d56/d67/f7a [597000,115247] 0 2024-04-24T23:13:39.094 INFO:tasks.workunit.client.0.smithi031.stdout:4/315: fdatasync d3/f11 0 2024-04-24T23:13:39.129 INFO:tasks.workunit.client.0.smithi031.stdout:6/453: mknod d13/d15/d1d/d23/d24/d43/ca4 0 2024-04-24T23:13:39.129 INFO:tasks.workunit.client.0.smithi031.stdout:2/254: creat d5/d28/d43/f57 x:0 0 0 2024-04-24T23:13:39.130 INFO:tasks.workunit.client.0.smithi031.stdout:0/277: creat da/d18/d5a/f5b x:0 0 0 2024-04-24T23:13:39.131 INFO:tasks.workunit.client.0.smithi031.stdout:7/275: mknod d1/d2/dd/d33/c57 0 2024-04-24T23:13:39.132 INFO:tasks.workunit.client.0.smithi031.stdout:2/255: chown d5/f36 201853556 1 2024-04-24T23:13:39.133 INFO:tasks.workunit.client.0.smithi031.stdout:6/454: write d13/d15/d1d/d23/f2b [3117313,42687] 0 2024-04-24T23:13:39.134 INFO:tasks.workunit.client.0.smithi031.stdout:7/276: write d1/f53 [745652,72880] 0 2024-04-24T23:13:39.144 INFO:tasks.workunit.client.0.smithi031.stdout:0/278: dwrite da/d18/d1a/d37/d41/f47 [0,4194304] 0 2024-04-24T23:13:39.144 INFO:tasks.workunit.client.0.smithi031.stdout:2/256: dwrite d5/d6/f16 [0,4194304] 0 2024-04-24T23:13:39.150 INFO:tasks.workunit.client.1.smithi040.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-24T23:13:39.150 INFO:tasks.workunit.client.0.smithi031.stdout:9/413: dread d2/d1b/d14/d1d/d45/d56/d67/d4a/d4b/f52 [0,4194304] 0 2024-04-24T23:13:39.151 INFO:tasks.workunit.client.0.smithi031.stdout:7/277: dread d1/f2b [0,4194304] 0 2024-04-24T23:13:39.152 INFO:tasks.workunit.client.0.smithi031.stdout:9/414: readlink d2/d1b/d14/l29 0 2024-04-24T23:13:39.152 INFO:tasks.workunit.client.0.smithi031.stdout:2/257: creat d5/f58 x:0 0 0 2024-04-24T23:13:39.156 INFO:tasks.workunit.client.0.smithi031.stdout:6/455: creat d13/d15/fa5 x:0 0 0 2024-04-24T23:13:39.156 INFO:tasks.workunit.client.0.smithi031.stdout:7/278: dread d1/f2b [0,4194304] 0 2024-04-24T23:13:39.157 INFO:tasks.workunit.client.1.smithi040.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T23:13:39.157 INFO:tasks.workunit.client.1.smithi040.stderr:+ make 2024-04-24T23:13:39.157 INFO:tasks.workunit.client.0.smithi031.stdout:2/258: symlink d5/d28/l59 0 2024-04-24T23:13:39.159 INFO:tasks.workunit.client.0.smithi031.stdout:0/279: dwrite da/d18/d1a/d37/d41/f47 [4194304,4194304] 0 2024-04-24T23:13:39.159 INFO:tasks.workunit.client.0.smithi031.stdout:4/316: write d3/d23/d3b/f43 [5145338,32321] 0 2024-04-24T23:13:39.162 INFO:tasks.workunit.client.0.smithi031.stdout:0/280: write da/f53 [425160,53573] 0 2024-04-24T23:13:39.162 INFO:tasks.workunit.client.0.smithi031.stdout:4/317: write d3/f4a [618366,98414] 0 2024-04-24T23:13:39.162 INFO:tasks.workunit.client.0.smithi031.stdout:7/279: fdatasync d1/d2/f7 0 2024-04-24T23:13:39.164 INFO:tasks.workunit.client.0.smithi031.stdout:9/415: rename d2/d1b/d14/d1d/d6b to d2/d1b/d14/d18/d7d 0 2024-04-24T23:13:39.165 INFO:tasks.workunit.client.0.smithi031.stdout:7/280: write d1/f2b [393210,57518] 0 2024-04-24T23:13:39.166 INFO:tasks.workunit.client.0.smithi031.stdout:2/259: dread d5/d8/f10 [0,4194304] 0 2024-04-24T23:13:39.167 INFO:tasks.workunit.client.0.smithi031.stdout:9/416: write d2/d1b/d14/f19 [297238,35942] 0 2024-04-24T23:13:39.170 INFO:tasks.workunit.client.0.smithi031.stdout:7/281: truncate d1/d2/dd/d33/f4e 812658 0 2024-04-24T23:13:39.170 INFO:tasks.workunit.client.0.smithi031.stdout:2/260: write d5/d8/d15/f22 [1280211,125966] 0 2024-04-24T23:13:39.172 INFO:tasks.workunit.client.0.smithi031.stdout:0/281: write da/d20/d4e/f51 [858518,113513] 0 2024-04-24T23:13:39.179 INFO:tasks.workunit.client.0.smithi031.stdout:6/456: mkdir d13/d15/d1d/d23/d44/da0/da6 0 2024-04-24T23:13:39.180 INFO:tasks.workunit.client.0.smithi031.stdout:4/318: write d3/f46 [5163029,51860] 0 2024-04-24T23:13:39.181 INFO:tasks.workunit.client.0.smithi031.stdout:6/457: write d13/d15/d1d/d23/d2d/f61 [5286005,104854] 0 2024-04-24T23:13:39.182 INFO:tasks.workunit.client.0.smithi031.stdout:9/417: mkdir d2/d1b/d14/d1d/d45/d56/d67/d7e 0 2024-04-24T23:13:39.185 INFO:tasks.workunit.client.0.smithi031.stdout:0/282: fdatasync da/fe 0 2024-04-24T23:13:39.187 INFO:tasks.workunit.client.0.smithi031.stdout:4/319: creat d3/d23/f64 x:0 0 0 2024-04-24T23:13:39.188 INFO:tasks.workunit.client.0.smithi031.stdout:7/282: mkdir d1/d2/d58 0 2024-04-24T23:13:39.189 INFO:tasks.workunit.client.0.smithi031.stdout:7/283: dread - d1/d1c/f3c zero size 2024-04-24T23:13:39.198 INFO:tasks.workunit.client.0.smithi031.stdout:9/418: mkdir d2/d1b/d14/d78/d7f 0 2024-04-24T23:13:39.199 INFO:tasks.workunit.client.0.smithi031.stdout:0/283: dread da/d18/d1a/d37/f3d [0,4194304] 0 2024-04-24T23:13:39.200 INFO:tasks.workunit.client.0.smithi031.stdout:0/284: fdatasync da/d11/f59 0 2024-04-24T23:13:39.201 INFO:tasks.workunit.client.0.smithi031.stdout:9/419: truncate d2/d1b/d14/d1d/d45/f73 524805 0 2024-04-24T23:13:39.202 INFO:tasks.workunit.client.0.smithi031.stdout:6/458: truncate d13/d15/d1d/d23/d44/d47/f4a 36258 0 2024-04-24T23:13:39.206 INFO:tasks.workunit.client.0.smithi031.stdout:0/285: mknod da/d20/d4e/c5c 0 2024-04-24T23:13:39.209 INFO:tasks.workunit.client.0.smithi031.stdout:6/459: dwrite d13/d15/d1d/f8c [0,4194304] 0 2024-04-24T23:13:39.210 INFO:tasks.workunit.client.0.smithi031.stdout:0/286: unlink da/d18/c50 0 2024-04-24T23:13:39.211 INFO:tasks.workunit.client.0.smithi031.stdout:0/287: fdatasync da/d18/d1a/f29 0 2024-04-24T23:13:39.212 INFO:tasks.workunit.client.0.smithi031.stdout:7/284: write d1/d2/f2d [1321067,12355] 0 2024-04-24T23:13:39.213 INFO:tasks.workunit.client.0.smithi031.stdout:4/320: dread d3/fb [0,4194304] 0 2024-04-24T23:13:39.214 INFO:tasks.workunit.client.0.smithi031.stdout:0/288: write da/d18/d1a/d37/f23 [2187543,36559] 0 2024-04-24T23:13:39.215 INFO:tasks.workunit.client.0.smithi031.stdout:0/289: dread - da/d22/f54 zero size 2024-04-24T23:13:39.216 INFO:tasks.workunit.client.0.smithi031.stdout:0/290: readlink l9 0 2024-04-24T23:13:39.219 INFO:tasks.workunit.client.0.smithi031.stdout:4/321: unlink d3/dd/d3f/f59 0 2024-04-24T23:13:39.221 INFO:tasks.workunit.client.0.smithi031.stdout:7/285: rename d1/d1c/c2f to d1/d2/dd/c59 0 2024-04-24T23:13:39.222 INFO:tasks.workunit.client.0.smithi031.stdout:9/420: write d2/fc [530912,9060] 0 2024-04-24T23:13:39.222 INFO:tasks.workunit.client.0.smithi031.stdout:7/286: chown d1/d2/dd/f23 41531 1 2024-04-24T23:13:39.223 INFO:tasks.workunit.client.0.smithi031.stdout:0/291: creat da/d11/f5d x:0 0 0 2024-04-24T23:13:39.225 INFO:tasks.workunit.client.1.smithi040.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-24T23:13:39.225 INFO:tasks.workunit.client.0.smithi031.stdout:6/460: truncate d13/f3f 53394 0 2024-04-24T23:13:39.228 INFO:tasks.workunit.client.0.smithi031.stdout:0/292: creat da/d18/d1a/d37/f5e x:0 0 0 2024-04-24T23:13:39.228 INFO:tasks.workunit.client.0.smithi031.stdout:4/322: creat d3/d20/d2d/d44/d5b/d5d/f65 x:0 0 0 2024-04-24T23:13:39.229 INFO:tasks.workunit.client.0.smithi031.stdout:9/421: symlink d2/d1b/d14/d1d/d45/d65/l80 0 2024-04-24T23:13:39.229 INFO:tasks.workunit.client.0.smithi031.stdout:6/461: dread d13/f6c [0,4194304] 0 2024-04-24T23:13:39.231 INFO:tasks.workunit.client.0.smithi031.stdout:9/422: chown d2/d1b/d14/l51 116353627 1 2024-04-24T23:13:39.232 INFO:tasks.workunit.client.0.smithi031.stdout:6/462: chown d13/d15/l58 993614 1 2024-04-24T23:13:39.234 INFO:tasks.workunit.client.0.smithi031.stdout:9/423: rmdir d2/d1b/d14/d1d/d45/d56 39 2024-04-24T23:13:39.234 INFO:tasks.workunit.client.0.smithi031.stdout:4/323: link d3/d23/l24 d3/d20/l66 0 2024-04-24T23:13:39.235 INFO:tasks.workunit.client.0.smithi031.stdout:0/293: dwrite da/d18/d1a/f1f [0,4194304] 0 2024-04-24T23:13:39.236 INFO:tasks.workunit.client.0.smithi031.stdout:0/294: stat da/d18/d1a/d37/f3d 0 2024-04-24T23:13:39.237 INFO:tasks.workunit.client.0.smithi031.stdout:9/424: mknod d2/c81 0 2024-04-24T23:13:39.238 INFO:tasks.workunit.client.0.smithi031.stdout:4/324: mknod d3/d23/d37/c67 0 2024-04-24T23:13:39.240 INFO:tasks.workunit.client.0.smithi031.stdout:0/295: creat da/d18/d1a/d37/f5f x:0 0 0 2024-04-24T23:13:39.245 INFO:tasks.workunit.client.0.smithi031.stdout:9/425: dwrite d2/d1b/d14/d1d/f24 [0,4194304] 0 2024-04-24T23:13:39.248 INFO:tasks.workunit.client.0.smithi031.stdout:4/325: dread d3/d20/d2d/f50 [0,4194304] 0 2024-04-24T23:13:39.249 INFO:tasks.workunit.client.0.smithi031.stdout:4/326: stat d3/d23/d37/c54 0 2024-04-24T23:13:39.254 INFO:tasks.workunit.client.0.smithi031.stdout:9/426: rename d2/d1b/d14/d1d/l3d to d2/d1b/d14/d1d/d45/d56/d67/d7e/l82 0 2024-04-24T23:13:39.256 INFO:tasks.workunit.client.0.smithi031.stdout:9/427: truncate d2/d1b/d14/d1d/d45/d56/f66 4637796 0 2024-04-24T23:13:39.257 INFO:tasks.workunit.client.0.smithi031.stdout:4/327: getdents d3/d20/d25/d60 0 2024-04-24T23:13:39.258 INFO:tasks.workunit.client.0.smithi031.stdout:0/296: write da/f19 [3432220,44665] 0 2024-04-24T23:13:39.259 INFO:tasks.workunit.client.0.smithi031.stdout:4/328: write d3/d23/d3b/f56 [1010439,83816] 0 2024-04-24T23:13:39.261 INFO:tasks.workunit.client.0.smithi031.stdout:4/329: write d3/d20/d2d/d44/f2c [1984201,120695] 0 2024-04-24T23:13:39.263 INFO:tasks.workunit.client.0.smithi031.stdout:9/428: dread d2/d1b/d14/d1d/f24 [0,4194304] 0 2024-04-24T23:13:39.264 INFO:tasks.workunit.client.0.smithi031.stdout:4/330: mkdir d3/d23/d37/d68 0 2024-04-24T23:13:39.264 INFO:tasks.workunit.client.0.smithi031.stdout:9/429: write d2/f64 [4288622,56406] 0 2024-04-24T23:13:39.268 INFO:tasks.workunit.client.0.smithi031.stdout:4/331: rename d3/dd to d3/d23/d69 0 2024-04-24T23:13:39.274 INFO:tasks.workunit.client.0.smithi031.stdout:0/297: dwrite da/fe [0,4194304] 0 2024-04-24T23:13:39.275 INFO:tasks.workunit.client.0.smithi031.stdout:9/430: write d2/f12 [167173,52121] 0 2024-04-24T23:13:39.277 INFO:tasks.workunit.client.0.smithi031.stdout:4/332: truncate d3/d23/d69/f31 264074 0 2024-04-24T23:13:39.278 INFO:tasks.workunit.client.0.smithi031.stdout:0/298: symlink da/l60 0 2024-04-24T23:13:39.279 INFO:tasks.workunit.client.0.smithi031.stdout:0/299: dread - da/d18/d5a/f5b zero size 2024-04-24T23:13:39.281 INFO:tasks.workunit.client.0.smithi031.stdout:4/333: mknod d3/d20/d2d/d44/c6a 0 2024-04-24T23:13:39.286 INFO:tasks.workunit.client.0.smithi031.stdout:4/334: rename d3/l5 to d3/d20/d25/d60/d63/l6b 0 2024-04-24T23:13:39.286 INFO:tasks.workunit.client.0.smithi031.stdout:4/335: fdatasync d3/d23/f3a 0 2024-04-24T23:13:39.287 INFO:tasks.workunit.client.0.smithi031.stdout:4/336: chown d3/d23/d69/f13 66 1 2024-04-24T23:13:39.288 INFO:tasks.workunit.client.0.smithi031.stdout:4/337: fdatasync d3/f62 0 2024-04-24T23:13:39.292 INFO:tasks.workunit.client.0.smithi031.stdout:9/431: dread d2/d1b/d14/d1d/f3c [0,4194304] 0 2024-04-24T23:13:39.292 INFO:tasks.workunit.client.0.smithi031.stdout:4/338: dread d3/f5c [0,4194304] 0 2024-04-24T23:13:39.293 INFO:tasks.workunit.client.0.smithi031.stdout:9/432: chown d2/d1b/f4f 1672 1 2024-04-24T23:13:39.293 INFO:tasks.workunit.client.0.smithi031.stdout:4/339: stat d3/d5e 0 2024-04-24T23:13:39.294 INFO:tasks.workunit.client.0.smithi031.stdout:9/433: write d2/f8 [3566058,8700] 0 2024-04-24T23:13:39.295 INFO:tasks.workunit.client.0.smithi031.stdout:4/340: symlink d3/d20/d2d/d44/d5b/l6c 0 2024-04-24T23:13:39.296 INFO:tasks.workunit.client.0.smithi031.stdout:0/300: dread da/d22/f27 [0,4194304] 0 2024-04-24T23:13:39.297 INFO:tasks.workunit.client.0.smithi031.stdout:9/434: symlink d2/d1b/d14/d1d/d45/d56/d67/d4a/d4b/l83 0 2024-04-24T23:13:39.298 INFO:tasks.workunit.client.0.smithi031.stdout:4/341: creat d3/d23/d69/d3f/f6d x:0 0 0 2024-04-24T23:13:39.298 INFO:tasks.workunit.client.0.smithi031.stdout:9/435: read d2/d43/f70 [347735,60581] 0 2024-04-24T23:13:39.298 INFO:tasks.workunit.client.0.smithi031.stdout:0/301: symlink da/d18/d1a/d37/d41/l61 0 2024-04-24T23:13:39.299 INFO:tasks.workunit.client.0.smithi031.stdout:0/302: stat da/f13 0 2024-04-24T23:13:39.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/290: sync 2024-04-24T23:13:39.309 INFO:tasks.workunit.client.0.smithi031.stdout:5/291: dread d0/d6/d3b/d4b/d1d/d2e/d28/f30 [0,4194304] 0 2024-04-24T23:13:39.312 INFO:tasks.workunit.client.0.smithi031.stdout:4/342: mkdir d3/d23/d37/d6e 0 2024-04-24T23:13:39.313 INFO:tasks.workunit.client.0.smithi031.stdout:4/343: write d3/d23/d3b/f43 [2836040,21956] 0 2024-04-24T23:13:39.314 INFO:tasks.workunit.client.0.smithi031.stdout:0/303: mkdir da/d22/d62 0 2024-04-24T23:13:39.317 INFO:tasks.workunit.client.0.smithi031.stdout:5/292: rename d0/d6/d3b/d4b/c2c to d0/d7/c5e 0 2024-04-24T23:13:39.317 INFO:tasks.workunit.client.0.smithi031.stdout:9/436: rmdir d2/d16 39 2024-04-24T23:13:39.324 INFO:tasks.workunit.client.0.smithi031.stdout:9/437: dwrite d2/d1b/d14/d1d/d45/d56/d67/f7a [0,4194304] 0 2024-04-24T23:13:39.325 INFO:tasks.workunit.client.0.smithi031.stdout:9/438: stat d2/d1b/f10 0 2024-04-24T23:13:39.337 INFO:tasks.workunit.client.0.smithi031.stdout:0/304: dwrite da/d18/d1a/d37/f26 [4194304,4194304] 0 2024-04-24T23:13:39.338 INFO:tasks.workunit.client.0.smithi031.stdout:4/344: dwrite d3/fc [0,4194304] 0 2024-04-24T23:13:39.342 INFO:tasks.workunit.client.0.smithi031.stdout:0/305: creat da/d18/d1a/f63 x:0 0 0 2024-04-24T23:13:39.345 INFO:tasks.workunit.client.0.smithi031.stdout:0/306: mknod da/d20/d4e/c64 0 2024-04-24T23:13:39.349 INFO:tasks.workunit.client.0.smithi031.stdout:4/345: getdents d3/d20/d2d/d44 0 2024-04-24T23:13:39.352 INFO:tasks.workunit.client.0.smithi031.stdout:4/346: symlink d3/d20/d2d/d44/l6f 0 2024-04-24T23:13:39.354 INFO:tasks.workunit.client.0.smithi031.stdout:0/307: dwrite da/d18/d1a/f1f [0,4194304] 0 2024-04-24T23:13:39.355 INFO:tasks.workunit.client.0.smithi031.stdout:0/308: fsync da/d22/f4b 0 2024-04-24T23:13:39.360 INFO:tasks.workunit.client.0.smithi031.stdout:0/309: unlink da/d20/l24 0 2024-04-24T23:13:39.361 INFO:tasks.workunit.client.0.smithi031.stdout:0/310: write da/d20/d4e/f57 [864421,5330] 0 2024-04-24T23:13:39.362 INFO:tasks.workunit.client.0.smithi031.stdout:4/347: truncate d3/d23/d69/f30 2402815 0 2024-04-24T23:13:39.369 INFO:tasks.workunit.client.0.smithi031.stdout:4/348: rename d3/d23/d69/lf to d3/d20/d2d/l70 0 2024-04-24T23:13:39.373 INFO:tasks.workunit.client.0.smithi031.stdout:0/311: link f6 da/d18/d1a/d37/f65 0 2024-04-24T23:13:39.381 INFO:tasks.workunit.client.0.smithi031.stdout:4/349: creat d3/f71 x:0 0 0 2024-04-24T23:13:39.381 INFO:tasks.workunit.client.0.smithi031.stdout:4/350: read d3/f9 [3847783,17437] 0 2024-04-24T23:13:39.387 INFO:tasks.workunit.client.0.smithi031.stdout:4/351: dwrite d3/d23/d3b/f41 [0,4194304] 0 2024-04-24T23:13:39.395 INFO:tasks.workunit.client.0.smithi031.stdout:4/352: dwrite d3/d23/d3b/f41 [0,4194304] 0 2024-04-24T23:13:39.403 INFO:tasks.workunit.client.0.smithi031.stdout:4/353: dwrite d3/d23/d3b/f43 [0,4194304] 0 2024-04-24T23:13:39.414 INFO:tasks.workunit.client.0.smithi031.stdout:4/354: getdents d3/d23/d69 0 2024-04-24T23:13:39.415 INFO:tasks.workunit.client.0.smithi031.stdout:0/312: dwrite da/d11/f33 [0,4194304] 0 2024-04-24T23:13:39.427 INFO:tasks.workunit.client.0.smithi031.stdout:4/355: truncate d3/f9 2939229 0 2024-04-24T23:13:39.432 INFO:tasks.workunit.client.0.smithi031.stdout:4/356: creat d3/d23/d3b/f72 x:0 0 0 2024-04-24T23:13:39.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/357: dread d3/d23/d3b/f41 [0,4194304] 0 2024-04-24T23:13:39.445 INFO:tasks.workunit.client.0.smithi031.stdout:0/313: truncate da/d18/d1a/d37/f26 4151539 0 2024-04-24T23:13:39.449 INFO:tasks.workunit.client.0.smithi031.stdout:0/314: dread da/fb [0,4194304] 0 2024-04-24T23:13:39.449 INFO:tasks.workunit.client.0.smithi031.stdout:4/358: truncate d3/d23/d69/d3f/f61 3405831 0 2024-04-24T23:13:39.450 INFO:tasks.workunit.client.0.smithi031.stdout:0/315: dread - da/d20/d4e/f48 zero size 2024-04-24T23:13:39.452 INFO:tasks.workunit.client.0.smithi031.stdout:0/316: creat da/d18/d1a/d37/d41/f66 x:0 0 0 2024-04-24T23:13:39.453 INFO:tasks.workunit.client.0.smithi031.stdout:0/317: chown da/d18/d1a/c42 107228 1 2024-04-24T23:13:39.453 INFO:tasks.workunit.client.0.smithi031.stdout:4/359: rmdir d3/d23/d37/d6e 0 2024-04-24T23:13:39.453 INFO:tasks.workunit.client.0.smithi031.stdout:0/318: write da/f19 [2388557,16165] 0 2024-04-24T23:13:39.466 INFO:tasks.workunit.client.0.smithi031.stdout:4/360: dwrite d3/d23/d69/f33 [0,4194304] 0 2024-04-24T23:13:39.468 INFO:tasks.workunit.client.0.smithi031.stdout:0/319: dwrite da/d18/d1a/d37/f26 [0,4194304] 0 2024-04-24T23:13:39.469 INFO:tasks.workunit.client.0.smithi031.stdout:4/361: symlink d3/d20/d25/l73 0 2024-04-24T23:13:39.471 INFO:tasks.workunit.client.0.smithi031.stdout:4/362: write d3/f29 [943502,111268] 0 2024-04-24T23:13:39.475 INFO:tasks.workunit.client.0.smithi031.stdout:4/363: rename d3/d20/d25/c3e to d3/d23/c74 0 2024-04-24T23:13:39.476 INFO:tasks.workunit.client.0.smithi031.stdout:0/320: dwrite da/d11/f5d [0,4194304] 0 2024-04-24T23:13:39.480 INFO:tasks.workunit.client.0.smithi031.stdout:0/321: dread da/d11/f33 [0,4194304] 0 2024-04-24T23:13:39.482 INFO:tasks.workunit.client.0.smithi031.stdout:0/322: write da/d18/d1a/f1f [2894134,32800] 0 2024-04-24T23:13:39.496 INFO:tasks.workunit.client.0.smithi031.stdout:1/282: sync 2024-04-24T23:13:39.497 INFO:tasks.workunit.client.0.smithi031.stdout:4/364: rename d3/f46 to d3/f75 0 2024-04-24T23:13:39.499 INFO:tasks.workunit.client.0.smithi031.stdout:0/323: mknod da/d18/c67 0 2024-04-24T23:13:39.500 INFO:tasks.workunit.client.0.smithi031.stdout:0/324: chown da/d22/c55 33205663 1 2024-04-24T23:13:39.587 INFO:tasks.workunit.client.0.smithi031.stdout:0/325: symlink da/d22/l68 0 2024-04-24T23:13:39.589 INFO:tasks.workunit.client.0.smithi031.stdout:0/326: creat da/d18/d5a/f69 x:0 0 0 2024-04-24T23:13:39.590 INFO:tasks.workunit.client.0.smithi031.stdout:1/283: getdents d1/da/d12 0 2024-04-24T23:13:39.590 INFO:tasks.workunit.client.0.smithi031.stdout:1/284: fdatasync d1/d33/f47 0 2024-04-24T23:13:39.596 INFO:tasks.workunit.client.0.smithi031.stdout:1/285: link d1/d1d/f35 d1/d3e/d46/f5c 0 2024-04-24T23:13:39.596 INFO:tasks.workunit.client.0.smithi031.stdout:4/365: write d3/f9 [2330779,30645] 0 2024-04-24T23:13:39.599 INFO:tasks.workunit.client.0.smithi031.stdout:1/286: unlink d1/dc/c4a 0 2024-04-24T23:13:39.599 INFO:tasks.workunit.client.0.smithi031.stdout:0/327: rmdir da/d18 39 2024-04-24T23:13:39.600 INFO:tasks.workunit.client.0.smithi031.stdout:4/366: mknod d3/d20/c76 0 2024-04-24T23:13:39.601 INFO:tasks.workunit.client.0.smithi031.stdout:0/328: mkdir da/d11/d6a 0 2024-04-24T23:13:39.601 INFO:tasks.workunit.client.0.smithi031.stdout:4/367: write d3/d23/d69/d3f/f57 [1172886,122421] 0 2024-04-24T23:13:39.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/329: write da/f3f [3608461,79185] 0 2024-04-24T23:13:39.604 INFO:tasks.workunit.client.0.smithi031.stdout:4/368: truncate d3/d23/f3a 2423317 0 2024-04-24T23:13:39.606 INFO:tasks.workunit.client.0.smithi031.stdout:0/330: truncate da/d18/d1a/d37/f65 4959337 0 2024-04-24T23:13:39.607 INFO:tasks.workunit.client.0.smithi031.stdout:4/369: creat d3/d20/d2d/d44/d5b/d5d/f77 x:0 0 0 2024-04-24T23:13:39.610 INFO:tasks.workunit.client.0.smithi031.stdout:4/370: rename d3/d23/f64 to d3/d5e/f78 0 2024-04-24T23:13:39.611 INFO:tasks.workunit.client.0.smithi031.stdout:4/371: fdatasync d3/f26 0 2024-04-24T23:13:39.612 INFO:tasks.workunit.client.0.smithi031.stdout:4/372: chown d3/d23/d69/d3f/f6d 0 1 2024-04-24T23:13:39.613 INFO:tasks.workunit.client.0.smithi031.stdout:4/373: fsync d3/f11 0 2024-04-24T23:13:39.614 INFO:tasks.workunit.client.0.smithi031.stdout:4/374: creat d3/d20/d2d/f79 x:0 0 0 2024-04-24T23:13:39.618 INFO:tasks.workunit.client.0.smithi031.stdout:4/375: link d3/d23/d37/c54 d3/c7a 0 2024-04-24T23:13:39.622 INFO:tasks.workunit.client.0.smithi031.stdout:0/331: dread da/f13 [0,4194304] 0 2024-04-24T23:13:39.623 INFO:tasks.workunit.client.0.smithi031.stdout:4/376: dread d3/d23/d69/d3f/f57 [0,4194304] 0 2024-04-24T23:13:39.624 INFO:tasks.workunit.client.0.smithi031.stdout:0/332: symlink da/d18/d1a/l6b 0 2024-04-24T23:13:39.625 INFO:tasks.workunit.client.0.smithi031.stdout:4/377: creat d3/d23/d37/f7b x:0 0 0 2024-04-24T23:13:39.626 INFO:tasks.workunit.client.0.smithi031.stdout:0/333: rename da/d20/c4c to da/d20/c6c 0 2024-04-24T23:13:39.627 INFO:tasks.workunit.client.0.smithi031.stdout:4/378: write d3/f75 [1221608,14807] 0 2024-04-24T23:13:39.628 INFO:tasks.workunit.client.0.smithi031.stdout:4/379: read d3/d23/d3b/f43 [2549924,41622] 0 2024-04-24T23:13:39.629 INFO:tasks.workunit.client.0.smithi031.stdout:4/380: fsync d3/d23/d69/f33 0 2024-04-24T23:13:39.633 INFO:tasks.workunit.client.0.smithi031.stdout:4/381: creat d3/d23/d37/f7c x:0 0 0 2024-04-24T23:13:39.634 INFO:tasks.workunit.client.0.smithi031.stdout:4/382: fdatasync d3/d23/d69/d3f/f6d 0 2024-04-24T23:13:39.634 INFO:tasks.workunit.client.1.smithi040.stderr:++ readlink -f fsstress 2024-04-24T23:13:39.636 INFO:tasks.workunit.client.1.smithi040.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-24T23:13:39.636 INFO:tasks.workunit.client.1.smithi040.stderr:+ popd 2024-04-24T23:13:39.637 INFO:tasks.workunit.client.1.smithi040.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-24T23:13:39.637 INFO:tasks.workunit.client.1.smithi040.stderr:+ popd 2024-04-24T23:13:39.637 INFO:tasks.workunit.client.1.smithi040.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-24T23:13:39.638 INFO:tasks.workunit.client.1.smithi040.stderr:++ mktemp -d -p . 2024-04-24T23:13:39.638 INFO:tasks.workunit.client.0.smithi031.stdout:4/383: rename d3/d20/d2d/d44/d5b to d3/d20/d2d/d44/d5b/d7d 22 2024-04-24T23:13:39.640 INFO:tasks.workunit.client.1.smithi040.stderr:+ T=./tmp.pF6GRdPU60 2024-04-24T23:13:39.640 INFO:tasks.workunit.client.1.smithi040.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.pF6GRdPU60 -l 1 -n 1000 -p 10 -v 2024-04-24T23:13:39.641 INFO:tasks.workunit.client.0.smithi031.stdout:0/334: dwrite da/d22/f27 [0,4194304] 0 2024-04-24T23:13:39.642 INFO:tasks.workunit.client.0.smithi031.stdout:4/384: mkdir d3/d20/d25/d7e 0 2024-04-24T23:13:39.642 INFO:tasks.workunit.client.0.smithi031.stdout:4/385: read d3/f11 [472692,44097] 0 2024-04-24T23:13:39.644 INFO:tasks.workunit.client.0.smithi031.stdout:0/335: rmdir da 39 2024-04-24T23:13:39.645 INFO:tasks.workunit.client.0.smithi031.stdout:4/386: mknod d3/d20/d25/c7f 0 2024-04-24T23:13:39.648 INFO:tasks.workunit.client.0.smithi031.stdout:0/336: write da/d18/d1a/f29 [5048400,3585] 0 2024-04-24T23:13:39.648 INFO:tasks.workunit.client.1.smithi040.stdout:seed = 1713403452 2024-04-24T23:13:39.649 INFO:tasks.workunit.client.0.smithi031.stdout:0/337: chown da/d18/d5a/f69 1 1 2024-04-24T23:13:39.650 INFO:tasks.workunit.client.0.smithi031.stdout:0/338: stat da/d20/d4e/f57 0 2024-04-24T23:13:39.656 INFO:tasks.workunit.client.0.smithi031.stdout:0/339: dwrite da/d20/d4e/f48 [0,4194304] 0 2024-04-24T23:13:39.657 INFO:tasks.workunit.client.1.smithi040.stdout:1/0: dwrite - no filename 2024-04-24T23:13:39.658 INFO:tasks.workunit.client.1.smithi040.stdout:0/0: symlink l0 0 2024-04-24T23:13:39.659 INFO:tasks.workunit.client.1.smithi040.stdout:2/0: rename - no filename 2024-04-24T23:13:39.659 INFO:tasks.workunit.client.1.smithi040.stdout:2/1: rename - no filename 2024-04-24T23:13:39.659 INFO:tasks.workunit.client.1.smithi040.stdout:2/2: write - no filename 2024-04-24T23:13:39.659 INFO:tasks.workunit.client.1.smithi040.stdout:2/3: write - no filename 2024-04-24T23:13:39.659 INFO:tasks.workunit.client.1.smithi040.stdout:2/4: truncate - no filename 2024-04-24T23:13:39.659 INFO:tasks.workunit.client.1.smithi040.stdout:2/5: read - no filename 2024-04-24T23:13:39.660 INFO:tasks.workunit.client.1.smithi040.stdout:2/6: write - no filename 2024-04-24T23:13:39.660 INFO:tasks.workunit.client.1.smithi040.stdout:2/7: unlink - no file 2024-04-24T23:13:39.660 INFO:tasks.workunit.client.1.smithi040.stdout:2/8: dwrite - no filename 2024-04-24T23:13:39.660 INFO:tasks.workunit.client.1.smithi040.stdout:2/9: rmdir - no directory 2024-04-24T23:13:39.660 INFO:tasks.workunit.client.1.smithi040.stdout:2/10: rename - no filename 2024-04-24T23:13:39.660 INFO:tasks.workunit.client.1.smithi040.stdout:2/11: stat - no entries 2024-04-24T23:13:39.660 INFO:tasks.workunit.client.0.smithi031.stdout:0/340: unlink da/f13 0 2024-04-24T23:13:39.661 INFO:tasks.workunit.client.1.smithi040.stdout:1/1: creat f0 x:0 0 0 2024-04-24T23:13:39.662 INFO:tasks.workunit.client.1.smithi040.stdout:0/1: unlink l0 0 2024-04-24T23:13:39.663 INFO:tasks.workunit.client.1.smithi040.stdout:0/2: dwrite - no filename 2024-04-24T23:13:39.665 INFO:tasks.workunit.client.0.smithi031.stdout:4/387: truncate d3/d20/d2d/d44/f4c 2444264 0 2024-04-24T23:13:39.665 INFO:tasks.workunit.client.0.smithi031.stdout:4/388: dread - d3/d20/f51 zero size 2024-04-24T23:13:39.668 INFO:tasks.workunit.client.0.smithi031.stdout:0/341: mknod da/c6d 0 2024-04-24T23:13:39.669 INFO:tasks.workunit.client.1.smithi040.stdout:1/2: rename f0 to f1 0 2024-04-24T23:13:39.670 INFO:tasks.workunit.client.0.smithi031.stdout:4/389: truncate d3/d20/d2d/d44/f2c 1208622 0 2024-04-24T23:13:39.671 INFO:tasks.workunit.client.1.smithi040.stdout:0/3: symlink l1 0 2024-04-24T23:13:39.671 INFO:tasks.workunit.client.1.smithi040.stdout:0/4: truncate - no filename 2024-04-24T23:13:39.672 INFO:tasks.workunit.client.0.smithi031.stdout:4/390: creat d3/d23/d37/f80 x:0 0 0 2024-04-24T23:13:39.673 INFO:tasks.workunit.client.1.smithi040.stdout:2/12: creat f0 x:0 0 0 2024-04-24T23:13:39.673 INFO:tasks.workunit.client.1.smithi040.stdout:2/13: rmdir - no directory 2024-04-24T23:13:39.674 INFO:tasks.workunit.client.0.smithi031.stdout:0/342: creat da/d11/d6a/f6e x:0 0 0 2024-04-24T23:13:39.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/343: truncate da/d22/f27 4244604 0 2024-04-24T23:13:39.676 INFO:tasks.workunit.client.0.smithi031.stdout:4/391: rename d3/f62 to d3/d20/d25/d60/d63/f81 0 2024-04-24T23:13:39.678 INFO:tasks.workunit.client.1.smithi040.stdout:1/3: write f1 [591442,92592] 0 2024-04-24T23:13:39.678 INFO:tasks.workunit.client.1.smithi040.stdout:0/5: creat f2 x:0 0 0 2024-04-24T23:13:39.679 INFO:tasks.workunit.client.0.smithi031.stdout:4/392: symlink d3/d20/d2d/d44/d5b/l82 0 2024-04-24T23:13:39.680 INFO:tasks.workunit.client.1.smithi040.stdout:5/0: truncate - no filename 2024-04-24T23:13:39.680 INFO:tasks.workunit.client.1.smithi040.stdout:5/1: dwrite - no filename 2024-04-24T23:13:39.680 INFO:tasks.workunit.client.0.smithi031.stdout:4/393: fdatasync d3/d23/d3b/f56 0 2024-04-24T23:13:39.681 INFO:tasks.workunit.client.1.smithi040.stdout:3/0: creat f0 x:0 0 0 2024-04-24T23:13:39.681 INFO:tasks.workunit.client.1.smithi040.stdout:1/4: chown f1 31553711 1 2024-04-24T23:13:39.682 INFO:tasks.workunit.client.1.smithi040.stdout:3/1: chown f0 139524726 1 2024-04-24T23:13:39.682 INFO:tasks.workunit.client.1.smithi040.stdout:3/2: stat f0 0 2024-04-24T23:13:39.682 INFO:tasks.workunit.client.1.smithi040.stdout:3/3: rmdir - no directory 2024-04-24T23:13:39.682 INFO:tasks.workunit.client.1.smithi040.stdout:2/14: dwrite f0 [0,4194304] 0 2024-04-24T23:13:39.684 INFO:tasks.workunit.client.0.smithi031.stdout:4/394: truncate d3/fb 595477 0 2024-04-24T23:13:39.685 INFO:tasks.workunit.client.1.smithi040.stdout:3/4: write f0 [907380,86458] 0 2024-04-24T23:13:39.687 INFO:tasks.workunit.client.1.smithi040.stdout:6/0: dread - no filename 2024-04-24T23:13:39.687 INFO:tasks.workunit.client.1.smithi040.stdout:6/1: rename - no filename 2024-04-24T23:13:39.692 INFO:tasks.workunit.client.1.smithi040.stdout:1/5: dwrite f1 [0,4194304] 0 2024-04-24T23:13:39.692 INFO:tasks.workunit.client.0.smithi031.stdout:4/395: creat d3/d20/d25/d7e/f83 x:0 0 0 2024-04-24T23:13:39.694 INFO:tasks.workunit.client.1.smithi040.stdout:1/6: write f1 [3499743,21675] 0 2024-04-24T23:13:39.696 INFO:tasks.workunit.client.1.smithi040.stdout:3/5: mkdir d1 0 2024-04-24T23:13:39.697 INFO:tasks.workunit.client.1.smithi040.stdout:3/6: chown f0 4466711 1 2024-04-24T23:13:39.698 INFO:tasks.workunit.client.1.smithi040.stdout:7/0: dread - no filename 2024-04-24T23:13:39.700 INFO:tasks.workunit.client.1.smithi040.stdout:5/2: creat f0 x:0 0 0 2024-04-24T23:13:39.700 INFO:tasks.workunit.client.1.smithi040.stdout:5/3: readlink - no filename 2024-04-24T23:13:39.701 INFO:tasks.workunit.client.1.smithi040.stdout:5/4: chown f0 1898627860 1 2024-04-24T23:13:39.701 INFO:tasks.workunit.client.0.smithi031.stdout:4/396: link d3/d23/d69/d3f/f57 d3/d20/d2d/d44/f84 0 2024-04-24T23:13:39.702 INFO:tasks.workunit.client.1.smithi040.stdout:0/6: fsync f2 0 2024-04-24T23:13:39.702 INFO:tasks.workunit.client.0.smithi031.stdout:4/397: chown d3/d20/d2d/d44/f84 63602 1 2024-04-24T23:13:39.703 INFO:tasks.workunit.client.0.smithi031.stdout:0/344: write da/fb [27586,109016] 0 2024-04-24T23:13:39.704 INFO:tasks.workunit.client.1.smithi040.stdout:1/7: dwrite f1 [0,4194304] 0 2024-04-24T23:13:39.706 INFO:tasks.workunit.client.1.smithi040.stdout:8/0: truncate - no filename 2024-04-24T23:13:39.706 INFO:tasks.workunit.client.1.smithi040.stdout:8/1: chown . 665 1 2024-04-24T23:13:39.707 INFO:tasks.workunit.client.1.smithi040.stdout:8/2: rename - no filename 2024-04-24T23:13:39.707 INFO:tasks.workunit.client.1.smithi040.stdout:8/3: dread - no filename 2024-04-24T23:13:39.707 INFO:tasks.workunit.client.0.smithi031.stdout:0/345: dread da/fe [0,4194304] 0 2024-04-24T23:13:39.709 INFO:tasks.workunit.client.1.smithi040.stdout:6/2: creat f0 x:0 0 0 2024-04-24T23:13:39.711 INFO:tasks.workunit.client.0.smithi031.stdout:0/346: dread da/d20/d4e/f48 [0,4194304] 0 2024-04-24T23:13:39.716 INFO:tasks.workunit.client.1.smithi040.stdout:5/5: rename f0 to f1 0 2024-04-24T23:13:39.716 INFO:tasks.workunit.client.1.smithi040.stdout:6/3: dwrite f0 [0,4194304] 0 2024-04-24T23:13:39.718 INFO:tasks.workunit.client.0.smithi031.stdout:0/347: dwrite da/d22/f27 [0,4194304] 0 2024-04-24T23:13:39.719 INFO:tasks.workunit.client.0.smithi031.stdout:0/348: chown da/fe 20397 1 2024-04-24T23:13:39.721 INFO:tasks.workunit.client.1.smithi040.stdout:0/7: mknod c3 0 2024-04-24T23:13:39.721 INFO:tasks.workunit.client.1.smithi040.stdout:0/8: fsync f2 0 2024-04-24T23:13:39.722 INFO:tasks.workunit.client.1.smithi040.stdout:0/9: fsync f2 0 2024-04-24T23:13:39.722 INFO:tasks.workunit.client.1.smithi040.stdout:0/10: stat l1 0 2024-04-24T23:13:39.723 INFO:tasks.workunit.client.1.smithi040.stdout:1/8: creat f2 x:0 0 0 2024-04-24T23:13:39.723 INFO:tasks.workunit.client.1.smithi040.stdout:1/9: dread - f2 zero size 2024-04-24T23:13:39.726 INFO:tasks.workunit.client.1.smithi040.stdout:9/0: truncate - no filename 2024-04-24T23:13:39.727 INFO:tasks.workunit.client.1.smithi040.stdout:9/1: fdatasync - no filename 2024-04-24T23:13:39.727 INFO:tasks.workunit.client.1.smithi040.stdout:9/2: write - no filename 2024-04-24T23:13:39.727 INFO:tasks.workunit.client.1.smithi040.stdout:9/3: write - no filename 2024-04-24T23:13:39.727 INFO:tasks.workunit.client.1.smithi040.stdout:9/4: dwrite - no filename 2024-04-24T23:13:39.727 INFO:tasks.workunit.client.0.smithi031.stdout:8/403: sync 2024-04-24T23:13:39.728 INFO:tasks.workunit.client.1.smithi040.stdout:3/7: getdents d1 0 2024-04-24T23:13:39.730 INFO:tasks.workunit.client.1.smithi040.stdout:7/1: creat f0 x:0 0 0 2024-04-24T23:13:39.730 INFO:tasks.workunit.client.1.smithi040.stdout:7/2: rmdir - no directory 2024-04-24T23:13:39.731 INFO:tasks.workunit.client.0.smithi031.stdout:4/398: rmdir d3/d23/d37/d68 0 2024-04-24T23:13:39.732 INFO:tasks.workunit.client.1.smithi040.stdout:3/8: dread f0 [0,4194304] 0 2024-04-24T23:13:39.733 INFO:tasks.workunit.client.1.smithi040.stdout:3/9: rename d1 to d1/d2 22 2024-04-24T23:13:39.733 INFO:tasks.workunit.client.0.smithi031.stdout:0/349: write da/d11/f33 [2239610,4721] 0 2024-04-24T23:13:39.738 INFO:tasks.workunit.client.1.smithi040.stdout:7/3: dwrite f0 [0,4194304] 0 2024-04-24T23:13:39.738 INFO:tasks.workunit.client.1.smithi040.stdout:6/4: creat f1 x:0 0 0 2024-04-24T23:13:39.739 INFO:tasks.workunit.client.0.smithi031.stdout:4/399: dwrite d3/d20/d2d/d44/d5b/d5d/f65 [0,4194304] 0 2024-04-24T23:13:39.740 INFO:tasks.workunit.client.1.smithi040.stdout:2/15: getdents . 0 2024-04-24T23:13:39.742 INFO:tasks.workunit.client.0.smithi031.stdout:0/350: read da/d18/d1a/d37/f4a [192953,43642] 0 2024-04-24T23:13:39.743 INFO:tasks.workunit.client.1.smithi040.stdout:0/11: symlink l4 0 2024-04-24T23:13:39.743 INFO:tasks.workunit.client.1.smithi040.stdout:7/4: dread f0 [0,4194304] 0 2024-04-24T23:13:39.744 INFO:tasks.workunit.client.1.smithi040.stdout:0/12: write f2 [277811,91234] 0 2024-04-24T23:13:39.745 INFO:tasks.workunit.client.1.smithi040.stdout:7/5: read f0 [1635517,23155] 0 2024-04-24T23:13:39.748 INFO:tasks.workunit.client.0.smithi031.stdout:0/351: symlink da/d20/d4e/l6f 0 2024-04-24T23:13:39.749 INFO:tasks.workunit.client.1.smithi040.stdout:2/16: dwrite f0 [0,4194304] 0 2024-04-24T23:13:39.751 INFO:tasks.workunit.client.0.smithi031.stdout:8/404: write d8/d15/d37/d43/f6f [629713,43061] 0 2024-04-24T23:13:39.754 INFO:tasks.workunit.client.0.smithi031.stdout:0/352: rmdir da/d20/d4d 39 2024-04-24T23:13:39.757 INFO:tasks.workunit.client.0.smithi031.stdout:4/400: creat d3/f85 x:0 0 0 2024-04-24T23:13:39.757 INFO:tasks.workunit.client.1.smithi040.stdout:0/13: fsync f2 0 2024-04-24T23:13:39.758 INFO:tasks.workunit.client.1.smithi040.stdout:0/14: fdatasync f2 0 2024-04-24T23:13:39.759 INFO:tasks.workunit.client.1.smithi040.stdout:5/6: unlink f1 0 2024-04-24T23:13:39.759 INFO:tasks.workunit.client.1.smithi040.stdout:5/7: chown . 13857311 1 2024-04-24T23:13:39.760 INFO:tasks.workunit.client.0.smithi031.stdout:8/405: symlink d8/d1a/d73/l99 0 2024-04-24T23:13:39.762 INFO:tasks.workunit.client.0.smithi031.stdout:0/353: creat da/d18/d1a/f70 x:0 0 0 2024-04-24T23:13:39.764 INFO:tasks.workunit.client.0.smithi031.stdout:0/354: chown da/d11/l46 2437 1 2024-04-24T23:13:39.768 INFO:tasks.workunit.client.1.smithi040.stdout:1/10: rename f1 to f3 0 2024-04-24T23:13:39.768 INFO:tasks.workunit.client.0.smithi031.stdout:8/406: dwrite d8/d1a/d1d/d35/d58/d5b/f91 [0,4194304] 0 2024-04-24T23:13:39.770 INFO:tasks.workunit.client.1.smithi040.stdout:7/6: symlink l1 0 2024-04-24T23:13:39.771 INFO:tasks.workunit.client.1.smithi040.stdout:7/7: readlink l1 0 2024-04-24T23:13:39.771 INFO:tasks.workunit.client.1.smithi040.stdout:7/8: chown l1 1288 1 2024-04-24T23:13:39.771 INFO:tasks.workunit.client.0.smithi031.stdout:0/355: mknod da/d18/d1a/d37/d41/c71 0 2024-04-24T23:13:39.772 INFO:tasks.workunit.client.1.smithi040.stdout:9/5: symlink l0 0 2024-04-24T23:13:39.772 INFO:tasks.workunit.client.1.smithi040.stdout:9/6: stat l0 0 2024-04-24T23:13:39.773 INFO:tasks.workunit.client.0.smithi031.stdout:8/407: dread d8/d15/d37/d43/f6f [0,4194304] 0 2024-04-24T23:13:39.774 INFO:tasks.workunit.client.0.smithi031.stdout:4/401: truncate d3/d23/d69/d3f/f61 1606857 0 2024-04-24T23:13:39.775 INFO:tasks.workunit.client.0.smithi031.stdout:4/402: dread - d3/d23/d3b/f72 zero size 2024-04-24T23:13:39.775 INFO:tasks.workunit.client.0.smithi031.stdout:4/403: chown d3/d23/d69 290 1 2024-04-24T23:13:39.776 INFO:tasks.workunit.client.1.smithi040.stdout:2/17: creat f1 x:0 0 0 2024-04-24T23:13:39.779 INFO:tasks.workunit.client.0.smithi031.stdout:0/356: write da/fd [5723944,59666] 0 2024-04-24T23:13:39.780 INFO:tasks.workunit.client.0.smithi031.stdout:0/357: write da/d18/d5a/f69 [523498,117631] 0 2024-04-24T23:13:39.781 INFO:tasks.workunit.client.1.smithi040.stdout:3/10: link f0 d1/f3 0 2024-04-24T23:13:39.782 INFO:tasks.workunit.client.0.smithi031.stdout:8/408: stat d8/c25 0 2024-04-24T23:13:39.785 INFO:tasks.workunit.client.1.smithi040.stdout:5/8: mkdir d2 0 2024-04-24T23:13:39.785 INFO:tasks.workunit.client.1.smithi040.stdout:5/9: stat d2 0 2024-04-24T23:13:39.785 INFO:tasks.workunit.client.1.smithi040.stdout:5/10: unlink - no file 2024-04-24T23:13:39.785 INFO:tasks.workunit.client.1.smithi040.stdout:5/11: dwrite - no filename 2024-04-24T23:13:39.785 INFO:tasks.workunit.client.1.smithi040.stdout:5/12: stat d2 0 2024-04-24T23:13:39.786 INFO:tasks.workunit.client.1.smithi040.stdout:6/5: unlink f1 0 2024-04-24T23:13:39.787 INFO:tasks.workunit.client.0.smithi031.stdout:8/409: mkdir d8/d15/d9a 0 2024-04-24T23:13:39.788 INFO:tasks.workunit.client.0.smithi031.stdout:8/410: dread - d8/d15/f3d zero size 2024-04-24T23:13:39.789 INFO:tasks.workunit.client.1.smithi040.stdout:1/11: mknod c4 0 2024-04-24T23:13:39.790 INFO:tasks.workunit.client.1.smithi040.stdout:7/9: write f0 [3284067,123776] 0 2024-04-24T23:13:39.790 INFO:tasks.workunit.client.0.smithi031.stdout:8/411: truncate d8/d1a/d1d/d35/d58/d47/f72 232421 0 2024-04-24T23:13:39.791 INFO:tasks.workunit.client.0.smithi031.stdout:8/412: dread - d8/d15/d37/f5d zero size 2024-04-24T23:13:39.792 INFO:tasks.workunit.client.1.smithi040.stdout:2/18: creat f2 x:0 0 0 2024-04-24T23:13:39.792 INFO:tasks.workunit.client.0.smithi031.stdout:4/404: getdents d3/d20/d25/d60/d63 0 2024-04-24T23:13:39.793 INFO:tasks.workunit.client.0.smithi031.stdout:8/413: truncate d8/d15/f40 1406152 0 2024-04-24T23:13:39.793 INFO:tasks.workunit.client.1.smithi040.stdout:6/6: dwrite f0 [0,4194304] 0 2024-04-24T23:13:39.793 INFO:tasks.workunit.client.0.smithi031.stdout:8/414: chown d8/d15/l5f 175 1 2024-04-24T23:13:39.794 INFO:tasks.workunit.client.0.smithi031.stdout:0/358: getdents da/d20 0 2024-04-24T23:13:39.795 INFO:tasks.workunit.client.1.smithi040.stdout:3/11: write d1/f3 [337114,49076] 0 2024-04-24T23:13:39.797 INFO:tasks.workunit.client.0.smithi031.stdout:8/415: creat d8/d1a/f9b x:0 0 0 2024-04-24T23:13:39.799 INFO:tasks.workunit.client.0.smithi031.stdout:4/405: dwrite d3/d23/d69/f33 [0,4194304] 0 2024-04-24T23:13:39.800 INFO:tasks.workunit.client.1.smithi040.stdout:1/12: dwrite f2 [0,4194304] 0 2024-04-24T23:13:39.800 INFO:tasks.workunit.client.1.smithi040.stdout:3/12: chown d1/f3 0 1 2024-04-24T23:13:39.800 INFO:tasks.workunit.client.0.smithi031.stdout:0/359: creat da/d18/d1a/f72 x:0 0 0 2024-04-24T23:13:39.801 INFO:tasks.workunit.client.0.smithi031.stdout:4/406: chown d3/d23/d69/f13 123431 1 2024-04-24T23:13:39.802 INFO:tasks.workunit.client.1.smithi040.stdout:2/19: unlink f2 0 2024-04-24T23:13:39.807 INFO:tasks.workunit.client.0.smithi031.stdout:8/416: dwrite d8/d15/f40 [0,4194304] 0 2024-04-24T23:13:39.812 INFO:tasks.workunit.client.0.smithi031.stdout:4/407: dread d3/d23/f3a [0,4194304] 0 2024-04-24T23:13:39.813 INFO:tasks.workunit.client.1.smithi040.stdout:5/13: mknod d2/c3 0 2024-04-24T23:13:39.814 INFO:tasks.workunit.client.1.smithi040.stdout:5/14: rename d2 to d2/d4 22 2024-04-24T23:13:39.814 INFO:tasks.workunit.client.1.smithi040.stdout:5/15: fsync - no filename 2024-04-24T23:13:39.814 INFO:tasks.workunit.client.1.smithi040.stdout:5/16: dwrite - no filename 2024-04-24T23:13:39.814 INFO:tasks.workunit.client.1.smithi040.stdout:9/7: link l0 l1 0 2024-04-24T23:13:39.814 INFO:tasks.workunit.client.1.smithi040.stdout:9/8: write - no filename 2024-04-24T23:13:39.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/417: symlink d8/d15/d37/d43/d7b/d89/l9c 0 2024-04-24T23:13:39.818 INFO:tasks.workunit.client.0.smithi031.stdout:8/418: write d8/f70 [6957768,74078] 0 2024-04-24T23:13:39.819 INFO:tasks.workunit.client.1.smithi040.stdout:7/10: link f0 f2 0 2024-04-24T23:13:39.819 INFO:tasks.workunit.client.0.smithi031.stdout:8/419: chown d8/d1a/d1d/d35/d58/d47/f77 27 1 2024-04-24T23:13:39.826 INFO:tasks.workunit.client.0.smithi031.stdout:0/360: rename da/d11/l36 to da/l73 0 2024-04-24T23:13:39.826 INFO:tasks.workunit.client.1.smithi040.stdout:3/13: dwrite f0 [0,4194304] 0 2024-04-24T23:13:39.827 INFO:tasks.workunit.client.1.smithi040.stdout:1/13: mknod c5 0 2024-04-24T23:13:39.828 INFO:tasks.workunit.client.1.smithi040.stdout:2/20: creat f3 x:0 0 0 2024-04-24T23:13:39.831 INFO:tasks.workunit.client.1.smithi040.stdout:0/15: getdents . 0 2024-04-24T23:13:39.832 INFO:tasks.workunit.client.1.smithi040.stdout:0/16: write f2 [1413686,93177] 0 2024-04-24T23:13:39.835 INFO:tasks.workunit.client.1.smithi040.stdout:5/17: symlink d2/l5 0 2024-04-24T23:13:39.836 INFO:tasks.workunit.client.1.smithi040.stdout:9/9: creat f2 x:0 0 0 2024-04-24T23:13:39.837 INFO:tasks.workunit.client.1.smithi040.stdout:9/10: fdatasync f2 0 2024-04-24T23:13:39.837 INFO:tasks.workunit.client.1.smithi040.stdout:7/11: mknod c3 0 2024-04-24T23:13:39.840 INFO:tasks.workunit.client.1.smithi040.stdout:1/14: fsync f3 0 2024-04-24T23:13:39.843 INFO:tasks.workunit.client.1.smithi040.stdout:0/17: rename l4 to l5 0 2024-04-24T23:13:39.844 INFO:tasks.workunit.client.1.smithi040.stdout:5/18: creat d2/f6 x:0 0 0 2024-04-24T23:13:39.845 INFO:tasks.workunit.client.1.smithi040.stdout:9/11: creat f3 x:0 0 0 2024-04-24T23:13:39.846 INFO:tasks.workunit.client.0.smithi031.stdout:4/408: dwrite d3/fc [0,4194304] 0 2024-04-24T23:13:39.850 INFO:tasks.workunit.client.0.smithi031.stdout:4/409: dread - d3/f71 zero size 2024-04-24T23:13:39.851 INFO:tasks.workunit.client.0.smithi031.stdout:4/410: mkdir d3/d23/d37/d86 0 2024-04-24T23:13:39.851 INFO:tasks.workunit.client.1.smithi040.stdout:3/14: mknod d1/c4 0 2024-04-24T23:13:39.851 INFO:tasks.workunit.client.1.smithi040.stdout:3/15: readlink - no filename 2024-04-24T23:13:39.851 INFO:tasks.workunit.client.1.smithi040.stdout:3/16: chown f0 583 1 2024-04-24T23:13:39.851 INFO:tasks.workunit.client.1.smithi040.stdout:1/15: symlink l6 0 2024-04-24T23:13:39.854 INFO:tasks.workunit.client.1.smithi040.stdout:0/18: dwrite f2 [0,4194304] 0 2024-04-24T23:13:39.854 INFO:tasks.workunit.client.1.smithi040.stdout:5/19: mkdir d2/d7 0 2024-04-24T23:13:39.856 INFO:tasks.workunit.client.1.smithi040.stdout:9/12: rename f3 to f4 0 2024-04-24T23:13:39.856 INFO:tasks.workunit.client.1.smithi040.stdout:5/20: chown d2/c3 66568051 1 2024-04-24T23:13:39.856 INFO:tasks.workunit.client.0.smithi031.stdout:4/411: getdents d3/d23/d37/d86 0 2024-04-24T23:13:39.857 INFO:tasks.workunit.client.0.smithi031.stdout:4/412: chown d3/d20/d2d/f50 1 1 2024-04-24T23:13:39.858 INFO:tasks.workunit.client.1.smithi040.stdout:3/17: rmdir d1 39 2024-04-24T23:13:39.859 INFO:tasks.workunit.client.0.smithi031.stdout:0/361: getdents da/d11 0 2024-04-24T23:13:39.863 INFO:tasks.workunit.client.0.smithi031.stdout:4/413: dread d3/fb [0,4194304] 0 2024-04-24T23:13:39.864 INFO:tasks.workunit.client.1.smithi040.stdout:5/21: dwrite d2/f6 [0,4194304] 0 2024-04-24T23:13:39.865 INFO:tasks.workunit.client.0.smithi031.stdout:0/362: mknod da/d20/d4d/c74 0 2024-04-24T23:13:39.866 INFO:tasks.workunit.client.0.smithi031.stdout:8/420: truncate d8/d1a/d1d/d35/d58/d5b/f91 1703472 0 2024-04-24T23:13:39.868 INFO:tasks.workunit.client.1.smithi040.stdout:9/13: symlink l5 0 2024-04-24T23:13:39.869 INFO:tasks.workunit.client.0.smithi031.stdout:4/414: creat d3/d20/d2d/d44/d5b/d5d/f87 x:0 0 0 2024-04-24T23:13:39.871 INFO:tasks.workunit.client.1.smithi040.stdout:6/7: truncate f0 138109 0 2024-04-24T23:13:39.873 INFO:tasks.workunit.client.1.smithi040.stdout:1/16: link f2 f7 0 2024-04-24T23:13:39.874 INFO:tasks.workunit.client.1.smithi040.stdout:1/17: chown l6 387677 1 2024-04-24T23:13:39.876 INFO:tasks.workunit.client.1.smithi040.stdout:0/19: readlink l5 0 2024-04-24T23:13:39.877 INFO:tasks.workunit.client.0.smithi031.stdout:4/415: mknod d3/d23/c88 0 2024-04-24T23:13:39.878 INFO:tasks.workunit.client.0.smithi031.stdout:4/416: stat d3/d23/f3d 0 2024-04-24T23:13:39.879 INFO:tasks.workunit.client.1.smithi040.stdout:0/20: dread f2 [0,4194304] 0 2024-04-24T23:13:39.880 INFO:tasks.workunit.client.1.smithi040.stdout:0/21: chown l1 224439 1 2024-04-24T23:13:39.881 INFO:tasks.workunit.client.1.smithi040.stdout:7/12: dwrite f0 [0,4194304] 0 2024-04-24T23:13:39.882 INFO:tasks.workunit.client.1.smithi040.stdout:3/18: creat d1/f5 x:0 0 0 2024-04-24T23:13:39.882 INFO:tasks.workunit.client.1.smithi040.stdout:3/19: write f0 [2360824,99303] 0 2024-04-24T23:13:39.884 INFO:tasks.workunit.client.1.smithi040.stdout:6/8: creat f2 x:0 0 0 2024-04-24T23:13:39.884 INFO:tasks.workunit.client.1.smithi040.stdout:2/21: truncate f0 1161221 0 2024-04-24T23:13:39.885 INFO:tasks.workunit.client.0.smithi031.stdout:4/417: dwrite d3/d23/d3b/f72 [0,4194304] 0 2024-04-24T23:13:39.887 INFO:tasks.workunit.client.0.smithi031.stdout:0/363: getdents da/d18 0 2024-04-24T23:13:39.891 INFO:tasks.workunit.client.1.smithi040.stdout:6/9: dwrite f2 [0,4194304] 0 2024-04-24T23:13:39.894 INFO:tasks.workunit.client.1.smithi040.stdout:6/10: dread f2 [0,4194304] 0 2024-04-24T23:13:39.895 INFO:tasks.workunit.client.1.smithi040.stdout:5/22: mkdir d2/d7/d8 0 2024-04-24T23:13:39.896 INFO:tasks.workunit.client.0.smithi031.stdout:0/364: dwrite da/f2b [0,4194304] 0 2024-04-24T23:13:39.896 INFO:tasks.workunit.client.0.smithi031.stdout:4/418: truncate d3/d23/f3a 2932113 0 2024-04-24T23:13:39.898 INFO:tasks.workunit.client.0.smithi031.stdout:4/419: readlink d3/d23/d69/d3f/l58 0 2024-04-24T23:13:39.899 INFO:tasks.workunit.client.0.smithi031.stdout:4/420: fsync d3/d20/d2d/f40 0 2024-04-24T23:13:39.904 INFO:tasks.workunit.client.1.smithi040.stdout:5/23: dwrite d2/f6 [0,4194304] 0 2024-04-24T23:13:39.904 INFO:tasks.workunit.client.0.smithi031.stdout:0/365: creat da/d18/d1a/d37/f75 x:0 0 0 2024-04-24T23:13:39.904 INFO:tasks.workunit.client.0.smithi031.stdout:0/366: stat l7 0 2024-04-24T23:13:39.906 INFO:tasks.workunit.client.1.smithi040.stdout:7/13: mknod c4 0 2024-04-24T23:13:39.906 INFO:tasks.workunit.client.0.smithi031.stdout:0/367: write da/d18/d1a/f29 [1648966,89111] 0 2024-04-24T23:13:39.907 INFO:tasks.workunit.client.1.smithi040.stdout:7/14: readlink l1 0 2024-04-24T23:13:39.914 INFO:tasks.workunit.client.0.smithi031.stdout:4/421: mknod d3/d20/d25/d60/c89 0 2024-04-24T23:13:39.917 INFO:tasks.workunit.client.1.smithi040.stdout:7/15: dwrite f2 [0,4194304] 0 2024-04-24T23:13:39.920 INFO:tasks.workunit.client.0.smithi031.stdout:8/421: dwrite d8/f38 [12582912,4194304] 0 2024-04-24T23:13:39.922 INFO:tasks.workunit.client.0.smithi031.stdout:0/368: dwrite da/d11/f25 [0,4194304] 0 2024-04-24T23:13:39.923 INFO:tasks.workunit.client.1.smithi040.stdout:1/18: dwrite f3 [0,4194304] 0 2024-04-24T23:13:39.923 INFO:tasks.workunit.client.1.smithi040.stdout:7/16: dread f0 [0,4194304] 0 2024-04-24T23:13:39.923 INFO:tasks.workunit.client.1.smithi040.stdout:7/17: readlink l1 0 2024-04-24T23:13:39.927 INFO:tasks.workunit.client.1.smithi040.stdout:6/11: mknod c3 0 2024-04-24T23:13:39.937 INFO:tasks.workunit.client.0.smithi031.stdout:4/422: unlink d3/d23/d69/l36 0 2024-04-24T23:13:39.939 INFO:tasks.workunit.client.0.smithi031.stdout:8/422: dread d8/d1a/d1d/d35/f6d [0,4194304] 0 2024-04-24T23:13:39.940 INFO:tasks.workunit.client.1.smithi040.stdout:5/24: symlink d2/d7/l9 0 2024-04-24T23:13:39.941 INFO:tasks.workunit.client.1.smithi040.stdout:5/25: chown d2/d7 28 1 2024-04-24T23:13:39.942 INFO:tasks.workunit.client.0.smithi031.stdout:0/369: mkdir da/d18/d1a/d76 0 2024-04-24T23:13:39.946 INFO:tasks.workunit.client.0.smithi031.stdout:8/423: dwrite d8/d15/d37/f5d [0,4194304] 0 2024-04-24T23:13:39.946 INFO:tasks.workunit.client.1.smithi040.stdout:5/26: dwrite d2/f6 [0,4194304] 0 2024-04-24T23:13:39.947 INFO:tasks.workunit.client.1.smithi040.stdout:5/27: stat d2/f6 0 2024-04-24T23:13:39.949 INFO:tasks.workunit.client.0.smithi031.stdout:0/370: mknod da/d22/c77 0 2024-04-24T23:13:39.950 INFO:tasks.workunit.client.0.smithi031.stdout:0/371: fdatasync da/f10 0 2024-04-24T23:13:39.953 INFO:tasks.workunit.client.1.smithi040.stdout:1/19: creat f8 x:0 0 0 2024-04-24T23:13:39.953 INFO:tasks.workunit.client.1.smithi040.stdout:1/20: chown f8 2360 1 2024-04-24T23:13:39.955 INFO:tasks.workunit.client.1.smithi040.stdout:7/18: creat f5 x:0 0 0 2024-04-24T23:13:39.955 INFO:tasks.workunit.client.0.smithi031.stdout:4/423: dwrite d3/d20/d25/d60/d63/f81 [0,4194304] 0 2024-04-24T23:13:39.957 INFO:tasks.workunit.client.1.smithi040.stdout:6/12: write f2 [1342148,80442] 0 2024-04-24T23:13:39.960 INFO:tasks.workunit.client.1.smithi040.stdout:7/19: dread f0 [0,4194304] 0 2024-04-24T23:13:39.961 INFO:tasks.workunit.client.1.smithi040.stdout:9/14: getdents . 0 2024-04-24T23:13:39.963 INFO:tasks.workunit.client.1.smithi040.stdout:6/13: dread f2 [0,4194304] 0 2024-04-24T23:13:39.964 INFO:tasks.workunit.client.0.smithi031.stdout:4/424: dwrite d3/d20/d2d/d44/d5b/d5d/f65 [0,4194304] 0 2024-04-24T23:13:39.964 INFO:tasks.workunit.client.1.smithi040.stdout:6/14: write f2 [3779077,102879] 0 2024-04-24T23:13:39.967 INFO:tasks.workunit.client.1.smithi040.stdout:5/28: symlink d2/la 0 2024-04-24T23:13:39.967 INFO:tasks.workunit.client.1.smithi040.stdout:5/29: chown d2/la 83265 1 2024-04-24T23:13:39.968 INFO:tasks.workunit.client.1.smithi040.stdout:5/30: chown d2/c3 115155 1 2024-04-24T23:13:39.972 INFO:tasks.workunit.client.0.smithi031.stdout:4/425: symlink d3/d20/d25/l8a 0 2024-04-24T23:13:39.973 INFO:tasks.workunit.client.0.smithi031.stdout:4/426: chown d3/d20/d2d/d44 1 1 2024-04-24T23:13:39.974 INFO:tasks.workunit.client.1.smithi040.stdout:2/22: dwrite f0 [0,4194304] 0 2024-04-24T23:13:39.974 INFO:tasks.workunit.client.1.smithi040.stdout:2/23: rmdir - no directory 2024-04-24T23:13:39.974 INFO:tasks.workunit.client.1.smithi040.stdout:2/24: rmdir - no directory 2024-04-24T23:13:39.975 INFO:tasks.workunit.client.1.smithi040.stdout:1/21: creat f9 x:0 0 0 2024-04-24T23:13:39.976 INFO:tasks.workunit.client.1.smithi040.stdout:7/20: creat f6 x:0 0 0 2024-04-24T23:13:39.977 INFO:tasks.workunit.client.0.smithi031.stdout:4/427: creat d3/d20/d25/d60/d63/f8b x:0 0 0 2024-04-24T23:13:39.977 INFO:tasks.workunit.client.1.smithi040.stdout:0/22: getdents . 0 2024-04-24T23:13:39.977 INFO:tasks.workunit.client.0.smithi031.stdout:3/340: sync 2024-04-24T23:13:39.978 INFO:tasks.workunit.client.1.smithi040.stdout:2/25: write f3 [172286,4626] 0 2024-04-24T23:13:39.979 INFO:tasks.workunit.client.1.smithi040.stdout:5/31: unlink d2/la 0 2024-04-24T23:13:39.982 INFO:tasks.workunit.client.0.smithi031.stdout:3/341: write d6/df/d16/d2a/f31 [384786,47854] 0 2024-04-24T23:13:39.983 INFO:tasks.workunit.client.0.smithi031.stdout:3/342: fdatasync d6/f8 0 2024-04-24T23:13:39.984 INFO:tasks.workunit.client.0.smithi031.stdout:2/261: sync 2024-04-24T23:13:39.985 INFO:tasks.workunit.client.0.smithi031.stdout:3/343: dread - d6/d14/d1d/d59/f5e zero size 2024-04-24T23:13:39.985 INFO:tasks.workunit.client.0.smithi031.stdout:2/262: fdatasync d5/d28/f40 0 2024-04-24T23:13:39.985 INFO:tasks.workunit.client.1.smithi040.stdout:3/20: truncate f0 325838 0 2024-04-24T23:13:39.986 INFO:tasks.workunit.client.1.smithi040.stdout:9/15: dwrite f4 [0,4194304] 0 2024-04-24T23:13:39.987 INFO:tasks.workunit.client.1.smithi040.stdout:0/23: dread f2 [0,4194304] 0 2024-04-24T23:13:39.987 INFO:tasks.workunit.client.1.smithi040.stdout:1/22: mkdir da 0 2024-04-24T23:13:39.987 INFO:tasks.workunit.client.1.smithi040.stdout:9/16: fsync f2 0 2024-04-24T23:13:39.988 INFO:tasks.workunit.client.1.smithi040.stdout:7/21: fsync f2 0 2024-04-24T23:13:39.991 INFO:tasks.workunit.client.0.smithi031.stdout:1/287: sync 2024-04-24T23:13:39.991 INFO:tasks.workunit.client.0.smithi031.stdout:6/463: sync 2024-04-24T23:13:39.991 INFO:tasks.workunit.client.0.smithi031.stdout:9/439: sync 2024-04-24T23:13:39.991 INFO:tasks.workunit.client.0.smithi031.stdout:7/287: sync 2024-04-24T23:13:39.991 INFO:tasks.workunit.client.0.smithi031.stdout:5/293: sync 2024-04-24T23:13:39.993 INFO:tasks.workunit.client.1.smithi040.stdout:2/26: unlink f1 0 2024-04-24T23:13:39.994 INFO:tasks.workunit.client.0.smithi031.stdout:1/288: chown d1/d1d 2726621 1 2024-04-24T23:13:39.994 INFO:tasks.workunit.client.1.smithi040.stdout:2/27: fsync f0 0 2024-04-24T23:13:39.994 INFO:tasks.workunit.client.1.smithi040.stdout:2/28: stat f3 0 2024-04-24T23:13:39.996 INFO:tasks.workunit.client.0.smithi031.stdout:9/440: fdatasync d2/d43/f5d 0 2024-04-24T23:13:39.998 INFO:tasks.workunit.client.0.smithi031.stdout:3/344: creat d6/d14/d1d/d59/f6c x:0 0 0 2024-04-24T23:13:40.000 INFO:tasks.workunit.client.0.smithi031.stdout:2/263: rename d5/d28/d43/f51 to d5/d6/d29/f5a 0 2024-04-24T23:13:40.001 INFO:tasks.workunit.client.0.smithi031.stdout:9/441: write d2/f32 [2460235,50506] 0 2024-04-24T23:13:40.004 INFO:tasks.workunit.client.0.smithi031.stdout:2/264: fsync d5/f58 0 2024-04-24T23:13:40.007 INFO:tasks.workunit.client.0.smithi031.stdout:6/464: dread d13/d1b/f46 [0,4194304] 0 2024-04-24T23:13:40.007 INFO:tasks.workunit.client.0.smithi031.stdout:8/424: dwrite d8/d1a/d1d/f7d [0,4194304] 0 2024-04-24T23:13:40.007 INFO:tasks.workunit.client.1.smithi040.stdout:0/24: creat f6 x:0 0 0 2024-04-24T23:13:40.008 INFO:tasks.workunit.client.1.smithi040.stdout:0/25: truncate f6 407379 0 2024-04-24T23:13:40.008 INFO:tasks.workunit.client.1.smithi040.stdout:0/26: chown c3 235579 1 2024-04-24T23:13:40.008 INFO:tasks.workunit.client.1.smithi040.stdout:0/27: rmdir - no directory 2024-04-24T23:13:40.009 INFO:tasks.workunit.client.0.smithi031.stdout:5/294: creat d0/d6/d32/f5f x:0 0 0 2024-04-24T23:13:40.010 INFO:tasks.workunit.client.0.smithi031.stdout:9/442: write d2/d1b/d14/d1d/f7b [7521,43724] 0 2024-04-24T23:13:40.015 INFO:tasks.workunit.client.0.smithi031.stdout:5/295: fdatasync d0/d6/d3b/d4b/dc/f24 0 2024-04-24T23:13:40.015 INFO:tasks.workunit.client.0.smithi031.stdout:9/443: fdatasync d2/d1b/d14/f69 0 2024-04-24T23:13:40.016 INFO:tasks.workunit.client.0.smithi031.stdout:0/372: dwrite da/d18/d1a/d37/f4a [0,4194304] 0 2024-04-24T23:13:40.017 INFO:tasks.workunit.client.1.smithi040.stdout:9/17: creat f6 x:0 0 0 2024-04-24T23:13:40.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/296: fsync d0/d6/d3b/f4d 0 2024-04-24T23:13:40.018 INFO:tasks.workunit.client.1.smithi040.stdout:9/18: read - f6 zero size 2024-04-24T23:13:40.020 INFO:tasks.workunit.client.0.smithi031.stdout:0/373: read da/d11/f5d [2903002,43911] 0 2024-04-24T23:13:40.028 INFO:tasks.workunit.client.0.smithi031.stdout:0/374: dwrite da/d11/f33 [0,4194304] 0 2024-04-24T23:13:40.029 INFO:tasks.workunit.client.0.smithi031.stdout:0/375: stat da/d18/d1a/d37 0 2024-04-24T23:13:40.033 INFO:tasks.workunit.client.0.smithi031.stdout:4/428: getdents d3/d20/d2d 0 2024-04-24T23:13:40.036 INFO:tasks.workunit.client.0.smithi031.stdout:0/376: truncate da/d18/d1a/d37/f5e 240438 0 2024-04-24T23:13:40.037 INFO:tasks.workunit.client.0.smithi031.stdout:8/425: dread d8/d1a/d1d/d35/d58/f44 [0,4194304] 0 2024-04-24T23:13:40.037 INFO:tasks.workunit.client.0.smithi031.stdout:8/426: readlink d8/d15/d37/l51 0 2024-04-24T23:13:40.038 INFO:tasks.workunit.client.1.smithi040.stdout:2/29: symlink l4 0 2024-04-24T23:13:40.039 INFO:tasks.workunit.client.1.smithi040.stdout:6/15: dwrite f0 [0,4194304] 0 2024-04-24T23:13:40.040 INFO:tasks.workunit.client.1.smithi040.stdout:5/32: mkdir d2/d7/d8/db 0 2024-04-24T23:13:40.042 INFO:tasks.workunit.client.0.smithi031.stdout:2/265: unlink d5/d6/c4a 0 2024-04-24T23:13:40.043 INFO:tasks.workunit.client.1.smithi040.stdout:3/21: fsync f0 0 2024-04-24T23:13:40.044 INFO:tasks.workunit.client.1.smithi040.stdout:3/22: chown d1 4589 1 2024-04-24T23:13:40.048 INFO:tasks.workunit.client.1.smithi040.stdout:2/30: dwrite f0 [0,4194304] 0 2024-04-24T23:13:40.048 INFO:tasks.workunit.client.1.smithi040.stdout:1/23: mknod da/cb 0 2024-04-24T23:13:40.049 INFO:tasks.workunit.client.0.smithi031.stdout:9/444: dread d2/fe [0,4194304] 0 2024-04-24T23:13:40.049 INFO:tasks.workunit.client.1.smithi040.stdout:2/31: readlink l4 0 2024-04-24T23:13:40.051 INFO:tasks.workunit.client.1.smithi040.stdout:9/19: mkdir d7 0 2024-04-24T23:13:40.051 INFO:tasks.workunit.client.0.smithi031.stdout:9/445: chown d2/d1b/d14/d1d/f24 181 1 2024-04-24T23:13:40.052 INFO:tasks.workunit.client.0.smithi031.stdout:7/288: dwrite d1/d1c/f35 [0,4194304] 0 2024-04-24T23:13:40.054 INFO:tasks.workunit.client.0.smithi031.stdout:7/289: fsync d1/f53 0 2024-04-24T23:13:40.055 INFO:tasks.workunit.client.1.smithi040.stdout:2/32: dread f3 [0,4194304] 0 2024-04-24T23:13:40.055 INFO:tasks.workunit.client.1.smithi040.stdout:5/33: dwrite d2/f6 [0,4194304] 0 2024-04-24T23:13:40.057 INFO:tasks.workunit.client.1.smithi040.stdout:7/22: truncate f2 395587 0 2024-04-24T23:13:40.061 INFO:tasks.workunit.client.1.smithi040.stdout:0/28: dwrite f2 [4194304,4194304] 0 2024-04-24T23:13:40.062 INFO:tasks.workunit.client.0.smithi031.stdout:5/297: creat d0/d6/d3b/d4b/dc/d25/f60 x:0 0 0 2024-04-24T23:13:40.062 INFO:tasks.workunit.client.1.smithi040.stdout:0/29: fsync f6 0 2024-04-24T23:13:40.064 INFO:tasks.workunit.client.1.smithi040.stdout:0/30: readlink l1 0 2024-04-24T23:13:40.066 INFO:tasks.workunit.client.1.smithi040.stdout:0/31: read f6 [147945,75719] 0 2024-04-24T23:13:40.070 INFO:tasks.workunit.client.1.smithi040.stdout:6/16: dread f2 [0,4194304] 0 2024-04-24T23:13:40.070 INFO:tasks.workunit.client.1.smithi040.stdout:3/23: symlink d1/l6 0 2024-04-24T23:13:40.072 INFO:tasks.workunit.client.1.smithi040.stdout:3/24: write d1/f5 [534571,66139] 0 2024-04-24T23:13:40.072 INFO:tasks.workunit.client.1.smithi040.stdout:0/32: dread f2 [4194304,4194304] 0 2024-04-24T23:13:40.073 INFO:tasks.workunit.client.1.smithi040.stdout:0/33: chown f6 229 1 2024-04-24T23:13:40.073 INFO:tasks.workunit.client.0.smithi031.stdout:3/345: link d6/d14/d1d/f24 d6/d14/d1d/d69/f6d 0 2024-04-24T23:13:40.075 INFO:tasks.workunit.client.1.smithi040.stdout:1/24: creat da/fc x:0 0 0 2024-04-24T23:13:40.076 INFO:tasks.workunit.client.1.smithi040.stdout:1/25: write f8 [333077,42327] 0 2024-04-24T23:13:40.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/377: mknod da/d18/d1a/d37/c78 0 2024-04-24T23:13:40.079 INFO:tasks.workunit.client.1.smithi040.stdout:0/34: dwrite f2 [8388608,4194304] 0 2024-04-24T23:13:40.081 INFO:tasks.workunit.client.0.smithi031.stdout:0/378: dread da/d18/d1a/d37/f23 [0,4194304] 0 2024-04-24T23:13:40.081 INFO:tasks.workunit.client.1.smithi040.stdout:5/34: symlink d2/lc 0 2024-04-24T23:13:40.083 INFO:tasks.workunit.client.1.smithi040.stdout:7/23: mknod c7 0 2024-04-24T23:13:40.084 INFO:tasks.workunit.client.1.smithi040.stdout:7/24: dread - f5 zero size 2024-04-24T23:13:40.085 INFO:tasks.workunit.client.1.smithi040.stdout:7/25: fsync f6 0 2024-04-24T23:13:40.085 INFO:tasks.workunit.client.1.smithi040.stdout:7/26: stat f0 0 2024-04-24T23:13:40.085 INFO:tasks.workunit.client.0.smithi031.stdout:0/379: dread da/d18/d1a/f1f [0,4194304] 0 2024-04-24T23:13:40.086 INFO:tasks.workunit.client.1.smithi040.stdout:7/27: dread - f6 zero size 2024-04-24T23:13:40.086 INFO:tasks.workunit.client.1.smithi040.stdout:7/28: chown f0 808263 1 2024-04-24T23:13:40.087 INFO:tasks.workunit.client.1.smithi040.stdout:7/29: fsync f6 0 2024-04-24T23:13:40.089 INFO:tasks.workunit.client.1.smithi040.stdout:6/17: creat f4 x:0 0 0 2024-04-24T23:13:40.091 INFO:tasks.workunit.client.1.smithi040.stdout:3/25: rename d1/l6 to d1/l7 0 2024-04-24T23:13:40.097 INFO:tasks.workunit.client.1.smithi040.stdout:9/20: getdents d7 0 2024-04-24T23:13:40.098 INFO:tasks.workunit.client.0.smithi031.stdout:3/346: unlink d6/d14/c41 0 2024-04-24T23:13:40.100 INFO:tasks.workunit.client.1.smithi040.stdout:2/33: link f0 f5 0 2024-04-24T23:13:40.102 INFO:tasks.workunit.client.1.smithi040.stdout:0/35: creat f7 x:0 0 0 2024-04-24T23:13:40.104 INFO:tasks.workunit.client.0.smithi031.stdout:8/427: mkdir d8/d15/d37/d62/d76/d9d 0 2024-04-24T23:13:40.105 INFO:tasks.workunit.client.0.smithi031.stdout:8/428: write d8/f38 [15325635,129956] 0 2024-04-24T23:13:40.106 INFO:tasks.workunit.client.0.smithi031.stdout:1/289: dwrite d1/da/f2c [0,4194304] 0 2024-04-24T23:13:40.109 INFO:tasks.workunit.client.1.smithi040.stdout:0/36: dwrite f6 [0,4194304] 0 2024-04-24T23:13:40.111 INFO:tasks.workunit.client.1.smithi040.stdout:1/26: rmdir da 39 2024-04-24T23:13:40.114 INFO:tasks.workunit.client.1.smithi040.stdout:7/30: rename l1 to l8 0 2024-04-24T23:13:40.123 INFO:tasks.workunit.client.0.smithi031.stdout:0/380: fsync da/d18/d1a/f1e 0 2024-04-24T23:13:40.129 INFO:tasks.workunit.client.0.smithi031.stdout:5/298: creat d0/d7/d56/f61 x:0 0 0 2024-04-24T23:13:40.130 INFO:tasks.workunit.client.0.smithi031.stdout:5/299: fsync d0/f49 0 2024-04-24T23:13:40.131 INFO:tasks.workunit.client.1.smithi040.stdout:2/34: creat f6 x:0 0 0 2024-04-24T23:13:40.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.130+0000 7f5fd5ffb700 1 -- 172.21.15.31:0/977640748 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fc0013b70 con 0x7f5fd80fcff0 2024-04-24T23:13:40.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.131+0000 7f5fdd370700 1 -- 172.21.15.31:0/977640748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fcff0 msgr2=0x7f5fd80fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.131+0000 7f5fdd370700 1 --2- 172.21.15.31:0/977640748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fcff0 0x7f5fd80fd460 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0004940 tx=0x7f5fc0009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.131+0000 7f5fdd370700 1 -- 172.21.15.31:0/977640748 shutdown_connections 2024-04-24T23:13:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.131+0000 7f5fdd370700 1 --2- 172.21.15.31:0/977640748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fcff0 0x7f5fd80fd460 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.131+0000 7f5fdd370700 1 --2- 172.21.15.31:0/977640748 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fd80fbd40 0x7f5fd80fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.131+0000 7f5fdd370700 1 -- 172.21.15.31:0/977640748 >> 172.21.15.31:0/977640748 conn(0x7f5fd80f74c0 msgr2=0x7f5fd80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.131+0000 7f5fdd370700 1 -- 172.21.15.31:0/977640748 shutdown_connections 2024-04-24T23:13:40.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.131+0000 7f5fdd370700 1 -- 172.21.15.31:0/977640748 wait complete. 2024-04-24T23:13:40.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.132+0000 7f5fdd370700 1 Processor -- start 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.132+0000 7f5fdd370700 1 -- start start 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.132+0000 7f5fdd370700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fbd40 0x7f5fd810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.132+0000 7f5fdd370700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fd80fcff0 0x7f5fd810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.132+0000 7f5fdd370700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810e4a0 con 0x7f5fd80fbd40 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.132+0000 7f5fdd370700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810ecf0 con 0x7f5fd80fcff0 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.133+0000 7f5fd6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fbd40 0x7f5fd810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.133+0000 7f5fd6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fbd40 0x7f5fd810d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58224/0 (socket says 172.21.15.31:58224) 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.133+0000 7f5fd6ffd700 1 -- 172.21.15.31:0/1873298614 learned_addr learned my addr 172.21.15.31:0/1873298614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:40.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.133+0000 7f5fd67fc700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fd80fcff0 0x7f5fd810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.133+0000 7f5fd6ffd700 1 -- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fd80fcff0 msgr2=0x7f5fd810dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.134+0000 7f5fd6ffd700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fd80fcff0 0x7f5fd810dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.134+0000 7f5fd6ffd700 1 -- 172.21.15.31:0/1873298614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc0005040 con 0x7f5fd80fbd40 2024-04-24T23:13:40.136 INFO:tasks.workunit.client.0.smithi031.stdout:3/347: creat d6/df/d16/d53/f6e x:0 0 0 2024-04-24T23:13:40.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.134+0000 7f5fd6ffd700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fbd40 0x7f5fd810d960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800eb40 tx=0x7f5fc800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.134+0000 7f5fcf7fe700 1 -- 172.21.15.31:0/1873298614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fc8007cc0 con 0x7f5fd80fbd40 2024-04-24T23:13:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.134+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd810ef10 con 0x7f5fd80fbd40 2024-04-24T23:13:40.137 INFO:tasks.workunit.client.0.smithi031.stdout:2/266: truncate d5/f27 714192 0 2024-04-24T23:13:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.134+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd810f4d0 con 0x7f5fd80fbd40 2024-04-24T23:13:40.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.134+0000 7f5fcf7fe700 1 -- 172.21.15.31:0/1873298614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fc800f040 con 0x7f5fd80fbd40 2024-04-24T23:13:40.138 INFO:tasks.workunit.client.0.smithi031.stdout:2/267: fsync d5/f46 0 2024-04-24T23:13:40.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.134+0000 7f5fcf7fe700 1 -- 172.21.15.31:0/1873298614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fc8009620 con 0x7f5fd80fbd40 2024-04-24T23:13:40.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.136+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb80052e0 con 0x7f5fd80fbd40 2024-04-24T23:13:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.138+0000 7f5fcf7fe700 1 -- 172.21.15.31:0/1873298614 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f5fc800cce0 con 0x7f5fd80fbd40 2024-04-24T23:13:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.138+0000 7f5fcf7fe700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5fc406b860 0x7f5fc406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:40.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.139+0000 7f5fd67fc700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5fc406b860 0x7f5fc406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:40.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/300: dwrite d0/d6/d3b/d4b/d1d/d2e/f50 [0,4194304] 0 2024-04-24T23:13:40.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.139+0000 7f5fcf7fe700 1 -- 172.21.15.31:0/1873298614 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5fc808a1c0 con 0x7f5fd80fbd40 2024-04-24T23:13:40.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.139+0000 7f5fd67fc700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5fc406b860 0x7f5fc406dd10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000b0d0 tx=0x7f5fc0009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:40.143 INFO:tasks.workunit.client.1.smithi040.stdout:0/37: creat f8 x:0 0 0 2024-04-24T23:13:40.145 INFO:tasks.workunit.client.1.smithi040.stdout:6/18: dwrite f2 [0,4194304] 0 2024-04-24T23:13:40.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.144+0000 7f5fcf7fe700 1 -- 172.21.15.31:0/1873298614 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f5fc8057650 con 0x7f5fd80fbd40 2024-04-24T23:13:40.147 INFO:tasks.workunit.client.0.smithi031.stdout:1/290: unlink d1/da/d10/f1c 0 2024-04-24T23:13:40.151 INFO:tasks.workunit.client.1.smithi040.stdout:0/38: dwrite f8 [0,4194304] 0 2024-04-24T23:13:40.151 INFO:tasks.workunit.client.1.smithi040.stdout:1/27: unlink l6 0 2024-04-24T23:13:40.152 INFO:tasks.workunit.client.0.smithi031.stdout:8/429: mknod d8/d15/d37/d43/d7b/d89/c9e 0 2024-04-24T23:13:40.154 INFO:tasks.workunit.client.0.smithi031.stdout:6/465: getdents d13/d15/d1d/d23 0 2024-04-24T23:13:40.155 INFO:tasks.workunit.client.1.smithi040.stdout:5/35: truncate d2/f6 3435806 0 2024-04-24T23:13:40.155 INFO:tasks.workunit.client.1.smithi040.stdout:5/36: readlink d2/l5 0 2024-04-24T23:13:40.162 INFO:tasks.workunit.client.1.smithi040.stdout:9/21: symlink d7/l8 0 2024-04-24T23:13:40.163 INFO:tasks.workunit.client.0.smithi031.stdout:0/381: creat da/d11/d6a/f79 x:0 0 0 2024-04-24T23:13:40.163 INFO:tasks.workunit.client.0.smithi031.stdout:0/382: readlink l7 0 2024-04-24T23:13:40.164 INFO:tasks.workunit.client.0.smithi031.stdout:0/383: fdatasync da/d18/d1a/f70 0 2024-04-24T23:13:40.165 INFO:tasks.workunit.client.1.smithi040.stdout:9/22: dread f4 [0,4194304] 0 2024-04-24T23:13:40.165 INFO:tasks.workunit.client.1.smithi040.stdout:2/35: unlink l4 0 2024-04-24T23:13:40.166 INFO:tasks.workunit.client.0.smithi031.stdout:7/290: link d1/d1c/d2c/d38/c56 d1/d2/d1e/d4f/c5a 0 2024-04-24T23:13:40.166 INFO:tasks.workunit.client.1.smithi040.stdout:9/23: dread - f2 zero size 2024-04-24T23:13:40.166 INFO:tasks.workunit.client.1.smithi040.stdout:9/24: dread - f2 zero size 2024-04-24T23:13:40.166 INFO:tasks.workunit.client.1.smithi040.stdout:9/25: write f2 [736494,3662] 0 2024-04-24T23:13:40.167 INFO:tasks.workunit.client.1.smithi040.stdout:6/19: creat f5 x:0 0 0 2024-04-24T23:13:40.167 INFO:tasks.workunit.client.0.smithi031.stdout:4/429: getdents d3/d5e 0 2024-04-24T23:13:40.169 INFO:tasks.workunit.client.0.smithi031.stdout:4/430: chown d3/d20/d2d/d44/d5b/l6c 290 1 2024-04-24T23:13:40.170 INFO:tasks.workunit.client.0.smithi031.stdout:4/431: write d3/d20/d2d/d44/d5b/d5d/f65 [286817,18833] 0 2024-04-24T23:13:40.170 INFO:tasks.workunit.client.1.smithi040.stdout:0/39: creat f9 x:0 0 0 2024-04-24T23:13:40.171 INFO:tasks.workunit.client.0.smithi031.stdout:4/432: chown d3/d23/d69/d3f/f6d 1790863 1 2024-04-24T23:13:40.172 INFO:tasks.workunit.client.0.smithi031.stdout:4/433: chown d3/d23/d37/c49 181 1 2024-04-24T23:13:40.172 INFO:tasks.workunit.client.0.smithi031.stdout:3/348: rename d6/d14/c27 to d6/d15/c6f 0 2024-04-24T23:13:40.175 INFO:tasks.workunit.client.1.smithi040.stdout:6/20: dwrite f5 [0,4194304] 0 2024-04-24T23:13:40.176 INFO:tasks.workunit.client.1.smithi040.stdout:6/21: read f2 [3074697,74819] 0 2024-04-24T23:13:40.176 INFO:tasks.workunit.client.1.smithi040.stdout:6/22: readlink - no filename 2024-04-24T23:13:40.176 INFO:tasks.workunit.client.1.smithi040.stdout:6/23: readlink - no filename 2024-04-24T23:13:40.178 INFO:tasks.workunit.client.1.smithi040.stdout:5/37: mknod d2/cd 0 2024-04-24T23:13:40.180 INFO:tasks.workunit.client.0.smithi031.stdout:1/291: symlink d1/d31/l5d 0 2024-04-24T23:13:40.190 INFO:tasks.workunit.client.0.smithi031.stdout:9/446: getdents d2/d1b/d14/d1d/d45/d56/d67/d4a/d4b 0 2024-04-24T23:13:40.192 INFO:tasks.workunit.client.1.smithi040.stdout:9/26: mknod d7/c9 0 2024-04-24T23:13:40.197 INFO:tasks.workunit.client.0.smithi031.stdout:9/447: dwrite d2/f75 [0,4194304] 0 2024-04-24T23:13:40.197 INFO:tasks.workunit.client.0.smithi031.stdout:0/384: symlink da/d18/d1a/l7a 0 2024-04-24T23:13:40.206 INFO:tasks.workunit.client.1.smithi040.stdout:6/24: mknod c6 0 2024-04-24T23:13:40.206 INFO:tasks.workunit.client.1.smithi040.stdout:6/25: fsync f5 0 2024-04-24T23:13:40.207 INFO:tasks.workunit.client.1.smithi040.stdout:7/31: dwrite f2 [0,4194304] 0 2024-04-24T23:13:40.209 INFO:tasks.workunit.client.0.smithi031.stdout:2/268: rename d5/d8/d15/d33 to d5/d8/d45/d5b 0 2024-04-24T23:13:40.212 INFO:tasks.workunit.client.0.smithi031.stdout:6/466: write d13/d1b/d32/f45 [297373,109404] 0 2024-04-24T23:13:40.214 INFO:tasks.workunit.client.1.smithi040.stdout:3/26: getdents d1 0 2024-04-24T23:13:40.214 INFO:tasks.workunit.client.1.smithi040.stdout:3/27: write f0 [1307838,87218] 0 2024-04-24T23:13:40.216 INFO:tasks.workunit.client.0.smithi031.stdout:8/430: mknod d8/d15/d37/d43/d7b/d86/c9f 0 2024-04-24T23:13:40.219 INFO:tasks.workunit.client.1.smithi040.stdout:2/36: link f6 f7 0 2024-04-24T23:13:40.220 INFO:tasks.workunit.client.1.smithi040.stdout:9/27: mkdir d7/da 0 2024-04-24T23:13:40.220 INFO:tasks.workunit.client.0.smithi031.stdout:8/431: read d8/d15/d37/f5e [3593246,66821] 0 2024-04-24T23:13:40.220 INFO:tasks.workunit.client.1.smithi040.stdout:9/28: dread - f6 zero size 2024-04-24T23:13:40.221 INFO:tasks.workunit.client.1.smithi040.stdout:9/29: chown l5 15125 1 2024-04-24T23:13:40.223 INFO:tasks.workunit.client.0.smithi031.stdout:0/385: mknod da/d11/d6a/c7b 0 2024-04-24T23:13:40.225 INFO:tasks.workunit.client.1.smithi040.stdout:3/28: dread f0 [0,4194304] 0 2024-04-24T23:13:40.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/291: truncate d1/d2/f7 1370891 0 2024-04-24T23:13:40.227 INFO:tasks.workunit.client.1.smithi040.stdout:6/26: unlink c6 0 2024-04-24T23:13:40.230 INFO:tasks.workunit.client.0.smithi031.stdout:0/386: dwrite da/d18/d1a/f29 [0,4194304] 0 2024-04-24T23:13:40.231 INFO:tasks.workunit.client.0.smithi031.stdout:0/387: read da/d11/f25 [1554838,10921] 0 2024-04-24T23:13:40.232 INFO:tasks.workunit.client.0.smithi031.stdout:5/301: rename d0/d6/d3b/d4b/dc/d25/d53/f57 to d0/d6/d3b/d4b/dc/d25/f62 0 2024-04-24T23:13:40.233 INFO:tasks.workunit.client.1.smithi040.stdout:3/29: dwrite d1/f3 [0,4194304] 0 2024-04-24T23:13:40.234 INFO:tasks.workunit.client.0.smithi031.stdout:2/269: unlink d5/d6/d29/d26/f38 0 2024-04-24T23:13:40.235 INFO:tasks.workunit.client.0.smithi031.stdout:2/270: fsync d5/f46 0 2024-04-24T23:13:40.236 INFO:tasks.workunit.client.1.smithi040.stdout:5/38: fsync d2/f6 0 2024-04-24T23:13:40.238 INFO:tasks.workunit.client.0.smithi031.stdout:3/349: mknod d6/df/d16/c70 0 2024-04-24T23:13:40.247 INFO:tasks.workunit.client.0.smithi031.stdout:9/448: chown d2/d16/l1e 3364242 1 2024-04-24T23:13:40.249 INFO:tasks.workunit.client.1.smithi040.stdout:6/27: unlink c3 0 2024-04-24T23:13:40.250 INFO:tasks.workunit.client.1.smithi040.stdout:6/28: write f4 [244184,23426] 0 2024-04-24T23:13:40.250 INFO:tasks.workunit.client.1.smithi040.stdout:6/29: rmdir - no directory 2024-04-24T23:13:40.251 INFO:tasks.workunit.client.1.smithi040.stdout:6/30: write f5 [1796310,49365] 0 2024-04-24T23:13:40.255 INFO:tasks.workunit.client.1.smithi040.stdout:3/30: creat d1/f8 x:0 0 0 2024-04-24T23:13:40.257 INFO:tasks.workunit.client.0.smithi031.stdout:4/434: dwrite d3/d23/f3a [0,4194304] 0 2024-04-24T23:13:40.263 INFO:tasks.workunit.client.1.smithi040.stdout:0/40: getdents . 0 2024-04-24T23:13:40.264 INFO:tasks.workunit.client.1.smithi040.stdout:0/41: dread - f7 zero size 2024-04-24T23:13:40.265 INFO:tasks.workunit.client.1.smithi040.stdout:9/30: rename l0 to d7/lb 0 2024-04-24T23:13:40.266 INFO:tasks.workunit.client.1.smithi040.stdout:9/31: stat d7 0 2024-04-24T23:13:40.267 INFO:tasks.workunit.client.0.smithi031.stdout:5/302: rmdir d0/d6/d3b/d4b/d1d/d2e/d28 39 2024-04-24T23:13:40.267 INFO:tasks.workunit.client.0.smithi031.stdout:5/303: dread - d0/d6/f40 zero size 2024-04-24T23:13:40.268 INFO:tasks.workunit.client.1.smithi040.stdout:6/31: mkdir d7 0 2024-04-24T23:13:40.269 INFO:tasks.workunit.client.1.smithi040.stdout:7/32: truncate f0 72758 0 2024-04-24T23:13:40.270 INFO:tasks.workunit.client.1.smithi040.stdout:2/37: dwrite f3 [0,4194304] 0 2024-04-24T23:13:40.271 INFO:tasks.workunit.client.1.smithi040.stdout:6/32: write f4 [184313,23733] 0 2024-04-24T23:13:40.271 INFO:tasks.workunit.client.0.smithi031.stdout:2/271: creat d5/f5c x:0 0 0 2024-04-24T23:13:40.272 INFO:tasks.workunit.client.1.smithi040.stdout:2/38: chown f3 260001 1 2024-04-24T23:13:40.272 INFO:tasks.workunit.client.1.smithi040.stdout:0/42: dread f6 [0,4194304] 0 2024-04-24T23:13:40.272 INFO:tasks.workunit.client.1.smithi040.stdout:6/33: write f0 [2284094,32170] 0 2024-04-24T23:13:40.273 INFO:tasks.workunit.client.1.smithi040.stdout:2/39: write f3 [589085,73057] 0 2024-04-24T23:13:40.274 INFO:tasks.workunit.client.0.smithi031.stdout:3/350: mkdir d6/d14/d71 0 2024-04-24T23:13:40.274 INFO:tasks.workunit.client.1.smithi040.stdout:0/43: readlink l1 0 2024-04-24T23:13:40.274 INFO:tasks.workunit.client.0.smithi031.stdout:5/304: dwrite d0/d6/d32/f46 [0,4194304] 0 2024-04-24T23:13:40.278 INFO:tasks.workunit.client.1.smithi040.stdout:9/32: write f4 [809941,52558] 0 2024-04-24T23:13:40.279 INFO:tasks.workunit.client.0.smithi031.stdout:8/432: creat d8/d15/d37/d43/d7b/d87/fa0 x:0 0 0 2024-04-24T23:13:40.282 INFO:tasks.workunit.client.1.smithi040.stdout:5/39: write d2/f6 [1480667,24535] 0 2024-04-24T23:13:40.283 INFO:tasks.workunit.client.0.smithi031.stdout:4/435: creat d3/d20/d2d/d44/f8c x:0 0 0 2024-04-24T23:13:40.284 INFO:tasks.workunit.client.0.smithi031.stdout:5/305: dwrite d0/d7/f58 [0,4194304] 0 2024-04-24T23:13:40.285 INFO:tasks.workunit.client.1.smithi040.stdout:6/34: dwrite f4 [0,4194304] 0 2024-04-24T23:13:40.285 INFO:tasks.workunit.client.0.smithi031.stdout:4/436: write d3/f26 [4909426,70773] 0 2024-04-24T23:13:40.287 INFO:tasks.workunit.client.0.smithi031.stdout:4/437: dread - d3/d20/d2d/d44/d5b/d5d/f87 zero size 2024-04-24T23:13:40.288 INFO:tasks.workunit.client.0.smithi031.stdout:4/438: fdatasync d3/d20/d2d/f52 0 2024-04-24T23:13:40.309 INFO:tasks.workunit.client.1.smithi040.stdout:6/35: fdatasync f4 0 2024-04-24T23:13:40.312 INFO:tasks.workunit.client.1.smithi040.stdout:6/36: dread f5 [0,4194304] 0 2024-04-24T23:13:40.316 INFO:tasks.workunit.client.1.smithi040.stdout:6/37: dread f4 [0,4194304] 0 2024-04-24T23:13:40.320 INFO:tasks.workunit.client.1.smithi040.stdout:6/38: dread f4 [0,4194304] 0 2024-04-24T23:13:40.341 INFO:tasks.workunit.client.1.smithi040.stdout:0/44: rename f7 to fa 0 2024-04-24T23:13:40.348 INFO:tasks.workunit.client.0.smithi031.stdout:2/272: dwrite d5/d8/d15/f34 [0,4194304] 0 2024-04-24T23:13:40.354 INFO:tasks.workunit.client.1.smithi040.stdout:9/33: mknod d7/cc 0 2024-04-24T23:13:40.356 INFO:tasks.workunit.client.1.smithi040.stdout:5/40: creat d2/d7/fe x:0 0 0 2024-04-24T23:13:40.358 INFO:tasks.workunit.client.1.smithi040.stdout:0/45: dwrite f8 [4194304,4194304] 0 2024-04-24T23:13:40.364 INFO:tasks.workunit.client.0.smithi031.stdout:5/306: creat d0/d6/d12/f63 x:0 0 0 2024-04-24T23:13:40.374 INFO:tasks.workunit.client.0.smithi031.stdout:6/467: getdents d13/d1b/d32/d9c 0 2024-04-24T23:13:40.383 INFO:tasks.workunit.client.1.smithi040.stdout:3/31: getdents d1 0 2024-04-24T23:13:40.383 INFO:tasks.workunit.client.1.smithi040.stdout:3/32: read - d1/f8 zero size 2024-04-24T23:13:40.384 INFO:tasks.workunit.client.0.smithi031.stdout:7/292: getdents d1/d1c/d2c/d38 0 2024-04-24T23:13:40.391 INFO:tasks.workunit.client.0.smithi031.stdout:0/388: getdents da/d20 0 2024-04-24T23:13:40.394 INFO:tasks.workunit.client.0.smithi031.stdout:7/293: unlink d1/d2/d1e/f54 0 2024-04-24T23:13:40.395 INFO:tasks.workunit.client.0.smithi031.stdout:6/468: symlink d13/d15/d1d/d23/d24/d43/la7 0 2024-04-24T23:13:40.397 INFO:tasks.workunit.client.0.smithi031.stdout:5/307: symlink d0/d6/d3b/d4b/dc/l64 0 2024-04-24T23:13:40.398 INFO:tasks.workunit.client.0.smithi031.stdout:5/308: write d0/d6/d32/f17 [2954509,125567] 0 2024-04-24T23:13:40.399 INFO:tasks.workunit.client.0.smithi031.stdout:0/389: mknod da/d22/c7c 0 2024-04-24T23:13:40.399 INFO:tasks.workunit.client.0.smithi031.stdout:5/309: chown d0/d6/d12/l13 332273344 1 2024-04-24T23:13:40.400 INFO:tasks.workunit.client.0.smithi031.stdout:0/390: truncate da/ff 4801436 0 2024-04-24T23:13:40.401 INFO:tasks.workunit.client.0.smithi031.stdout:5/310: fsync d0/d6/d3b/d4b/dc/d25/d53/f5a 0 2024-04-24T23:13:40.404 INFO:tasks.workunit.client.0.smithi031.stdout:7/294: rmdir d1/d1c/d2c/d38 39 2024-04-24T23:13:40.405 INFO:tasks.workunit.client.0.smithi031.stdout:0/391: dread da/d18/d1a/f29 [0,4194304] 0 2024-04-24T23:13:40.406 INFO:tasks.workunit.client.0.smithi031.stdout:0/392: write da/d20/f52 [618467,37047] 0 2024-04-24T23:13:40.408 INFO:tasks.workunit.client.0.smithi031.stdout:6/469: mknod d13/d1b/d32/d9c/ca8 0 2024-04-24T23:13:40.410 INFO:tasks.workunit.client.0.smithi031.stdout:6/470: read d13/d15/d1d/d23/d4d/d7b/f82 [444026,29639] 0 2024-04-24T23:13:40.411 INFO:tasks.workunit.client.0.smithi031.stdout:6/471: dread - d13/d15/d1d/d23/d24/d8b/f9d zero size 2024-04-24T23:13:40.422 INFO:tasks.workunit.client.0.smithi031.stdout:6/472: mknod d13/d15/d1d/d23/d24/d63/ca9 0 2024-04-24T23:13:40.426 INFO:tasks.workunit.client.0.smithi031.stdout:5/311: symlink d0/l65 0 2024-04-24T23:13:40.428 INFO:tasks.workunit.client.0.smithi031.stdout:1/292: rename d1/da/l3f to d1/dc/d11/l5e 0 2024-04-24T23:13:40.429 INFO:tasks.workunit.client.0.smithi031.stdout:7/295: creat d1/d45/f5b x:0 0 0 2024-04-24T23:13:40.434 INFO:tasks.workunit.client.0.smithi031.stdout:6/473: creat d13/d15/d1d/d23/d44/d47/d69/faa x:0 0 0 2024-04-24T23:13:40.434 INFO:tasks.workunit.client.0.smithi031.stdout:6/474: chown d13/c34 15 1 2024-04-24T23:13:40.437 INFO:tasks.workunit.client.0.smithi031.stdout:5/312: symlink d0/d6/d3b/d4b/dc/d25/d53/l66 0 2024-04-24T23:13:40.438 INFO:tasks.workunit.client.0.smithi031.stdout:2/273: rmdir d5/d6/df 39 2024-04-24T23:13:40.442 INFO:tasks.workunit.client.0.smithi031.stdout:9/449: rename d2/d1b/d14/d1d/d45/d56/d67 to d2/d1b/d14/d78/d84 0 2024-04-24T23:13:40.442 INFO:tasks.workunit.client.0.smithi031.stdout:6/475: dwrite d13/d15/d1d/d23/d44/d47/d69/f89 [0,4194304] 0 2024-04-24T23:13:40.443 INFO:tasks.workunit.client.0.smithi031.stdout:6/476: stat d13/d75/f39 0 2024-04-24T23:13:40.451 INFO:tasks.workunit.client.0.smithi031.stdout:5/313: unlink d0/d6/d3b/d4b/dc/d25/f29 0 2024-04-24T23:13:40.453 INFO:tasks.workunit.client.0.smithi031.stdout:2/274: creat d5/d8/d45/d5b/d49/d2f/f5d x:0 0 0 2024-04-24T23:13:40.455 INFO:tasks.workunit.client.0.smithi031.stdout:2/275: truncate d5/d6/d29/d26/f3f 50050 0 2024-04-24T23:13:40.457 INFO:tasks.workunit.client.0.smithi031.stdout:4/439: rename d3/d23/d69/f33 to d3/d5e/f8d 0 2024-04-24T23:13:40.460 INFO:tasks.workunit.client.0.smithi031.stdout:9/450: unlink d2/d1b/d14/d78/d84/d4a/c6a 0 2024-04-24T23:13:40.460 INFO:tasks.workunit.client.0.smithi031.stdout:7/296: rmdir d1/d2/dd 39 2024-04-24T23:13:40.463 INFO:tasks.workunit.client.1.smithi040.stdout:2/40: dwrite f5 [0,4194304] 0 2024-04-24T23:13:40.464 INFO:tasks.workunit.client.0.smithi031.stdout:0/393: getdents da/d20/d4e 0 2024-04-24T23:13:40.466 INFO:tasks.workunit.client.0.smithi031.stdout:0/394: write da/d18/d1a/d37/f5e [587734,36580] 0 2024-04-24T23:13:40.466 INFO:tasks.workunit.client.0.smithi031.stdout:5/314: creat d0/d6/d3b/d4b/dc/d25/f67 x:0 0 0 2024-04-24T23:13:40.477 INFO:tasks.workunit.client.0.smithi031.stdout:6/477: mkdir d13/d15/d1d/d23/d2d/d9e/dab 0 2024-04-24T23:13:40.477 INFO:tasks.workunit.client.0.smithi031.stdout:6/478: chown d13/d15/f76 14002824 1 2024-04-24T23:13:40.478 INFO:tasks.workunit.client.0.smithi031.stdout:6/479: readlink d13/d75/l4f 0 2024-04-24T23:13:40.482 INFO:tasks.workunit.client.0.smithi031.stdout:6/480: dread d13/d15/d1d/d23/d4d/d7b/f82 [0,4194304] 0 2024-04-24T23:13:40.485 INFO:tasks.workunit.client.0.smithi031.stdout:4/440: symlink d3/d23/d37/d86/l8e 0 2024-04-24T23:13:40.486 INFO:tasks.workunit.client.0.smithi031.stdout:0/395: dread f6 [0,4194304] 0 2024-04-24T23:13:40.490 INFO:tasks.workunit.client.0.smithi031.stdout:4/441: mknod d3/d20/d25/d60/c8f 0 2024-04-24T23:13:40.493 INFO:tasks.workunit.client.0.smithi031.stdout:9/451: dread d2/d16/f2a [0,4194304] 0 2024-04-24T23:13:40.495 INFO:tasks.workunit.client.0.smithi031.stdout:0/396: mkdir da/d18/d5a/d7d 0 2024-04-24T23:13:40.499 INFO:tasks.workunit.client.0.smithi031.stdout:5/315: getdents d0/d7/d56 0 2024-04-24T23:13:40.502 INFO:tasks.workunit.client.0.smithi031.stdout:0/397: mknod da/d20/c7e 0 2024-04-24T23:13:40.503 INFO:tasks.workunit.client.0.smithi031.stdout:8/433: truncate d8/d1a/f2f 2528984 0 2024-04-24T23:13:40.508 INFO:tasks.workunit.client.0.smithi031.stdout:5/316: creat d0/d6/d3b/f68 x:0 0 0 2024-04-24T23:13:40.511 INFO:tasks.workunit.client.0.smithi031.stdout:0/398: creat da/d11/f7f x:0 0 0 2024-04-24T23:13:40.512 INFO:tasks.workunit.client.0.smithi031.stdout:0/399: write da/d11/f59 [65900,50709] 0 2024-04-24T23:13:40.513 INFO:tasks.workunit.client.0.smithi031.stdout:8/434: mknod d8/d1a/d1d/d35/ca1 0 2024-04-24T23:13:40.514 INFO:tasks.workunit.client.0.smithi031.stdout:8/435: dread - d8/d15/d37/d62/f7f zero size 2024-04-24T23:13:40.521 INFO:tasks.workunit.client.0.smithi031.stdout:8/436: dwrite d8/d15/d37/f85 [0,4194304] 0 2024-04-24T23:13:40.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.520+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fb8000bf0 con 0x7f5fc406b860 2024-04-24T23:13:40.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.521+0000 7f5fcf7fe700 1 -- 172.21.15.31:0/1873298614 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f5fb8000bf0 con 0x7f5fc406b860 2024-04-24T23:13:40.523 INFO:tasks.workunit.client.0.smithi031.stdout:8/437: write d8/d1a/d1d/d35/d58/f64 [495260,76994] 0 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5fc406b860 msgr2=0x7f5fc406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5fc406b860 0x7f5fc406dd10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000b0d0 tx=0x7f5fc0009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fbd40 msgr2=0x7f5fd810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fbd40 0x7f5fd810d960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800eb40 tx=0x7f5fc800b040 comp rx=0 tx=0).stop 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 shutdown_connections 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5fc406b860 0x7f5fc406dd10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5fd80fbd40 0x7f5fd810d960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 --2- 172.21.15.31:0/1873298614 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5fd80fcff0 0x7f5fd810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 >> 172.21.15.31:0/1873298614 conn(0x7f5fd80f74c0 msgr2=0x7f5fd80f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:40.529 INFO:tasks.workunit.client.0.smithi031.stdout:0/400: symlink da/l80 0 2024-04-24T23:13:40.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 shutdown_connections 2024-04-24T23:13:40.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.526+0000 7f5fdd370700 1 -- 172.21.15.31:0/1873298614 wait complete. 2024-04-24T23:13:40.531 INFO:tasks.workunit.client.0.smithi031.stdout:8/438: mkdir d8/d15/d37/d43/da2 0 2024-04-24T23:13:40.534 INFO:tasks.workunit.client.0.smithi031.stdout:8/439: chown d8/d1a/d1d/d35/d58/f44 34 1 2024-04-24T23:13:40.536 INFO:tasks.workunit.client.0.smithi031.stdout:0/401: dwrite da/d18/d1a/d37/f5f [0,4194304] 0 2024-04-24T23:13:40.544 INFO:tasks.workunit.client.0.smithi031.stdout:8/440: creat d8/d15/d37/d62/d76/d9d/fa3 x:0 0 0 2024-04-24T23:13:40.545 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:13:40.545 INFO:tasks.workunit.client.0.smithi031.stdout:0/402: symlink da/d18/d1a/d76/l81 0 2024-04-24T23:13:40.549 INFO:tasks.workunit.client.0.smithi031.stdout:8/441: symlink d8/d15/d37/d43/d7b/la4 0 2024-04-24T23:13:40.549 INFO:tasks.workunit.client.0.smithi031.stdout:8/442: readlink l4 0 2024-04-24T23:13:40.554 INFO:tasks.workunit.client.0.smithi031.stdout:8/443: mkdir d8/d15/d37/d43/da5 0 2024-04-24T23:13:40.561 INFO:tasks.workunit.client.0.smithi031.stdout:8/444: dwrite d8/d15/d37/f5d [0,4194304] 0 2024-04-24T23:13:40.562 INFO:tasks.workunit.client.0.smithi031.stdout:8/445: write d8/f22 [216749,78281] 0 2024-04-24T23:13:40.563 INFO:tasks.workunit.client.0.smithi031.stdout:8/446: readlink d8/d1a/d1d/l59 0 2024-04-24T23:13:40.564 INFO:tasks.workunit.client.0.smithi031.stdout:0/403: dread f3 [0,4194304] 0 2024-04-24T23:13:40.565 INFO:tasks.workunit.client.0.smithi031.stdout:0/404: dread - da/d18/d1a/d37/f75 zero size 2024-04-24T23:13:40.569 INFO:tasks.workunit.client.0.smithi031.stdout:6/481: rmdir d13/d15/d1d/d23/d44/d47/d69 39 2024-04-24T23:13:40.574 INFO:tasks.workunit.client.0.smithi031.stdout:0/405: mkdir da/d18/d1a/d76/d82 0 2024-04-24T23:13:40.580 INFO:tasks.workunit.client.0.smithi031.stdout:0/406: creat da/d18/d1a/d37/d41/f83 x:0 0 0 2024-04-24T23:13:40.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/482: fdatasync f9 0 2024-04-24T23:13:40.583 INFO:tasks.workunit.client.0.smithi031.stdout:1/293: dwrite d1/dc/fe [0,4194304] 0 2024-04-24T23:13:40.591 INFO:tasks.workunit.client.0.smithi031.stdout:6/483: dwrite d13/d15/d1d/d23/d44/f4e [0,4194304] 0 2024-04-24T23:13:40.591 INFO:tasks.workunit.client.0.smithi031.stdout:1/294: dread - d1/dc/d11/f24 zero size 2024-04-24T23:13:40.591 INFO:tasks.workunit.client.0.smithi031.stdout:0/407: dread da/d18/d1a/d37/f23 [0,4194304] 0 2024-04-24T23:13:40.601 INFO:tasks.workunit.client.0.smithi031.stdout:0/408: rename da/d18/d1a/d37/d41/l61 to da/d18/d1a/d76/l84 0 2024-04-24T23:13:40.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/409: truncate da/d11/f33 4505250 0 2024-04-24T23:13:40.603 INFO:tasks.workunit.client.0.smithi031.stdout:0/410: fdatasync da/d20/f44 0 2024-04-24T23:13:40.609 INFO:tasks.workunit.client.0.smithi031.stdout:0/411: dwrite da/d18/d1a/d37/f75 [0,4194304] 0 2024-04-24T23:13:40.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.665+0000 7f0669664700 1 -- 172.21.15.31:0/547514755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 msgr2=0x7f06640fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:40.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.665+0000 7f0669664700 1 --2- 172.21.15.31:0/547514755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 0x7f06640fd270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0654004940 tx=0x7f0654009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:40.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.665+0000 7f0669664700 1 -- 172.21.15.31:0/547514755 shutdown_connections 2024-04-24T23:13:40.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.665+0000 7f0669664700 1 --2- 172.21.15.31:0/547514755 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06640fdfd0 0x7f06640fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.665+0000 7f0669664700 1 --2- 172.21.15.31:0/547514755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 0x7f06640fd270 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.665+0000 7f0669664700 1 -- 172.21.15.31:0/547514755 >> 172.21.15.31:0/547514755 conn(0x7f06640f8540 msgr2=0x7f06640fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:40.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.665+0000 7f0669664700 1 -- 172.21.15.31:0/547514755 shutdown_connections 2024-04-24T23:13:40.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.665+0000 7f0669664700 1 -- 172.21.15.31:0/547514755 wait complete. 2024-04-24T23:13:40.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.666+0000 7f0669664700 1 Processor -- start 2024-04-24T23:13:40.668 INFO:tasks.workunit.client.0.smithi031.stdout:7/297: rmdir d1 39 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.666+0000 7f0669664700 1 -- start start 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f0669664700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 0x7f0664071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f0669664700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06640fdfd0 0x7f06640702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f0669664700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0664072200 con 0x7f06640fce60 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f0669664700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0664111610 con 0x7f06640fdfd0 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f0662ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 0x7f0664071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f0662ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 0x7f0664071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58254/0 (socket says 172.21.15.31:58254) 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f0662ffd700 1 -- 172.21.15.31:0/1125775029 learned_addr learned my addr 172.21.15.31:0/1125775029 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f06627fc700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06640fdfd0 0x7f06640702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f06627fc700 1 -- 172.21.15.31:0/1125775029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 msgr2=0x7f0664071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f06627fc700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 0x7f0664071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:40.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.667+0000 7f06627fc700 1 -- 172.21.15.31:0/1125775029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654005040 con 0x7f06640fdfd0 2024-04-24T23:13:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.668+0000 7f06627fc700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06640fdfd0 0x7f06640702b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0658002820 tx=0x7f0658004570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.668+0000 7f064b7fe700 1 -- 172.21.15.31:0/1125775029 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0658007cc0 con 0x7f06640fdfd0 2024-04-24T23:13:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.668+0000 7f064b7fe700 1 -- 172.21.15.31:0/1125775029 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f065800f040 con 0x7f06640fdfd0 2024-04-24T23:13:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.668+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06641118c0 con 0x7f06640fdfd0 2024-04-24T23:13:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.668+0000 7f064b7fe700 1 -- 172.21.15.31:0/1125775029 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0658013920 con 0x7f06640fdfd0 2024-04-24T23:13:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.668+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0664111e80 con 0x7f06640fdfd0 2024-04-24T23:13:40.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.669+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066404c710 con 0x7f06640fdfd0 2024-04-24T23:13:40.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.669+0000 7f064b7fe700 1 -- 172.21.15.31:0/1125775029 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f0658013ac0 con 0x7f06640fdfd0 2024-04-24T23:13:40.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.670+0000 7f064b7fe700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f064c06b7a0 0x7f064c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:40.672 INFO:tasks.workunit.client.0.smithi031.stdout:1/295: symlink d1/dc/d4c/d4e/l5f 0 2024-04-24T23:13:40.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.670+0000 7f064b7fe700 1 -- 172.21.15.31:0/1125775029 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f065808a120 con 0x7f06640fdfd0 2024-04-24T23:13:40.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.670+0000 7f0662ffd700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f064c06b7a0 0x7f064c06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:40.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.671+0000 7f0662ffd700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f064c06b7a0 0x7f064c06dc50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0654004f90 tx=0x7f0654009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:40.675 INFO:tasks.workunit.client.0.smithi031.stdout:6/484: unlink d13/d15/d1d/d23/d44/d47/d69/f89 0 2024-04-24T23:13:40.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:40.675+0000 7f064b7fe700 1 -- 172.21.15.31:0/1125775029 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0658057710 con 0x7f06640fdfd0 2024-04-24T23:13:40.677 INFO:tasks.workunit.client.0.smithi031.stdout:6/485: write d13/d15/f27 [2237076,82084] 0 2024-04-24T23:13:40.679 INFO:tasks.workunit.client.0.smithi031.stdout:6/486: chown d13/d15/d1d/d23/d24/d8b 17813 1 2024-04-24T23:13:40.679 INFO:tasks.workunit.client.0.smithi031.stdout:1/296: dwrite d1/dc/d11/d21/f4b [0,4194304] 0 2024-04-24T23:13:40.681 INFO:tasks.workunit.client.0.smithi031.stdout:1/297: read d1/d33/f48 [43333,73045] 0 2024-04-24T23:13:40.681 INFO:tasks.workunit.client.0.smithi031.stdout:0/412: creat da/d18/f85 x:0 0 0 2024-04-24T23:13:40.683 INFO:tasks.workunit.client.0.smithi031.stdout:0/413: write da/f3f [3702340,85332] 0 2024-04-24T23:13:40.698 INFO:tasks.workunit.client.1.smithi040.stdout:7/33: link c4 c9 0 2024-04-24T23:13:40.698 INFO:tasks.workunit.client.1.smithi040.stdout:7/34: stat c7 0 2024-04-24T23:13:40.700 INFO:tasks.workunit.client.0.smithi031.stdout:0/414: fsync da/d18/d1a/f1e 0 2024-04-24T23:13:40.701 INFO:tasks.workunit.client.1.smithi040.stdout:0/46: creat fb x:0 0 0 2024-04-24T23:13:40.704 INFO:tasks.workunit.client.1.smithi040.stdout:5/41: rename d2/lc to d2/lf 0 2024-04-24T23:13:40.704 INFO:tasks.workunit.client.1.smithi040.stdout:5/42: read d2/f6 [1761505,78752] 0 2024-04-24T23:13:40.705 INFO:tasks.workunit.client.0.smithi031.stdout:4/442: write d3/d23/d69/d3f/f57 [636538,65024] 0 2024-04-24T23:13:40.708 INFO:tasks.workunit.client.1.smithi040.stdout:3/33: mkdir d1/d9 0 2024-04-24T23:13:40.709 INFO:tasks.workunit.client.0.smithi031.stdout:6/487: truncate d13/d15/d1d/d23/d2d/f90 1141971 0 2024-04-24T23:13:40.710 INFO:tasks.workunit.client.0.smithi031.stdout:9/452: write d2/d1b/d14/f39 [526842,118556] 0 2024-04-24T23:13:40.711 INFO:tasks.workunit.client.0.smithi031.stdout:4/443: dread d3/d20/d25/d60/d63/f81 [0,4194304] 0 2024-04-24T23:13:40.711 INFO:tasks.workunit.client.1.smithi040.stdout:5/43: dwrite d2/d7/fe [0,4194304] 0 2024-04-24T23:13:40.712 INFO:tasks.workunit.client.0.smithi031.stdout:4/444: stat d3/d20/d25/d60/d63 0 2024-04-24T23:13:40.714 INFO:tasks.workunit.client.1.smithi040.stdout:7/35: mknod ca 0 2024-04-24T23:13:40.718 INFO:tasks.workunit.client.0.smithi031.stdout:0/415: write f6 [2255689,68308] 0 2024-04-24T23:13:40.718 INFO:tasks.workunit.client.1.smithi040.stdout:9/34: creat d7/da/fd x:0 0 0 2024-04-24T23:13:40.719 INFO:tasks.workunit.client.0.smithi031.stdout:9/453: dwrite d2/d1b/d14/d1d/f7b [0,4194304] 0 2024-04-24T23:13:40.722 INFO:tasks.workunit.client.1.smithi040.stdout:0/47: unlink fa 0 2024-04-24T23:13:40.730 INFO:tasks.workunit.client.1.smithi040.stdout:3/34: fsync f0 0 2024-04-24T23:13:40.730 INFO:tasks.workunit.client.1.smithi040.stdout:3/35: stat f0 0 2024-04-24T23:13:40.734 INFO:tasks.workunit.client.1.smithi040.stdout:5/44: mkdir d2/d10 0 2024-04-24T23:13:40.734 INFO:tasks.workunit.client.0.smithi031.stdout:5/317: truncate d0/d6/d32/f17 1687748 0 2024-04-24T23:13:40.736 INFO:tasks.workunit.client.0.smithi031.stdout:7/298: rename d1/c4a to d1/d2/dd/d33/c5c 0 2024-04-24T23:13:40.738 INFO:tasks.workunit.client.1.smithi040.stdout:7/36: mkdir db 0 2024-04-24T23:13:40.739 INFO:tasks.workunit.client.1.smithi040.stdout:9/35: creat d7/fe x:0 0 0 2024-04-24T23:13:40.740 INFO:tasks.workunit.client.1.smithi040.stdout:9/36: dread - d7/fe zero size 2024-04-24T23:13:40.740 INFO:tasks.workunit.client.1.smithi040.stdout:0/48: fsync f6 0 2024-04-24T23:13:40.741 INFO:tasks.workunit.client.0.smithi031.stdout:0/416: mkdir da/d86 0 2024-04-24T23:13:40.743 INFO:tasks.workunit.client.0.smithi031.stdout:5/318: creat d0/d6/d3b/d4b/dc/d25/f69 x:0 0 0 2024-04-24T23:13:40.745 INFO:tasks.workunit.client.0.smithi031.stdout:5/319: dread - d0/f49 zero size 2024-04-24T23:13:40.746 INFO:tasks.workunit.client.0.smithi031.stdout:4/445: rename d3/d23/d37/f80 to d3/d20/d25/f90 0 2024-04-24T23:13:40.747 INFO:tasks.workunit.client.0.smithi031.stdout:5/320: write d0/d6/d12/f31 [353731,41036] 0 2024-04-24T23:13:40.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:40 smithi031 ceph-mon[26411]: pgmap v23: 65 pgs: 65 active+clean; 689 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.3 MiB/s rd, 61 MiB/s wr, 240 op/s 2024-04-24T23:13:40.749 INFO:tasks.workunit.client.0.smithi031.stdout:5/321: stat d0/d6/d32/c3e 0 2024-04-24T23:13:40.749 INFO:tasks.workunit.client.0.smithi031.stdout:0/417: dwrite da/d11/d6a/f6e [0,4194304] 0 2024-04-24T23:13:40.749 INFO:tasks.workunit.client.1.smithi040.stdout:5/45: mkdir d2/d7/d8/d11 0 2024-04-24T23:13:40.750 INFO:tasks.workunit.client.0.smithi031.stdout:4/446: read d3/d23/d69/d3f/f61 [747791,119692] 0 2024-04-24T23:13:40.751 INFO:tasks.workunit.client.1.smithi040.stdout:7/37: readlink l8 0 2024-04-24T23:13:40.752 INFO:tasks.workunit.client.0.smithi031.stdout:5/322: write d0/d6/d3b/f4d [687424,41401] 0 2024-04-24T23:13:40.755 INFO:tasks.workunit.client.0.smithi031.stdout:5/323: write d0/d6/d12/f21 [4757208,63147] 0 2024-04-24T23:13:40.756 INFO:tasks.workunit.client.1.smithi040.stdout:9/37: chown l1 17848059 1 2024-04-24T23:13:40.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/447: rmdir d8/d15/d37/d62 39 2024-04-24T23:13:40.762 INFO:tasks.workunit.client.1.smithi040.stdout:3/36: mkdir d1/d9/da 0 2024-04-24T23:13:40.765 INFO:tasks.workunit.client.0.smithi031.stdout:8/448: dwrite d8/d1a/f7e [0,4194304] 0 2024-04-24T23:13:40.766 INFO:tasks.workunit.client.1.smithi040.stdout:3/37: dread f0 [0,4194304] 0 2024-04-24T23:13:40.767 INFO:tasks.workunit.client.1.smithi040.stdout:7/38: dread f2 [0,4194304] 0 2024-04-24T23:13:40.768 INFO:tasks.workunit.client.0.smithi031.stdout:7/299: dread d1/d2/d1e/f36 [0,4194304] 0 2024-04-24T23:13:40.768 INFO:tasks.workunit.client.0.smithi031.stdout:4/447: symlink d3/d23/d37/l91 0 2024-04-24T23:13:40.770 INFO:tasks.workunit.client.0.smithi031.stdout:4/448: fsync d3/d20/d25/d7e/f83 0 2024-04-24T23:13:40.771 INFO:tasks.workunit.client.1.smithi040.stdout:9/38: rename f6 to d7/ff 0 2024-04-24T23:13:40.772 INFO:tasks.workunit.client.0.smithi031.stdout:4/449: fdatasync d3/d20/d25/d7e/f83 0 2024-04-24T23:13:40.772 INFO:tasks.workunit.client.0.smithi031.stdout:4/450: write d3/f2b [847347,102516] 0 2024-04-24T23:13:40.776 INFO:tasks.workunit.client.0.smithi031.stdout:5/324: symlink d0/d6/d3b/d4b/l6a 0 2024-04-24T23:13:40.777 INFO:tasks.workunit.client.0.smithi031.stdout:5/325: fsync d0/d6/d32/f46 0 2024-04-24T23:13:40.781 INFO:tasks.workunit.client.0.smithi031.stdout:5/326: dread d0/d6/d12/f21 [4194304,4194304] 0 2024-04-24T23:13:40.781 INFO:tasks.workunit.client.1.smithi040.stdout:9/39: mknod d7/c10 0 2024-04-24T23:13:40.783 INFO:tasks.workunit.client.1.smithi040.stdout:0/49: link l5 lc 0 2024-04-24T23:13:40.783 INFO:tasks.workunit.client.1.smithi040.stdout:0/50: chown f8 144880691 1 2024-04-24T23:13:40.784 INFO:tasks.workunit.client.0.smithi031.stdout:8/449: dread - d8/d15/d37/d62/f7f zero size 2024-04-24T23:13:40.785 INFO:tasks.workunit.client.1.smithi040.stdout:5/46: link d2/c3 d2/d7/d8/c12 0 2024-04-24T23:13:40.785 INFO:tasks.workunit.client.0.smithi031.stdout:8/450: fdatasync d8/d15/f2a 0 2024-04-24T23:13:40.786 INFO:tasks.workunit.client.1.smithi040.stdout:5/47: chown d2/d10 319411 1 2024-04-24T23:13:40.786 INFO:tasks.workunit.client.0.smithi031.stdout:8/451: stat d8/d15/d37/d43/c75 0 2024-04-24T23:13:40.787 INFO:tasks.workunit.client.1.smithi040.stdout:7/39: symlink db/lc 0 2024-04-24T23:13:40.791 INFO:tasks.workunit.client.1.smithi040.stdout:0/51: mknod cd 0 2024-04-24T23:13:40.793 INFO:tasks.workunit.client.1.smithi040.stdout:7/40: creat db/fd x:0 0 0 2024-04-24T23:13:40.794 INFO:tasks.workunit.client.1.smithi040.stdout:7/41: dread f2 [0,4194304] 0 2024-04-24T23:13:40.794 INFO:tasks.workunit.client.1.smithi040.stdout:0/52: dread f2 [4194304,4194304] 0 2024-04-24T23:13:40.794 INFO:tasks.workunit.client.1.smithi040.stdout:0/53: stat cd 0 2024-04-24T23:13:40.795 INFO:tasks.workunit.client.1.smithi040.stdout:7/42: dread - db/fd zero size 2024-04-24T23:13:40.795 INFO:tasks.workunit.client.1.smithi040.stdout:0/54: chown fb 6 1 2024-04-24T23:13:40.795 INFO:tasks.workunit.client.1.smithi040.stdout:0/55: fsync f8 0 2024-04-24T23:13:40.796 INFO:tasks.workunit.client.0.smithi031.stdout:9/454: rename d2/d1b/d14/d78/d7f to d2/d1b/d85 0 2024-04-24T23:13:40.798 INFO:tasks.workunit.client.1.smithi040.stdout:7/43: fdatasync f2 0 2024-04-24T23:13:40.799 INFO:tasks.workunit.client.0.smithi031.stdout:5/327: rmdir d0/d6/d3b/d4b/dc 39 2024-04-24T23:13:40.803 INFO:tasks.workunit.client.1.smithi040.stdout:0/56: readlink lc 0 2024-04-24T23:13:40.805 INFO:tasks.workunit.client.0.smithi031.stdout:8/452: unlink d8/d1a/d1d/d35/d58/d5b/f79 0 2024-04-24T23:13:40.807 INFO:tasks.workunit.client.0.smithi031.stdout:7/300: creat d1/d2/d1e/d4f/f5d x:0 0 0 2024-04-24T23:13:40.808 INFO:tasks.workunit.client.1.smithi040.stdout:7/44: rename c4 to db/ce 0 2024-04-24T23:13:40.809 INFO:tasks.workunit.client.0.smithi031.stdout:0/418: link da/d18/d1a/d76/l81 da/d18/d5a/d7d/l87 0 2024-04-24T23:13:40.810 INFO:tasks.workunit.client.0.smithi031.stdout:0/419: read da/fb [13242,53727] 0 2024-04-24T23:13:40.810 INFO:tasks.workunit.client.0.smithi031.stdout:0/420: readlink da/d18/d1a/l6b 0 2024-04-24T23:13:40.811 INFO:tasks.workunit.client.1.smithi040.stdout:0/57: rename f9 to fe 0 2024-04-24T23:13:40.812 INFO:tasks.workunit.client.0.smithi031.stdout:4/451: truncate d3/d20/d2d/d44/f2c 1709724 0 2024-04-24T23:13:40.816 INFO:tasks.workunit.client.1.smithi040.stdout:0/58: mkdir df 0 2024-04-24T23:13:40.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/328: mkdir d0/d6/d32/d6b 0 2024-04-24T23:13:40.819 INFO:tasks.workunit.client.0.smithi031.stdout:5/329: read - d0/d6/d12/f63 zero size 2024-04-24T23:13:40.820 INFO:tasks.workunit.client.0.smithi031.stdout:8/453: mkdir d8/d1a/d73/da6 0 2024-04-24T23:13:40.822 INFO:tasks.workunit.client.1.smithi040.stdout:0/59: symlink df/l10 0 2024-04-24T23:13:40.823 INFO:tasks.workunit.client.1.smithi040.stdout:0/60: write f8 [8528828,5568] 0 2024-04-24T23:13:40.824 INFO:tasks.workunit.client.0.smithi031.stdout:1/298: write d1/da/d10/f4d [3305825,109866] 0 2024-04-24T23:13:40.825 INFO:tasks.workunit.client.0.smithi031.stdout:4/452: creat d3/d20/d2d/f92 x:0 0 0 2024-04-24T23:13:40.826 INFO:tasks.workunit.client.1.smithi040.stdout:0/61: mkdir df/d11 0 2024-04-24T23:13:40.827 INFO:tasks.workunit.client.0.smithi031.stdout:5/330: mknod d0/d7/c6c 0 2024-04-24T23:13:40.828 INFO:tasks.workunit.client.0.smithi031.stdout:7/301: fsync d1/d2/f3b 0 2024-04-24T23:13:40.829 INFO:tasks.workunit.client.1.smithi040.stdout:0/62: readlink l5 0 2024-04-24T23:13:40.830 INFO:tasks.workunit.client.0.smithi031.stdout:1/299: dread d1/dc/d11/d21/f28 [0,4194304] 0 2024-04-24T23:13:40.831 INFO:tasks.workunit.client.0.smithi031.stdout:0/421: creat da/d22/d62/f88 x:0 0 0 2024-04-24T23:13:40.831 INFO:tasks.workunit.client.0.smithi031.stdout:0/422: readlink da/d18/l2c 0 2024-04-24T23:13:40.831 INFO:tasks.workunit.client.0.smithi031.stdout:4/453: read d3/d23/d69/d3f/f61 [2648564,128173] 0 2024-04-24T23:13:40.832 INFO:tasks.workunit.client.0.smithi031.stdout:9/455: rmdir d2/d1b/d79 0 2024-04-24T23:13:40.833 INFO:tasks.workunit.client.0.smithi031.stdout:4/454: write d3/d23/d3b/f56 [1091028,67223] 0 2024-04-24T23:13:40.834 INFO:tasks.workunit.client.0.smithi031.stdout:8/454: mknod d8/d15/d37/d43/da5/ca7 0 2024-04-24T23:13:40.835 INFO:tasks.workunit.client.1.smithi040.stdout:0/63: dwrite fb [0,4194304] 0 2024-04-24T23:13:40.835 INFO:tasks.workunit.client.1.smithi040.stdout:0/64: write f6 [4087641,27175] 0 2024-04-24T23:13:40.836 INFO:tasks.workunit.client.0.smithi031.stdout:8/455: write d8/f70 [5292053,110863] 0 2024-04-24T23:13:40.836 INFO:tasks.workunit.client.0.smithi031.stdout:7/302: mkdir d1/d2/dd/d33/d5e 0 2024-04-24T23:13:40.839 INFO:tasks.workunit.client.0.smithi031.stdout:8/456: truncate d8/d15/d37/d62/f81 688879 0 2024-04-24T23:13:40.839 INFO:tasks.workunit.client.0.smithi031.stdout:0/423: write da/d18/d1a/d37/f3d [5063372,58174] 0 2024-04-24T23:13:40.840 INFO:tasks.workunit.client.1.smithi040.stdout:0/65: dread f8 [0,4194304] 0 2024-04-24T23:13:40.841 INFO:tasks.workunit.client.0.smithi031.stdout:9/456: creat d2/d1b/d85/f86 x:0 0 0 2024-04-24T23:13:40.842 INFO:tasks.workunit.client.0.smithi031.stdout:4/455: mknod d3/d20/d2d/d44/d5b/d5d/c93 0 2024-04-24T23:13:40.843 INFO:tasks.workunit.client.0.smithi031.stdout:9/457: dread - d2/d1b/d14/d78/d84/d4a/f76 zero size 2024-04-24T23:13:40.844 INFO:tasks.workunit.client.0.smithi031.stdout:4/456: chown d3/d20/d25/d60/d63/l6b 36032974 1 2024-04-24T23:13:40.845 INFO:tasks.workunit.client.0.smithi031.stdout:9/458: truncate d2/d1b/f2f 845618 0 2024-04-24T23:13:40.846 INFO:tasks.workunit.client.0.smithi031.stdout:9/459: write d2/f8 [2771212,5056] 0 2024-04-24T23:13:40.846 INFO:tasks.workunit.client.0.smithi031.stdout:7/303: rename d1/d1c/f29 to d1/d2/d1e/f5f 0 2024-04-24T23:13:40.848 INFO:tasks.workunit.client.0.smithi031.stdout:7/304: fdatasync d1/d1c/d2c/f4c 0 2024-04-24T23:13:40.850 INFO:tasks.workunit.client.0.smithi031.stdout:0/424: dread da/d22/f30 [0,4194304] 0 2024-04-24T23:13:40.852 INFO:tasks.workunit.client.0.smithi031.stdout:8/457: creat d8/fa8 x:0 0 0 2024-04-24T23:13:40.853 INFO:tasks.workunit.client.1.smithi040.stdout:0/66: fsync fb 0 2024-04-24T23:13:40.853 INFO:tasks.workunit.client.1.smithi040.stdout:9/40: dread f2 [0,4194304] 0 2024-04-24T23:13:40.855 INFO:tasks.workunit.client.1.smithi040.stdout:9/41: write d7/fe [417035,16723] 0 2024-04-24T23:13:40.855 INFO:tasks.workunit.client.0.smithi031.stdout:8/458: chown d8/d15/d37/f85 122406690 1 2024-04-24T23:13:40.856 INFO:tasks.workunit.client.0.smithi031.stdout:0/425: symlink da/d18/d1a/d37/l89 0 2024-04-24T23:13:40.856 INFO:tasks.workunit.client.0.smithi031.stdout:9/460: dwrite d2/d1b/d14/d78/d84/d4a/f71 [0,4194304] 0 2024-04-24T23:13:40.858 INFO:tasks.workunit.client.1.smithi040.stdout:0/67: mkdir df/d12 0 2024-04-24T23:13:40.858 INFO:tasks.workunit.client.1.smithi040.stdout:0/68: fdatasync fb 0 2024-04-24T23:13:40.862 INFO:tasks.workunit.client.1.smithi040.stdout:9/42: rename d7/c10 to d7/da/c11 0 2024-04-24T23:13:40.863 INFO:tasks.workunit.client.0.smithi031.stdout:4/457: getdents d3/d20 0 2024-04-24T23:13:40.865 INFO:tasks.workunit.client.1.smithi040.stdout:0/69: fdatasync f8 0 2024-04-24T23:13:40.866 INFO:tasks.workunit.client.1.smithi040.stdout:0/70: write f8 [7130344,23692] 0 2024-04-24T23:13:40.867 INFO:tasks.workunit.client.1.smithi040.stdout:9/43: mknod d7/c12 0 2024-04-24T23:13:40.869 INFO:tasks.workunit.client.1.smithi040.stdout:2/41: dwrite f7 [0,4194304] 0 2024-04-24T23:13:40.872 INFO:tasks.workunit.client.0.smithi031.stdout:6/488: write d13/d15/d1d/d23/d24/f50 [298893,40491] 0 2024-04-24T23:13:40.873 INFO:tasks.workunit.client.1.smithi040.stdout:2/42: dread f6 [0,4194304] 0 2024-04-24T23:13:40.877 INFO:tasks.workunit.client.1.smithi040.stdout:9/44: rename d7/lb to d7/da/l13 0 2024-04-24T23:13:40.882 INFO:tasks.workunit.client.0.smithi031.stdout:6/489: dwrite d13/d15/d1d/d23/d2d/f61 [0,4194304] 0 2024-04-24T23:13:40.884 INFO:tasks.workunit.client.0.smithi031.stdout:7/305: rename d1/d1c/d2c/d38 to d1/d2/d60 0 2024-04-24T23:13:40.885 INFO:tasks.workunit.client.1.smithi040.stdout:0/71: creat df/d11/f13 x:0 0 0 2024-04-24T23:13:40.886 INFO:tasks.workunit.client.1.smithi040.stdout:0/72: chown f6 181816108 1 2024-04-24T23:13:40.888 INFO:tasks.workunit.client.1.smithi040.stdout:2/43: creat f8 x:0 0 0 2024-04-24T23:13:40.890 INFO:tasks.workunit.client.1.smithi040.stdout:9/45: symlink d7/da/l14 0 2024-04-24T23:13:40.893 INFO:tasks.workunit.client.0.smithi031.stdout:8/459: rename d8/d15/d37/d43/d7b/d86/c9f to d8/d15/ca9 0 2024-04-24T23:13:40.894 INFO:tasks.workunit.client.1.smithi040.stdout:0/73: write fe [1031975,107388] 0 2024-04-24T23:13:40.896 INFO:tasks.workunit.client.1.smithi040.stdout:2/44: rename f0 to f9 0 2024-04-24T23:13:40.897 INFO:tasks.workunit.client.1.smithi040.stdout:0/74: dread f2 [4194304,4194304] 0 2024-04-24T23:13:40.897 INFO:tasks.workunit.client.0.smithi031.stdout:9/461: getdents d2/d1b/d14/d18/d7d 0 2024-04-24T23:13:40.898 INFO:tasks.workunit.client.1.smithi040.stdout:0/75: write f6 [1443398,5244] 0 2024-04-24T23:13:40.899 INFO:tasks.workunit.client.1.smithi040.stdout:9/46: creat d7/f15 x:0 0 0 2024-04-24T23:13:40.903 INFO:tasks.workunit.client.0.smithi031.stdout:0/426: rename da/d18/d5a/f69 to da/d11/d6a/f8a 0 2024-04-24T23:13:40.904 INFO:tasks.workunit.client.0.smithi031.stdout:0/427: fsync da/d11/f7f 0 2024-04-24T23:13:40.905 INFO:tasks.workunit.client.1.smithi040.stdout:2/45: mkdir da 0 2024-04-24T23:13:40.906 INFO:tasks.workunit.client.1.smithi040.stdout:9/47: dwrite d7/f15 [0,4194304] 0 2024-04-24T23:13:40.906 INFO:tasks.workunit.client.0.smithi031.stdout:7/306: symlink d1/d2/dd/d33/d5e/l61 0 2024-04-24T23:13:40.909 INFO:tasks.workunit.client.0.smithi031.stdout:9/462: creat d2/d1b/d14/d78/d84/d4a/d4b/f87 x:0 0 0 2024-04-24T23:13:40.910 INFO:tasks.workunit.client.1.smithi040.stdout:0/76: mkdir df/d14 0 2024-04-24T23:13:40.911 INFO:tasks.workunit.client.1.smithi040.stdout:0/77: stat cd 0 2024-04-24T23:13:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:40 smithi040 ceph-mon[33128]: pgmap v23: 65 pgs: 65 active+clean; 689 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 5.3 MiB/s rd, 61 MiB/s wr, 240 op/s 2024-04-24T23:13:40.912 INFO:tasks.workunit.client.0.smithi031.stdout:8/460: mknod d8/d1a/d1d/caa 0 2024-04-24T23:13:40.913 INFO:tasks.workunit.client.1.smithi040.stdout:9/48: dread d7/fe [0,4194304] 0 2024-04-24T23:13:40.917 INFO:tasks.workunit.client.1.smithi040.stdout:2/46: rename f6 to da/fb 0 2024-04-24T23:13:40.917 INFO:tasks.workunit.client.1.smithi040.stdout:2/47: readlink - no filename 2024-04-24T23:13:40.918 INFO:tasks.workunit.client.1.smithi040.stdout:0/78: dwrite f6 [0,4194304] 0 2024-04-24T23:13:40.918 INFO:tasks.workunit.client.0.smithi031.stdout:0/428: mknod da/d18/d5a/c8b 0 2024-04-24T23:13:40.920 INFO:tasks.workunit.client.0.smithi031.stdout:0/429: write da/d18/d1a/d37/d41/f83 [803888,115741] 0 2024-04-24T23:13:40.923 INFO:tasks.workunit.client.0.smithi031.stdout:7/307: truncate d1/d2/dd/f23 351152 0 2024-04-24T23:13:40.928 INFO:tasks.workunit.client.1.smithi040.stdout:9/49: write f2 [873939,101021] 0 2024-04-24T23:13:40.929 INFO:tasks.workunit.client.0.smithi031.stdout:0/430: dwrite f6 [0,4194304] 0 2024-04-24T23:13:40.933 INFO:tasks.workunit.client.1.smithi040.stdout:0/79: readlink lc 0 2024-04-24T23:13:40.933 INFO:tasks.workunit.client.1.smithi040.stdout:0/80: chown l5 2893 1 2024-04-24T23:13:40.934 INFO:tasks.workunit.client.1.smithi040.stdout:7/45: rmdir db 39 2024-04-24T23:13:40.935 INFO:tasks.workunit.client.1.smithi040.stdout:2/48: mknod da/cc 0 2024-04-24T23:13:40.936 INFO:tasks.workunit.client.1.smithi040.stdout:2/49: chown f5 5637193 1 2024-04-24T23:13:40.939 INFO:tasks.workunit.client.1.smithi040.stdout:9/50: mkdir d7/d16 0 2024-04-24T23:13:40.942 INFO:tasks.workunit.client.1.smithi040.stdout:0/81: symlink df/d11/l15 0 2024-04-24T23:13:40.945 INFO:tasks.workunit.client.1.smithi040.stdout:2/50: symlink da/ld 0 2024-04-24T23:13:40.945 INFO:tasks.workunit.client.0.smithi031.stdout:3/351: sync 2024-04-24T23:13:40.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/276: sync 2024-04-24T23:13:40.948 INFO:tasks.workunit.client.1.smithi040.stdout:9/51: mkdir d7/da/d17 0 2024-04-24T23:13:40.949 INFO:tasks.workunit.client.1.smithi040.stdout:7/46: mknod db/cf 0 2024-04-24T23:13:40.950 INFO:tasks.workunit.client.1.smithi040.stdout:7/47: dread - f6 zero size 2024-04-24T23:13:40.950 INFO:tasks.workunit.client.1.smithi040.stdout:7/48: dread - f6 zero size 2024-04-24T23:13:40.953 INFO:tasks.workunit.client.1.smithi040.stdout:9/52: rename d7/c12 to d7/da/c18 0 2024-04-24T23:13:40.956 INFO:tasks.workunit.client.0.smithi031.stdout:1/300: write d1/dc/d11/d21/f28 [1254101,111563] 0 2024-04-24T23:13:40.957 INFO:tasks.workunit.client.0.smithi031.stdout:1/301: fdatasync d1/dc/fe 0 2024-04-24T23:13:40.959 INFO:tasks.workunit.client.0.smithi031.stdout:1/302: truncate d1/dc/d11/d21/f28 1445854 0 2024-04-24T23:13:40.960 INFO:tasks.workunit.client.0.smithi031.stdout:7/308: creat d1/d2/dd/d33/f62 x:0 0 0 2024-04-24T23:13:40.962 INFO:tasks.workunit.client.0.smithi031.stdout:1/303: dread d1/dc/d11/d21/f28 [0,4194304] 0 2024-04-24T23:13:40.964 INFO:tasks.workunit.client.1.smithi040.stdout:9/53: rename d7/cc to d7/d16/c19 0 2024-04-24T23:13:40.970 INFO:tasks.workunit.client.0.smithi031.stdout:3/352: unlink d6/d15/d29/l32 0 2024-04-24T23:13:40.976 INFO:tasks.workunit.client.0.smithi031.stdout:7/309: mkdir d1/d45/d63 0 2024-04-24T23:13:40.979 INFO:tasks.workunit.client.0.smithi031.stdout:0/431: link da/d18/d1a/d37/f4a da/d18/d5a/d7d/f8c 0 2024-04-24T23:13:40.980 INFO:tasks.workunit.client.0.smithi031.stdout:0/432: chown da/d18/d1a/d37/f75 12093138 1 2024-04-24T23:13:40.981 INFO:tasks.workunit.client.0.smithi031.stdout:1/304: symlink d1/d3e/l60 0 2024-04-24T23:13:40.983 INFO:tasks.workunit.client.0.smithi031.stdout:2/277: mkdir d5/d28/d4e/d5e 0 2024-04-24T23:13:40.985 INFO:tasks.workunit.client.0.smithi031.stdout:2/278: chown d5/d6/d29/l2e 857664 1 2024-04-24T23:13:40.986 INFO:tasks.workunit.client.0.smithi031.stdout:2/279: stat d5/f13 0 2024-04-24T23:13:40.987 INFO:tasks.workunit.client.0.smithi031.stdout:0/433: dwrite da/d11/d6a/f79 [0,4194304] 0 2024-04-24T23:13:40.991 INFO:tasks.workunit.client.0.smithi031.stdout:1/305: rmdir d1/da/d10 39 2024-04-24T23:13:40.994 INFO:tasks.workunit.client.0.smithi031.stdout:4/458: truncate d3/d23/d3b/f41 3996100 0 2024-04-24T23:13:41.002 INFO:tasks.workunit.client.0.smithi031.stdout:0/434: rename da/d18/d5a/d7d/f8c to da/d18/d1a/d76/f8d 0 2024-04-24T23:13:41.004 INFO:tasks.workunit.client.0.smithi031.stdout:4/459: symlink d3/d20/l94 0 2024-04-24T23:13:41.008 INFO:tasks.workunit.client.0.smithi031.stdout:0/435: mkdir da/d20/d8e 0 2024-04-24T23:13:41.009 INFO:tasks.workunit.client.0.smithi031.stdout:0/436: dread - da/d22/d62/f88 zero size 2024-04-24T23:13:41.010 INFO:tasks.workunit.client.0.smithi031.stdout:1/306: mkdir d1/da/d61 0 2024-04-24T23:13:41.011 INFO:tasks.workunit.client.0.smithi031.stdout:1/307: read d1/dc/d11/d21/f4b [3298047,30865] 0 2024-04-24T23:13:41.011 INFO:tasks.workunit.client.1.smithi040.stdout:9/54: write d7/f15 [5133530,29583] 0 2024-04-24T23:13:41.013 INFO:tasks.workunit.client.0.smithi031.stdout:2/280: creat d5/d8/d15/f5f x:0 0 0 2024-04-24T23:13:41.017 INFO:tasks.workunit.client.1.smithi040.stdout:9/55: symlink d7/d16/l1a 0 2024-04-24T23:13:41.017 INFO:tasks.workunit.client.0.smithi031.stdout:4/460: dwrite d3/f5c [0,4194304] 0 2024-04-24T23:13:41.022 INFO:tasks.workunit.client.0.smithi031.stdout:0/437: dwrite da/d18/d1a/d37/f4a [0,4194304] 0 2024-04-24T23:13:41.023 INFO:tasks.workunit.client.1.smithi040.stdout:9/56: mkdir d7/d1b 0 2024-04-24T23:13:41.030 INFO:tasks.workunit.client.1.smithi040.stdout:9/57: dwrite d7/da/fd [0,4194304] 0 2024-04-24T23:13:41.033 INFO:tasks.workunit.client.0.smithi031.stdout:2/281: creat d5/d28/d4e/d5e/f60 x:0 0 0 2024-04-24T23:13:41.035 INFO:tasks.workunit.client.1.smithi040.stdout:9/58: dread d7/da/fd [0,4194304] 0 2024-04-24T23:13:41.035 INFO:tasks.workunit.client.0.smithi031.stdout:4/461: truncate d3/fc 2708531 0 2024-04-24T23:13:41.035 INFO:tasks.workunit.client.1.smithi040.stdout:9/59: chown d7/ff 588897 1 2024-04-24T23:13:41.036 INFO:tasks.workunit.client.0.smithi031.stdout:1/308: mkdir d1/da/d10/d52/d62 0 2024-04-24T23:13:41.040 INFO:tasks.workunit.client.1.smithi040.stdout:9/60: stat d7/da/l13 0 2024-04-24T23:13:41.040 INFO:tasks.workunit.client.1.smithi040.stdout:9/61: chown f4 864 1 2024-04-24T23:13:41.043 INFO:tasks.workunit.client.1.smithi040.stdout:9/62: write d7/da/fd [183955,83496] 0 2024-04-24T23:13:41.044 INFO:tasks.workunit.client.1.smithi040.stdout:9/63: write f2 [3444,117631] 0 2024-04-24T23:13:41.045 INFO:tasks.workunit.client.0.smithi031.stdout:4/462: mkdir d3/d20/d25/d95 0 2024-04-24T23:13:41.046 INFO:tasks.workunit.client.0.smithi031.stdout:4/463: chown d3/d20/d25/d60/c8f 1790 1 2024-04-24T23:13:41.047 INFO:tasks.workunit.client.0.smithi031.stdout:4/464: write d3/f85 [585565,31994] 0 2024-04-24T23:13:41.048 INFO:tasks.workunit.client.0.smithi031.stdout:1/309: dread d1/dc/d11/d21/f28 [0,4194304] 0 2024-04-24T23:13:41.049 INFO:tasks.workunit.client.0.smithi031.stdout:4/465: readlink d3/d20/d25/d60/d63/l6b 0 2024-04-24T23:13:41.050 INFO:tasks.workunit.client.0.smithi031.stdout:4/466: chown d3/d20/d2d/d44/d5b 14 1 2024-04-24T23:13:41.051 INFO:tasks.workunit.client.0.smithi031.stdout:2/282: rename d5/d6/df/f4c to d5/d6/d29/f61 0 2024-04-24T23:13:41.052 INFO:tasks.workunit.client.0.smithi031.stdout:2/283: chown d5/d6/f1a 4634 1 2024-04-24T23:13:41.052 INFO:tasks.workunit.client.1.smithi040.stdout:9/64: symlink d7/d16/l1c 0 2024-04-24T23:13:41.053 INFO:tasks.workunit.client.1.smithi040.stdout:9/65: write f2 [765497,2556] 0 2024-04-24T23:13:41.053 INFO:tasks.workunit.client.1.smithi040.stdout:7/49: write f0 [27904,412] 0 2024-04-24T23:13:41.055 INFO:tasks.workunit.client.0.smithi031.stdout:1/310: mknod d1/d3e/c63 0 2024-04-24T23:13:41.057 INFO:tasks.workunit.client.1.smithi040.stdout:2/51: truncate f3 3518043 0 2024-04-24T23:13:41.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.057+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0664111a50 con 0x7f064c06b7a0 2024-04-24T23:13:41.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.058+0000 7f064b7fe700 1 -- 172.21.15.31:0/1125775029 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f0664111a50 con 0x7f064c06b7a0 2024-04-24T23:13:41.060 INFO:tasks.workunit.client.0.smithi031.stdout:2/284: dwrite d5/d28/d43/f55 [0,4194304] 0 2024-04-24T23:13:41.060 INFO:tasks.workunit.client.0.smithi031.stdout:1/311: unlink d1/d3e/f4f 0 2024-04-24T23:13:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f064c06b7a0 msgr2=0x7f064c06dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f064c06b7a0 0x7f064c06dc50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0654004f90 tx=0x7f0654009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06640fdfd0 msgr2=0x7f06640702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06640fdfd0 0x7f06640702b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0658002820 tx=0x7f0658004570 comp rx=0 tx=0).stop 2024-04-24T23:13:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 shutdown_connections 2024-04-24T23:13:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f064c06b7a0 0x7f064c06dc50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06640fce60 0x7f0664071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 --2- 172.21.15.31:0/1125775029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f06640fdfd0 0x7f06640702b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 >> 172.21.15.31:0/1125775029 conn(0x7f06640f8540 msgr2=0x7f06640f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:41.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.061+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 shutdown_connections 2024-04-24T23:13:41.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.062+0000 7f0669664700 1 -- 172.21.15.31:0/1125775029 wait complete. 2024-04-24T23:13:41.065 INFO:tasks.workunit.client.0.smithi031.stdout:3/353: rename d6/d15 to d6/d72 0 2024-04-24T23:13:41.068 INFO:tasks.workunit.client.1.smithi040.stdout:2/52: unlink da/ld 0 2024-04-24T23:13:41.069 INFO:tasks.workunit.client.1.smithi040.stdout:2/53: readlink - no filename 2024-04-24T23:13:41.071 INFO:tasks.workunit.client.1.smithi040.stdout:2/54: dread f7 [0,4194304] 0 2024-04-24T23:13:41.072 INFO:tasks.workunit.client.1.smithi040.stdout:7/50: link f0 db/f10 0 2024-04-24T23:13:41.074 INFO:tasks.workunit.client.1.smithi040.stdout:2/55: dread f9 [0,4194304] 0 2024-04-24T23:13:41.075 INFO:tasks.workunit.client.1.smithi040.stdout:9/66: symlink d7/da/d17/l1d 0 2024-04-24T23:13:41.075 INFO:tasks.workunit.client.1.smithi040.stdout:2/56: write f8 [42510,117262] 0 2024-04-24T23:13:41.077 INFO:tasks.workunit.client.1.smithi040.stdout:7/51: mknod db/c11 0 2024-04-24T23:13:41.079 INFO:tasks.workunit.client.1.smithi040.stdout:9/67: mkdir d7/d16/d1e 0 2024-04-24T23:13:41.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/461: dwrite d8/d1a/f2f [0,4194304] 0 2024-04-24T23:13:41.083 INFO:tasks.workunit.client.1.smithi040.stdout:2/57: rename da/fb to da/fe 0 2024-04-24T23:13:41.092 INFO:tasks.workunit.client.1.smithi040.stdout:2/58: fdatasync da/fe 0 2024-04-24T23:13:41.093 INFO:tasks.workunit.client.0.smithi031.stdout:1/312: symlink d1/da/d61/l64 0 2024-04-24T23:13:41.094 INFO:tasks.workunit.client.1.smithi040.stdout:7/52: dwrite db/f10 [0,4194304] 0 2024-04-24T23:13:41.094 INFO:tasks.workunit.client.1.smithi040.stdout:2/59: write f7 [613152,27824] 0 2024-04-24T23:13:41.095 INFO:tasks.workunit.client.0.smithi031.stdout:8/462: symlink d8/d15/d37/d62/d76/d9d/lab 0 2024-04-24T23:13:41.095 INFO:tasks.workunit.client.1.smithi040.stdout:2/60: chown f5 741997 1 2024-04-24T23:13:41.098 INFO:tasks.workunit.client.0.smithi031.stdout:1/313: chown d1/dc/d11/l5e 1930160646 1 2024-04-24T23:13:41.099 INFO:tasks.workunit.client.1.smithi040.stdout:2/61: dread f7 [0,4194304] 0 2024-04-24T23:13:41.099 INFO:tasks.workunit.client.1.smithi040.stdout:7/53: mkdir db/d12 0 2024-04-24T23:13:41.099 INFO:tasks.workunit.client.1.smithi040.stdout:7/54: dread - db/fd zero size 2024-04-24T23:13:41.104 INFO:tasks.workunit.client.1.smithi040.stdout:2/62: unlink f5 0 2024-04-24T23:13:41.105 INFO:tasks.workunit.client.0.smithi031.stdout:1/314: read d1/dc/d11/f20 [1857002,119676] 0 2024-04-24T23:13:41.107 INFO:tasks.workunit.client.0.smithi031.stdout:7/310: dwrite d1/d2/dd/f23 [0,4194304] 0 2024-04-24T23:13:41.108 INFO:tasks.workunit.client.1.smithi040.stdout:7/55: rename l8 to db/d12/l13 0 2024-04-24T23:13:41.109 INFO:tasks.workunit.client.0.smithi031.stdout:8/463: dwrite d8/d1a/d1d/d35/f8f [0,4194304] 0 2024-04-24T23:13:41.109 INFO:tasks.workunit.client.0.smithi031.stdout:7/311: chown d1/d2/cc 1067114 1 2024-04-24T23:13:41.109 INFO:tasks.workunit.client.1.smithi040.stdout:7/56: truncate f5 880265 0 2024-04-24T23:13:41.109 INFO:tasks.workunit.client.1.smithi040.stdout:7/57: dread - f6 zero size 2024-04-24T23:13:41.110 INFO:tasks.workunit.client.1.smithi040.stdout:7/58: write f6 [3813,35277] 0 2024-04-24T23:13:41.115 INFO:tasks.workunit.client.0.smithi031.stdout:0/438: truncate da/d18/d1a/d37/d41/f47 2076457 0 2024-04-24T23:13:41.118 INFO:tasks.workunit.client.1.smithi040.stdout:2/63: read f9 [2239953,127283] 0 2024-04-24T23:13:41.118 INFO:tasks.workunit.client.1.smithi040.stdout:2/64: chown da 23713 1 2024-04-24T23:13:41.122 INFO:tasks.workunit.client.0.smithi031.stdout:7/312: write d1/f4b [324896,43462] 0 2024-04-24T23:13:41.123 INFO:tasks.workunit.client.0.smithi031.stdout:7/313: stat d1/d2/dd/f23 0 2024-04-24T23:13:41.124 INFO:tasks.workunit.client.1.smithi040.stdout:7/59: creat db/f14 x:0 0 0 2024-04-24T23:13:41.128 INFO:tasks.workunit.client.1.smithi040.stdout:2/65: symlink da/lf 0 2024-04-24T23:13:41.130 INFO:tasks.workunit.client.1.smithi040.stdout:7/60: symlink db/d12/l15 0 2024-04-24T23:13:41.130 INFO:tasks.workunit.client.0.smithi031.stdout:1/315: rmdir d1/d3e 39 2024-04-24T23:13:41.131 INFO:tasks.workunit.client.1.smithi040.stdout:7/61: read f6 [30707,118994] 0 2024-04-24T23:13:41.131 INFO:tasks.workunit.client.0.smithi031.stdout:4/467: write d3/d20/d2d/d44/f2c [2290188,27194] 0 2024-04-24T23:13:41.136 INFO:tasks.workunit.client.1.smithi040.stdout:2/66: dread f7 [0,4194304] 0 2024-04-24T23:13:41.138 INFO:tasks.workunit.client.0.smithi031.stdout:0/439: dwrite da/d11/f25 [4194304,4194304] 0 2024-04-24T23:13:41.139 INFO:tasks.workunit.client.1.smithi040.stdout:7/62: creat db/d12/f16 x:0 0 0 2024-04-24T23:13:41.139 INFO:tasks.workunit.client.1.smithi040.stdout:7/63: write db/f14 [789088,76887] 0 2024-04-24T23:13:41.141 INFO:tasks.workunit.client.1.smithi040.stdout:9/68: rmdir d7/da 39 2024-04-24T23:13:41.144 INFO:tasks.workunit.client.1.smithi040.stdout:2/67: mkdir da/d10 0 2024-04-24T23:13:41.146 INFO:tasks.workunit.client.0.smithi031.stdout:1/316: mkdir d1/da/d12/d65 0 2024-04-24T23:13:41.148 INFO:tasks.workunit.client.0.smithi031.stdout:3/354: write d6/d72/d1a/f22 [1765791,33157] 0 2024-04-24T23:13:41.151 INFO:tasks.workunit.client.0.smithi031.stdout:3/355: read d6/d14/d1d/d69/f6d [2970202,664] 0 2024-04-24T23:13:41.152 INFO:tasks.workunit.client.0.smithi031.stdout:7/314: unlink d1/f21 0 2024-04-24T23:13:41.153 INFO:tasks.workunit.client.1.smithi040.stdout:7/64: write f0 [1367239,107389] 0 2024-04-24T23:13:41.155 INFO:tasks.workunit.client.0.smithi031.stdout:2/285: truncate d5/f13 827510 0 2024-04-24T23:13:41.156 INFO:tasks.workunit.client.1.smithi040.stdout:2/68: creat da/f11 x:0 0 0 2024-04-24T23:13:41.158 INFO:tasks.workunit.client.0.smithi031.stdout:0/440: dread da/d18/d1a/d37/f26 [0,4194304] 0 2024-04-24T23:13:41.160 INFO:tasks.workunit.client.0.smithi031.stdout:8/464: rmdir d8/d1a/d73/da6 0 2024-04-24T23:13:41.161 INFO:tasks.workunit.client.1.smithi040.stdout:7/65: creat db/f17 x:0 0 0 2024-04-24T23:13:41.166 INFO:tasks.workunit.client.0.smithi031.stdout:4/468: link d3/d20/d2d/f39 d3/d20/d2d/d44/d5b/d5d/f96 0 2024-04-24T23:13:41.169 INFO:tasks.workunit.client.1.smithi040.stdout:7/66: creat db/f18 x:0 0 0 2024-04-24T23:13:41.169 INFO:tasks.workunit.client.1.smithi040.stdout:7/67: readlink db/lc 0 2024-04-24T23:13:41.188 INFO:tasks.workunit.client.1.smithi040.stdout:9/69: truncate d7/da/fd 1487095 0 2024-04-24T23:13:41.188 INFO:tasks.workunit.client.1.smithi040.stdout:2/69: dwrite f3 [0,4194304] 0 2024-04-24T23:13:41.188 INFO:tasks.workunit.client.1.smithi040.stdout:7/68: dwrite db/f10 [4194304,4194304] 0 2024-04-24T23:13:41.189 INFO:tasks.workunit.client.1.smithi040.stdout:2/70: stat da/fe 0 2024-04-24T23:13:41.190 INFO:tasks.workunit.client.1.smithi040.stdout:2/71: readlink da/lf 0 2024-04-24T23:13:41.190 INFO:tasks.workunit.client.1.smithi040.stdout:2/72: read f7 [124727,40978] 0 2024-04-24T23:13:41.192 INFO:tasks.workunit.client.1.smithi040.stdout:2/73: write f8 [270709,80336] 0 2024-04-24T23:13:41.193 INFO:tasks.workunit.client.1.smithi040.stdout:9/70: write d7/ff [757488,33273] 0 2024-04-24T23:13:41.193 INFO:tasks.workunit.client.1.smithi040.stdout:2/74: readlink da/lf 0 2024-04-24T23:13:41.194 INFO:tasks.workunit.client.1.smithi040.stdout:7/69: creat db/f19 x:0 0 0 2024-04-24T23:13:41.197 INFO:tasks.workunit.client.1.smithi040.stdout:9/71: unlink d7/d16/l1c 0 2024-04-24T23:13:41.198 INFO:tasks.workunit.client.1.smithi040.stdout:2/75: mknod da/d10/c12 0 2024-04-24T23:13:41.200 INFO:tasks.workunit.client.1.smithi040.stdout:2/76: symlink da/l13 0 2024-04-24T23:13:41.204 INFO:tasks.workunit.client.1.smithi040.stdout:2/77: rename f3 to da/f14 0 2024-04-24T23:13:41.204 INFO:tasks.workunit.client.1.smithi040.stdout:9/72: dwrite d7/ff [0,4194304] 0 2024-04-24T23:13:41.207 INFO:tasks.workunit.client.1.smithi040.stdout:2/78: mkdir da/d10/d15 0 2024-04-24T23:13:41.208 INFO:tasks.workunit.client.1.smithi040.stdout:9/73: mkdir d7/d16/d1e/d1f 0 2024-04-24T23:13:41.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.209+0000 7fdc5199c700 1 -- 172.21.15.31:0/1874644545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 msgr2=0x7fdc4c0fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.209+0000 7fdc5199c700 1 --2- 172.21.15.31:0/1874644545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 0x7fdc4c0fd400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdc40004940 tx=0x7fdc40009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:41.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.209+0000 7fdc5199c700 1 -- 172.21.15.31:0/1874644545 shutdown_connections 2024-04-24T23:13:41.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.209+0000 7fdc5199c700 1 --2- 172.21.15.31:0/1874644545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 0x7fdc4c0fd400 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.209+0000 7fdc5199c700 1 --2- 172.21.15.31:0/1874644545 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc4c0fbce0 0x7fdc4c0fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.209+0000 7fdc5199c700 1 -- 172.21.15.31:0/1874644545 >> 172.21.15.31:0/1874644545 conn(0x7fdc4c0f7480 msgr2=0x7fdc4c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:41.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.210+0000 7fdc5199c700 1 -- 172.21.15.31:0/1874644545 shutdown_connections 2024-04-24T23:13:41.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.210+0000 7fdc5199c700 1 -- 172.21.15.31:0/1874644545 wait complete. 2024-04-24T23:13:41.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.210+0000 7fdc5199c700 1 Processor -- start 2024-04-24T23:13:41.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.210+0000 7fdc5199c700 1 -- start start 2024-04-24T23:13:41.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.210+0000 7fdc5199c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc4c0fbce0 0x7fdc4c10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.210+0000 7fdc5199c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 0x7fdc4c10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.210+0000 7fdc5199c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c10e3f0 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.210+0000 7fdc5199c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c10e560 con 0x7fdc4c0fbce0 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc4a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 0x7fdc4c10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc4a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 0x7fdc4c10ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58260/0 (socket says 172.21.15.31:58260) 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc4a7fc700 1 -- 172.21.15.31:0/1294969751 learned_addr learned my addr 172.21.15.31:0/1294969751 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc4affd700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc4c0fbce0 0x7fdc4c10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc4a7fc700 1 -- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc4c0fbce0 msgr2=0x7fdc4c10d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc4a7fc700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc4c0fbce0 0x7fdc4c10d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc4a7fc700 1 -- 172.21.15.31:0/1294969751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc40005040 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc4a7fc700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 0x7fdc4c10ddf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdc40013870 tx=0x7fdc40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc337fe700 1 -- 172.21.15.31:0/1294969751 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc40021070 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c112f60 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc337fe700 1 -- 172.21.15.31:0/1294969751 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc40009430 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c113470 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.211+0000 7fdc337fe700 1 -- 172.21.15.31:0/1294969751 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4000e070 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.213+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c101b20 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.213+0000 7fdc337fe700 1 -- 172.21.15.31:0/1294969751 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7fdc4001c750 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.214+0000 7fdc337fe700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdc3406b730 0x7fdc3406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.214+0000 7fdc337fe700 1 -- 172.21.15.31:0/1294969751 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fdc4008b600 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.214+0000 7fdc4affd700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdc3406b730 0x7fdc3406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.214+0000 7fdc4affd700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdc3406b730 0x7fdc3406dbe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009b80 tx=0x7fdc3c00b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:41.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.219+0000 7fdc337fe700 1 -- 172.21.15.31:0/1294969751 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fdc40058d90 con 0x7fdc4c0fcf90 2024-04-24T23:13:41.226 INFO:tasks.workunit.client.1.smithi040.stdout:2/79: mknod da/d10/d15/c16 0 2024-04-24T23:13:41.227 INFO:tasks.workunit.client.0.smithi031.stdout:4/469: chown d3/d20/d2d/d44/d5b/d5d/f96 4 1 2024-04-24T23:13:41.227 INFO:tasks.workunit.client.1.smithi040.stdout:2/80: chown f8 0 1 2024-04-24T23:13:41.227 INFO:tasks.workunit.client.0.smithi031.stdout:7/315: truncate d1/d1c/f3c 521667 0 2024-04-24T23:13:41.228 INFO:tasks.workunit.client.1.smithi040.stdout:9/74: symlink d7/d16/d1e/l20 0 2024-04-24T23:13:41.228 INFO:tasks.workunit.client.0.smithi031.stdout:7/316: chown d1/d1c 160 1 2024-04-24T23:13:41.228 INFO:tasks.workunit.client.0.smithi031.stdout:4/470: stat d3/d20/d2d/f79 0 2024-04-24T23:13:41.230 INFO:tasks.workunit.client.1.smithi040.stdout:9/75: dread f2 [0,4194304] 0 2024-04-24T23:13:41.232 INFO:tasks.workunit.client.1.smithi040.stdout:2/81: dread f9 [0,4194304] 0 2024-04-24T23:13:41.234 INFO:tasks.workunit.client.0.smithi031.stdout:7/317: dread d1/f4b [0,4194304] 0 2024-04-24T23:13:41.235 INFO:tasks.workunit.client.1.smithi040.stdout:2/82: dread f7 [0,4194304] 0 2024-04-24T23:13:41.236 INFO:tasks.workunit.client.0.smithi031.stdout:0/441: dread da/f3f [4194304,4194304] 0 2024-04-24T23:13:41.239 INFO:tasks.workunit.client.0.smithi031.stdout:8/465: dread d8/d1a/d1d/d35/d58/d47/f49 [0,4194304] 0 2024-04-24T23:13:41.318 INFO:tasks.workunit.client.0.smithi031.stdout:3/356: truncate d6/df/d16/f1c 448778 0 2024-04-24T23:13:41.319 INFO:tasks.workunit.client.1.smithi040.stdout:7/70: truncate f5 765881 0 2024-04-24T23:13:41.319 INFO:tasks.workunit.client.1.smithi040.stdout:7/71: chown db/f14 633235 1 2024-04-24T23:13:41.320 INFO:tasks.workunit.client.1.smithi040.stdout:9/76: symlink d7/da/l21 0 2024-04-24T23:13:41.321 INFO:tasks.workunit.client.0.smithi031.stdout:2/286: creat d5/d3d/f62 x:0 0 0 2024-04-24T23:13:41.321 INFO:tasks.workunit.client.0.smithi031.stdout:2/287: readlink d5/d6/d29/l14 0 2024-04-24T23:13:41.323 INFO:tasks.workunit.client.1.smithi040.stdout:9/77: dread d7/ff [0,4194304] 0 2024-04-24T23:13:41.324 INFO:tasks.workunit.client.1.smithi040.stdout:7/72: rename db/d12/l15 to db/l1a 0 2024-04-24T23:13:41.329 INFO:tasks.workunit.client.0.smithi031.stdout:1/317: creat d1/f66 x:0 0 0 2024-04-24T23:13:41.330 INFO:tasks.workunit.client.0.smithi031.stdout:1/318: chown d1/da/d61 316810505 1 2024-04-24T23:13:41.332 INFO:tasks.workunit.client.0.smithi031.stdout:5/331: sync 2024-04-24T23:13:41.333 INFO:tasks.workunit.client.1.smithi040.stdout:7/73: mkdir db/d1b 0 2024-04-24T23:13:41.333 INFO:tasks.workunit.client.1.smithi040.stdout:7/74: write f0 [5356875,3663] 0 2024-04-24T23:13:41.336 INFO:tasks.workunit.client.1.smithi040.stdout:9/78: mknod d7/c22 0 2024-04-24T23:13:41.337 INFO:tasks.workunit.client.0.smithi031.stdout:1/319: dwrite d1/dc/d11/f38 [0,4194304] 0 2024-04-24T23:13:41.343 INFO:tasks.workunit.client.0.smithi031.stdout:7/318: read d1/f13 [1360280,106741] 0 2024-04-24T23:13:41.346 INFO:tasks.workunit.client.1.smithi040.stdout:7/75: creat db/f1c x:0 0 0 2024-04-24T23:13:41.347 INFO:tasks.workunit.client.0.smithi031.stdout:0/442: mknod da/d18/d1a/c8f 0 2024-04-24T23:13:41.358 INFO:tasks.workunit.client.0.smithi031.stdout:2/288: rename d5/d6/d29/f21 to d5/d6/d29/d26/f63 0 2024-04-24T23:13:41.358 INFO:tasks.workunit.client.1.smithi040.stdout:7/76: symlink db/d1b/l1d 0 2024-04-24T23:13:41.360 INFO:tasks.workunit.client.0.smithi031.stdout:4/471: link d3/d20/d2d/d44/f84 d3/d20/d2d/d44/d5b/d5d/f97 0 2024-04-24T23:13:41.360 INFO:tasks.workunit.client.0.smithi031.stdout:2/289: dread d5/d6/d29/d26/f63 [0,4194304] 0 2024-04-24T23:13:41.362 INFO:tasks.workunit.client.0.smithi031.stdout:5/332: chown d0/d6/d12/l41 3 1 2024-04-24T23:13:41.367 INFO:tasks.workunit.client.0.smithi031.stdout:1/320: mknod d1/dc/d4c/d4e/c67 0 2024-04-24T23:13:41.371 INFO:tasks.workunit.client.1.smithi040.stdout:9/79: write d7/fe [1048805,63052] 0 2024-04-24T23:13:41.374 INFO:tasks.workunit.client.0.smithi031.stdout:0/443: truncate da/d18/d1a/f1e 94491 0 2024-04-24T23:13:41.376 INFO:tasks.workunit.client.0.smithi031.stdout:8/466: mknod d8/d15/d37/cac 0 2024-04-24T23:13:41.378 INFO:tasks.workunit.client.0.smithi031.stdout:7/319: dwrite d1/d2/d1e/f5f [0,4194304] 0 2024-04-24T23:13:41.379 INFO:tasks.workunit.client.0.smithi031.stdout:7/320: read d1/d1c/f35 [631014,95574] 0 2024-04-24T23:13:41.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/472: creat d3/d5e/f98 x:0 0 0 2024-04-24T23:13:41.386 INFO:tasks.workunit.client.0.smithi031.stdout:4/473: readlink d3/d20/d25/d60/d63/l6b 0 2024-04-24T23:13:41.389 INFO:tasks.workunit.client.0.smithi031.stdout:2/290: mkdir d5/d28/d4e/d64 0 2024-04-24T23:13:41.391 INFO:tasks.workunit.client.0.smithi031.stdout:5/333: dread - d0/d6/d3b/d4b/dc/d25/f67 zero size 2024-04-24T23:13:41.396 INFO:tasks.workunit.client.0.smithi031.stdout:0/444: rename da/d18/d1a/d37/f65 to da/d11/f90 0 2024-04-24T23:13:41.396 INFO:tasks.workunit.client.0.smithi031.stdout:0/445: fdatasync da/d11/f33 0 2024-04-24T23:13:41.398 INFO:tasks.workunit.client.0.smithi031.stdout:8/467: mkdir d8/d15/d37/d43/d7b/d87/dad 0 2024-04-24T23:13:41.404 INFO:tasks.workunit.client.0.smithi031.stdout:7/321: mkdir d1/d2/dd/d33/d5e/d64 0 2024-04-24T23:13:41.410 INFO:tasks.workunit.client.0.smithi031.stdout:1/321: dread d1/f8 [0,4194304] 0 2024-04-24T23:13:41.414 INFO:tasks.workunit.client.0.smithi031.stdout:5/334: unlink d0/d7/f3a 0 2024-04-24T23:13:41.416 INFO:tasks.workunit.client.0.smithi031.stdout:2/291: dwrite d5/d6/f4d [0,4194304] 0 2024-04-24T23:13:41.416 INFO:tasks.workunit.client.0.smithi031.stdout:0/446: dread da/f19 [4194304,4194304] 0 2024-04-24T23:13:41.417 INFO:tasks.workunit.client.0.smithi031.stdout:5/335: chown d0/d6/d3b/d4b/dc/f44 30 1 2024-04-24T23:13:41.419 INFO:tasks.workunit.client.0.smithi031.stdout:5/336: write d0/d6/d32/f4 [4280982,77501] 0 2024-04-24T23:13:41.424 INFO:tasks.workunit.client.0.smithi031.stdout:9/463: sync 2024-04-24T23:13:41.425 INFO:tasks.workunit.client.0.smithi031.stdout:6/490: sync 2024-04-24T23:13:41.425 INFO:tasks.workunit.client.0.smithi031.stdout:9/464: fdatasync d2/d1b/d14/f39 0 2024-04-24T23:13:41.447 INFO:tasks.workunit.client.0.smithi031.stdout:3/357: getdents d6/d14/d1d/d59 0 2024-04-24T23:13:41.450 INFO:tasks.workunit.client.0.smithi031.stdout:7/322: dread d1/d2/dd/f17 [0,4194304] 0 2024-04-24T23:13:41.457 INFO:tasks.workunit.client.0.smithi031.stdout:7/323: dwrite d1/d45/f5b [0,4194304] 0 2024-04-24T23:13:41.459 INFO:tasks.workunit.client.0.smithi031.stdout:0/447: mknod da/d22/c91 0 2024-04-24T23:13:41.460 INFO:tasks.workunit.client.0.smithi031.stdout:0/448: write da/d20/d4e/f51 [69231,98672] 0 2024-04-24T23:13:41.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/337: unlink d0/d6/c1c 0 2024-04-24T23:13:41.473 INFO:tasks.workunit.client.0.smithi031.stdout:6/491: write d13/f7c [4677218,59805] 0 2024-04-24T23:13:41.484 INFO:tasks.workunit.client.0.smithi031.stdout:4/474: dwrite d3/d23/d69/f31 [0,4194304] 0 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:8/4: sync 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:8/5: truncate - no filename 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:8/6: write - no filename 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:8/7: rmdir - no directory 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:1/28: sync 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:6/39: sync 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:4/0: sync 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:1/29: readlink - no filename 2024-04-24T23:13:41.488 INFO:tasks.workunit.client.1.smithi040.stdout:8/8: chown . 353 1 2024-04-24T23:13:41.491 INFO:tasks.workunit.client.0.smithi031.stdout:0/449: mkdir da/d18/d1a/d37/d41/d92 0 2024-04-24T23:13:41.495 INFO:tasks.workunit.client.0.smithi031.stdout:8/468: link d8/d15/d37/cac d8/d1a/d1d/d35/d58/d47/cae 0 2024-04-24T23:13:41.501 INFO:tasks.workunit.client.0.smithi031.stdout:3/358: creat d6/d55/f73 x:0 0 0 2024-04-24T23:13:41.504 INFO:tasks.workunit.client.0.smithi031.stdout:1/322: link d1/dc/d4c/d4e/l5f d1/da/d10/d52/d62/l68 0 2024-04-24T23:13:41.506 INFO:tasks.workunit.client.1.smithi040.stdout:3/38: sync 2024-04-24T23:13:41.506 INFO:tasks.workunit.client.1.smithi040.stdout:5/48: sync 2024-04-24T23:13:41.507 INFO:tasks.workunit.client.1.smithi040.stdout:2/83: sync 2024-04-24T23:13:41.507 INFO:tasks.workunit.client.1.smithi040.stdout:0/82: sync 2024-04-24T23:13:41.507 INFO:tasks.workunit.client.0.smithi031.stdout:4/475: symlink d3/d23/d37/d86/l99 0 2024-04-24T23:13:41.507 INFO:tasks.workunit.client.0.smithi031.stdout:4/476: stat d3/d20/d2d/l55 0 2024-04-24T23:13:41.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.512+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc4c02b360 con 0x7fdc3406b730 2024-04-24T23:13:41.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/450: write da/d11/d6a/f8a [312221,72906] 0 2024-04-24T23:13:41.517 INFO:tasks.workunit.client.0.smithi031.stdout:0/451: truncate da/d22/f4b 10413 0 2024-04-24T23:13:41.519 INFO:tasks.workunit.client.0.smithi031.stdout:9/465: write d2/d1b/d14/d18/f21 [1372726,107529] 0 2024-04-24T23:13:41.520 INFO:tasks.workunit.client.0.smithi031.stdout:9/466: chown d2/d1b/d14/f46 445740842 1 2024-04-24T23:13:41.521 INFO:tasks.workunit.client.0.smithi031.stdout:0/452: dread da/d18/d1a/d37/f5e [0,4194304] 0 2024-04-24T23:13:41.525 INFO:tasks.workunit.client.0.smithi031.stdout:8/469: dread d8/d15/d37/d43/f4b [0,4194304] 0 2024-04-24T23:13:41.526 INFO:tasks.workunit.client.0.smithi031.stdout:4/477: dread d3/f2b [0,4194304] 0 2024-04-24T23:13:41.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.524+0000 7fdc337fe700 1 -- 172.21.15.31:0/1294969751 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fdc4c02b360 con 0x7fdc3406b730 2024-04-24T23:13:41.526 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:13:41.526 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (7m) 25s ago 8m 24.1M - 0.25.0 c8568f914cd2 87840bb14366 2024-04-24T23:13:41.526 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (8m) 25s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (7m) 32s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 25s ago 8m 84.1M - 9.4.7 954c08fa6188 44ecf082adf8 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (2m) 25s ago 2m 135M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (2m) 25s ago 2m 11.6M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (2m) 32s ago 2m 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (2m) 32s ago 2m 13.2M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:9283 running (9m) 25s ago 9m 322M - 17.2.7-902-gc61472c8 0de407c1b3af 7b9a4ca2c955 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (62s) 32s ago 7m 491M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8b888d3a97fe 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (9m) 25s ago 10m 47.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (7m) 32s ago 7m 38.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 starting - - - - 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (7m) 32s ago 7m 16.0M - 1.5.0 0da6a335fe13 7bd2678617bc 2024-04-24T23:13:41.527 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 25s ago 6m 95.4M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:13:41.528 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (6m) 25s ago 6m 99.8M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:13:41.528 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 25s ago 5m 87.7M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:13:41.528 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (5m) 32s ago 5m 104M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:13:41.528 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (4m) 32s ago 4m 99.5M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:13:41.528 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (4m) 32s ago 4m 93.9M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:13:41.528 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 25s ago 8m 58.2M - 2.43.0 a07b618ecd1d aac36626882b 2024-04-24T23:13:41.528 INFO:tasks.workunit.client.0.smithi031.stdout:1/323: chown d1/da/d10/l5b 46674442 1 2024-04-24T23:13:41.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdc3406b730 msgr2=0x7fdc3406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdc3406b730 0x7fdc3406dbe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c009b80 tx=0x7fdc3c00b040 comp rx=0 tx=0).stop 2024-04-24T23:13:41.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 msgr2=0x7fdc4c10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 0x7fdc4c10ddf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdc40013870 tx=0x7fdc40009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:41.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 shutdown_connections 2024-04-24T23:13:41.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fdc3406b730 0x7fdc3406dbe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc4c0fbce0 0x7fdc4c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 --2- 172.21.15.31:0/1294969751 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc4c0fcf90 0x7fdc4c10ddf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.527+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 >> 172.21.15.31:0/1294969751 conn(0x7fdc4c0f7480 msgr2=0x7fdc4c0f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:41.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.528+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 shutdown_connections 2024-04-24T23:13:41.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.528+0000 7fdc5199c700 1 -- 172.21.15.31:0/1294969751 wait complete. 2024-04-24T23:13:41.536 INFO:tasks.workunit.client.0.smithi031.stdout:5/338: dread d0/d6/d3b/d4b/d1d/d2e/d28/f30 [0,4194304] 0 2024-04-24T23:13:41.538 INFO:tasks.workunit.client.0.smithi031.stdout:0/453: mknod da/d11/d6a/c93 0 2024-04-24T23:13:41.545 INFO:tasks.workunit.client.0.smithi031.stdout:7/324: truncate d1/d2/f2a 420419 0 2024-04-24T23:13:41.546 INFO:tasks.workunit.client.0.smithi031.stdout:4/478: creat d3/d23/d37/d86/f9a x:0 0 0 2024-04-24T23:13:41.547 INFO:tasks.workunit.client.0.smithi031.stdout:1/324: mkdir d1/da/d42/d69 0 2024-04-24T23:13:41.553 INFO:tasks.workunit.client.0.smithi031.stdout:6/492: getdents d13/d15/d1d/d23/d24/d63 0 2024-04-24T23:13:41.556 INFO:tasks.workunit.client.0.smithi031.stdout:7/325: truncate d1/d2/dd/f15 5518230 0 2024-04-24T23:13:41.557 INFO:tasks.workunit.client.0.smithi031.stdout:7/326: stat d1/d45/f47 0 2024-04-24T23:13:41.558 INFO:tasks.workunit.client.0.smithi031.stdout:1/325: stat d1/d3e/d46/f59 0 2024-04-24T23:13:41.561 INFO:tasks.workunit.client.0.smithi031.stdout:6/493: dwrite d13/d15/d1d/d23/d24/f9f [0,4194304] 0 2024-04-24T23:13:41.562 INFO:tasks.workunit.client.0.smithi031.stdout:6/494: stat d13/d15/d1d/d23/d24/d63 0 2024-04-24T23:13:41.563 INFO:tasks.workunit.client.0.smithi031.stdout:0/454: link da/d18/d1a/f63 da/d11/d6a/f94 0 2024-04-24T23:13:41.565 INFO:tasks.workunit.client.0.smithi031.stdout:0/455: stat da/d18/d1a/d37/d41/f83 0 2024-04-24T23:13:41.566 INFO:tasks.workunit.client.0.smithi031.stdout:8/470: rmdir d8/d15/d37/d43/da2 0 2024-04-24T23:13:41.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/327: fsync d1/d2/f46 0 2024-04-24T23:13:41.569 INFO:tasks.workunit.client.0.smithi031.stdout:7/328: dread d1/d2/dd/f17 [0,4194304] 0 2024-04-24T23:13:41.570 INFO:tasks.workunit.client.0.smithi031.stdout:4/479: link d3/d5e/f98 d3/d20/d2d/d44/d5b/f9b 0 2024-04-24T23:13:41.575 INFO:tasks.workunit.client.0.smithi031.stdout:9/467: getdents d2/d1b/d14/d78 0 2024-04-24T23:13:41.576 INFO:tasks.workunit.client.0.smithi031.stdout:7/329: dwrite d1/f53 [0,4194304] 0 2024-04-24T23:13:41.577 INFO:tasks.workunit.client.0.smithi031.stdout:7/330: chown d1/d2/d1e/l30 8 1 2024-04-24T23:13:41.578 INFO:tasks.workunit.client.0.smithi031.stdout:7/331: stat d1/d2/dd/d33/d5e/d64 0 2024-04-24T23:13:41.580 INFO:tasks.workunit.client.0.smithi031.stdout:6/495: truncate d13/d15/d1d/d23/d2d/f41 2565425 0 2024-04-24T23:13:41.581 INFO:tasks.workunit.client.0.smithi031.stdout:7/332: read d1/d2/dd/d33/f4e [575215,127305] 0 2024-04-24T23:13:41.584 INFO:tasks.workunit.client.0.smithi031.stdout:8/471: chown d8/lf 230950955 1 2024-04-24T23:13:41.589 INFO:tasks.workunit.client.0.smithi031.stdout:0/456: rename da/d18/d1a/d37/d41/c45 to da/d86/c95 0 2024-04-24T23:13:41.593 INFO:tasks.workunit.client.0.smithi031.stdout:4/480: write d3/f9 [2375693,69391] 0 2024-04-24T23:13:41.595 INFO:tasks.workunit.client.0.smithi031.stdout:4/481: dread - d3/d20/d25/f90 zero size 2024-04-24T23:13:41.598 INFO:tasks.workunit.client.0.smithi031.stdout:0/457: dwrite da/d18/d1a/d37/f3d [4194304,4194304] 0 2024-04-24T23:13:41.601 INFO:tasks.workunit.client.0.smithi031.stdout:4/482: dread d3/f4a [0,4194304] 0 2024-04-24T23:13:41.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/339: getdents d0/d6/d3b/d4b/dc/d25/d53 0 2024-04-24T23:13:41.609 INFO:tasks.workunit.client.0.smithi031.stdout:6/496: rmdir d13/d15/d1d/d23/d24/d8b 39 2024-04-24T23:13:41.612 INFO:tasks.workunit.client.0.smithi031.stdout:8/472: stat d8/l10 0 2024-04-24T23:13:41.615 INFO:tasks.workunit.client.1.smithi040.stdout:2/84: sync 2024-04-24T23:13:41.618 INFO:tasks.workunit.client.0.smithi031.stdout:0/458: readlink da/d18/d1a/d76/l81 0 2024-04-24T23:13:41.621 INFO:tasks.workunit.client.0.smithi031.stdout:4/483: stat d3/d23/l24 0 2024-04-24T23:13:41.623 INFO:tasks.workunit.client.0.smithi031.stdout:0/459: dread da/d20/d4e/f51 [0,4194304] 0 2024-04-24T23:13:41.628 INFO:tasks.workunit.client.0.smithi031.stdout:6/497: chown f12 1 1 2024-04-24T23:13:41.630 INFO:tasks.workunit.client.0.smithi031.stdout:7/333: mkdir d1/d1c/d65 0 2024-04-24T23:13:41.633 INFO:tasks.workunit.client.0.smithi031.stdout:8/473: mkdir d8/d15/d37/d43/daf 0 2024-04-24T23:13:41.636 INFO:tasks.workunit.client.0.smithi031.stdout:1/326: write d1/d31/f39 [731180,129135] 0 2024-04-24T23:13:41.638 INFO:tasks.workunit.client.0.smithi031.stdout:1/327: write d1/f66 [349210,28032] 0 2024-04-24T23:13:41.641 INFO:tasks.workunit.client.0.smithi031.stdout:1/328: dread d1/dc/d11/d21/f28 [0,4194304] 0 2024-04-24T23:13:41.645 INFO:tasks.workunit.client.0.smithi031.stdout:0/460: fsync da/d18/d1a/d37/f5e 0 2024-04-24T23:13:41.648 INFO:tasks.workunit.client.0.smithi031.stdout:0/461: dread da/d20/d4e/f57 [0,4194304] 0 2024-04-24T23:13:41.649 INFO:tasks.workunit.client.0.smithi031.stdout:6/498: creat d13/d15/d1d/d23/d2d/fac x:0 0 0 2024-04-24T23:13:41.653 INFO:tasks.workunit.client.0.smithi031.stdout:8/474: rename d8/d1a/d1d/d50/c84 to d8/d15/d37/d43/cb0 0 2024-04-24T23:13:41.654 INFO:tasks.workunit.client.0.smithi031.stdout:8/475: chown d8/d15/d37/d43/d5c 292 1 2024-04-24T23:13:41.655 INFO:tasks.workunit.client.0.smithi031.stdout:9/468: write d2/f1c [900991,61444] 0 2024-04-24T23:13:41.655 INFO:tasks.workunit.client.0.smithi031.stdout:8/476: fsync d8/fb 0 2024-04-24T23:13:41.664 INFO:tasks.workunit.client.0.smithi031.stdout:6/499: chown d13/d75/d79/c97 71241137 1 2024-04-24T23:13:41.668 INFO:tasks.workunit.client.0.smithi031.stdout:0/462: rename da/d22/d62/f88 to da/d22/f96 0 2024-04-24T23:13:41.674 INFO:tasks.workunit.client.0.smithi031.stdout:0/463: dwrite da/d20/f44 [0,4194304] 0 2024-04-24T23:13:41.675 INFO:tasks.workunit.client.0.smithi031.stdout:9/469: creat d2/d1b/d14/f88 x:0 0 0 2024-04-24T23:13:41.680 INFO:tasks.workunit.client.0.smithi031.stdout:0/464: dread da/f3f [4194304,4194304] 0 2024-04-24T23:13:41.684 INFO:tasks.workunit.client.0.smithi031.stdout:6/500: mknod d13/d75/cad 0 2024-04-24T23:13:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.683+0000 7f6335311700 1 -- 172.21.15.31:0/728396894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 msgr2=0x7f6330101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.683+0000 7f6335311700 1 --2- 172.21.15.31:0/728396894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 0x7f6330101e00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6320004940 tx=0x7f6320009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.683+0000 7f6335311700 1 -- 172.21.15.31:0/728396894 shutdown_connections 2024-04-24T23:13:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.683+0000 7f6335311700 1 --2- 172.21.15.31:0/728396894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330102340 0x7f6330104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.683+0000 7f6335311700 1 --2- 172.21.15.31:0/728396894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 0x7f6330101e00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.683+0000 7f6335311700 1 -- 172.21.15.31:0/728396894 >> 172.21.15.31:0/728396894 conn(0x7f63300f7430 msgr2=0x7f63300f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:41.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.684+0000 7f6335311700 1 -- 172.21.15.31:0/728396894 shutdown_connections 2024-04-24T23:13:41.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.684+0000 7f6335311700 1 -- 172.21.15.31:0/728396894 wait complete. 2024-04-24T23:13:41.686 INFO:tasks.workunit.client.0.smithi031.stdout:6/501: fdatasync d13/d15/d1d/d23/d4d/f6a 0 2024-04-24T23:13:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f6335311700 1 Processor -- start 2024-04-24T23:13:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f6335311700 1 -- start start 2024-04-24T23:13:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f6335311700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 0x7f633010fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f632effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 0x7f633010fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f632effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 0x7f633010fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58282/0 (socket says 172.21.15.31:58282) 2024-04-24T23:13:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f632effd700 1 -- 172.21.15.31:0/473902929 learned_addr learned my addr 172.21.15.31:0/473902929 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f6335311700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330102340 0x7f6330110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330110730 con 0x7f6330066260 2024-04-24T23:13:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.685+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330110f30 con 0x7f6330102340 2024-04-24T23:13:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.686+0000 7f632effd700 1 -- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330102340 msgr2=0x7f6330110130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:41.688 INFO:tasks.workunit.client.0.smithi031.stdout:6/502: write d13/d15/d1d/d23/d2d/f3d [2735932,55879] 0 2024-04-24T23:13:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.686+0000 7f632effd700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330102340 0x7f6330110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.686+0000 7f632effd700 1 -- 172.21.15.31:0/473902929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6320005040 con 0x7f6330066260 2024-04-24T23:13:41.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.686+0000 7f632effd700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 0x7f633010fbf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6320000c00 tx=0x7f6320009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:41.689 INFO:tasks.workunit.client.0.smithi031.stdout:7/334: rmdir d1/d45/d63 0 2024-04-24T23:13:41.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.686+0000 7f63177fe700 1 -- 172.21.15.31:0/473902929 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f632000e030 con 0x7f6330066260 2024-04-24T23:13:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.686+0000 7f63177fe700 1 -- 172.21.15.31:0/473902929 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f632001c400 con 0x7f6330066260 2024-04-24T23:13:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.686+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63301111b0 con 0x7f6330066260 2024-04-24T23:13:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.686+0000 7f63177fe700 1 -- 172.21.15.31:0/473902929 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f632000b650 con 0x7f6330066260 2024-04-24T23:13:41.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.687+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63301116f0 con 0x7f6330066260 2024-04-24T23:13:41.690 INFO:tasks.workunit.client.0.smithi031.stdout:5/340: write d0/d6/d32/f17 [1436598,81673] 0 2024-04-24T23:13:41.691 INFO:tasks.workunit.client.1.smithi040.stdout:7/77: creat db/f1e x:0 0 0 2024-04-24T23:13:41.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.689+0000 7f63177fe700 1 -- 172.21.15.31:0/473902929 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f632001ca90 con 0x7f6330066260 2024-04-24T23:13:41.691 INFO:tasks.workunit.client.0.smithi031.stdout:5/341: fsync d0/d6/d3b/d4b/d1d/d2e/f52 0 2024-04-24T23:13:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.690+0000 7f63177fe700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f631806b860 0x7f631806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.690+0000 7f63177fe700 1 -- 172.21.15.31:0/473902929 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f632008bdc0 con 0x7f6330066260 2024-04-24T23:13:41.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.691+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633005f5c0 con 0x7f6330066260 2024-04-24T23:13:41.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.691+0000 7f632e7fc700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f631806b860 0x7f631806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:41.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.692+0000 7f632e7fc700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f631806b860 0x7f631806dd10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6324009fd0 tx=0x7f632400b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:41.697 INFO:tasks.workunit.client.1.smithi040.stdout:9/80: mknod d7/da/c23 0 2024-04-24T23:13:41.698 INFO:tasks.workunit.client.0.smithi031.stdout:9/470: symlink d2/d1b/d14/d78/d84/l89 0 2024-04-24T23:13:41.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:41.698+0000 7f63177fe700 1 -- 172.21.15.31:0/473902929 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6320059550 con 0x7f6330066260 2024-04-24T23:13:41.701 INFO:tasks.workunit.client.0.smithi031.stdout:0/465: fsync da/f3f 0 2024-04-24T23:13:41.705 INFO:tasks.workunit.client.1.smithi040.stdout:6/40: rename f4 to d7/f8 0 2024-04-24T23:13:41.709 INFO:tasks.workunit.client.1.smithi040.stdout:0/83: unlink fe 0 2024-04-24T23:13:41.712 INFO:tasks.workunit.client.1.smithi040.stdout:3/39: mkdir d1/d9/db 0 2024-04-24T23:13:41.712 INFO:tasks.workunit.client.1.smithi040.stdout:3/40: fdatasync d1/f5 0 2024-04-24T23:13:41.714 INFO:tasks.workunit.client.0.smithi031.stdout:7/335: write d1/f28 [218561,39746] 0 2024-04-24T23:13:41.714 INFO:tasks.workunit.client.1.smithi040.stdout:5/49: creat d2/d7/d8/f13 x:0 0 0 2024-04-24T23:13:41.718 INFO:tasks.workunit.client.1.smithi040.stdout:2/85: unlink da/f14 0 2024-04-24T23:13:41.718 INFO:tasks.workunit.client.1.smithi040.stdout:3/41: dwrite f0 [0,4194304] 0 2024-04-24T23:13:41.719 INFO:tasks.workunit.client.1.smithi040.stdout:3/42: read f0 [1100918,62272] 0 2024-04-24T23:13:41.719 INFO:tasks.workunit.client.1.smithi040.stdout:7/78: symlink db/d12/l1f 0 2024-04-24T23:13:41.721 INFO:tasks.workunit.client.0.smithi031.stdout:1/329: getdents d1/dc/d4c/d4e 0 2024-04-24T23:13:41.723 INFO:tasks.workunit.client.0.smithi031.stdout:0/466: symlink da/d22/d62/l97 0 2024-04-24T23:13:41.724 INFO:tasks.workunit.client.1.smithi040.stdout:4/1: symlink l0 0 2024-04-24T23:13:41.725 INFO:tasks.workunit.client.1.smithi040.stdout:4/2: fdatasync - no filename 2024-04-24T23:13:41.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/477: dwrite d8/d15/d37/f2e [0,4194304] 0 2024-04-24T23:13:41.729 INFO:tasks.workunit.client.1.smithi040.stdout:8/9: mkdir d0 0 2024-04-24T23:13:41.730 INFO:tasks.workunit.client.0.smithi031.stdout:0/467: dread da/d18/d1a/d37/f4a [0,4194304] 0 2024-04-24T23:13:41.732 INFO:tasks.workunit.client.0.smithi031.stdout:8/478: dread d8/d15/d37/f5d [0,4194304] 0 2024-04-24T23:13:41.733 INFO:tasks.workunit.client.0.smithi031.stdout:8/479: chown d8/d15/f3d 711411 1 2024-04-24T23:13:41.734 INFO:tasks.workunit.client.0.smithi031.stdout:8/480: dread - d8/d15/f3d zero size 2024-04-24T23:13:41.735 INFO:tasks.workunit.client.0.smithi031.stdout:8/481: write d8/fd [6994222,44056] 0 2024-04-24T23:13:41.736 INFO:tasks.workunit.client.0.smithi031.stdout:5/342: mknod d0/d6/d3b/d4f/c6d 0 2024-04-24T23:13:41.738 INFO:tasks.workunit.client.1.smithi040.stdout:6/41: creat d7/f9 x:0 0 0 2024-04-24T23:13:41.742 INFO:tasks.workunit.client.1.smithi040.stdout:0/84: rename df/d11/f13 to df/d14/f16 0 2024-04-24T23:13:41.743 INFO:tasks.workunit.client.1.smithi040.stdout:1/30: write f7 [137984,11997] 0 2024-04-24T23:13:41.743 INFO:tasks.workunit.client.1.smithi040.stdout:0/85: write fb [942188,15484] 0 2024-04-24T23:13:41.746 INFO:tasks.workunit.client.1.smithi040.stdout:5/50: chown d2/d7/d8/c12 15392015 1 2024-04-24T23:13:41.746 INFO:tasks.workunit.client.1.smithi040.stdout:5/51: fdatasync d2/d7/fe 0 2024-04-24T23:13:41.747 INFO:tasks.workunit.client.1.smithi040.stdout:5/52: chown d2/d7/d8/c12 4 1 2024-04-24T23:13:41.748 INFO:tasks.workunit.client.0.smithi031.stdout:7/336: mkdir d1/d1c/d66 0 2024-04-24T23:13:41.748 INFO:tasks.workunit.client.1.smithi040.stdout:5/53: readlink d2/d7/l9 0 2024-04-24T23:13:41.748 INFO:tasks.workunit.client.0.smithi031.stdout:7/337: rename d1/d2 to d1/d2/d1e/d4f/d67 22 2024-04-24T23:13:41.749 INFO:tasks.workunit.client.1.smithi040.stdout:2/86: symlink da/l17 0 2024-04-24T23:13:41.749 INFO:tasks.workunit.client.1.smithi040.stdout:2/87: write f8 [758503,124219] 0 2024-04-24T23:13:41.750 INFO:tasks.workunit.client.0.smithi031.stdout:0/468: mknod da/d20/c98 0 2024-04-24T23:13:41.751 INFO:tasks.workunit.client.0.smithi031.stdout:0/469: fsync da/d11/d6a/f79 0 2024-04-24T23:13:41.755 INFO:tasks.workunit.client.0.smithi031.stdout:8/482: stat d8/c42 0 2024-04-24T23:13:41.756 INFO:tasks.workunit.client.1.smithi040.stdout:7/79: chown db/ce 52 1 2024-04-24T23:13:41.757 INFO:tasks.workunit.client.1.smithi040.stdout:7/80: write db/f1c [240731,76192] 0 2024-04-24T23:13:41.758 INFO:tasks.workunit.client.1.smithi040.stdout:7/81: truncate db/f1e 196942 0 2024-04-24T23:13:41.758 INFO:tasks.workunit.client.1.smithi040.stdout:7/82: readlink db/d12/l1f 0 2024-04-24T23:13:41.761 INFO:tasks.workunit.client.1.smithi040.stdout:4/3: symlink l1 0 2024-04-24T23:13:41.761 INFO:tasks.workunit.client.0.smithi031.stdout:6/503: truncate d13/d15/d1d/d23/d24/f77 3394844 0 2024-04-24T23:13:41.765 INFO:tasks.workunit.client.1.smithi040.stdout:6/42: mkdir d7/da 0 2024-04-24T23:13:41.766 INFO:tasks.workunit.client.0.smithi031.stdout:7/338: symlink d1/d2/d1e/d4f/l68 0 2024-04-24T23:13:41.772 INFO:tasks.workunit.client.1.smithi040.stdout:0/86: rmdir df 39 2024-04-24T23:13:41.772 INFO:tasks.workunit.client.1.smithi040.stdout:0/87: stat f8 0 2024-04-24T23:13:41.774 INFO:tasks.workunit.client.0.smithi031.stdout:1/330: link d1/f8 d1/dc/d4c/f6a 0 2024-04-24T23:13:41.774 INFO:tasks.workunit.client.1.smithi040.stdout:9/81: write d7/da/fd [2381178,53142] 0 2024-04-24T23:13:41.775 INFO:tasks.workunit.client.0.smithi031.stdout:1/331: write d1/d31/f39 [2507581,85136] 0 2024-04-24T23:13:41.777 INFO:tasks.workunit.client.0.smithi031.stdout:6/504: fdatasync d13/d15/d1d/d23/d24/d43/f4b 0 2024-04-24T23:13:41.782 INFO:tasks.workunit.client.1.smithi040.stdout:2/88: creat da/d10/f18 x:0 0 0 2024-04-24T23:13:41.785 INFO:tasks.workunit.client.1.smithi040.stdout:3/43: mknod d1/d9/da/cc 0 2024-04-24T23:13:41.785 INFO:tasks.workunit.client.0.smithi031.stdout:8/483: write d8/d1a/d1d/d35/d58/d47/f53 [4905255,44102] 0 2024-04-24T23:13:41.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/343: creat d0/d6/d3b/d4b/dc/f6e x:0 0 0 2024-04-24T23:13:41.788 INFO:tasks.workunit.client.0.smithi031.stdout:5/344: write d0/d6/d3b/d4b/dc/f24 [1091126,128050] 0 2024-04-24T23:13:41.789 INFO:tasks.workunit.client.1.smithi040.stdout:7/83: readlink db/l1a 0 2024-04-24T23:13:41.791 INFO:tasks.workunit.client.0.smithi031.stdout:1/332: symlink d1/d31/l6b 0 2024-04-24T23:13:41.791 INFO:tasks.workunit.client.1.smithi040.stdout:7/84: read db/f1c [169762,75998] 0 2024-04-24T23:13:41.792 INFO:tasks.workunit.client.0.smithi031.stdout:1/333: dread - d1/dc/f2b zero size 2024-04-24T23:13:41.792 INFO:tasks.workunit.client.1.smithi040.stdout:4/4: creat f2 x:0 0 0 2024-04-24T23:13:41.792 INFO:tasks.workunit.client.1.smithi040.stdout:7/85: write f0 [4432350,94473] 0 2024-04-24T23:13:41.793 INFO:tasks.workunit.client.1.smithi040.stdout:4/5: dread - f2 zero size 2024-04-24T23:13:41.793 INFO:tasks.workunit.client.1.smithi040.stdout:8/10: creat d0/f1 x:0 0 0 2024-04-24T23:13:41.796 INFO:tasks.workunit.client.0.smithi031.stdout:6/505: dread d13/d1b/f22 [0,4194304] 0 2024-04-24T23:13:41.797 INFO:tasks.workunit.client.1.smithi040.stdout:6/43: rename f5 to d7/fb 0 2024-04-24T23:13:41.798 INFO:tasks.workunit.client.0.smithi031.stdout:7/339: creat d1/d1c/d66/f69 x:0 0 0 2024-04-24T23:13:41.799 INFO:tasks.workunit.client.0.smithi031.stdout:1/334: dwrite d1/dc/fe [0,4194304] 0 2024-04-24T23:13:41.800 INFO:tasks.workunit.client.1.smithi040.stdout:1/31: rename da/fc to da/fd 0 2024-04-24T23:13:41.801 INFO:tasks.workunit.client.0.smithi031.stdout:0/470: rename da/d18/d1a/f70 to da/f99 0 2024-04-24T23:13:41.803 INFO:tasks.workunit.client.0.smithi031.stdout:8/484: unlink d8/d1a/d1d/l59 0 2024-04-24T23:13:41.805 INFO:tasks.workunit.client.0.smithi031.stdout:5/345: unlink d0/d6/d3b/d4b/dc/d25/f67 0 2024-04-24T23:13:41.810 INFO:tasks.workunit.client.1.smithi040.stdout:3/44: creat d1/fd x:0 0 0 2024-04-24T23:13:41.811 INFO:tasks.workunit.client.0.smithi031.stdout:8/485: dwrite d8/d1a/d1d/f78 [8388608,4194304] 0 2024-04-24T23:13:41.811 INFO:tasks.workunit.client.1.smithi040.stdout:3/45: write d1/f5 [728504,113061] 0 2024-04-24T23:13:41.815 INFO:tasks.workunit.client.1.smithi040.stdout:7/86: mkdir db/d12/d20 0 2024-04-24T23:13:41.816 INFO:tasks.workunit.client.0.smithi031.stdout:5/346: dwrite d0/d6/d3b/d4b/dc/f6e [0,4194304] 0 2024-04-24T23:13:41.817 INFO:tasks.workunit.client.1.smithi040.stdout:7/87: write db/f1e [926153,121867] 0 2024-04-24T23:13:41.817 INFO:tasks.workunit.client.0.smithi031.stdout:5/347: readlink d0/l16 0 2024-04-24T23:13:41.818 INFO:tasks.workunit.client.1.smithi040.stdout:4/6: creat f3 x:0 0 0 2024-04-24T23:13:41.819 INFO:tasks.workunit.client.0.smithi031.stdout:5/348: truncate d0/d6/d12/f63 250092 0 2024-04-24T23:13:41.820 INFO:tasks.workunit.client.0.smithi031.stdout:0/471: creat da/d18/f9a x:0 0 0 2024-04-24T23:13:41.820 INFO:tasks.workunit.client.1.smithi040.stdout:8/11: symlink d0/l2 0 2024-04-24T23:13:41.822 INFO:tasks.workunit.client.1.smithi040.stdout:8/12: write d0/f1 [151466,124104] 0 2024-04-24T23:13:41.822 INFO:tasks.workunit.client.1.smithi040.stdout:6/44: mknod d7/cc 0 2024-04-24T23:13:41.823 INFO:tasks.workunit.client.1.smithi040.stdout:8/13: chown d0/l2 4426129 1 2024-04-24T23:13:41.827 INFO:tasks.workunit.client.1.smithi040.stdout:5/54: rmdir d2/d10 0 2024-04-24T23:13:41.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/292: sync 2024-04-24T23:13:41.833 INFO:tasks.workunit.client.0.smithi031.stdout:5/349: fdatasync d0/d6/d3b/d4b/dc/f44 0 2024-04-24T23:13:41.836 INFO:tasks.workunit.client.1.smithi040.stdout:3/46: mkdir d1/d9/da/de 0 2024-04-24T23:13:41.837 INFO:tasks.workunit.client.0.smithi031.stdout:0/472: creat da/d18/d5a/d7d/f9b x:0 0 0 2024-04-24T23:13:41.838 INFO:tasks.workunit.client.0.smithi031.stdout:5/350: dread d0/d7/f58 [0,4194304] 0 2024-04-24T23:13:41.838 INFO:tasks.workunit.client.0.smithi031.stdout:0/473: chown da/d11/d6a/f8a 649131344 1 2024-04-24T23:13:41.840 INFO:tasks.workunit.client.0.smithi031.stdout:8/486: creat d8/d15/d9a/fb1 x:0 0 0 2024-04-24T23:13:41.841 INFO:tasks.workunit.client.0.smithi031.stdout:8/487: write d8/d1a/d1d/d35/d58/f4c [1376000,7193] 0 2024-04-24T23:13:41.842 INFO:tasks.workunit.client.1.smithi040.stdout:4/7: creat f4 x:0 0 0 2024-04-24T23:13:41.842 INFO:tasks.workunit.client.1.smithi040.stdout:4/8: stat f4 0 2024-04-24T23:13:41.842 INFO:tasks.workunit.client.0.smithi031.stdout:8/488: fsync d8/d15/f2a 0 2024-04-24T23:13:41.848 INFO:tasks.workunit.client.1.smithi040.stdout:7/88: rename f0 to db/d12/d20/f21 0 2024-04-24T23:13:41.848 INFO:tasks.workunit.client.0.smithi031.stdout:7/340: rename d1/d1c to d1/d6a 0 2024-04-24T23:13:41.849 INFO:tasks.workunit.client.0.smithi031.stdout:8/489: fsync d8/d1a/d1d/d35/d58/d47/f77 0 2024-04-24T23:13:41.850 INFO:tasks.workunit.client.0.smithi031.stdout:7/341: write d1/d2/d1e/f5f [973992,96445] 0 2024-04-24T23:13:41.854 INFO:tasks.workunit.client.1.smithi040.stdout:7/89: dwrite db/d12/f16 [0,4194304] 0 2024-04-24T23:13:41.855 INFO:tasks.workunit.client.1.smithi040.stdout:0/88: creat df/d12/f17 x:0 0 0 2024-04-24T23:13:41.858 INFO:tasks.workunit.client.1.smithi040.stdout:8/14: symlink d0/l3 0 2024-04-24T23:13:41.861 INFO:tasks.workunit.client.1.smithi040.stdout:2/89: dwrite f7 [0,4194304] 0 2024-04-24T23:13:41.861 INFO:tasks.workunit.client.1.smithi040.stdout:7/90: dread db/f10 [4194304,4194304] 0 2024-04-24T23:13:41.863 INFO:tasks.workunit.client.1.smithi040.stdout:8/15: dread d0/f1 [0,4194304] 0 2024-04-24T23:13:41.863 INFO:tasks.workunit.client.1.smithi040.stdout:8/16: fsync d0/f1 0 2024-04-24T23:13:41.864 INFO:tasks.workunit.client.0.smithi031.stdout:6/506: dwrite f11 [0,4194304] 0 2024-04-24T23:13:41.864 INFO:tasks.workunit.client.1.smithi040.stdout:8/17: write d0/f1 [1142024,63577] 0 2024-04-24T23:13:41.865 INFO:tasks.workunit.client.1.smithi040.stdout:2/90: dread f9 [0,4194304] 0 2024-04-24T23:13:41.866 INFO:tasks.workunit.client.0.smithi031.stdout:0/474: mkdir da/d22/d62/d9c 0 2024-04-24T23:13:41.868 INFO:tasks.workunit.client.0.smithi031.stdout:1/335: write d1/d33/f48 [39088,81825] 0 2024-04-24T23:13:41.875 INFO:tasks.workunit.client.0.smithi031.stdout:1/336: write d1/dc/d11/f55 [949843,29806] 0 2024-04-24T23:13:41.877 INFO:tasks.workunit.client.0.smithi031.stdout:2/293: mkdir d5/d8/d15/d65 0 2024-04-24T23:13:41.878 INFO:tasks.workunit.client.0.smithi031.stdout:6/507: dread f11 [0,4194304] 0 2024-04-24T23:13:41.880 INFO:tasks.workunit.client.0.smithi031.stdout:6/508: fsync d13/d15/d1d/d23/d24/f9f 0 2024-04-24T23:13:41.882 INFO:tasks.workunit.client.0.smithi031.stdout:6/509: fdatasync d13/d1b/f54 0 2024-04-24T23:13:41.886 INFO:tasks.workunit.client.0.smithi031.stdout:8/490: rename d8/d1a/d1d/d35/d58/d5b/f8b to d8/d15/d9a/fb2 0 2024-04-24T23:13:41.886 INFO:tasks.workunit.client.0.smithi031.stdout:5/351: dread d0/d6/d12/f21 [0,4194304] 0 2024-04-24T23:13:41.886 INFO:tasks.workunit.client.0.smithi031.stdout:2/294: dwrite d5/d6/f3c [0,4194304] 0 2024-04-24T23:13:41.889 INFO:tasks.workunit.client.0.smithi031.stdout:2/295: read - d5/d3d/f62 zero size 2024-04-24T23:13:41.891 INFO:tasks.workunit.client.1.smithi040.stdout:1/32: sync 2024-04-24T23:13:41.894 INFO:tasks.workunit.client.0.smithi031.stdout:0/475: mknod da/d86/c9d 0 2024-04-24T23:13:41.896 INFO:tasks.workunit.client.1.smithi040.stdout:0/89: mknod df/c18 0 2024-04-24T23:13:41.896 INFO:tasks.workunit.client.1.smithi040.stdout:0/90: fsync f8 0 2024-04-24T23:13:41.901 INFO:tasks.workunit.client.1.smithi040.stdout:5/55: mkdir d2/d7/d8/d11/d14 0 2024-04-24T23:13:41.902 INFO:tasks.workunit.client.1.smithi040.stdout:7/91: creat db/d12/d20/f22 x:0 0 0 2024-04-24T23:13:41.904 INFO:tasks.workunit.client.1.smithi040.stdout:9/82: getdents d7/d16 0 2024-04-24T23:13:41.905 INFO:tasks.workunit.client.1.smithi040.stdout:8/18: symlink d0/l4 0 2024-04-24T23:13:41.906 INFO:tasks.workunit.client.1.smithi040.stdout:8/19: stat d0/l3 0 2024-04-24T23:13:41.906 INFO:tasks.workunit.client.0.smithi031.stdout:6/510: creat d13/d15/d1d/d23/d2d/fae x:0 0 0 2024-04-24T23:13:41.908 INFO:tasks.workunit.client.1.smithi040.stdout:2/91: rename f9 to da/f19 0 2024-04-24T23:13:41.910 INFO:tasks.workunit.client.1.smithi040.stdout:9/83: dwrite f4 [0,4194304] 0 2024-04-24T23:13:41.910 INFO:tasks.workunit.client.0.smithi031.stdout:1/337: rename d1/d33/d36/f3b to d1/da/d42/d69/f6c 0 2024-04-24T23:13:41.911 INFO:tasks.workunit.client.1.smithi040.stdout:3/47: getdents d1/d9/db 0 2024-04-24T23:13:41.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:41 smithi040 ceph-mon[33128]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:41.912 INFO:tasks.workunit.client.0.smithi031.stdout:8/491: creat d8/d15/d37/d62/fb3 x:0 0 0 2024-04-24T23:13:41.913 INFO:tasks.workunit.client.0.smithi031.stdout:8/492: readlink l5 0 2024-04-24T23:13:41.914 INFO:tasks.workunit.client.0.smithi031.stdout:6/511: dwrite d13/d15/d1d/d23/d44/f4e [0,4194304] 0 2024-04-24T23:13:41.915 INFO:tasks.workunit.client.0.smithi031.stdout:5/352: creat d0/d6/d12/f6f x:0 0 0 2024-04-24T23:13:41.915 INFO:tasks.workunit.client.0.smithi031.stdout:6/512: stat d13/f7c 0 2024-04-24T23:13:41.917 INFO:tasks.workunit.client.1.smithi040.stdout:9/84: dwrite d7/f15 [0,4194304] 0 2024-04-24T23:13:41.917 INFO:tasks.workunit.client.1.smithi040.stdout:4/9: rename f3 to f5 0 2024-04-24T23:13:41.918 INFO:tasks.workunit.client.1.smithi040.stdout:4/10: chown f5 1 1 2024-04-24T23:13:41.918 INFO:tasks.workunit.client.1.smithi040.stdout:9/85: chown d7/d16/d1e/l20 1192 1 2024-04-24T23:13:41.922 INFO:tasks.workunit.client.0.smithi031.stdout:8/493: truncate d8/d15/d37/d43/d7b/d89/f92 394691 0 2024-04-24T23:13:41.923 INFO:tasks.workunit.client.0.smithi031.stdout:5/353: read d0/d6/d32/f17 [928872,87697] 0 2024-04-24T23:13:41.924 INFO:tasks.workunit.client.1.smithi040.stdout:6/45: rmdir d7/da 0 2024-04-24T23:13:41.926 INFO:tasks.workunit.client.0.smithi031.stdout:7/342: link d1/d6a/c32 d1/d2/dd/c6b 0 2024-04-24T23:13:41.927 INFO:tasks.workunit.client.1.smithi040.stdout:4/11: dwrite f2 [0,4194304] 0 2024-04-24T23:13:41.929 INFO:tasks.workunit.client.1.smithi040.stdout:0/91: mkdir df/d12/d19 0 2024-04-24T23:13:41.933 INFO:tasks.workunit.client.0.smithi031.stdout:1/338: truncate d1/da/d12/f23 878332 0 2024-04-24T23:13:41.934 INFO:tasks.workunit.client.0.smithi031.stdout:1/339: stat d1/dc/d11/d21/f28 0 2024-04-24T23:13:41.935 INFO:tasks.workunit.client.1.smithi040.stdout:7/92: creat db/d1b/f23 x:0 0 0 2024-04-24T23:13:41.936 INFO:tasks.workunit.client.0.smithi031.stdout:1/340: write d1/d31/f39 [3451969,119491] 0 2024-04-24T23:13:41.942 INFO:tasks.workunit.client.0.smithi031.stdout:1/341: read d1/f2 [5431759,29426] 0 2024-04-24T23:13:41.943 INFO:tasks.workunit.client.1.smithi040.stdout:8/20: rmdir d0 39 2024-04-24T23:13:41.945 INFO:tasks.workunit.client.1.smithi040.stdout:2/92: mknod da/d10/c1a 0 2024-04-24T23:13:41.946 INFO:tasks.workunit.client.1.smithi040.stdout:7/93: dread db/f14 [0,4194304] 0 2024-04-24T23:13:41.954 INFO:tasks.workunit.client.1.smithi040.stdout:9/86: rename d7/l8 to d7/da/l24 0 2024-04-24T23:13:41.954 INFO:tasks.workunit.client.1.smithi040.stdout:2/93: rename da to da/d1b 22 2024-04-24T23:13:41.958 INFO:tasks.workunit.client.1.smithi040.stdout:6/46: creat d7/fd x:0 0 0 2024-04-24T23:13:41.965 INFO:tasks.workunit.client.1.smithi040.stdout:4/12: creat f6 x:0 0 0 2024-04-24T23:13:41.969 INFO:tasks.workunit.client.1.smithi040.stdout:0/92: mkdir df/d12/d1a 0 2024-04-24T23:13:41.970 INFO:tasks.workunit.client.0.smithi031.stdout:2/296: mknod d5/d8/c66 0 2024-04-24T23:13:41.971 INFO:tasks.workunit.client.0.smithi031.stdout:2/297: write d5/d6/d29/d26/f3f [305904,70044] 0 2024-04-24T23:13:41.975 INFO:tasks.workunit.client.1.smithi040.stdout:1/33: truncate f2 962983 0 2024-04-24T23:13:41.978 INFO:tasks.workunit.client.1.smithi040.stdout:7/94: creat db/f24 x:0 0 0 2024-04-24T23:13:41.979 INFO:tasks.workunit.client.1.smithi040.stdout:7/95: chown db/f1c 14642 1 2024-04-24T23:13:41.979 INFO:tasks.workunit.client.1.smithi040.stdout:7/96: truncate db/d12/d20/f22 252114 0 2024-04-24T23:13:41.980 INFO:tasks.workunit.client.1.smithi040.stdout:3/48: mknod d1/d9/da/de/cf 0 2024-04-24T23:13:41.980 INFO:tasks.workunit.client.1.smithi040.stdout:7/97: stat db/f1e 0 2024-04-24T23:13:41.983 INFO:tasks.workunit.client.1.smithi040.stdout:3/49: dread d1/f5 [0,4194304] 0 2024-04-24T23:13:41.984 INFO:tasks.workunit.client.1.smithi040.stdout:4/13: mknod c7 0 2024-04-24T23:13:41.987 INFO:tasks.workunit.client.1.smithi040.stdout:0/93: rename df/d11 to df/d14/d1b 0 2024-04-24T23:13:41.988 INFO:tasks.workunit.client.1.smithi040.stdout:5/56: link d2/cd d2/d7/d8/d11/c15 0 2024-04-24T23:13:41.988 INFO:tasks.workunit.client.1.smithi040.stdout:4/14: dread f2 [0,4194304] 0 2024-04-24T23:13:41.992 INFO:tasks.workunit.client.1.smithi040.stdout:1/34: mknod da/ce 0 2024-04-24T23:13:41.993 INFO:tasks.workunit.client.1.smithi040.stdout:4/15: dread f2 [0,4194304] 0 2024-04-24T23:13:41.994 INFO:tasks.workunit.client.0.smithi031.stdout:1/342: write d1/d1d/f35 [4465852,3997] 0 2024-04-24T23:13:41.994 INFO:tasks.workunit.client.1.smithi040.stdout:0/94: dwrite fb [0,4194304] 0 2024-04-24T23:13:41.998 INFO:tasks.workunit.client.1.smithi040.stdout:7/98: mknod db/c25 0 2024-04-24T23:13:41.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:41 smithi031 ceph-mon[26411]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:41.999 INFO:tasks.workunit.client.0.smithi031.stdout:5/354: creat d0/d6/d32/d6b/f70 x:0 0 0 2024-04-24T23:13:42.000 INFO:tasks.workunit.client.1.smithi040.stdout:0/95: dread f6 [0,4194304] 0 2024-04-24T23:13:42.000 INFO:tasks.workunit.client.1.smithi040.stdout:0/96: readlink l5 0 2024-04-24T23:13:42.000 INFO:tasks.workunit.client.1.smithi040.stdout:0/97: write f8 [3065779,127407] 0 2024-04-24T23:13:42.001 INFO:tasks.workunit.client.1.smithi040.stdout:9/87: mknod d7/d16/d1e/d1f/c25 0 2024-04-24T23:13:42.001 INFO:tasks.workunit.client.1.smithi040.stdout:0/98: chown df/c18 25 1 2024-04-24T23:13:42.001 INFO:tasks.workunit.client.1.smithi040.stdout:9/88: fdatasync d7/da/fd 0 2024-04-24T23:13:42.007 INFO:tasks.workunit.client.0.smithi031.stdout:2/298: rename d5/d8/d15/f3b to d5/d28/d43/f67 0 2024-04-24T23:13:42.008 INFO:tasks.workunit.client.1.smithi040.stdout:3/50: symlink d1/d9/l10 0 2024-04-24T23:13:42.008 INFO:tasks.workunit.client.0.smithi031.stdout:2/299: write d5/d8/d15/f34 [123180,110561] 0 2024-04-24T23:13:42.010 INFO:tasks.workunit.client.0.smithi031.stdout:6/513: truncate d13/d75/d79/f85 2745940 0 2024-04-24T23:13:42.011 INFO:tasks.workunit.client.1.smithi040.stdout:8/21: dwrite d0/f1 [0,4194304] 0 2024-04-24T23:13:42.011 INFO:tasks.workunit.client.0.smithi031.stdout:8/494: creat d8/fb4 x:0 0 0 2024-04-24T23:13:42.012 INFO:tasks.workunit.client.1.smithi040.stdout:8/22: chown d0/l2 0 1 2024-04-24T23:13:42.013 INFO:tasks.workunit.client.0.smithi031.stdout:5/355: creat d0/d6/d12/f71 x:0 0 0 2024-04-24T23:13:42.018 INFO:tasks.workunit.client.1.smithi040.stdout:1/35: rename f3 to da/ff 0 2024-04-24T23:13:42.018 INFO:tasks.workunit.client.1.smithi040.stdout:8/23: dwrite d0/f1 [0,4194304] 0 2024-04-24T23:13:42.019 INFO:tasks.workunit.client.1.smithi040.stdout:2/94: truncate f7 1616889 0 2024-04-24T23:13:42.019 INFO:tasks.workunit.client.0.smithi031.stdout:7/343: dwrite d1/d2/dd/f15 [0,4194304] 0 2024-04-24T23:13:42.020 INFO:tasks.workunit.client.1.smithi040.stdout:2/95: dread - da/f11 zero size 2024-04-24T23:13:42.021 INFO:tasks.workunit.client.0.smithi031.stdout:7/344: dread - d1/d2/d1e/d4f/f5d zero size 2024-04-24T23:13:42.022 INFO:tasks.workunit.client.1.smithi040.stdout:4/16: mknod c8 0 2024-04-24T23:13:42.031 INFO:tasks.workunit.client.0.smithi031.stdout:2/300: truncate d5/d8/d15/f2b 4210011 0 2024-04-24T23:13:42.033 INFO:tasks.workunit.client.1.smithi040.stdout:6/47: sync 2024-04-24T23:13:42.036 INFO:tasks.workunit.client.0.smithi031.stdout:8/495: mkdir d8/d15/d37/d43/d5c/db5 0 2024-04-24T23:13:42.038 INFO:tasks.workunit.client.1.smithi040.stdout:7/99: symlink db/d12/l26 0 2024-04-24T23:13:42.039 INFO:tasks.workunit.client.1.smithi040.stdout:7/100: chown c7 18 1 2024-04-24T23:13:42.039 INFO:tasks.workunit.client.1.smithi040.stdout:7/101: chown db/ce 876 1 2024-04-24T23:13:42.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/496: dread d8/d1a/f7e [0,4194304] 0 2024-04-24T23:13:42.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/497: chown d8/fa8 458666 1 2024-04-24T23:13:42.043 INFO:tasks.workunit.client.0.smithi031.stdout:7/345: chown d1/lb 180 1 2024-04-24T23:13:42.044 INFO:tasks.workunit.client.0.smithi031.stdout:7/346: read d1/d2/d1e/f36 [1173451,19621] 0 2024-04-24T23:13:42.044 INFO:tasks.workunit.client.1.smithi040.stdout:0/99: mknod df/d14/d1b/c1c 0 2024-04-24T23:13:42.045 INFO:tasks.workunit.client.0.smithi031.stdout:7/347: write d1/d2/dd/f23 [2024984,44102] 0 2024-04-24T23:13:42.048 INFO:tasks.workunit.client.1.smithi040.stdout:9/89: truncate d7/ff 4320058 0 2024-04-24T23:13:42.049 INFO:tasks.workunit.client.0.smithi031.stdout:6/514: symlink d13/d15/d1d/d23/d2d/d9e/dab/laf 0 2024-04-24T23:13:42.054 INFO:tasks.workunit.client.0.smithi031.stdout:8/498: creat d8/d15/d37/d43/d7b/d86/fb6 x:0 0 0 2024-04-24T23:13:42.054 INFO:tasks.workunit.client.1.smithi040.stdout:9/90: dwrite d7/f15 [4194304,4194304] 0 2024-04-24T23:13:42.054 INFO:tasks.workunit.client.0.smithi031.stdout:7/348: dread d1/f28 [0,4194304] 0 2024-04-24T23:13:42.055 INFO:tasks.workunit.client.1.smithi040.stdout:5/57: mknod d2/d7/d8/db/c16 0 2024-04-24T23:13:42.058 INFO:tasks.workunit.client.1.smithi040.stdout:1/36: write da/fd [246586,101899] 0 2024-04-24T23:13:42.061 INFO:tasks.workunit.client.1.smithi040.stdout:3/51: rename d1/l7 to d1/d9/l11 0 2024-04-24T23:13:42.062 INFO:tasks.workunit.client.1.smithi040.stdout:9/91: rename d7/da to d7/da/d26 22 2024-04-24T23:13:42.064 INFO:tasks.workunit.client.0.smithi031.stdout:1/343: getdents d1/dc/d11 0 2024-04-24T23:13:42.065 INFO:tasks.workunit.client.1.smithi040.stdout:8/24: creat d0/f5 x:0 0 0 2024-04-24T23:13:42.066 INFO:tasks.workunit.client.1.smithi040.stdout:8/25: read - d0/f5 zero size 2024-04-24T23:13:42.066 INFO:tasks.workunit.client.1.smithi040.stdout:9/92: dread d7/da/fd [0,4194304] 0 2024-04-24T23:13:42.067 INFO:tasks.workunit.client.1.smithi040.stdout:4/17: creat f9 x:0 0 0 2024-04-24T23:13:42.068 INFO:tasks.workunit.client.1.smithi040.stdout:9/93: write d7/da/fd [3424736,130801] 0 2024-04-24T23:13:42.070 INFO:tasks.workunit.client.0.smithi031.stdout:1/344: dwrite d1/d33/f47 [0,4194304] 0 2024-04-24T23:13:42.073 INFO:tasks.workunit.client.1.smithi040.stdout:6/48: mkdir d7/de 0 2024-04-24T23:13:42.073 INFO:tasks.workunit.client.1.smithi040.stdout:6/49: write d7/f9 [723912,48629] 0 2024-04-24T23:13:42.075 INFO:tasks.workunit.client.0.smithi031.stdout:5/356: creat d0/d6/d3b/d4b/f72 x:0 0 0 2024-04-24T23:13:42.076 INFO:tasks.workunit.client.1.smithi040.stdout:7/102: dread db/f14 [0,4194304] 0 2024-04-24T23:13:42.077 INFO:tasks.workunit.client.0.smithi031.stdout:5/357: read d0/d6/d3b/d4b/dc/f6e [1696019,32939] 0 2024-04-24T23:13:42.077 INFO:tasks.workunit.client.1.smithi040.stdout:7/103: write db/d12/f16 [452865,36087] 0 2024-04-24T23:13:42.078 INFO:tasks.workunit.client.0.smithi031.stdout:6/515: truncate d13/d15/d1d/d23/d24/d43/f5a 3529635 0 2024-04-24T23:13:42.079 INFO:tasks.workunit.client.0.smithi031.stdout:5/358: fdatasync d0/d6/d3b/d4b/d1d/d2e/f52 0 2024-04-24T23:13:42.080 INFO:tasks.workunit.client.1.smithi040.stdout:0/100: mknod df/c1d 0 2024-04-24T23:13:42.080 INFO:tasks.workunit.client.1.smithi040.stdout:0/101: dread - df/d14/f16 zero size 2024-04-24T23:13:42.081 INFO:tasks.workunit.client.1.smithi040.stdout:0/102: dread - df/d12/f17 zero size 2024-04-24T23:13:42.084 INFO:tasks.workunit.client.0.smithi031.stdout:8/499: rename d8/d15/f3e to d8/d1a/d1d/d50/fb7 0 2024-04-24T23:13:42.084 INFO:tasks.workunit.client.0.smithi031.stdout:8/500: chown d8/d15/d37 21 1 2024-04-24T23:13:42.089 INFO:tasks.workunit.client.0.smithi031.stdout:2/301: dwrite d5/f27 [0,4194304] 0 2024-04-24T23:13:42.090 INFO:tasks.workunit.client.0.smithi031.stdout:1/345: creat d1/d1d/f6d x:0 0 0 2024-04-24T23:13:42.092 INFO:tasks.workunit.client.1.smithi040.stdout:3/52: dread d1/f5 [0,4194304] 0 2024-04-24T23:13:42.098 INFO:tasks.workunit.client.1.smithi040.stdout:8/26: symlink d0/l6 0 2024-04-24T23:13:42.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/346: dread d1/d1d/f35 [0,4194304] 0 2024-04-24T23:13:42.100 INFO:tasks.workunit.client.0.smithi031.stdout:8/501: dread d8/fb [0,4194304] 0 2024-04-24T23:13:42.102 INFO:tasks.workunit.client.1.smithi040.stdout:4/18: mkdir da 0 2024-04-24T23:13:42.103 INFO:tasks.workunit.client.1.smithi040.stdout:4/19: truncate f4 921866 0 2024-04-24T23:13:42.103 INFO:tasks.workunit.client.1.smithi040.stdout:4/20: fdatasync f6 0 2024-04-24T23:13:42.104 INFO:tasks.workunit.client.1.smithi040.stdout:8/27: dwrite d0/f1 [0,4194304] 0 2024-04-24T23:13:42.109 INFO:tasks.workunit.client.1.smithi040.stdout:7/104: mknod db/d12/d20/c27 0 2024-04-24T23:13:42.110 INFO:tasks.workunit.client.1.smithi040.stdout:0/103: fsync df/d14/f16 0 2024-04-24T23:13:42.110 INFO:tasks.workunit.client.1.smithi040.stdout:0/104: readlink l5 0 2024-04-24T23:13:42.112 INFO:tasks.workunit.client.1.smithi040.stdout:5/58: creat d2/d7/d8/d11/d14/f17 x:0 0 0 2024-04-24T23:13:42.114 INFO:tasks.workunit.client.1.smithi040.stdout:8/28: dwrite d0/f1 [0,4194304] 0 2024-04-24T23:13:42.114 INFO:tasks.workunit.client.1.smithi040.stdout:0/105: chown df/d14/d1b/l15 25840 1 2024-04-24T23:13:42.115 INFO:tasks.workunit.client.1.smithi040.stdout:1/37: fsync f2 0 2024-04-24T23:13:42.115 INFO:tasks.workunit.client.1.smithi040.stdout:0/106: fsync df/d14/f16 0 2024-04-24T23:13:42.118 INFO:tasks.workunit.client.1.smithi040.stdout:5/59: truncate d2/d7/d8/d11/d14/f17 959937 0 2024-04-24T23:13:42.121 INFO:tasks.workunit.client.1.smithi040.stdout:3/53: rename d1/d9/l10 to d1/d9/db/l12 0 2024-04-24T23:13:42.122 INFO:tasks.workunit.client.1.smithi040.stdout:3/54: dread - d1/fd zero size 2024-04-24T23:13:42.123 INFO:tasks.workunit.client.1.smithi040.stdout:8/29: dwrite d0/f5 [0,4194304] 0 2024-04-24T23:13:42.127 INFO:tasks.workunit.client.0.smithi031.stdout:2/302: mkdir d5/d6/d68 0 2024-04-24T23:13:42.131 INFO:tasks.workunit.client.0.smithi031.stdout:7/349: link d1/c1b d1/d2/dd/c6c 0 2024-04-24T23:13:42.132 INFO:tasks.workunit.client.1.smithi040.stdout:6/50: dwrite f2 [0,4194304] 0 2024-04-24T23:13:42.132 INFO:tasks.workunit.client.1.smithi040.stdout:2/96: dwrite f7 [0,4194304] 0 2024-04-24T23:13:42.133 INFO:tasks.workunit.client.1.smithi040.stdout:7/105: creat db/d1b/f28 x:0 0 0 2024-04-24T23:13:42.133 INFO:tasks.workunit.client.1.smithi040.stdout:2/97: dread - da/d10/f18 zero size 2024-04-24T23:13:42.134 INFO:tasks.workunit.client.0.smithi031.stdout:7/350: dread - d1/d6a/d66/f69 zero size 2024-04-24T23:13:42.136 INFO:tasks.workunit.client.0.smithi031.stdout:6/516: dwrite d13/d15/d1d/d23/d2d/f90 [0,4194304] 0 2024-04-24T23:13:42.137 INFO:tasks.workunit.client.0.smithi031.stdout:5/359: rename d0/d6/d3b/d4b/dc/d25/d53 to d0/d6/d3b/d4b/dc/d73 0 2024-04-24T23:13:42.141 INFO:tasks.workunit.client.0.smithi031.stdout:2/303: mknod d5/d3d/c69 0 2024-04-24T23:13:42.144 INFO:tasks.workunit.client.1.smithi040.stdout:0/107: mkdir df/d1e 0 2024-04-24T23:13:42.148 INFO:tasks.workunit.client.0.smithi031.stdout:1/347: link d1/d31/c3c d1/dc/d11/c6e 0 2024-04-24T23:13:42.148 INFO:tasks.workunit.client.1.smithi040.stdout:1/38: dwrite f2 [0,4194304] 0 2024-04-24T23:13:42.149 INFO:tasks.workunit.client.1.smithi040.stdout:5/60: mknod d2/d7/d8/d11/d14/c18 0 2024-04-24T23:13:42.151 INFO:tasks.workunit.client.1.smithi040.stdout:3/55: rmdir d1 39 2024-04-24T23:13:42.154 INFO:tasks.workunit.client.0.smithi031.stdout:5/360: mkdir d0/d7/d74 0 2024-04-24T23:13:42.154 INFO:tasks.workunit.client.1.smithi040.stdout:9/94: dwrite d7/ff [4194304,4194304] 0 2024-04-24T23:13:42.155 INFO:tasks.workunit.client.0.smithi031.stdout:5/361: write d0/d6/d3b/f4d [279027,124274] 0 2024-04-24T23:13:42.156 INFO:tasks.workunit.client.1.smithi040.stdout:8/30: creat d0/f7 x:0 0 0 2024-04-24T23:13:42.157 INFO:tasks.workunit.client.0.smithi031.stdout:5/362: dread - d0/d6/d3b/f68 zero size 2024-04-24T23:13:42.157 INFO:tasks.workunit.client.0.smithi031.stdout:6/517: unlink d13/d15/d1d/d23/d2d/fae 0 2024-04-24T23:13:42.157 INFO:tasks.workunit.client.0.smithi031.stdout:5/363: chown d0/d7/c1f 30 1 2024-04-24T23:13:42.160 INFO:tasks.workunit.client.0.smithi031.stdout:5/364: read d0/d6/d3b/d4b/d1d/d2e/d28/f30 [1041277,92998] 0 2024-04-24T23:13:42.161 INFO:tasks.workunit.client.1.smithi040.stdout:4/21: creat da/fb x:0 0 0 2024-04-24T23:13:42.162 INFO:tasks.workunit.client.0.smithi031.stdout:3/359: sync 2024-04-24T23:13:42.163 INFO:tasks.workunit.client.0.smithi031.stdout:8/502: getdents d8/d1a/d1d 0 2024-04-24T23:13:42.166 INFO:tasks.workunit.client.0.smithi031.stdout:8/503: truncate d8/d15/d9a/fb2 442542 0 2024-04-24T23:13:42.168 INFO:tasks.workunit.client.0.smithi031.stdout:7/351: mkdir d1/d2/dd/d33/d5e/d64/d6d 0 2024-04-24T23:13:42.170 INFO:tasks.workunit.client.1.smithi040.stdout:7/106: creat db/d12/f29 x:0 0 0 2024-04-24T23:13:42.172 INFO:tasks.workunit.client.0.smithi031.stdout:8/504: dwrite d8/f28 [0,4194304] 0 2024-04-24T23:13:42.180 INFO:tasks.workunit.client.1.smithi040.stdout:0/108: rename f8 to df/d12/f1f 0 2024-04-24T23:13:42.182 INFO:tasks.workunit.client.1.smithi040.stdout:3/56: chown d1/f3 14817 1 2024-04-24T23:13:42.184 INFO:tasks.workunit.client.0.smithi031.stdout:3/360: creat d6/d72/d1a/d25/f74 x:0 0 0 2024-04-24T23:13:42.185 INFO:tasks.workunit.client.0.smithi031.stdout:3/361: chown d6/d72/c30 189697076 1 2024-04-24T23:13:42.189 INFO:tasks.workunit.client.1.smithi040.stdout:9/95: symlink d7/da/d17/l27 0 2024-04-24T23:13:42.192 INFO:tasks.workunit.client.1.smithi040.stdout:8/31: mkdir d0/d8 0 2024-04-24T23:13:42.194 INFO:tasks.workunit.client.0.smithi031.stdout:7/352: mknod d1/d2/d1e/c6e 0 2024-04-24T23:13:42.198 INFO:tasks.workunit.client.0.smithi031.stdout:8/505: write d8/d1a/f7e [2950074,104187] 0 2024-04-24T23:13:42.198 INFO:tasks.workunit.client.0.smithi031.stdout:5/365: write d0/d6/d3b/d4b/dc/f6e [1268590,56334] 0 2024-04-24T23:13:42.201 INFO:tasks.workunit.client.0.smithi031.stdout:2/304: link c2 d5/d8/d15/d65/c6a 0 2024-04-24T23:13:42.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.200+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6330107a90 con 0x7f6330066260 2024-04-24T23:13:42.202 INFO:tasks.workunit.client.1.smithi040.stdout:7/107: mknod db/d12/d20/c2a 0 2024-04-24T23:13:42.203 INFO:tasks.workunit.client.0.smithi031.stdout:2/305: chown d5/d8/d15/f5f 4259998 1 2024-04-24T23:13:42.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.201+0000 7f63177fe700 1 -- 172.21.15.31:0/473902929 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f63200590e0 con 0x7f6330066260 2024-04-24T23:13:42.203 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:13:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:13:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:13:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:13:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:13:42.203 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:13:42.204 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:13:42.205 INFO:tasks.workunit.client.1.smithi040.stdout:7/108: write db/d12/f16 [4797316,94774] 0 2024-04-24T23:13:42.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f631806b860 msgr2=0x7f631806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f631806b860 0x7f631806dd10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6324009fd0 tx=0x7f632400b040 comp rx=0 tx=0).stop 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 msgr2=0x7f633010fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 0x7f633010fbf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6320000c00 tx=0x7f6320009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 shutdown_connections 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f631806b860 0x7f631806dd10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6330066260 0x7f633010fbf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 --2- 172.21.15.31:0/473902929 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6330102340 0x7f6330110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 >> 172.21.15.31:0/473902929 conn(0x7f63300f7430 msgr2=0x7f63300f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 shutdown_connections 2024-04-24T23:13:42.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.205+0000 7f6335311700 1 -- 172.21.15.31:0/473902929 wait complete. 2024-04-24T23:13:42.208 INFO:tasks.workunit.client.1.smithi040.stdout:6/51: write d7/fb [558856,20792] 0 2024-04-24T23:13:42.208 INFO:tasks.workunit.client.1.smithi040.stdout:6/52: write d7/fb [1059796,73788] 0 2024-04-24T23:13:42.208 INFO:tasks.workunit.client.1.smithi040.stdout:2/98: truncate f8 871872 0 2024-04-24T23:13:42.208 INFO:tasks.workunit.client.0.smithi031.stdout:1/348: dwrite d1/da/d12/f23 [0,4194304] 0 2024-04-24T23:13:42.209 INFO:tasks.workunit.client.1.smithi040.stdout:6/53: truncate d7/f8 4377369 0 2024-04-24T23:13:42.210 INFO:tasks.workunit.client.1.smithi040.stdout:1/39: rename f9 to da/f10 0 2024-04-24T23:13:42.210 INFO:tasks.workunit.client.1.smithi040.stdout:1/40: chown da 319551664 1 2024-04-24T23:13:42.212 INFO:tasks.workunit.client.0.smithi031.stdout:7/353: symlink d1/d2/dd/d33/d5e/l6f 0 2024-04-24T23:13:42.214 INFO:tasks.workunit.client.0.smithi031.stdout:2/306: dwrite d5/d6/df/f44 [0,4194304] 0 2024-04-24T23:13:42.214 INFO:tasks.workunit.client.0.smithi031.stdout:8/506: mknod d8/d15/d37/d62/cb8 0 2024-04-24T23:13:42.215 INFO:tasks.workunit.client.1.smithi040.stdout:3/57: dread d1/f3 [0,4194304] 0 2024-04-24T23:13:42.215 INFO:tasks.workunit.client.0.smithi031.stdout:2/307: chown d5/d28/f40 1723 1 2024-04-24T23:13:42.217 INFO:tasks.workunit.client.1.smithi040.stdout:9/96: dread f2 [0,4194304] 0 2024-04-24T23:13:42.217 INFO:tasks.workunit.client.1.smithi040.stdout:9/97: write f4 [1467931,71775] 0 2024-04-24T23:13:42.220 INFO:tasks.workunit.client.1.smithi040.stdout:8/32: unlink d0/f1 0 2024-04-24T23:13:42.224 INFO:tasks.workunit.client.0.smithi031.stdout:1/349: mknod d1/dc/d4c/c6f 0 2024-04-24T23:13:42.224 INFO:tasks.workunit.client.0.smithi031.stdout:8/507: dwrite d8/d1a/f7e [0,4194304] 0 2024-04-24T23:13:42.225 INFO:tasks.workunit.client.0.smithi031.stdout:8/508: fsync d8/fa8 0 2024-04-24T23:13:42.225 INFO:tasks.workunit.client.1.smithi040.stdout:7/109: creat db/f2b x:0 0 0 2024-04-24T23:13:42.234 INFO:tasks.workunit.client.0.smithi031.stdout:1/350: dread d1/d33/d36/f44 [0,4194304] 0 2024-04-24T23:13:42.235 INFO:tasks.workunit.client.1.smithi040.stdout:2/99: mkdir da/d1c 0 2024-04-24T23:13:42.237 INFO:tasks.workunit.client.0.smithi031.stdout:5/366: link d0/d6/d12/f71 d0/d6/d3b/d4b/dc/d25/d55/f75 0 2024-04-24T23:13:42.238 INFO:tasks.workunit.client.0.smithi031.stdout:6/518: link d13/d15/d1d/d23/d24/d43/ca4 d13/d15/d1d/cb0 0 2024-04-24T23:13:42.240 INFO:tasks.workunit.client.1.smithi040.stdout:1/41: mknod da/c11 0 2024-04-24T23:13:42.240 INFO:tasks.workunit.client.0.smithi031.stdout:6/519: write d13/d15/d1d/d23/d44/f4e [4316228,55436] 0 2024-04-24T23:13:42.242 INFO:tasks.workunit.client.0.smithi031.stdout:7/354: mknod d1/d2/dd/d33/d5e/d64/d6d/c70 0 2024-04-24T23:13:42.243 INFO:tasks.workunit.client.1.smithi040.stdout:5/61: link d2/d7/d8/c12 d2/d7/d8/d11/d14/c19 0 2024-04-24T23:13:42.243 INFO:tasks.workunit.client.0.smithi031.stdout:7/355: fsync d1/d2/d1e/d4f/f5d 0 2024-04-24T23:13:42.244 INFO:tasks.workunit.client.0.smithi031.stdout:7/356: truncate d1/d6a/d66/f69 753037 0 2024-04-24T23:13:42.245 INFO:tasks.workunit.client.1.smithi040.stdout:3/58: mkdir d1/d9/da/de/d13 0 2024-04-24T23:13:42.247 INFO:tasks.workunit.client.0.smithi031.stdout:2/308: creat d5/d6/d68/f6b x:0 0 0 2024-04-24T23:13:42.252 INFO:tasks.workunit.client.1.smithi040.stdout:9/98: mkdir d7/d16/d1e/d28 0 2024-04-24T23:13:42.255 INFO:tasks.workunit.client.1.smithi040.stdout:9/99: dread d7/fe [0,4194304] 0 2024-04-24T23:13:42.255 INFO:tasks.workunit.client.1.smithi040.stdout:8/33: unlink d0/f5 0 2024-04-24T23:13:42.255 INFO:tasks.workunit.client.1.smithi040.stdout:9/100: chown d7 74859 1 2024-04-24T23:13:42.256 INFO:tasks.workunit.client.1.smithi040.stdout:8/34: write d0/f7 [178624,26253] 0 2024-04-24T23:13:42.256 INFO:tasks.workunit.client.1.smithi040.stdout:9/101: write d7/f15 [4794081,92036] 0 2024-04-24T23:13:42.258 INFO:tasks.workunit.client.0.smithi031.stdout:8/509: creat d8/d15/d37/d43/daf/fb9 x:0 0 0 2024-04-24T23:13:42.259 INFO:tasks.workunit.client.1.smithi040.stdout:9/102: fdatasync d7/f15 0 2024-04-24T23:13:42.262 INFO:tasks.workunit.client.0.smithi031.stdout:5/367: mknod d0/d6/d3b/d4b/d1d/d2e/c76 0 2024-04-24T23:13:42.263 INFO:tasks.workunit.client.1.smithi040.stdout:5/62: symlink d2/d7/d8/db/l1a 0 2024-04-24T23:13:42.264 INFO:tasks.workunit.client.0.smithi031.stdout:6/520: mkdir d13/d15/d1d/db1 0 2024-04-24T23:13:42.265 INFO:tasks.workunit.client.0.smithi031.stdout:6/521: write d13/d15/d1d/d23/d44/d47/d69/faa [704450,8674] 0 2024-04-24T23:13:42.266 INFO:tasks.workunit.client.1.smithi040.stdout:3/59: unlink f0 0 2024-04-24T23:13:42.267 INFO:tasks.workunit.client.1.smithi040.stdout:1/42: dwrite da/f10 [0,4194304] 0 2024-04-24T23:13:42.268 INFO:tasks.workunit.client.1.smithi040.stdout:4/22: sync 2024-04-24T23:13:42.269 INFO:tasks.workunit.client.0.smithi031.stdout:8/510: rmdir d8/d15/d37/d62/d76 39 2024-04-24T23:13:42.279 INFO:tasks.workunit.client.0.smithi031.stdout:6/522: dread - d13/d15/d1d/f74 zero size 2024-04-24T23:13:42.281 INFO:tasks.workunit.client.0.smithi031.stdout:7/357: link d1/d2/d1e/c6e d1/d6a/d65/c71 0 2024-04-24T23:13:42.282 INFO:tasks.workunit.client.1.smithi040.stdout:2/100: mknod da/d1c/c1d 0 2024-04-24T23:13:42.284 INFO:tasks.workunit.client.0.smithi031.stdout:8/511: creat d8/d15/d37/d43/d5c/fba x:0 0 0 2024-04-24T23:13:42.284 INFO:tasks.workunit.client.1.smithi040.stdout:6/54: truncate d7/f8 1696317 0 2024-04-24T23:13:42.287 INFO:tasks.workunit.client.0.smithi031.stdout:2/309: creat d5/d8/f6c x:0 0 0 2024-04-24T23:13:42.288 INFO:tasks.workunit.client.0.smithi031.stdout:2/310: truncate d5/d8/d15/f5f 641855 0 2024-04-24T23:13:42.289 INFO:tasks.workunit.client.0.smithi031.stdout:5/368: getdents d0/d7/d74 0 2024-04-24T23:13:42.289 INFO:tasks.workunit.client.0.smithi031.stdout:2/311: write d5/d28/f54 [1102390,32466] 0 2024-04-24T23:13:42.290 INFO:tasks.workunit.client.1.smithi040.stdout:5/63: creat d2/d7/d8/f1b x:0 0 0 2024-04-24T23:13:42.291 INFO:tasks.workunit.client.1.smithi040.stdout:5/64: write d2/d7/fe [1043799,62358] 0 2024-04-24T23:13:42.292 INFO:tasks.workunit.client.1.smithi040.stdout:3/60: rmdir d1/d9 39 2024-04-24T23:13:42.292 INFO:tasks.workunit.client.1.smithi040.stdout:3/61: dread - d1/f8 zero size 2024-04-24T23:13:42.293 INFO:tasks.workunit.client.0.smithi031.stdout:6/523: mknod d13/d75/d79/cb2 0 2024-04-24T23:13:42.295 INFO:tasks.workunit.client.1.smithi040.stdout:1/43: mkdir da/d12 0 2024-04-24T23:13:42.297 INFO:tasks.workunit.client.0.smithi031.stdout:7/358: truncate d1/d2/f12 728601 0 2024-04-24T23:13:42.298 INFO:tasks.workunit.client.1.smithi040.stdout:7/110: write f5 [775810,58093] 0 2024-04-24T23:13:42.299 INFO:tasks.workunit.client.1.smithi040.stdout:4/23: mknod da/cc 0 2024-04-24T23:13:42.299 INFO:tasks.workunit.client.1.smithi040.stdout:3/62: dwrite d1/f8 [0,4194304] 0 2024-04-24T23:13:42.300 INFO:tasks.workunit.client.1.smithi040.stdout:7/111: chown db/d12/d20/f22 82 1 2024-04-24T23:13:42.301 INFO:tasks.workunit.client.1.smithi040.stdout:4/24: write da/fb [1040348,8653] 0 2024-04-24T23:13:42.301 INFO:tasks.workunit.client.0.smithi031.stdout:1/351: dwrite d1/dc/d11/f24 [0,4194304] 0 2024-04-24T23:13:42.302 INFO:tasks.workunit.client.1.smithi040.stdout:1/44: dread f7 [0,4194304] 0 2024-04-24T23:13:42.304 INFO:tasks.workunit.client.0.smithi031.stdout:5/369: mkdir d0/d6/d3b/d4b/d1d/d2e/d28/d77 0 2024-04-24T23:13:42.306 INFO:tasks.workunit.client.0.smithi031.stdout:2/312: creat d5/d8/d45/d5b/d49/f6d x:0 0 0 2024-04-24T23:13:42.307 INFO:tasks.workunit.client.0.smithi031.stdout:2/313: fdatasync d5/f25 0 2024-04-24T23:13:42.309 INFO:tasks.workunit.client.1.smithi040.stdout:2/101: symlink da/d1c/l1e 0 2024-04-24T23:13:42.310 INFO:tasks.workunit.client.0.smithi031.stdout:6/524: rename f10 to d13/d15/d1d/d23/d24/d43/fb3 0 2024-04-24T23:13:42.312 INFO:tasks.workunit.client.1.smithi040.stdout:9/103: creat d7/d16/d1e/d28/f29 x:0 0 0 2024-04-24T23:13:42.313 INFO:tasks.workunit.client.0.smithi031.stdout:2/314: dread d5/d28/d43/f55 [0,4194304] 0 2024-04-24T23:13:42.315 INFO:tasks.workunit.client.0.smithi031.stdout:7/359: symlink d1/d2/dd/d33/l72 0 2024-04-24T23:13:42.319 INFO:tasks.workunit.client.1.smithi040.stdout:9/104: dwrite d7/d16/d1e/d28/f29 [0,4194304] 0 2024-04-24T23:13:42.320 INFO:tasks.workunit.client.0.smithi031.stdout:2/315: dwrite d5/d8/d15/f34 [0,4194304] 0 2024-04-24T23:13:42.321 INFO:tasks.workunit.client.1.smithi040.stdout:5/65: rename d2/c3 to d2/d7/d8/d11/d14/c1c 0 2024-04-24T23:13:42.322 INFO:tasks.workunit.client.0.smithi031.stdout:2/316: read - d5/f46 zero size 2024-04-24T23:13:42.323 INFO:tasks.workunit.client.0.smithi031.stdout:2/317: write d5/d8/f6c [54735,82185] 0 2024-04-24T23:13:42.325 INFO:tasks.workunit.client.1.smithi040.stdout:9/105: dread d7/da/fd [0,4194304] 0 2024-04-24T23:13:42.328 INFO:tasks.workunit.client.1.smithi040.stdout:5/66: dread d2/d7/fe [0,4194304] 0 2024-04-24T23:13:42.331 INFO:tasks.workunit.client.1.smithi040.stdout:5/67: truncate d2/d7/d8/f13 843476 0 2024-04-24T23:13:42.332 INFO:tasks.workunit.client.1.smithi040.stdout:9/106: dread d7/d16/d1e/d28/f29 [0,4194304] 0 2024-04-24T23:13:42.338 INFO:tasks.workunit.client.0.smithi031.stdout:5/370: rename d0/d6/d3b/d4b/d1d/d2e/d28/l38 to d0/d6/d3b/l78 0 2024-04-24T23:13:42.341 INFO:tasks.workunit.client.0.smithi031.stdout:6/525: unlink d13/d15/f76 0 2024-04-24T23:13:42.342 INFO:tasks.workunit.client.1.smithi040.stdout:2/102: symlink da/d10/l1f 0 2024-04-24T23:13:42.345 INFO:tasks.workunit.client.0.smithi031.stdout:7/360: symlink d1/d2/dd/d33/l73 0 2024-04-24T23:13:42.345 INFO:tasks.workunit.client.1.smithi040.stdout:7/112: dwrite db/d12/d20/f21 [0,4194304] 0 2024-04-24T23:13:42.345 INFO:tasks.workunit.client.1.smithi040.stdout:6/55: creat d7/de/ff x:0 0 0 2024-04-24T23:13:42.346 INFO:tasks.workunit.client.0.smithi031.stdout:8/512: creat d8/d15/d37/d62/fbb x:0 0 0 2024-04-24T23:13:42.352 INFO:tasks.workunit.client.1.smithi040.stdout:4/25: rename c8 to da/cd 0 2024-04-24T23:13:42.353 INFO:tasks.workunit.client.1.smithi040.stdout:4/26: truncate f6 361525 0 2024-04-24T23:13:42.357 INFO:tasks.workunit.client.1.smithi040.stdout:0/109: sync 2024-04-24T23:13:42.360 INFO:tasks.workunit.client.0.smithi031.stdout:6/526: rename d13/d75 to d13/d1b/d32/db4 0 2024-04-24T23:13:42.367 INFO:tasks.workunit.client.0.smithi031.stdout:2/318: mkdir d5/d8/d6e 0 2024-04-24T23:13:42.369 INFO:tasks.workunit.client.0.smithi031.stdout:4/484: sync 2024-04-24T23:13:42.372 INFO:tasks.workunit.client.0.smithi031.stdout:1/352: link d1/f2 d1/dc/f70 0 2024-04-24T23:13:42.373 INFO:tasks.workunit.client.1.smithi040.stdout:5/68: rmdir d2/d7/d8 39 2024-04-24T23:13:42.373 INFO:tasks.workunit.client.1.smithi040.stdout:6/56: fsync d7/de/ff 0 2024-04-24T23:13:42.373 INFO:tasks.workunit.client.1.smithi040.stdout:8/35: getdents d0 0 2024-04-24T23:13:42.374 INFO:tasks.workunit.client.1.smithi040.stdout:6/57: stat d7 0 2024-04-24T23:13:42.374 INFO:tasks.workunit.client.1.smithi040.stdout:3/63: fdatasync d1/fd 0 2024-04-24T23:13:42.376 INFO:tasks.workunit.client.0.smithi031.stdout:7/361: write d1/d2/dd/f17 [746409,87276] 0 2024-04-24T23:13:42.380 INFO:tasks.workunit.client.0.smithi031.stdout:2/319: mkdir d5/d8/d45/d5b/d49/d6f 0 2024-04-24T23:13:42.381 INFO:tasks.workunit.client.1.smithi040.stdout:8/36: dwrite d0/f7 [0,4194304] 0 2024-04-24T23:13:42.383 INFO:tasks.workunit.client.0.smithi031.stdout:4/485: creat d3/d23/d3b/f9c x:0 0 0 2024-04-24T23:13:42.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.383+0000 7fa7b0b43700 1 -- 172.21.15.31:0/2231058018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac066200 msgr2=0x7fa7ac0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:42.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.383+0000 7fa7b0b43700 1 --2- 172.21.15.31:0/2231058018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac066200 0x7fa7ac0ffa60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0004880 tx=0x7fa7a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:42.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.383+0000 7fa7b0b43700 1 -- 172.21.15.31:0/2231058018 shutdown_connections 2024-04-24T23:13:42.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.383+0000 7fa7b0b43700 1 --2- 172.21.15.31:0/2231058018 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7ac0fffa0 0x7fa7ac102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:42.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.383+0000 7fa7b0b43700 1 --2- 172.21.15.31:0/2231058018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac066200 0x7fa7ac0ffa60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:42.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.383+0000 7fa7b0b43700 1 -- 172.21.15.31:0/2231058018 >> 172.21.15.31:0/2231058018 conn(0x7fa7ac0f7430 msgr2=0x7fa7ac0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:42.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.383+0000 7fa7b0b43700 1 -- 172.21.15.31:0/2231058018 shutdown_connections 2024-04-24T23:13:42.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.383+0000 7fa7b0b43700 1 -- 172.21.15.31:0/2231058018 wait complete. 2024-04-24T23:13:42.386 INFO:tasks.workunit.client.1.smithi040.stdout:4/27: mknod da/ce 0 2024-04-24T23:13:42.386 INFO:tasks.workunit.client.1.smithi040.stdout:4/28: dread - f5 zero size 2024-04-24T23:13:42.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.384+0000 7fa7b0b43700 1 Processor -- start 2024-04-24T23:13:42.386 INFO:tasks.workunit.client.0.smithi031.stdout:8/513: symlink d8/d1a/d1d/d35/d58/lbc 0 2024-04-24T23:13:42.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.384+0000 7fa7b0b43700 1 -- start start 2024-04-24T23:13:42.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.385+0000 7fa7b0b43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7ac066200 0x7fa7ac0fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:42.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.385+0000 7fa7b0b43700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac0fffa0 0x7fa7ac0fb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:42.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.385+0000 7fa7b0b43700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0fbf10 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.385+0000 7fa7b0b43700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0fc080 con 0x7fa7ac066200 2024-04-24T23:13:42.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.385+0000 7fa7a9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac0fffa0 0x7fa7ac0fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:42.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.385+0000 7fa7a9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac0fffa0 0x7fa7ac0fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58310/0 (socket says 172.21.15.31:58310) 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.385+0000 7fa7a9d9b700 1 -- 172.21.15.31:0/33267953 learned_addr learned my addr 172.21.15.31:0/33267953 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.385+0000 7fa7aa59c700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7ac066200 0x7fa7ac0fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa7a9d9b700 1 -- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7ac066200 msgr2=0x7fa7ac0fd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa7a9d9b700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7ac066200 0x7fa7ac0fd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa7a9d9b700 1 -- 172.21.15.31:0/33267953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a0005040 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa7a9d9b700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac0fffa0 0x7fa7ac0fb960 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa79c002a80 tx=0x7fa79c004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa79affd700 1 -- 172.21.15.31:0/33267953 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79c007cc0 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa79affd700 1 -- 172.21.15.31:0/33267953 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa79c00f040 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac0fc360 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa79affd700 1 -- 172.21.15.31:0/33267953 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79c00da30 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.386+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac1159c0 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.390 INFO:tasks.workunit.client.1.smithi040.stdout:0/110: write f2 [9916468,118490] 0 2024-04-24T23:13:42.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.388+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c0052e0 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.390 INFO:tasks.workunit.client.0.smithi031.stdout:5/371: creat d0/d6/d3b/d4b/d1d/d2e/f79 x:0 0 0 2024-04-24T23:13:42.390 INFO:tasks.workunit.client.0.smithi031.stdout:7/362: creat d1/d2/dd/f74 x:0 0 0 2024-04-24T23:13:42.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.390+0000 7fa79affd700 1 -- 172.21.15.31:0/33267953 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7fa79c00db90 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.391 INFO:tasks.workunit.client.0.smithi031.stdout:5/372: write d0/d6/d12/f31 [563904,116729] 0 2024-04-24T23:13:42.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.390+0000 7fa79affd700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fa79406b860 0x7fa79406dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:13:42.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.390+0000 7fa79affd700 1 -- 172.21.15.31:0/33267953 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fa79c08a710 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.391+0000 7fa7aa59c700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fa79406b860 0x7fa79406dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:13:42.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.392+0000 7fa7aa59c700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fa79406b860 0x7fa79406dd10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009fd0 tx=0x7fa7a0009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:13:42.395 INFO:tasks.workunit.client.0.smithi031.stdout:8/514: mkdir d8/d15/d37/d43/dbd 0 2024-04-24T23:13:42.399 INFO:tasks.workunit.client.0.smithi031.stdout:7/363: dwrite d1/f53 [0,4194304] 0 2024-04-24T23:13:42.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.397+0000 7fa79affd700 1 -- 172.21.15.31:0/33267953 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa79c054250 con 0x7fa7ac0fffa0 2024-04-24T23:13:42.401 INFO:tasks.workunit.client.0.smithi031.stdout:2/320: getdents d5/d28/d4e/d64 0 2024-04-24T23:13:42.404 INFO:tasks.workunit.client.0.smithi031.stdout:4/486: fsync d3/fc 0 2024-04-24T23:13:42.410 INFO:tasks.workunit.client.0.smithi031.stdout:2/321: dwrite d5/d28/f54 [0,4194304] 0 2024-04-24T23:13:42.410 INFO:tasks.workunit.client.0.smithi031.stdout:8/515: rmdir d8/d1a/d1d/d35/d58 39 2024-04-24T23:13:42.416 INFO:tasks.workunit.client.0.smithi031.stdout:7/364: mkdir d1/d45/d75 0 2024-04-24T23:13:42.420 INFO:tasks.workunit.client.0.smithi031.stdout:4/487: dwrite d3/d20/d2d/f92 [0,4194304] 0 2024-04-24T23:13:42.428 INFO:tasks.workunit.client.0.smithi031.stdout:2/322: dwrite d5/d6/d29/d26/f3f [0,4194304] 0 2024-04-24T23:13:42.433 INFO:tasks.workunit.client.0.smithi031.stdout:1/353: dwrite d1/dc/d4c/f6a [4194304,4194304] 0 2024-04-24T23:13:42.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/488: symlink d3/d20/d2d/d44/d5b/d5d/l9d 0 2024-04-24T23:13:42.443 INFO:tasks.workunit.client.0.smithi031.stdout:8/516: write d8/d15/d37/d43/f4b [1382559,28715] 0 2024-04-24T23:13:42.445 INFO:tasks.workunit.client.0.smithi031.stdout:8/517: read d8/d15/d9a/fb2 [347494,31610] 0 2024-04-24T23:13:42.446 INFO:tasks.workunit.client.0.smithi031.stdout:8/518: readlink d8/d1a/l32 0 2024-04-24T23:13:42.446 INFO:tasks.workunit.client.0.smithi031.stdout:8/519: dread - d8/d1a/f9b zero size 2024-04-24T23:13:42.448 INFO:tasks.workunit.client.0.smithi031.stdout:8/520: write d8/fa8 [411395,112093] 0 2024-04-24T23:13:42.452 INFO:tasks.workunit.client.0.smithi031.stdout:5/373: getdents d0/d6/d3b/d4b/d1d/d2e/d28 0 2024-04-24T23:13:42.454 INFO:tasks.workunit.client.0.smithi031.stdout:2/323: mkdir d5/d8/d6e/d70 0 2024-04-24T23:13:42.454 INFO:tasks.workunit.client.0.smithi031.stdout:5/374: dread d0/d6/d12/f36 [0,4194304] 0 2024-04-24T23:13:42.455 INFO:tasks.workunit.client.0.smithi031.stdout:5/375: dread - d0/d6/d3b/f68 zero size 2024-04-24T23:13:42.458 INFO:tasks.workunit.client.0.smithi031.stdout:2/324: symlink d5/d28/d4e/d5e/l71 0 2024-04-24T23:13:42.458 INFO:tasks.workunit.client.1.smithi040.stdout:1/45: sync 2024-04-24T23:13:42.460 INFO:tasks.workunit.client.0.smithi031.stdout:8/521: creat d8/d15/d37/d62/d76/d9d/fbe x:0 0 0 2024-04-24T23:13:42.462 INFO:tasks.workunit.client.0.smithi031.stdout:8/522: read d8/d15/d37/f5e [1106458,89281] 0 2024-04-24T23:13:42.472 INFO:tasks.workunit.client.0.smithi031.stdout:2/325: dwrite d5/d6/d29/f61 [0,4194304] 0 2024-04-24T23:13:42.473 INFO:tasks.workunit.client.0.smithi031.stdout:5/376: getdents d0/d6/d3b/d4b/dc/d25/d55 0 2024-04-24T23:13:42.474 INFO:tasks.workunit.client.0.smithi031.stdout:4/489: dread d3/d20/d25/d60/d63/f81 [0,4194304] 0 2024-04-24T23:13:42.476 INFO:tasks.workunit.client.0.smithi031.stdout:5/377: readlink d0/d6/d3b/d4b/dc/l64 0 2024-04-24T23:13:42.477 INFO:tasks.workunit.client.1.smithi040.stdout:2/103: sync 2024-04-24T23:13:42.477 INFO:tasks.workunit.client.1.smithi040.stdout:7/113: sync 2024-04-24T23:13:42.477 INFO:tasks.workunit.client.0.smithi031.stdout:7/365: dwrite d1/d2/f2a [0,4194304] 0 2024-04-24T23:13:42.482 INFO:tasks.workunit.client.0.smithi031.stdout:2/326: creat d5/d6/d29/f72 x:0 0 0 2024-04-24T23:13:42.484 INFO:tasks.workunit.client.0.smithi031.stdout:2/327: write d5/d6/f1a [1688733,113091] 0 2024-04-24T23:13:42.486 INFO:tasks.workunit.client.0.smithi031.stdout:4/490: dread d3/f4a [0,4194304] 0 2024-04-24T23:13:42.491 INFO:tasks.workunit.client.0.smithi031.stdout:7/366: symlink d1/d2/dd/d33/d5e/d64/l76 0 2024-04-24T23:13:42.498 INFO:tasks.workunit.client.0.smithi031.stdout:4/491: rmdir d3/d20/d25/d7e 39 2024-04-24T23:13:42.499 INFO:tasks.workunit.client.0.smithi031.stdout:4/492: chown d3/d23/d3b/c5a 827300 1 2024-04-24T23:13:42.500 INFO:tasks.workunit.client.0.smithi031.stdout:4/493: chown d3/d20/d2d/d44/d5b/d5d/f97 248982006 1 2024-04-24T23:13:42.502 INFO:tasks.workunit.client.0.smithi031.stdout:7/367: creat d1/d2/dd/d33/f77 x:0 0 0 2024-04-24T23:13:42.503 INFO:tasks.workunit.client.0.smithi031.stdout:7/368: read - d1/d2/dd/d33/f62 zero size 2024-04-24T23:13:42.506 INFO:tasks.workunit.client.0.smithi031.stdout:5/378: link d0/d6/d3b/d4b/f51 d0/d6/d3b/d4b/f7a 0 2024-04-24T23:13:42.512 INFO:tasks.workunit.client.0.smithi031.stdout:5/379: creat d0/d6/d32/d6b/f7b x:0 0 0 2024-04-24T23:13:42.517 INFO:tasks.workunit.client.0.smithi031.stdout:2/328: getdents d5/d28/d4e 0 2024-04-24T23:13:42.528 INFO:tasks.workunit.client.0.smithi031.stdout:7/369: write d1/d2/f12 [1156019,111103] 0 2024-04-24T23:13:42.530 INFO:tasks.workunit.client.0.smithi031.stdout:4/494: dwrite d3/d23/d3b/f41 [0,4194304] 0 2024-04-24T23:13:42.532 INFO:tasks.workunit.client.0.smithi031.stdout:4/495: write d3/d23/d69/d3f/f6d [264028,80416] 0 2024-04-24T23:13:42.535 INFO:tasks.workunit.client.0.smithi031.stdout:7/370: dread d1/d2/f2d [0,4194304] 0 2024-04-24T23:13:42.536 INFO:tasks.workunit.client.0.smithi031.stdout:4/496: chown d3/d20/d25/d60/d63/f8b 37406198 1 2024-04-24T23:13:42.539 INFO:tasks.workunit.client.0.smithi031.stdout:7/371: creat d1/d6a/d66/f78 x:0 0 0 2024-04-24T23:13:42.541 INFO:tasks.workunit.client.0.smithi031.stdout:4/497: mkdir d3/d23/d69/d3f/d9e 0 2024-04-24T23:13:42.542 INFO:tasks.workunit.client.0.smithi031.stdout:4/498: readlink d3/d20/d2d/d44/d5b/l82 0 2024-04-24T23:13:42.543 INFO:tasks.workunit.client.0.smithi031.stdout:4/499: fsync d3/d23/f3a 0 2024-04-24T23:13:42.543 INFO:tasks.workunit.client.0.smithi031.stdout:7/372: symlink d1/d2/dd/d33/l79 0 2024-04-24T23:13:42.544 INFO:tasks.workunit.client.0.smithi031.stdout:7/373: fdatasync d1/f2b 0 2024-04-24T23:13:42.544 INFO:tasks.workunit.client.0.smithi031.stdout:7/374: stat d1/d2/d60 0 2024-04-24T23:13:42.548 INFO:tasks.workunit.client.0.smithi031.stdout:4/500: symlink d3/d20/l9f 0 2024-04-24T23:13:42.552 INFO:tasks.workunit.client.0.smithi031.stdout:4/501: symlink d3/d23/la0 0 2024-04-24T23:13:42.553 INFO:tasks.workunit.client.0.smithi031.stdout:4/502: dread - d3/d20/d2d/f52 zero size 2024-04-24T23:13:42.559 INFO:tasks.workunit.client.0.smithi031.stdout:4/503: creat d3/d20/d25/d7e/fa1 x:0 0 0 2024-04-24T23:13:42.562 INFO:tasks.workunit.client.0.smithi031.stdout:4/504: creat d3/d20/fa2 x:0 0 0 2024-04-24T23:13:42.564 INFO:tasks.workunit.client.0.smithi031.stdout:4/505: truncate d3/d20/d2d/d44/d5b/d5d/f96 774898 0 2024-04-24T23:13:42.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/506: mkdir d3/da3 0 2024-04-24T23:13:42.569 INFO:tasks.workunit.client.0.smithi031.stdout:7/375: write d1/f9 [1700666,124681] 0 2024-04-24T23:13:42.574 INFO:tasks.workunit.client.0.smithi031.stdout:4/507: rename d3/fc to d3/d20/fa4 0 2024-04-24T23:13:42.575 INFO:tasks.workunit.client.0.smithi031.stdout:7/376: symlink d1/d2/dd/d33/l7a 0 2024-04-24T23:13:42.576 INFO:tasks.workunit.client.0.smithi031.stdout:7/377: read d1/f53 [2862859,49489] 0 2024-04-24T23:13:42.576 INFO:tasks.workunit.client.0.smithi031.stdout:4/508: symlink d3/d20/d2d/d44/d5b/la5 0 2024-04-24T23:13:42.581 INFO:tasks.workunit.client.0.smithi031.stdout:7/378: truncate d1/d2/dd/d33/f4e 1261475 0 2024-04-24T23:13:42.582 INFO:tasks.workunit.client.0.smithi031.stdout:4/509: dread d3/d20/d2d/f40 [0,4194304] 0 2024-04-24T23:13:42.585 INFO:tasks.workunit.client.0.smithi031.stdout:7/379: symlink d1/d2/l7b 0 2024-04-24T23:13:42.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/380: creat d1/d6a/f7c x:0 0 0 2024-04-24T23:13:42.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/381: readlink d1/d2/dd/d33/l79 0 2024-04-24T23:13:42.595 INFO:tasks.workunit.client.0.smithi031.stdout:7/382: unlink d1/d45/f47 0 2024-04-24T23:13:42.621 INFO:tasks.workunit.client.0.smithi031.stdout:9/471: sync 2024-04-24T23:13:42.621 INFO:tasks.workunit.client.0.smithi031.stdout:3/362: sync 2024-04-24T23:13:42.622 INFO:tasks.workunit.client.0.smithi031.stdout:0/476: sync 2024-04-24T23:13:42.622 INFO:tasks.workunit.client.0.smithi031.stdout:0/477: chown da/d11/d6a 0 1 2024-04-24T23:13:42.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/363: dread - d6/d14/d1d/d59/f5e zero size 2024-04-24T23:13:42.623 INFO:tasks.workunit.client.0.smithi031.stdout:7/383: creat d1/d2/dd/f7d x:0 0 0 2024-04-24T23:13:42.632 INFO:tasks.workunit.client.0.smithi031.stdout:3/364: symlink d6/l75 0 2024-04-24T23:13:42.634 INFO:tasks.workunit.client.0.smithi031.stdout:3/365: dread - d6/df/d16/d57/f65 zero size 2024-04-24T23:13:42.635 INFO:tasks.workunit.client.0.smithi031.stdout:7/384: mknod d1/d2/c7e 0 2024-04-24T23:13:42.640 INFO:tasks.workunit.client.0.smithi031.stdout:9/472: dread d2/d43/f58 [0,4194304] 0 2024-04-24T23:13:42.641 INFO:tasks.workunit.client.0.smithi031.stdout:3/366: dwrite d6/df/d16/d57/f65 [0,4194304] 0 2024-04-24T23:13:42.646 INFO:tasks.workunit.client.0.smithi031.stdout:9/473: truncate d2/d16/f2a 3650960 0 2024-04-24T23:13:42.651 INFO:tasks.workunit.client.0.smithi031.stdout:4/510: truncate d3/f2b 2479297 0 2024-04-24T23:13:42.658 INFO:tasks.workunit.client.0.smithi031.stdout:4/511: mknod d3/d20/d25/d95/ca6 0 2024-04-24T23:13:42.662 INFO:tasks.workunit.client.0.smithi031.stdout:4/512: unlink d3/d20/d2d/d44/d5b/f9b 0 2024-04-24T23:13:42.663 INFO:tasks.workunit.client.0.smithi031.stdout:4/513: truncate d3/d23/d37/f7c 600770 0 2024-04-24T23:13:42.664 INFO:tasks.workunit.client.0.smithi031.stdout:4/514: write d3/d23/d69/d3f/f6d [107135,100649] 0 2024-04-24T23:13:42.669 INFO:tasks.workunit.client.0.smithi031.stdout:4/515: truncate d3/d20/d25/d60/d63/f81 4332483 0 2024-04-24T23:13:42.669 INFO:tasks.workunit.client.0.smithi031.stdout:4/516: stat d3/f75 0 2024-04-24T23:13:42.673 INFO:tasks.workunit.client.0.smithi031.stdout:0/478: dwrite da/d18/d1a/d76/f8d [4194304,4194304] 0 2024-04-24T23:13:42.674 INFO:tasks.workunit.client.0.smithi031.stdout:0/479: chown da/d18/f2f 15 1 2024-04-24T23:13:42.681 INFO:tasks.workunit.client.0.smithi031.stdout:0/480: dwrite da/fd [0,4194304] 0 2024-04-24T23:13:42.687 INFO:tasks.workunit.client.0.smithi031.stdout:4/517: dread d3/f5c [0,4194304] 0 2024-04-24T23:13:42.689 INFO:tasks.workunit.client.0.smithi031.stdout:9/474: write d2/d43/f70 [1476911,60751] 0 2024-04-24T23:13:42.694 INFO:tasks.workunit.client.0.smithi031.stdout:3/367: truncate d6/df/d16/d57/f65 4029386 0 2024-04-24T23:13:42.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/518: dwrite d3/d23/d3b/f43 [0,4194304] 0 2024-04-24T23:13:42.696 INFO:tasks.workunit.client.0.smithi031.stdout:3/368: chown d6/d14/d1d/d59/f5e 910 1 2024-04-24T23:13:42.699 INFO:tasks.workunit.client.0.smithi031.stdout:9/475: symlink d2/d1b/d14/d78/d84/d4a/l8a 0 2024-04-24T23:13:42.702 INFO:tasks.workunit.client.0.smithi031.stdout:4/519: write d3/d20/d25/f90 [725757,51381] 0 2024-04-24T23:13:42.705 INFO:tasks.workunit.client.0.smithi031.stdout:3/369: dwrite d6/d72/d1a/f62 [0,4194304] 0 2024-04-24T23:13:42.710 INFO:tasks.workunit.client.0.smithi031.stdout:4/520: unlink d3/d20/d2d/f92 0 2024-04-24T23:13:42.716 INFO:tasks.workunit.client.0.smithi031.stdout:4/521: mkdir d3/d20/da7 0 2024-04-24T23:13:42.716 INFO:tasks.workunit.client.0.smithi031.stdout:4/522: stat d3/d23/l3c 0 2024-04-24T23:13:42.721 INFO:tasks.workunit.client.0.smithi031.stdout:3/370: creat d6/df/d16/f76 x:0 0 0 2024-04-24T23:13:42.722 INFO:tasks.workunit.client.0.smithi031.stdout:4/523: mknod d3/d20/d2d/d44/d5b/ca8 0 2024-04-24T23:13:42.730 INFO:tasks.workunit.client.0.smithi031.stdout:4/524: link d3/d20/f51 d3/d23/d37/fa9 0 2024-04-24T23:13:42.737 INFO:tasks.workunit.client.0.smithi031.stdout:3/371: link d6/d14/l21 d6/df/l77 0 2024-04-24T23:13:42.738 INFO:tasks.workunit.client.0.smithi031.stdout:4/525: dwrite d3/d20/d25/d60/d63/f8b [0,4194304] 0 2024-04-24T23:13:42.740 INFO:tasks.workunit.client.0.smithi031.stdout:9/476: write d2/d1b/d14/d1d/f47 [1020819,64554] 0 2024-04-24T23:13:42.745 INFO:tasks.workunit.client.0.smithi031.stdout:9/477: symlink d2/d1b/d14/d78/d84/d7e/l8b 0 2024-04-24T23:13:42.746 INFO:tasks.workunit.client.0.smithi031.stdout:9/478: fdatasync d2/d1b/d14/d1d/f7b 0 2024-04-24T23:13:42.754 INFO:tasks.workunit.client.0.smithi031.stdout:9/479: symlink d2/d1b/d14/d78/d84/l8c 0 2024-04-24T23:13:42.755 INFO:tasks.workunit.client.0.smithi031.stdout:3/372: link d6/d14/l6a d6/l78 0 2024-04-24T23:13:42.757 INFO:tasks.workunit.client.0.smithi031.stdout:3/373: write d6/df/d16/d2a/f5a [95356,17183] 0 2024-04-24T23:13:42.758 INFO:tasks.workunit.client.0.smithi031.stdout:9/480: dread d2/f1c [0,4194304] 0 2024-04-24T23:13:42.763 INFO:tasks.workunit.client.0.smithi031.stdout:3/374: creat d6/d72/d29/f79 x:0 0 0 2024-04-24T23:13:42.766 INFO:tasks.workunit.client.0.smithi031.stdout:3/375: mknod d6/d14/d1d/d59/c7a 0 2024-04-24T23:13:42.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.767+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa78c000bf0 con 0x7fa79406b860 2024-04-24T23:13:42.769 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:42 smithi031 ceph-mon[26411]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:42.769 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:42 smithi031 ceph-mon[26411]: pgmap v24: 65 pgs: 65 active+clean; 1.0 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 18 MiB/s rd, 125 MiB/s wr, 403 op/s 2024-04-24T23:13:42.769 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:42 smithi031 ceph-mon[26411]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:42.769 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:42 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/473902929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.768+0000 7fa79affd700 1 -- 172.21.15.31:0/33267953 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fa78c000bf0 con 0x7fa79406b860 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-04-24T23:13:42.770 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:13:42.771 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:13:42.772 INFO:tasks.workunit.client.0.smithi031.stdout:3/376: dwrite d6/df/d16/d2a/f4f [0,4194304] 0 2024-04-24T23:13:42.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fa79406b860 msgr2=0x7fa79406dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fa79406b860 0x7fa79406dd10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009fd0 tx=0x7fa7a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac0fffa0 msgr2=0x7fa7ac0fb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac0fffa0 0x7fa7ac0fb960 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa79c002a80 tx=0x7fa79c004630 comp rx=0 tx=0).stop 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 shutdown_connections 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fa79406b860 0x7fa79406dd10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa7ac066200 0x7fa7ac0fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 --2- 172.21.15.31:0/33267953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7ac0fffa0 0x7fa7ac0fb960 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa79c002a80 tx=0x7fa79c004630 comp rx=0 tx=0).stop 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.771+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 >> 172.21.15.31:0/33267953 conn(0x7fa7ac0f7430 msgr2=0x7fa7ac0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.772+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 shutdown_connections 2024-04-24T23:13:42.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:13:42.772+0000 7fa7b0b43700 1 -- 172.21.15.31:0/33267953 wait complete. 2024-04-24T23:13:42.775 INFO:tasks.workunit.client.0.smithi031.stdout:3/377: write d6/d72/d1a/d25/f54 [24251,104702] 0 2024-04-24T23:13:42.775 INFO:tasks.workunit.client.0.smithi031.stdout:9/481: creat d2/d1b/d14/d78/d84/d4a/f8d x:0 0 0 2024-04-24T23:13:42.786 INFO:tasks.workunit.client.0.smithi031.stdout:3/378: creat d6/f7b x:0 0 0 2024-04-24T23:13:42.786 INFO:tasks.workunit.client.0.smithi031.stdout:6/527: sync 2024-04-24T23:13:42.787 INFO:tasks.workunit.client.0.smithi031.stdout:6/528: chown d13/d15/d1d/c95 14740 1 2024-04-24T23:13:42.792 INFO:tasks.workunit.client.0.smithi031.stdout:3/379: dwrite d6/d72/d60/f64 [0,4194304] 0 2024-04-24T23:13:42.797 INFO:tasks.workunit.client.0.smithi031.stdout:3/380: rename d6/df/d16/f76 to d6/d72/f7c 0 2024-04-24T23:13:42.801 INFO:tasks.workunit.client.0.smithi031.stdout:9/482: dread d2/d1b/d14/f39 [0,4194304] 0 2024-04-24T23:13:42.805 INFO:tasks.workunit.client.0.smithi031.stdout:3/381: read d6/d14/d1d/f24 [1874988,48531] 0 2024-04-24T23:13:42.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:42 smithi040 ceph-mon[33128]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:42.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:42 smithi040 ceph-mon[33128]: pgmap v24: 65 pgs: 65 active+clean; 1.0 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 18 MiB/s rd, 125 MiB/s wr, 403 op/s 2024-04-24T23:13:42.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:42 smithi040 ceph-mon[33128]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:42.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:42 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/473902929' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:13:42.931 INFO:tasks.workunit.client.0.smithi031.stdout:9/483: dwrite d2/f1c [0,4194304] 0 2024-04-24T23:13:42.932 INFO:tasks.workunit.client.0.smithi031.stdout:9/484: write d2/f32 [1028353,65233] 0 2024-04-24T23:13:42.935 INFO:tasks.workunit.client.0.smithi031.stdout:4/526: write d3/f2b [2167661,63623] 0 2024-04-24T23:13:42.943 INFO:tasks.workunit.client.0.smithi031.stdout:6/529: rmdir d13/d15/d1d/db1 0 2024-04-24T23:13:42.951 INFO:tasks.workunit.client.0.smithi031.stdout:6/530: chown d13/d15/d1d/d23/d24/c38 1 1 2024-04-24T23:13:42.958 INFO:tasks.workunit.client.0.smithi031.stdout:6/531: dwrite d13/f6c [0,4194304] 0 2024-04-24T23:13:42.962 INFO:tasks.workunit.client.0.smithi031.stdout:9/485: dread d2/d16/f20 [0,4194304] 0 2024-04-24T23:13:42.965 INFO:tasks.workunit.client.0.smithi031.stdout:9/486: mknod d2/d43/d6e/c8e 0 2024-04-24T23:13:42.974 INFO:tasks.workunit.client.0.smithi031.stdout:9/487: creat d2/d1b/f8f x:0 0 0 2024-04-24T23:13:42.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/354: sync 2024-04-24T23:13:42.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/355: chown d1 22955 1 2024-04-24T23:13:42.980 INFO:tasks.workunit.client.0.smithi031.stdout:9/488: rename d2/d1b/d14/d1d/l3e to d2/d43/l90 0 2024-04-24T23:13:42.982 INFO:tasks.workunit.client.0.smithi031.stdout:1/356: creat d1/d33/d36/f71 x:0 0 0 2024-04-24T23:13:42.989 INFO:tasks.workunit.client.0.smithi031.stdout:4/527: dwrite d3/f4a [0,4194304] 0 2024-04-24T23:13:42.992 INFO:tasks.workunit.client.1.smithi040.stdout:2/104: write da/f19 [2872760,94304] 0 2024-04-24T23:13:42.993 INFO:tasks.workunit.client.1.smithi040.stdout:7/114: unlink db/f14 0 2024-04-24T23:13:42.996 INFO:tasks.workunit.client.1.smithi040.stdout:9/107: creat d7/f2a x:0 0 0 2024-04-24T23:13:42.999 INFO:tasks.workunit.client.0.smithi031.stdout:4/528: link d3/d23/d3b/f72 d3/d20/d25/d7e/faa 0 2024-04-24T23:13:42.999 INFO:tasks.workunit.client.0.smithi031.stdout:4/529: stat d3/d20/l66 0 2024-04-24T23:13:43.000 INFO:tasks.workunit.client.0.smithi031.stdout:4/530: fdatasync d3/f9 0 2024-04-24T23:13:43.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/531: readlink d3/d20/l9f 0 2024-04-24T23:13:43.002 INFO:tasks.workunit.client.1.smithi040.stdout:6/58: link f2 d7/f10 0 2024-04-24T23:13:43.003 INFO:tasks.workunit.client.0.smithi031.stdout:9/489: dread d2/d16/f33 [0,4194304] 0 2024-04-24T23:13:43.003 INFO:tasks.workunit.client.1.smithi040.stdout:8/37: getdents d0/d8 0 2024-04-24T23:13:43.004 INFO:tasks.workunit.client.0.smithi031.stdout:9/490: read - d2/d1b/d85/f86 zero size 2024-04-24T23:13:43.004 INFO:tasks.workunit.client.1.smithi040.stdout:6/59: write d7/fb [244654,63943] 0 2024-04-24T23:13:43.005 INFO:tasks.workunit.client.1.smithi040.stdout:3/64: dwrite d1/f8 [0,4194304] 0 2024-04-24T23:13:43.005 INFO:tasks.workunit.client.0.smithi031.stdout:6/532: write d13/d1b/d32/db4/d79/f85 [1007564,47924] 0 2024-04-24T23:13:43.006 INFO:tasks.workunit.client.0.smithi031.stdout:4/532: mknod d3/d20/d25/d60/cab 0 2024-04-24T23:13:43.007 INFO:tasks.workunit.client.0.smithi031.stdout:6/533: dread - d13/d15/d1d/d23/d24/d63/f67 zero size 2024-04-24T23:13:43.008 INFO:tasks.workunit.client.1.smithi040.stdout:0/111: mkdir df/d1e/d20 0 2024-04-24T23:13:43.009 INFO:tasks.workunit.client.0.smithi031.stdout:4/533: write d3/d20/d2d/d44/d5b/d5d/f87 [121286,77081] 0 2024-04-24T23:13:43.009 INFO:tasks.workunit.client.1.smithi040.stdout:1/46: getdents da/d12 0 2024-04-24T23:13:43.010 INFO:tasks.workunit.client.0.smithi031.stdout:9/491: truncate d2/d16/f77 125738 0 2024-04-24T23:13:43.013 INFO:tasks.workunit.client.1.smithi040.stdout:2/105: unlink da/d1c/l1e 0 2024-04-24T23:13:43.013 INFO:tasks.workunit.client.1.smithi040.stdout:2/106: fsync da/f11 0 2024-04-24T23:13:43.013 INFO:tasks.workunit.client.1.smithi040.stdout:2/107: fsync f7 0 2024-04-24T23:13:43.014 INFO:tasks.workunit.client.1.smithi040.stdout:7/115: mknod db/d12/d20/c2c 0 2024-04-24T23:13:43.016 INFO:tasks.workunit.client.0.smithi031.stdout:6/534: creat d13/d15/d1d/d23/d24/d43/d68/fb5 x:0 0 0 2024-04-24T23:13:43.018 INFO:tasks.workunit.client.0.smithi031.stdout:4/534: dwrite d3/d20/d2d/f40 [0,4194304] 0 2024-04-24T23:13:43.019 INFO:tasks.workunit.client.1.smithi040.stdout:7/116: dwrite f2 [4194304,4194304] 0 2024-04-24T23:13:43.021 INFO:tasks.workunit.client.1.smithi040.stdout:9/108: symlink d7/da/l2b 0 2024-04-24T23:13:43.024 INFO:tasks.workunit.client.0.smithi031.stdout:1/357: write d1/f34 [800485,91032] 0 2024-04-24T23:13:43.025 INFO:tasks.workunit.client.0.smithi031.stdout:1/358: write d1/d3e/d46/f59 [14046,79191] 0 2024-04-24T23:13:43.030 INFO:tasks.workunit.client.0.smithi031.stdout:1/359: dread d1/d3e/d46/f5c [0,4194304] 0 2024-04-24T23:13:43.031 INFO:tasks.workunit.client.1.smithi040.stdout:6/60: unlink d7/f10 0 2024-04-24T23:13:43.032 INFO:tasks.workunit.client.1.smithi040.stdout:6/61: truncate d7/fd 262874 0 2024-04-24T23:13:43.032 INFO:tasks.workunit.client.0.smithi031.stdout:9/492: creat d2/d1b/d14/d78/d84/d4a/f91 x:0 0 0 2024-04-24T23:13:43.036 INFO:tasks.workunit.client.0.smithi031.stdout:6/535: link d13/d15/d1d/d23/d2d/f4c d13/d15/d1d/d23/d44/da0/da6/fb6 0 2024-04-24T23:13:43.041 INFO:tasks.workunit.client.1.smithi040.stdout:1/47: symlink da/l13 0 2024-04-24T23:13:43.042 INFO:tasks.workunit.client.0.smithi031.stdout:9/493: dread - d2/d1b/d14/d78/d84/d4a/d4b/f57 zero size 2024-04-24T23:13:43.044 INFO:tasks.workunit.client.0.smithi031.stdout:6/536: creat d13/d15/d1d/d23/d2d/d9e/fb7 x:0 0 0 2024-04-24T23:13:43.046 INFO:tasks.workunit.client.1.smithi040.stdout:2/108: creat da/f20 x:0 0 0 2024-04-24T23:13:43.046 INFO:tasks.workunit.client.0.smithi031.stdout:4/535: link d3/d20/d2d/c42 d3/d20/cac 0 2024-04-24T23:13:43.046 INFO:tasks.workunit.client.1.smithi040.stdout:2/109: truncate da/f11 867285 0 2024-04-24T23:13:43.049 INFO:tasks.workunit.client.1.smithi040.stdout:5/69: getdents d2 0 2024-04-24T23:13:43.050 INFO:tasks.workunit.client.0.smithi031.stdout:9/494: readlink d2/d16/l26 0 2024-04-24T23:13:43.051 INFO:tasks.workunit.client.1.smithi040.stdout:7/117: mknod db/d12/c2d 0 2024-04-24T23:13:43.052 INFO:tasks.workunit.client.0.smithi031.stdout:4/536: dread d3/d20/d2d/d44/f84 [0,4194304] 0 2024-04-24T23:13:43.053 INFO:tasks.workunit.client.0.smithi031.stdout:6/537: mkdir d13/d15/d1d/d23/d24/d43/d68/db8 0 2024-04-24T23:13:43.054 INFO:tasks.workunit.client.0.smithi031.stdout:4/537: read d3/d23/d69/d3f/f6d [217659,52027] 0 2024-04-24T23:13:43.054 INFO:tasks.workunit.client.0.smithi031.stdout:6/538: write d13/f7c [3745109,52208] 0 2024-04-24T23:13:43.055 INFO:tasks.workunit.client.1.smithi040.stdout:9/109: rmdir d7/da/d17 39 2024-04-24T23:13:43.056 INFO:tasks.workunit.client.0.smithi031.stdout:6/539: read d13/d15/d1d/d23/d24/f40 [1407687,41580] 0 2024-04-24T23:13:43.057 INFO:tasks.workunit.client.0.smithi031.stdout:6/540: chown d13/d15/fa5 1152831 1 2024-04-24T23:13:43.057 INFO:tasks.workunit.client.1.smithi040.stdout:8/38: symlink d0/d8/l9 0 2024-04-24T23:13:43.057 INFO:tasks.workunit.client.1.smithi040.stdout:6/62: mknod d7/c11 0 2024-04-24T23:13:43.058 INFO:tasks.workunit.client.0.smithi031.stdout:6/541: write d13/d15/d1d/d23/d4d/f6a [4796685,108162] 0 2024-04-24T23:13:43.059 INFO:tasks.workunit.client.1.smithi040.stdout:4/29: getdents da 0 2024-04-24T23:13:43.059 INFO:tasks.workunit.client.0.smithi031.stdout:6/542: chown d13/c34 9709 1 2024-04-24T23:13:43.061 INFO:tasks.workunit.client.0.smithi031.stdout:1/360: write d1/dc/d11/d21/f4b [442996,74769] 0 2024-04-24T23:13:43.063 INFO:tasks.workunit.client.1.smithi040.stdout:0/112: mkdir df/d1e/d20/d21 0 2024-04-24T23:13:43.063 INFO:tasks.workunit.client.1.smithi040.stdout:0/113: chown l5 20603 1 2024-04-24T23:13:43.063 INFO:tasks.workunit.client.1.smithi040.stdout:0/114: chown df/d14 115887814 1 2024-04-24T23:13:43.066 INFO:tasks.workunit.client.0.smithi031.stdout:6/543: dwrite d13/d15/d1d/d23/d2d/f90 [0,4194304] 0 2024-04-24T23:13:43.067 INFO:tasks.workunit.client.0.smithi031.stdout:4/538: fdatasync d3/d20/d2d/f39 0 2024-04-24T23:13:43.068 INFO:tasks.workunit.client.0.smithi031.stdout:6/544: fsync d13/d15/d1d/d23/d2d/f3d 0 2024-04-24T23:13:43.068 INFO:tasks.workunit.client.1.smithi040.stdout:2/110: rmdir da 39 2024-04-24T23:13:43.070 INFO:tasks.workunit.client.1.smithi040.stdout:3/65: write d1/f3 [225296,23375] 0 2024-04-24T23:13:43.073 INFO:tasks.workunit.client.1.smithi040.stdout:7/118: rename db/f19 to db/f2e 0 2024-04-24T23:13:43.074 INFO:tasks.workunit.client.1.smithi040.stdout:7/119: chown f2 136 1 2024-04-24T23:13:43.076 INFO:tasks.workunit.client.1.smithi040.stdout:9/110: fsync d7/da/fd 0 2024-04-24T23:13:43.077 INFO:tasks.workunit.client.1.smithi040.stdout:8/39: mknod d0/ca 0 2024-04-24T23:13:43.078 INFO:tasks.workunit.client.1.smithi040.stdout:8/40: chown d0/l3 781 1 2024-04-24T23:13:43.079 INFO:tasks.workunit.client.0.smithi031.stdout:1/361: write d1/dc/d11/f13 [2328419,8580] 0 2024-04-24T23:13:43.083 INFO:tasks.workunit.client.1.smithi040.stdout:5/70: dwrite d2/d7/fe [0,4194304] 0 2024-04-24T23:13:43.083 INFO:tasks.workunit.client.1.smithi040.stdout:4/30: creat da/ff x:0 0 0 2024-04-24T23:13:43.084 INFO:tasks.workunit.client.1.smithi040.stdout:8/41: dread d0/f7 [0,4194304] 0 2024-04-24T23:13:43.084 INFO:tasks.workunit.client.1.smithi040.stdout:0/115: creat df/d12/f22 x:0 0 0 2024-04-24T23:13:43.085 INFO:tasks.workunit.client.0.smithi031.stdout:4/539: mknod d3/d20/d25/d60/d63/cad 0 2024-04-24T23:13:43.086 INFO:tasks.workunit.client.1.smithi040.stdout:1/48: mkdir da/d12/d14 0 2024-04-24T23:13:43.088 INFO:tasks.workunit.client.0.smithi031.stdout:9/495: creat d2/f92 x:0 0 0 2024-04-24T23:13:43.090 INFO:tasks.workunit.client.0.smithi031.stdout:9/496: write d2/d1b/f2f [1745941,58283] 0 2024-04-24T23:13:43.091 INFO:tasks.workunit.client.0.smithi031.stdout:1/362: creat d1/dc/d4c/d4e/f72 x:0 0 0 2024-04-24T23:13:43.092 INFO:tasks.workunit.client.1.smithi040.stdout:3/66: chown d1/d9/da/de 152 1 2024-04-24T23:13:43.092 INFO:tasks.workunit.client.0.smithi031.stdout:4/540: dwrite d3/d23/d3b/f41 [0,4194304] 0 2024-04-24T23:13:43.094 INFO:tasks.workunit.client.1.smithi040.stdout:7/120: mkdir db/d2f 0 2024-04-24T23:13:43.094 INFO:tasks.workunit.client.0.smithi031.stdout:9/497: chown d2/d1b/d14/d1d/f35 9547869 1 2024-04-24T23:13:43.094 INFO:tasks.workunit.client.0.smithi031.stdout:4/541: readlink d3/d20/d25/l73 0 2024-04-24T23:13:43.096 INFO:tasks.workunit.client.0.smithi031.stdout:9/498: write d2/d1b/d14/f46 [1013901,73179] 0 2024-04-24T23:13:43.099 INFO:tasks.workunit.client.0.smithi031.stdout:9/499: chown d2/d1b/d14/d1d/d45/f5e 2433971 1 2024-04-24T23:13:43.100 INFO:tasks.workunit.client.1.smithi040.stdout:4/31: dwrite f6 [0,4194304] 0 2024-04-24T23:13:43.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/363: symlink d1/d1d/l73 0 2024-04-24T23:13:43.102 INFO:tasks.workunit.client.1.smithi040.stdout:6/63: truncate d7/f8 2321172 0 2024-04-24T23:13:43.102 INFO:tasks.workunit.client.0.smithi031.stdout:6/545: creat d13/d15/fb9 x:0 0 0 2024-04-24T23:13:43.103 INFO:tasks.workunit.client.0.smithi031.stdout:6/546: readlink l6 0 2024-04-24T23:13:43.104 INFO:tasks.workunit.client.0.smithi031.stdout:4/542: dwrite d3/f85 [0,4194304] 0 2024-04-24T23:13:43.106 INFO:tasks.workunit.client.1.smithi040.stdout:0/116: dwrite fb [4194304,4194304] 0 2024-04-24T23:13:43.107 INFO:tasks.workunit.client.0.smithi031.stdout:1/364: unlink d1/da/d61/l64 0 2024-04-24T23:13:43.110 INFO:tasks.workunit.client.1.smithi040.stdout:1/49: dwrite f8 [0,4194304] 0 2024-04-24T23:13:43.111 INFO:tasks.workunit.client.1.smithi040.stdout:5/71: write d2/d7/d8/f1b [376653,94256] 0 2024-04-24T23:13:43.112 INFO:tasks.workunit.client.1.smithi040.stdout:8/42: symlink d0/lb 0 2024-04-24T23:13:43.113 INFO:tasks.workunit.client.1.smithi040.stdout:0/117: fdatasync df/d12/f22 0 2024-04-24T23:13:43.118 INFO:tasks.workunit.client.1.smithi040.stdout:3/67: dwrite d1/f3 [0,4194304] 0 2024-04-24T23:13:43.120 INFO:tasks.workunit.client.1.smithi040.stdout:0/118: dread - df/d14/f16 zero size 2024-04-24T23:13:43.121 INFO:tasks.workunit.client.0.smithi031.stdout:8/523: sync 2024-04-24T23:13:43.123 INFO:tasks.workunit.client.1.smithi040.stdout:0/119: fsync df/d12/f17 0 2024-04-24T23:13:43.123 INFO:tasks.workunit.client.0.smithi031.stdout:4/543: truncate d3/f9 1725044 0 2024-04-24T23:13:43.124 INFO:tasks.workunit.client.1.smithi040.stdout:4/32: dwrite da/fb [0,4194304] 0 2024-04-24T23:13:43.126 INFO:tasks.workunit.client.0.smithi031.stdout:1/365: dread d1/dc/d11/f55 [0,4194304] 0 2024-04-24T23:13:43.127 INFO:tasks.workunit.client.1.smithi040.stdout:9/111: mknod d7/da/d17/c2c 0 2024-04-24T23:13:43.127 INFO:tasks.workunit.client.0.smithi031.stdout:1/366: write d1/d3e/d46/f59 [750648,72269] 0 2024-04-24T23:13:43.128 INFO:tasks.workunit.client.0.smithi031.stdout:9/500: rename d2/d1b/d85/f86 to d2/d1b/d14/d1d/d45/f93 0 2024-04-24T23:13:43.129 INFO:tasks.workunit.client.1.smithi040.stdout:0/120: dread f2 [4194304,4194304] 0 2024-04-24T23:13:43.134 INFO:tasks.workunit.client.0.smithi031.stdout:6/547: creat d13/d15/d1d/d23/d24/fba x:0 0 0 2024-04-24T23:13:43.135 INFO:tasks.workunit.client.0.smithi031.stdout:8/524: mkdir d8/d15/d37/d43/d7b/dbf 0 2024-04-24T23:13:43.139 INFO:tasks.workunit.client.1.smithi040.stdout:2/111: write f8 [708778,129118] 0 2024-04-24T23:13:43.140 INFO:tasks.workunit.client.1.smithi040.stdout:5/72: symlink d2/d7/d8/l1d 0 2024-04-24T23:13:43.142 INFO:tasks.workunit.client.1.smithi040.stdout:3/68: creat d1/d9/da/de/f14 x:0 0 0 2024-04-24T23:13:43.144 INFO:tasks.workunit.client.0.smithi031.stdout:1/367: mkdir d1/d3e/d46/d74 0 2024-04-24T23:13:43.145 INFO:tasks.workunit.client.1.smithi040.stdout:7/121: creat db/d2f/f30 x:0 0 0 2024-04-24T23:13:43.145 INFO:tasks.workunit.client.0.smithi031.stdout:1/368: fsync d1/dc/d11/f13 0 2024-04-24T23:13:43.147 INFO:tasks.workunit.client.0.smithi031.stdout:6/548: unlink d13/d15/d1d/d23/d44/d47/l91 0 2024-04-24T23:13:43.148 INFO:tasks.workunit.client.1.smithi040.stdout:4/33: symlink da/l10 0 2024-04-24T23:13:43.149 INFO:tasks.workunit.client.1.smithi040.stdout:4/34: chown da/l10 384006 1 2024-04-24T23:13:43.151 INFO:tasks.workunit.client.0.smithi031.stdout:8/525: rename d8/d15/d37/d62/d76 to d8/d1a/d1d/d50/dc0 0 2024-04-24T23:13:43.152 INFO:tasks.workunit.client.0.smithi031.stdout:9/501: dread d2/f64 [0,4194304] 0 2024-04-24T23:13:43.153 INFO:tasks.workunit.client.1.smithi040.stdout:9/112: symlink d7/da/d17/l2d 0 2024-04-24T23:13:43.154 INFO:tasks.workunit.client.1.smithi040.stdout:0/121: unlink df/d14/d1b/c1c 0 2024-04-24T23:13:43.155 INFO:tasks.workunit.client.1.smithi040.stdout:4/35: dwrite da/ff [0,4194304] 0 2024-04-24T23:13:43.161 INFO:tasks.workunit.client.1.smithi040.stdout:1/50: symlink da/d12/d14/l15 0 2024-04-24T23:13:43.162 INFO:tasks.workunit.client.0.smithi031.stdout:1/369: symlink d1/l75 0 2024-04-24T23:13:43.164 INFO:tasks.workunit.client.1.smithi040.stdout:1/51: chown da/c11 2111944108 1 2024-04-24T23:13:43.165 INFO:tasks.workunit.client.1.smithi040.stdout:5/73: mkdir d2/d7/d8/db/d1e 0 2024-04-24T23:13:43.166 INFO:tasks.workunit.client.1.smithi040.stdout:2/112: dread f8 [0,4194304] 0 2024-04-24T23:13:43.168 INFO:tasks.workunit.client.1.smithi040.stdout:3/69: mkdir d1/d9/d15 0 2024-04-24T23:13:43.169 INFO:tasks.workunit.client.0.smithi031.stdout:8/526: getdents d8/d15/d37/d43/d7b/dbf 0 2024-04-24T23:13:43.172 INFO:tasks.workunit.client.1.smithi040.stdout:7/122: creat db/f31 x:0 0 0 2024-04-24T23:13:43.173 INFO:tasks.workunit.client.1.smithi040.stdout:7/123: chown db/ce 29305 1 2024-04-24T23:13:43.173 INFO:tasks.workunit.client.0.smithi031.stdout:1/370: write d1/dc/d11/f38 [283597,116214] 0 2024-04-24T23:13:43.173 INFO:tasks.workunit.client.0.smithi031.stdout:5/380: sync 2024-04-24T23:13:43.173 INFO:tasks.workunit.client.0.smithi031.stdout:7/385: sync 2024-04-24T23:13:43.174 INFO:tasks.workunit.client.0.smithi031.stdout:6/549: link l3 d13/d1b/d32/db4/lbb 0 2024-04-24T23:13:43.175 INFO:tasks.workunit.client.1.smithi040.stdout:3/70: dwrite d1/f8 [0,4194304] 0 2024-04-24T23:13:43.176 INFO:tasks.workunit.client.0.smithi031.stdout:4/544: dread d3/f11 [0,4194304] 0 2024-04-24T23:13:43.176 INFO:tasks.workunit.client.1.smithi040.stdout:9/113: truncate d7/fe 289963 0 2024-04-24T23:13:43.177 INFO:tasks.workunit.client.0.smithi031.stdout:8/527: symlink d8/d15/lc1 0 2024-04-24T23:13:43.184 INFO:tasks.workunit.client.0.smithi031.stdout:1/371: unlink d1/d33/f58 0 2024-04-24T23:13:43.186 INFO:tasks.workunit.client.0.smithi031.stdout:7/386: symlink d1/d6a/d66/l7f 0 2024-04-24T23:13:43.188 INFO:tasks.workunit.client.1.smithi040.stdout:4/36: mknod da/c11 0 2024-04-24T23:13:43.190 INFO:tasks.workunit.client.0.smithi031.stdout:5/381: mkdir d0/d7/d56/d7c 0 2024-04-24T23:13:43.190 INFO:tasks.workunit.client.1.smithi040.stdout:4/37: dread f4 [0,4194304] 0 2024-04-24T23:13:43.190 INFO:tasks.workunit.client.1.smithi040.stdout:6/64: write f2 [1823455,113137] 0 2024-04-24T23:13:43.190 INFO:tasks.workunit.client.1.smithi040.stdout:4/38: chown f6 4735776 1 2024-04-24T23:13:43.191 INFO:tasks.workunit.client.0.smithi031.stdout:5/382: truncate d0/f49 178640 0 2024-04-24T23:13:43.192 INFO:tasks.workunit.client.0.smithi031.stdout:9/502: write d2/f5 [4491783,55927] 0 2024-04-24T23:13:43.192 INFO:tasks.workunit.client.1.smithi040.stdout:1/52: rmdir da/d12 39 2024-04-24T23:13:43.194 INFO:tasks.workunit.client.0.smithi031.stdout:9/503: truncate d2/d1b/d14/f69 607779 0 2024-04-24T23:13:43.195 INFO:tasks.workunit.client.1.smithi040.stdout:5/74: mkdir d2/d7/d8/d1f 0 2024-04-24T23:13:43.195 INFO:tasks.workunit.client.0.smithi031.stdout:9/504: readlink d2/d1b/d14/d78/d84/d4a/l8a 0 2024-04-24T23:13:43.198 INFO:tasks.workunit.client.1.smithi040.stdout:2/113: chown da/f19 96871674 1 2024-04-24T23:13:43.198 INFO:tasks.workunit.client.1.smithi040.stdout:2/114: dread - da/f20 zero size 2024-04-24T23:13:43.200 INFO:tasks.workunit.client.1.smithi040.stdout:7/124: mknod db/d2f/c32 0 2024-04-24T23:13:43.201 INFO:tasks.workunit.client.1.smithi040.stdout:7/125: dread - db/f2e zero size 2024-04-24T23:13:43.201 INFO:tasks.workunit.client.0.smithi031.stdout:1/372: rmdir d1/da/d10/d52/d62 39 2024-04-24T23:13:43.204 INFO:tasks.workunit.client.1.smithi040.stdout:3/71: readlink d1/d9/db/l12 0 2024-04-24T23:13:43.208 INFO:tasks.workunit.client.1.smithi040.stdout:8/43: getdents d0/d8 0 2024-04-24T23:13:43.209 INFO:tasks.workunit.client.1.smithi040.stdout:7/126: dwrite db/d12/d20/f22 [0,4194304] 0 2024-04-24T23:13:43.210 INFO:tasks.workunit.client.1.smithi040.stdout:9/114: dread d7/da/fd [0,4194304] 0 2024-04-24T23:13:43.211 INFO:tasks.workunit.client.0.smithi031.stdout:4/545: link d3/d20/d25/d7e/fa1 d3/da3/fae 0 2024-04-24T23:13:43.212 INFO:tasks.workunit.client.1.smithi040.stdout:6/65: mknod d7/c12 0 2024-04-24T23:13:43.214 INFO:tasks.workunit.client.0.smithi031.stdout:8/528: link d8/d15/d37/d43/f4b d8/d15/d9a/fc2 0 2024-04-24T23:13:43.216 INFO:tasks.workunit.client.1.smithi040.stdout:4/39: rename f2 to da/f12 0 2024-04-24T23:13:43.217 INFO:tasks.workunit.client.1.smithi040.stdout:4/40: write f9 [160056,105068] 0 2024-04-24T23:13:43.220 INFO:tasks.workunit.client.0.smithi031.stdout:9/505: write d2/d1b/f4f [1956242,74470] 0 2024-04-24T23:13:43.220 INFO:tasks.workunit.client.1.smithi040.stdout:5/75: rmdir d2 39 2024-04-24T23:13:43.223 INFO:tasks.workunit.client.1.smithi040.stdout:2/115: write f8 [785397,129367] 0 2024-04-24T23:13:43.225 INFO:tasks.workunit.client.1.smithi040.stdout:4/41: dwrite da/ff [0,4194304] 0 2024-04-24T23:13:43.232 INFO:tasks.workunit.client.0.smithi031.stdout:1/373: write d1/d33/d36/f44 [375223,83599] 0 2024-04-24T23:13:43.234 INFO:tasks.workunit.client.1.smithi040.stdout:4/42: dwrite f6 [4194304,4194304] 0 2024-04-24T23:13:43.234 INFO:tasks.workunit.client.0.smithi031.stdout:7/387: truncate d1/d6a/f3c 95310 0 2024-04-24T23:13:43.235 INFO:tasks.workunit.client.1.smithi040.stdout:8/44: dwrite d0/f7 [0,4194304] 0 2024-04-24T23:13:43.236 INFO:tasks.workunit.client.0.smithi031.stdout:5/383: rename d0/f49 to d0/d6/d3b/d4b/f7d 0 2024-04-24T23:13:43.238 INFO:tasks.workunit.client.0.smithi031.stdout:5/384: fsync d0/d6/d32/f4 0 2024-04-24T23:13:43.239 INFO:tasks.workunit.client.1.smithi040.stdout:6/66: rmdir d7/de 39 2024-04-24T23:13:43.240 INFO:tasks.workunit.client.0.smithi031.stdout:5/385: chown d0/d6/d3b 97562 1 2024-04-24T23:13:43.240 INFO:tasks.workunit.client.0.smithi031.stdout:9/506: dwrite d2/d1b/d14/d1d/d45/f72 [0,4194304] 0 2024-04-24T23:13:43.241 INFO:tasks.workunit.client.0.smithi031.stdout:6/550: link f12 d13/d15/d1d/d23/d2d/d9e/dab/fbc 0 2024-04-24T23:13:43.241 INFO:tasks.workunit.client.0.smithi031.stdout:9/507: stat d2/d1b/d14/d78/d84/d4a 0 2024-04-24T23:13:43.245 INFO:tasks.workunit.client.0.smithi031.stdout:4/546: creat d3/da3/faf x:0 0 0 2024-04-24T23:13:43.250 INFO:tasks.workunit.client.1.smithi040.stdout:0/122: getdents df/d1e 0 2024-04-24T23:13:43.252 INFO:tasks.workunit.client.0.smithi031.stdout:7/388: read d1/f53 [3788723,33801] 0 2024-04-24T23:13:43.253 INFO:tasks.workunit.client.1.smithi040.stdout:0/123: truncate df/d12/f17 758895 0 2024-04-24T23:13:43.253 INFO:tasks.workunit.client.1.smithi040.stdout:5/76: dread d2/f6 [0,4194304] 0 2024-04-24T23:13:43.254 INFO:tasks.workunit.client.1.smithi040.stdout:2/116: symlink da/d1c/l21 0 2024-04-24T23:13:43.259 INFO:tasks.workunit.client.0.smithi031.stdout:1/374: creat d1/dc/d11/d21/d5a/f76 x:0 0 0 2024-04-24T23:13:43.261 INFO:tasks.workunit.client.1.smithi040.stdout:2/117: dwrite da/f20 [0,4194304] 0 2024-04-24T23:13:43.262 INFO:tasks.workunit.client.1.smithi040.stdout:2/118: truncate da/f11 1552126 0 2024-04-24T23:13:43.263 INFO:tasks.workunit.client.1.smithi040.stdout:8/45: mkdir d0/d8/dc 0 2024-04-24T23:13:43.263 INFO:tasks.workunit.client.1.smithi040.stdout:2/119: fsync da/f19 0 2024-04-24T23:13:43.264 INFO:tasks.workunit.client.1.smithi040.stdout:2/120: chown da/d1c/c1d 5 1 2024-04-24T23:13:43.265 INFO:tasks.workunit.client.1.smithi040.stdout:2/121: rename da to da/d10/d22 22 2024-04-24T23:13:43.269 INFO:tasks.workunit.client.1.smithi040.stdout:2/122: write da/fe [4391441,47264] 0 2024-04-24T23:13:43.269 INFO:tasks.workunit.client.1.smithi040.stdout:6/67: write d7/de/ff [635037,541] 0 2024-04-24T23:13:43.270 INFO:tasks.workunit.client.1.smithi040.stdout:2/123: read da/fe [832807,59935] 0 2024-04-24T23:13:43.270 INFO:tasks.workunit.client.0.smithi031.stdout:1/375: creat d1/da/d42/f77 x:0 0 0 2024-04-24T23:13:43.271 INFO:tasks.workunit.client.0.smithi031.stdout:1/376: fsync d1/dc/d4c/d4e/f72 0 2024-04-24T23:13:43.272 INFO:tasks.workunit.client.1.smithi040.stdout:5/77: truncate d2/f6 529108 0 2024-04-24T23:13:43.274 INFO:tasks.workunit.client.0.smithi031.stdout:8/529: truncate d8/d15/d37/f2e 2136402 0 2024-04-24T23:13:43.276 INFO:tasks.workunit.client.1.smithi040.stdout:1/53: truncate f2 3701314 0 2024-04-24T23:13:43.277 INFO:tasks.workunit.client.1.smithi040.stdout:2/124: dwrite da/f11 [0,4194304] 0 2024-04-24T23:13:43.277 INFO:tasks.workunit.client.0.smithi031.stdout:9/508: rename d2/fe to d2/d1b/d14/d78/d84/d4a/f94 0 2024-04-24T23:13:43.279 INFO:tasks.workunit.client.1.smithi040.stdout:3/72: truncate d1/f8 309690 0 2024-04-24T23:13:43.280 INFO:tasks.workunit.client.1.smithi040.stdout:3/73: stat d1/d9/d15 0 2024-04-24T23:13:43.280 INFO:tasks.workunit.client.1.smithi040.stdout:3/74: chown d1/d9/da/de/d13 14 1 2024-04-24T23:13:43.281 INFO:tasks.workunit.client.0.smithi031.stdout:7/389: creat d1/d6a/f80 x:0 0 0 2024-04-24T23:13:43.283 INFO:tasks.workunit.client.0.smithi031.stdout:7/390: read d1/d2/dd/f15 [747685,75486] 0 2024-04-24T23:13:43.283 INFO:tasks.workunit.client.1.smithi040.stdout:4/43: truncate f4 461785 0 2024-04-24T23:13:43.286 INFO:tasks.workunit.client.1.smithi040.stdout:8/46: creat d0/d8/fd x:0 0 0 2024-04-24T23:13:43.287 INFO:tasks.workunit.client.0.smithi031.stdout:6/551: link d13/d15/d1d/d23/d24/c2a d13/d1b/d32/cbd 0 2024-04-24T23:13:43.287 INFO:tasks.workunit.client.1.smithi040.stdout:8/47: write d0/f7 [1657966,48450] 0 2024-04-24T23:13:43.288 INFO:tasks.workunit.client.0.smithi031.stdout:5/386: write d0/d6/d3b/d4b/dc/d25/d55/f75 [233293,103878] 0 2024-04-24T23:13:43.288 INFO:tasks.workunit.client.1.smithi040.stdout:3/75: dwrite d1/d9/da/de/f14 [0,4194304] 0 2024-04-24T23:13:43.289 INFO:tasks.workunit.client.0.smithi031.stdout:6/552: dread - d13/d15/d1d/d23/d24/d43/d68/fb5 zero size 2024-04-24T23:13:43.290 INFO:tasks.workunit.client.0.smithi031.stdout:5/387: write d0/d6/d3b/f4d [1750152,130139] 0 2024-04-24T23:13:43.292 INFO:tasks.workunit.client.1.smithi040.stdout:7/127: truncate db/d12/d20/f22 3517661 0 2024-04-24T23:13:43.293 INFO:tasks.workunit.client.0.smithi031.stdout:1/377: dread d1/dc/d11/d21/f4b [0,4194304] 0 2024-04-24T23:13:43.293 INFO:tasks.workunit.client.0.smithi031.stdout:6/553: chown d13/d15/d1d/d23/d2d/d9e/dab/laf 31960204 1 2024-04-24T23:13:43.294 INFO:tasks.workunit.client.1.smithi040.stdout:8/48: fdatasync d0/f7 0 2024-04-24T23:13:43.294 INFO:tasks.workunit.client.0.smithi031.stdout:1/378: chown d1/d33 23897 1 2024-04-24T23:13:43.294 INFO:tasks.workunit.client.0.smithi031.stdout:9/509: truncate d2/d1b/d14/d78/d84/d4a/d4b/f57 372062 0 2024-04-24T23:13:43.296 INFO:tasks.workunit.client.0.smithi031.stdout:8/530: dread d8/d15/d37/d43/f6f [0,4194304] 0 2024-04-24T23:13:43.297 INFO:tasks.workunit.client.0.smithi031.stdout:9/510: stat d2/d1b/d14/d18/c1a 0 2024-04-24T23:13:43.303 INFO:tasks.workunit.client.1.smithi040.stdout:9/115: link d7/da/c11 d7/d16/d1e/c2e 0 2024-04-24T23:13:43.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/391: dread d1/d45/f5b [0,4194304] 0 2024-04-24T23:13:43.309 INFO:tasks.workunit.client.0.smithi031.stdout:6/554: dread - d13/d1b/f49 zero size 2024-04-24T23:13:43.310 INFO:tasks.workunit.client.1.smithi040.stdout:2/125: mknod da/d10/c23 0 2024-04-24T23:13:43.313 INFO:tasks.workunit.client.1.smithi040.stdout:4/44: rename da/ff to da/f13 0 2024-04-24T23:13:43.314 INFO:tasks.workunit.client.1.smithi040.stdout:4/45: truncate da/f13 5067234 0 2024-04-24T23:13:43.321 INFO:tasks.workunit.client.0.smithi031.stdout:9/511: mknod d2/d1b/d14/d78/d84/d4a/d4b/c95 0 2024-04-24T23:13:43.322 INFO:tasks.workunit.client.1.smithi040.stdout:0/124: link df/c1d df/d1e/c23 0 2024-04-24T23:13:43.322 INFO:tasks.workunit.client.0.smithi031.stdout:9/512: chown d2/d43/d6e/c8e 13987429 1 2024-04-24T23:13:43.322 INFO:tasks.workunit.client.1.smithi040.stdout:0/125: chown cd 160 1 2024-04-24T23:13:43.324 INFO:tasks.workunit.client.1.smithi040.stdout:0/126: write df/d14/f16 [299189,17898] 0 2024-04-24T23:13:43.324 INFO:tasks.workunit.client.1.smithi040.stdout:7/128: creat db/d12/f33 x:0 0 0 2024-04-24T23:13:43.327 INFO:tasks.workunit.client.0.smithi031.stdout:6/555: write d13/d15/d1d/d23/d24/d8b/f9d [40258,19182] 0 2024-04-24T23:13:43.329 INFO:tasks.workunit.client.0.smithi031.stdout:6/556: chown d13/d15/d1d/d23/d44/da0/da6/fb6 52890 1 2024-04-24T23:13:43.330 INFO:tasks.workunit.client.0.smithi031.stdout:6/557: chown d13/d15/d1d/d23/d4d 64183737 1 2024-04-24T23:13:43.331 INFO:tasks.workunit.client.1.smithi040.stdout:5/78: link d2/d7/fe d2/d7/d8/d11/f20 0 2024-04-24T23:13:43.332 INFO:tasks.workunit.client.0.smithi031.stdout:5/388: write d0/d7/f58 [4320534,122314] 0 2024-04-24T23:13:43.334 INFO:tasks.workunit.client.1.smithi040.stdout:9/116: rename d7/d16/d1e to d7/d1b/d2f 0 2024-04-24T23:13:43.338 INFO:tasks.workunit.client.0.smithi031.stdout:9/513: truncate d2/d1b/d14/d78/d84/f60 182548 0 2024-04-24T23:13:43.338 INFO:tasks.workunit.client.0.smithi031.stdout:7/392: mkdir d1/d45/d75/d81 0 2024-04-24T23:13:43.338 INFO:tasks.workunit.client.1.smithi040.stdout:2/126: rmdir da/d1c 39 2024-04-24T23:13:43.339 INFO:tasks.workunit.client.0.smithi031.stdout:7/393: write d1/d2/dd/f74 [509062,92160] 0 2024-04-24T23:13:43.341 INFO:tasks.workunit.client.0.smithi031.stdout:1/379: link d1/da/l1a d1/dc/d11/d21/l78 0 2024-04-24T23:13:43.342 INFO:tasks.workunit.client.0.smithi031.stdout:1/380: dread - d1/d1d/f6d zero size 2024-04-24T23:13:43.343 INFO:tasks.workunit.client.0.smithi031.stdout:1/381: write d1/da/d10/f51 [640733,31275] 0 2024-04-24T23:13:43.343 INFO:tasks.workunit.client.0.smithi031.stdout:6/558: symlink d13/d15/d1d/d23/d44/lbe 0 2024-04-24T23:13:43.345 INFO:tasks.workunit.client.1.smithi040.stdout:3/76: fsync d1/f5 0 2024-04-24T23:13:43.349 INFO:tasks.workunit.client.1.smithi040.stdout:7/129: mknod db/d1b/c34 0 2024-04-24T23:13:43.349 INFO:tasks.workunit.client.0.smithi031.stdout:9/514: mkdir d2/d1b/d14/d78/d84/d4a/d4b/d96 0 2024-04-24T23:13:43.350 INFO:tasks.workunit.client.1.smithi040.stdout:8/49: mkdir d0/d8/dc/de 0 2024-04-24T23:13:43.351 INFO:tasks.workunit.client.1.smithi040.stdout:8/50: write d0/d8/fd [325989,109434] 0 2024-04-24T23:13:43.352 INFO:tasks.workunit.client.1.smithi040.stdout:8/51: chown d0/d8/dc 492 1 2024-04-24T23:13:43.352 INFO:tasks.workunit.client.0.smithi031.stdout:7/394: rename d1/d2/dd/d33/d5e/d64/d6d/c70 to d1/d2/d60/c82 0 2024-04-24T23:13:43.359 INFO:tasks.workunit.client.1.smithi040.stdout:5/79: mkdir d2/d7/d8/d11/d21 0 2024-04-24T23:13:43.360 INFO:tasks.workunit.client.0.smithi031.stdout:6/559: mkdir d13/d15/d1d/d23/d2d/dbf 0 2024-04-24T23:13:43.361 INFO:tasks.workunit.client.0.smithi031.stdout:8/531: link d8/c5a d8/d15/d37/d43/dbd/cc3 0 2024-04-24T23:13:43.364 INFO:tasks.workunit.client.1.smithi040.stdout:0/127: rename df/d1e/c23 to df/d1e/d20/c24 0 2024-04-24T23:13:43.365 INFO:tasks.workunit.client.1.smithi040.stdout:0/128: dread - df/d12/f22 zero size 2024-04-24T23:13:43.370 INFO:tasks.workunit.client.1.smithi040.stdout:9/117: truncate f2 553860 0 2024-04-24T23:13:43.379 INFO:tasks.workunit.client.0.smithi031.stdout:6/560: rename lb to d13/d15/d1d/d23/d99/lc0 0 2024-04-24T23:13:43.382 INFO:tasks.workunit.client.0.smithi031.stdout:7/395: mkdir d1/d2/d83 0 2024-04-24T23:13:43.384 INFO:tasks.workunit.client.1.smithi040.stdout:3/77: rmdir d1/d9/db 39 2024-04-24T23:13:43.384 INFO:tasks.workunit.client.1.smithi040.stdout:3/78: chown d1/d9/da/cc 1 1 2024-04-24T23:13:43.384 INFO:tasks.workunit.client.0.smithi031.stdout:1/382: dwrite d1/da/d10/f4d [0,4194304] 0 2024-04-24T23:13:43.385 INFO:tasks.workunit.client.1.smithi040.stdout:3/79: chown d1/d9/da 597 1 2024-04-24T23:13:43.385 INFO:tasks.workunit.client.0.smithi031.stdout:9/515: dread d2/d1b/f17 [0,4194304] 0 2024-04-24T23:13:43.386 INFO:tasks.workunit.client.0.smithi031.stdout:1/383: write d1/ff [771505,69480] 0 2024-04-24T23:13:43.386 INFO:tasks.workunit.client.0.smithi031.stdout:9/516: read - d2/d1b/d14/f88 zero size 2024-04-24T23:13:43.387 INFO:tasks.workunit.client.0.smithi031.stdout:5/389: truncate d0/d7/f58 2180508 0 2024-04-24T23:13:43.389 INFO:tasks.workunit.client.0.smithi031.stdout:5/390: fsync d0/d6/d12/f36 0 2024-04-24T23:13:43.391 INFO:tasks.workunit.client.0.smithi031.stdout:6/561: creat d13/d15/d1d/d23/d2d/d9e/fc1 x:0 0 0 2024-04-24T23:13:43.392 INFO:tasks.workunit.client.1.smithi040.stdout:8/52: unlink d0/ca 0 2024-04-24T23:13:43.393 INFO:tasks.workunit.client.1.smithi040.stdout:6/68: getdents d7/de 0 2024-04-24T23:13:43.394 INFO:tasks.workunit.client.1.smithi040.stdout:5/80: fsync d2/d7/fe 0 2024-04-24T23:13:43.396 INFO:tasks.workunit.client.1.smithi040.stdout:2/127: rename da/d10/c12 to da/d10/d15/c24 0 2024-04-24T23:13:43.397 INFO:tasks.workunit.client.0.smithi031.stdout:1/384: mknod d1/d33/d36/c79 0 2024-04-24T23:13:43.398 INFO:tasks.workunit.client.1.smithi040.stdout:0/129: creat df/d12/f25 x:0 0 0 2024-04-24T23:13:43.399 INFO:tasks.workunit.client.1.smithi040.stdout:0/130: write df/d12/f17 [1072997,104098] 0 2024-04-24T23:13:43.400 INFO:tasks.workunit.client.1.smithi040.stdout:0/131: write df/d12/f17 [88970,96805] 0 2024-04-24T23:13:43.401 INFO:tasks.workunit.client.1.smithi040.stdout:0/132: stat df/d1e/d20 0 2024-04-24T23:13:43.401 INFO:tasks.workunit.client.0.smithi031.stdout:5/391: rename d0/d6/d3b/d4b/dc/d73 to d0/d7/d56/d7c/d7e 0 2024-04-24T23:13:43.404 INFO:tasks.workunit.client.1.smithi040.stdout:0/133: dread df/d14/f16 [0,4194304] 0 2024-04-24T23:13:43.404 INFO:tasks.workunit.client.1.smithi040.stdout:9/118: creat d7/d1b/d2f/f30 x:0 0 0 2024-04-24T23:13:43.404 INFO:tasks.workunit.client.0.smithi031.stdout:6/562: mknod d13/d1b/d32/cc2 0 2024-04-24T23:13:43.405 INFO:tasks.workunit.client.0.smithi031.stdout:6/563: stat d13/d15/d1d/d23/d24/d43/fb3 0 2024-04-24T23:13:43.406 INFO:tasks.workunit.client.0.smithi031.stdout:1/385: rmdir d1/da/d42/d69 39 2024-04-24T23:13:43.406 INFO:tasks.workunit.client.0.smithi031.stdout:6/564: chown d13/d15/d1d/d23/d44/d47/l94 2 1 2024-04-24T23:13:43.407 INFO:tasks.workunit.client.0.smithi031.stdout:1/386: chown d1/dc/f2b 1135 1 2024-04-24T23:13:43.411 INFO:tasks.workunit.client.0.smithi031.stdout:6/565: creat d13/d1b/d32/fc3 x:0 0 0 2024-04-24T23:13:43.415 INFO:tasks.workunit.client.0.smithi031.stdout:1/387: link d1/dc/d4c/f6a d1/d1d/f7a 0 2024-04-24T23:13:43.419 INFO:tasks.workunit.client.0.smithi031.stdout:5/392: dread d0/d6/d32/f46 [0,4194304] 0 2024-04-24T23:13:43.420 INFO:tasks.workunit.client.0.smithi031.stdout:6/566: rename d13/d1b/d32/f83 to d13/d1b/d32/d9c/fc4 0 2024-04-24T23:13:43.423 INFO:tasks.workunit.client.0.smithi031.stdout:7/396: write d1/d6a/f35 [4191297,107949] 0 2024-04-24T23:13:43.427 INFO:tasks.workunit.client.0.smithi031.stdout:1/388: rename d1/dc/d4c to d1/d49/d7b 0 2024-04-24T23:13:43.430 INFO:tasks.workunit.client.0.smithi031.stdout:7/397: dwrite d1/d2/dd/d33/f77 [0,4194304] 0 2024-04-24T23:13:43.438 INFO:tasks.workunit.client.0.smithi031.stdout:9/517: dwrite d2/d1b/f41 [0,4194304] 0 2024-04-24T23:13:43.439 INFO:tasks.workunit.client.0.smithi031.stdout:7/398: mkdir d1/d45/d84 0 2024-04-24T23:13:43.440 INFO:tasks.workunit.client.0.smithi031.stdout:9/518: fdatasync d2/d1b/d14/d1d/f7b 0 2024-04-24T23:13:43.444 INFO:tasks.workunit.client.0.smithi031.stdout:7/399: dread d1/f53 [0,4194304] 0 2024-04-24T23:13:43.444 INFO:tasks.workunit.client.0.smithi031.stdout:6/567: rename d13/d1b/f6e to d13/fc5 0 2024-04-24T23:13:43.446 INFO:tasks.workunit.client.0.smithi031.stdout:6/568: chown d13/d15/d1d/d23/d44/f5b 53676 1 2024-04-24T23:13:43.448 INFO:tasks.workunit.client.0.smithi031.stdout:9/519: mknod d2/d1b/d14/d78/d84/d4a/d4b/d96/c97 0 2024-04-24T23:13:43.449 INFO:tasks.workunit.client.0.smithi031.stdout:9/520: write d2/d16/f77 [971513,60517] 0 2024-04-24T23:13:43.450 INFO:tasks.workunit.client.0.smithi031.stdout:6/569: creat d13/d1b/d32/db4/fc6 x:0 0 0 2024-04-24T23:13:43.451 INFO:tasks.workunit.client.0.smithi031.stdout:7/400: mknod d1/d45/d84/c85 0 2024-04-24T23:13:43.452 INFO:tasks.workunit.client.0.smithi031.stdout:9/521: write d2/d1b/d14/d78/d84/d4a/f71 [4324813,29938] 0 2024-04-24T23:13:43.453 INFO:tasks.workunit.client.0.smithi031.stdout:7/401: chown d1/d2/d1e/d4f/f5d 0 1 2024-04-24T23:13:43.456 INFO:tasks.workunit.client.0.smithi031.stdout:6/570: dread d13/d15/d1d/d23/d44/d47/d69/faa [0,4194304] 0 2024-04-24T23:13:43.457 INFO:tasks.workunit.client.0.smithi031.stdout:9/522: dread d2/f1c [0,4194304] 0 2024-04-24T23:13:43.457 INFO:tasks.workunit.client.0.smithi031.stdout:7/402: read d1/d6a/f35 [2147931,88358] 0 2024-04-24T23:13:43.465 INFO:tasks.workunit.client.0.smithi031.stdout:0/481: sync 2024-04-24T23:13:43.465 INFO:tasks.workunit.client.0.smithi031.stdout:3/382: sync 2024-04-24T23:13:43.465 INFO:tasks.workunit.client.0.smithi031.stdout:2/329: sync 2024-04-24T23:13:43.466 INFO:tasks.workunit.client.0.smithi031.stdout:6/571: mknod d13/d1b/d32/db4/cc7 0 2024-04-24T23:13:43.467 INFO:tasks.workunit.client.0.smithi031.stdout:2/330: write d5/d8/d15/f34 [3024256,27320] 0 2024-04-24T23:13:43.467 INFO:tasks.workunit.client.0.smithi031.stdout:0/482: fdatasync da/d18/d1a/d37/f5f 0 2024-04-24T23:13:43.468 INFO:tasks.workunit.client.0.smithi031.stdout:2/331: fdatasync d5/d8/d15/f5f 0 2024-04-24T23:13:43.470 INFO:tasks.workunit.client.0.smithi031.stdout:2/332: write d5/d8/d15/f22 [937704,30987] 0 2024-04-24T23:13:43.478 INFO:tasks.workunit.client.0.smithi031.stdout:7/403: dread d1/d6a/f35 [0,4194304] 0 2024-04-24T23:13:43.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/383: dread d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:43.479 INFO:tasks.workunit.client.0.smithi031.stdout:9/523: mknod d2/d1b/d14/d1d/d45/c98 0 2024-04-24T23:13:43.480 INFO:tasks.workunit.client.0.smithi031.stdout:7/404: write d1/d2/d1e/d4f/f5d [489505,73411] 0 2024-04-24T23:13:43.481 INFO:tasks.workunit.client.0.smithi031.stdout:5/393: truncate d0/d6/d3b/f4d 1718044 0 2024-04-24T23:13:43.490 INFO:tasks.workunit.client.0.smithi031.stdout:6/572: mknod d13/d15/d1d/cc8 0 2024-04-24T23:13:43.491 INFO:tasks.workunit.client.0.smithi031.stdout:1/389: truncate d1/dc/d11/f38 2780654 0 2024-04-24T23:13:43.491 INFO:tasks.workunit.client.0.smithi031.stdout:6/573: chown d13/d15/d1d/d23/d24/f50 2 1 2024-04-24T23:13:43.493 INFO:tasks.workunit.client.0.smithi031.stdout:3/384: creat d6/d14/f7d x:0 0 0 2024-04-24T23:13:43.495 INFO:tasks.workunit.client.0.smithi031.stdout:9/524: chown d2/d1b/d14/d1d/f35 6032795 1 2024-04-24T23:13:43.500 INFO:tasks.workunit.client.0.smithi031.stdout:5/394: mknod d0/d6/d12/c7f 0 2024-04-24T23:13:43.507 INFO:tasks.workunit.client.0.smithi031.stdout:6/574: rename d13/d1b/d32/d9c/ca8 to d13/d15/d1d/d23/d99/cc9 0 2024-04-24T23:13:43.508 INFO:tasks.workunit.client.0.smithi031.stdout:6/575: truncate d13/d15/fb9 903479 0 2024-04-24T23:13:43.509 INFO:tasks.workunit.client.0.smithi031.stdout:3/385: rmdir d6/d72/d60 39 2024-04-24T23:13:43.513 INFO:tasks.workunit.client.0.smithi031.stdout:7/405: fsync d1/d2/f3b 0 2024-04-24T23:13:43.515 INFO:tasks.workunit.client.0.smithi031.stdout:3/386: dwrite d6/df/d16/d2a/f4f [0,4194304] 0 2024-04-24T23:13:43.518 INFO:tasks.workunit.client.0.smithi031.stdout:1/390: rename d1/da/d42/d69/f6c to d1/d31/f7c 0 2024-04-24T23:13:43.519 INFO:tasks.workunit.client.0.smithi031.stdout:1/391: stat d1/l75 0 2024-04-24T23:13:43.520 INFO:tasks.workunit.client.0.smithi031.stdout:1/392: write d1/da/f2c [4303615,108241] 0 2024-04-24T23:13:43.538 INFO:tasks.workunit.client.0.smithi031.stdout:0/483: write da/d11/d6a/f94 [163589,73321] 0 2024-04-24T23:13:43.540 INFO:tasks.workunit.client.0.smithi031.stdout:7/406: symlink d1/d2/l86 0 2024-04-24T23:13:43.540 INFO:tasks.workunit.client.0.smithi031.stdout:7/407: chown d1/d45/d75 649913 1 2024-04-24T23:13:43.542 INFO:tasks.workunit.client.0.smithi031.stdout:5/395: creat d0/d6/d3b/d4b/dc/f80 x:0 0 0 2024-04-24T23:13:43.546 INFO:tasks.workunit.client.0.smithi031.stdout:0/484: dwrite da/d18/d1a/d37/d41/f83 [0,4194304] 0 2024-04-24T23:13:43.546 INFO:tasks.workunit.client.0.smithi031.stdout:6/576: creat d13/d15/d1d/d23/d44/d47/fca x:0 0 0 2024-04-24T23:13:43.549 INFO:tasks.workunit.client.0.smithi031.stdout:7/408: dread d1/f53 [0,4194304] 0 2024-04-24T23:13:43.551 INFO:tasks.workunit.client.0.smithi031.stdout:9/525: getdents d2/d1b/d14/d78/d84/d4a 0 2024-04-24T23:13:43.551 INFO:tasks.workunit.client.0.smithi031.stdout:6/577: truncate d13/d15/d1d/d23/d2d/fac 162580 0 2024-04-24T23:13:43.554 INFO:tasks.workunit.client.0.smithi031.stdout:9/526: stat d2/d1b/d14/d78/d84/l89 0 2024-04-24T23:13:43.554 INFO:tasks.workunit.client.0.smithi031.stdout:5/396: dwrite d0/d6/d32/f5f [0,4194304] 0 2024-04-24T23:13:43.609 INFO:tasks.workunit.client.0.smithi031.stdout:3/387: rename d6/d14/d1d/f4c to d6/f7e 0 2024-04-24T23:13:43.610 INFO:tasks.workunit.client.0.smithi031.stdout:3/388: chown d6/df/f34 6176 1 2024-04-24T23:13:43.612 INFO:tasks.workunit.client.0.smithi031.stdout:3/389: dread d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:43.616 INFO:tasks.workunit.client.0.smithi031.stdout:7/409: dread d1/d2/dd/d33/f4e [0,4194304] 0 2024-04-24T23:13:43.618 INFO:tasks.workunit.client.0.smithi031.stdout:9/527: unlink d2/d1b/d14/d78/d84/f60 0 2024-04-24T23:13:43.619 INFO:tasks.workunit.client.1.smithi040.stdout:0/134: sync 2024-04-24T23:13:43.620 INFO:tasks.workunit.client.0.smithi031.stdout:3/390: dwrite d6/df/d16/d53/f67 [0,4194304] 0 2024-04-24T23:13:43.625 INFO:tasks.workunit.client.0.smithi031.stdout:1/393: rename d1/da/l16 to d1/da/d61/l7d 0 2024-04-24T23:13:43.634 INFO:tasks.workunit.client.0.smithi031.stdout:7/410: creat d1/d45/d75/f87 x:0 0 0 2024-04-24T23:13:43.642 INFO:tasks.workunit.client.0.smithi031.stdout:1/394: dread d1/d33/f47 [0,4194304] 0 2024-04-24T23:13:43.647 INFO:tasks.workunit.client.0.smithi031.stdout:9/528: mkdir d2/d1b/d14/d1d/d45/d99 0 2024-04-24T23:13:43.648 INFO:tasks.workunit.client.0.smithi031.stdout:9/529: stat d2/d1b/f41 0 2024-04-24T23:13:43.651 INFO:tasks.workunit.client.0.smithi031.stdout:7/411: rename d1/d2/c40 to d1/d2/d1e/c88 0 2024-04-24T23:13:43.656 INFO:tasks.workunit.client.0.smithi031.stdout:9/530: unlink d2/d1b/d14/c53 0 2024-04-24T23:13:43.659 INFO:tasks.workunit.client.0.smithi031.stdout:1/395: rename d1/d49 to d1/da/d12/d7e 0 2024-04-24T23:13:43.660 INFO:tasks.workunit.client.0.smithi031.stdout:1/396: truncate d1/da/d10/f51 1362831 0 2024-04-24T23:13:43.661 INFO:tasks.workunit.client.0.smithi031.stdout:7/412: mknod d1/d45/c89 0 2024-04-24T23:13:43.664 INFO:tasks.workunit.client.0.smithi031.stdout:6/578: truncate d13/d15/fb9 60034 0 2024-04-24T23:13:43.666 INFO:tasks.workunit.client.0.smithi031.stdout:9/531: creat d2/d1b/d14/d78/d84/f9a x:0 0 0 2024-04-24T23:13:43.666 INFO:tasks.workunit.client.0.smithi031.stdout:9/532: chown d2/d1b/d14/d18 0 1 2024-04-24T23:13:43.670 INFO:tasks.workunit.client.0.smithi031.stdout:7/413: rename d1/d2/f3b to d1/d6a/d65/f8a 0 2024-04-24T23:13:43.672 INFO:tasks.workunit.client.0.smithi031.stdout:4/547: sync 2024-04-24T23:13:43.673 INFO:tasks.workunit.client.0.smithi031.stdout:5/397: truncate d0/d6/d12/f31 119565 0 2024-04-24T23:13:43.680 INFO:tasks.workunit.client.0.smithi031.stdout:9/533: dread d2/d1b/f10 [0,4194304] 0 2024-04-24T23:13:43.681 INFO:tasks.workunit.client.0.smithi031.stdout:6/579: rename d13/d15/d1d/d23/d2d/d9e/fb7 to d13/d15/d1d/d23/d24/d43/fcb 0 2024-04-24T23:13:43.687 INFO:tasks.workunit.client.0.smithi031.stdout:1/397: link d1/da/d12/c14 d1/da/d12/c7f 0 2024-04-24T23:13:43.692 INFO:tasks.workunit.client.0.smithi031.stdout:6/580: fsync d13/d15/d1d/d23/d44/f5b 0 2024-04-24T23:13:43.694 INFO:tasks.workunit.client.0.smithi031.stdout:9/534: rmdir d2/d1b/d14/d78/d84/d4a/d4b/d96 39 2024-04-24T23:13:43.695 INFO:tasks.workunit.client.0.smithi031.stdout:3/391: write d6/f7e [2549395,57219] 0 2024-04-24T23:13:43.700 INFO:tasks.workunit.client.0.smithi031.stdout:1/398: mkdir d1/da/d61/d80 0 2024-04-24T23:13:43.700 INFO:tasks.workunit.client.0.smithi031.stdout:1/399: fdatasync d1/dc/d11/f13 0 2024-04-24T23:13:43.705 INFO:tasks.workunit.client.0.smithi031.stdout:9/535: creat d2/d1b/d85/f9b x:0 0 0 2024-04-24T23:13:43.707 INFO:tasks.workunit.client.0.smithi031.stdout:3/392: truncate d6/df/d16/f1c 1121807 0 2024-04-24T23:13:43.722 INFO:tasks.workunit.client.1.smithi040.stdout:4/46: link da/cd da/c14 0 2024-04-24T23:13:43.725 INFO:tasks.workunit.client.1.smithi040.stdout:7/130: link f5 db/d2f/f35 0 2024-04-24T23:13:43.725 INFO:tasks.workunit.client.1.smithi040.stdout:6/69: rmdir d7/de 39 2024-04-24T23:13:43.725 INFO:tasks.workunit.client.1.smithi040.stdout:7/131: read - db/fd zero size 2024-04-24T23:13:43.726 INFO:tasks.workunit.client.1.smithi040.stdout:8/53: rename d0/d8/dc/de to d0/d8/dc/df 0 2024-04-24T23:13:43.728 INFO:tasks.workunit.client.1.smithi040.stdout:1/54: getdents da/d12 0 2024-04-24T23:13:43.730 INFO:tasks.workunit.client.1.smithi040.stdout:6/70: dwrite f2 [0,4194304] 0 2024-04-24T23:13:43.730 INFO:tasks.workunit.client.1.smithi040.stdout:0/135: unlink df/c18 0 2024-04-24T23:13:43.734 INFO:tasks.workunit.client.1.smithi040.stdout:5/81: mkdir d2/d7/d8/db/d1e/d22 0 2024-04-24T23:13:43.735 INFO:tasks.workunit.client.1.smithi040.stdout:2/128: symlink da/l25 0 2024-04-24T23:13:43.738 INFO:tasks.workunit.client.1.smithi040.stdout:1/55: rename da/fd to da/d12/d14/f16 0 2024-04-24T23:13:43.742 INFO:tasks.workunit.client.1.smithi040.stdout:3/80: link d1/fd d1/f16 0 2024-04-24T23:13:43.743 INFO:tasks.workunit.client.1.smithi040.stdout:0/136: write df/d12/f1f [4466299,46992] 0 2024-04-24T23:13:43.744 INFO:tasks.workunit.client.1.smithi040.stdout:0/137: chown df/d14/d1b/l15 38249905 1 2024-04-24T23:13:43.744 INFO:tasks.workunit.client.1.smithi040.stdout:5/82: mkdir d2/d7/d8/d11/d23 0 2024-04-24T23:13:43.745 INFO:tasks.workunit.client.1.smithi040.stdout:2/129: creat da/d10/f26 x:0 0 0 2024-04-24T23:13:43.746 INFO:tasks.workunit.client.1.smithi040.stdout:1/56: chown f7 115438 1 2024-04-24T23:13:43.748 INFO:tasks.workunit.client.1.smithi040.stdout:6/71: mknod d7/de/c13 0 2024-04-24T23:13:43.749 INFO:tasks.workunit.client.1.smithi040.stdout:3/81: read d1/f5 [732764,80707] 0 2024-04-24T23:13:43.749 INFO:tasks.workunit.client.1.smithi040.stdout:7/132: link c3 db/c36 0 2024-04-24T23:13:43.749 INFO:tasks.workunit.client.1.smithi040.stdout:7/133: readlink db/l1a 0 2024-04-24T23:13:43.751 INFO:tasks.workunit.client.1.smithi040.stdout:9/119: getdents d7/d1b/d2f/d28 0 2024-04-24T23:13:43.753 INFO:tasks.workunit.client.1.smithi040.stdout:6/72: dread d7/f9 [0,4194304] 0 2024-04-24T23:13:43.754 INFO:tasks.workunit.client.1.smithi040.stdout:9/120: readlink d7/d1b/d2f/l20 0 2024-04-24T23:13:43.754 INFO:tasks.workunit.client.1.smithi040.stdout:4/47: getdents da 0 2024-04-24T23:13:43.755 INFO:tasks.workunit.client.1.smithi040.stdout:9/121: write f4 [384711,75454] 0 2024-04-24T23:13:43.755 INFO:tasks.workunit.client.1.smithi040.stdout:6/73: chown d7/de/ff 23756 1 2024-04-24T23:13:43.756 INFO:tasks.workunit.client.1.smithi040.stdout:7/134: mkdir db/d2f/d37 0 2024-04-24T23:13:43.756 INFO:tasks.workunit.client.1.smithi040.stdout:5/83: dread d2/d7/fe [0,4194304] 0 2024-04-24T23:13:43.759 INFO:tasks.workunit.client.1.smithi040.stdout:6/74: dread d7/de/ff [0,4194304] 0 2024-04-24T23:13:43.760 INFO:tasks.workunit.client.1.smithi040.stdout:9/122: dread d7/da/fd [0,4194304] 0 2024-04-24T23:13:43.760 INFO:tasks.workunit.client.1.smithi040.stdout:4/48: rename c7 to da/c15 0 2024-04-24T23:13:43.761 INFO:tasks.workunit.client.1.smithi040.stdout:5/84: rename d2/d7/d8/db to d2/d7/d8/db/d24 22 2024-04-24T23:13:43.761 INFO:tasks.workunit.client.1.smithi040.stdout:4/49: truncate f9 1072541 0 2024-04-24T23:13:43.762 INFO:tasks.workunit.client.1.smithi040.stdout:4/50: write f6 [5974233,21686] 0 2024-04-24T23:13:43.762 INFO:tasks.workunit.client.1.smithi040.stdout:4/51: chown da/fb 2286 1 2024-04-24T23:13:43.766 INFO:tasks.workunit.client.1.smithi040.stdout:9/123: chown d7/d1b/d2f/c2e 3469 1 2024-04-24T23:13:43.766 INFO:tasks.workunit.client.1.smithi040.stdout:3/82: creat d1/d9/f17 x:0 0 0 2024-04-24T23:13:43.770 INFO:tasks.workunit.client.1.smithi040.stdout:5/85: rename d2/d7/d8/d11/f20 to d2/d7/d8/db/d1e/d22/f25 0 2024-04-24T23:13:43.770 INFO:tasks.workunit.client.1.smithi040.stdout:4/52: symlink da/l16 0 2024-04-24T23:13:43.773 INFO:tasks.workunit.client.1.smithi040.stdout:6/75: unlink d7/f8 0 2024-04-24T23:13:43.774 INFO:tasks.workunit.client.1.smithi040.stdout:9/124: mkdir d7/d1b/d2f/d1f/d31 0 2024-04-24T23:13:43.776 INFO:tasks.workunit.client.1.smithi040.stdout:5/86: mknod d2/d7/d8/db/d1e/d22/c26 0 2024-04-24T23:13:43.777 INFO:tasks.workunit.client.0.smithi031.stdout:5/398: creat d0/d6/d3b/d4b/dc/f81 x:0 0 0 2024-04-24T23:13:43.777 INFO:tasks.workunit.client.0.smithi031.stdout:1/400: mkdir d1/d33/d36/d81 0 2024-04-24T23:13:43.779 INFO:tasks.workunit.client.1.smithi040.stdout:6/76: rmdir d7/de 39 2024-04-24T23:13:43.780 INFO:tasks.workunit.client.0.smithi031.stdout:9/536: write d2/d1b/d14/d1d/d45/f93 [59603,78733] 0 2024-04-24T23:13:43.780 INFO:tasks.workunit.client.1.smithi040.stdout:6/77: write f0 [2400419,17317] 0 2024-04-24T23:13:43.781 INFO:tasks.workunit.client.0.smithi031.stdout:5/399: write d0/d6/d3b/d4b/dc/f6e [2143876,82004] 0 2024-04-24T23:13:43.782 INFO:tasks.workunit.client.0.smithi031.stdout:7/414: rename d1/d2/dd/c6c to d1/d6a/c8b 0 2024-04-24T23:13:43.783 INFO:tasks.workunit.client.1.smithi040.stdout:5/87: dwrite d2/d7/d8/d11/d14/f17 [0,4194304] 0 2024-04-24T23:13:43.785 INFO:tasks.workunit.client.0.smithi031.stdout:4/548: dwrite d3/d23/f3d [0,4194304] 0 2024-04-24T23:13:43.785 INFO:tasks.workunit.client.0.smithi031.stdout:6/581: link d13/d15/d1d/d23/d24/d43/f71 d13/d15/d1d/d23/d24/d43/d68/db8/fcc 0 2024-04-24T23:13:43.786 INFO:tasks.workunit.client.0.smithi031.stdout:6/582: chown fe 0 1 2024-04-24T23:13:43.797 INFO:tasks.workunit.client.1.smithi040.stdout:0/138: write f6 [4060118,126388] 0 2024-04-24T23:13:43.798 INFO:tasks.workunit.client.1.smithi040.stdout:1/57: write f2 [4527795,64957] 0 2024-04-24T23:13:43.799 INFO:tasks.workunit.client.1.smithi040.stdout:1/58: write f7 [1602256,21513] 0 2024-04-24T23:13:43.799 INFO:tasks.workunit.client.1.smithi040.stdout:6/78: write d7/f9 [705670,40256] 0 2024-04-24T23:13:43.803 INFO:tasks.workunit.client.1.smithi040.stdout:9/125: fdatasync f2 0 2024-04-24T23:13:43.805 INFO:tasks.workunit.client.1.smithi040.stdout:9/126: chown d7/d1b/d2f/d1f/c25 189371102 1 2024-04-24T23:13:43.805 INFO:tasks.workunit.client.0.smithi031.stdout:1/401: rename d1/dc/d11/f55 to d1/da/d12/f82 0 2024-04-24T23:13:43.806 INFO:tasks.workunit.client.1.smithi040.stdout:9/127: dread - d7/d1b/d2f/f30 zero size 2024-04-24T23:13:43.807 INFO:tasks.workunit.client.0.smithi031.stdout:7/415: fdatasync d1/f4b 0 2024-04-24T23:13:43.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/416: truncate d1/d6a/f80 129240 0 2024-04-24T23:13:43.811 INFO:tasks.workunit.client.0.smithi031.stdout:6/583: creat d13/d15/d1d/d23/d24/d8b/fcd x:0 0 0 2024-04-24T23:13:43.814 INFO:tasks.workunit.client.0.smithi031.stdout:5/400: truncate d0/d7/f58 1175241 0 2024-04-24T23:13:43.817 INFO:tasks.workunit.client.0.smithi031.stdout:1/402: fdatasync d1/d1d/f7a 0 2024-04-24T23:13:43.818 INFO:tasks.workunit.client.1.smithi040.stdout:7/135: dwrite db/d12/d20/f22 [0,4194304] 0 2024-04-24T23:13:43.818 INFO:tasks.workunit.client.0.smithi031.stdout:1/403: write d1/d33/f48 [330965,120570] 0 2024-04-24T23:13:43.819 INFO:tasks.workunit.client.1.smithi040.stdout:7/136: write db/d12/f33 [103302,52073] 0 2024-04-24T23:13:43.819 INFO:tasks.workunit.client.1.smithi040.stdout:3/83: rename d1/d9/db to d1/d9/d18 0 2024-04-24T23:13:43.820 INFO:tasks.workunit.client.0.smithi031.stdout:9/537: mkdir d2/d1b/d14/d1d/d45/d99/d9c 0 2024-04-24T23:13:43.820 INFO:tasks.workunit.client.1.smithi040.stdout:7/137: fdatasync db/f17 0 2024-04-24T23:13:43.821 INFO:tasks.workunit.client.1.smithi040.stdout:9/128: readlink d7/da/l24 0 2024-04-24T23:13:43.822 INFO:tasks.workunit.client.0.smithi031.stdout:7/417: symlink d1/d2/d60/l8c 0 2024-04-24T23:13:43.823 INFO:tasks.workunit.client.1.smithi040.stdout:4/53: getdents da 0 2024-04-24T23:13:43.824 INFO:tasks.workunit.client.1.smithi040.stdout:7/138: dread db/d2f/f35 [0,4194304] 0 2024-04-24T23:13:43.824 INFO:tasks.workunit.client.0.smithi031.stdout:6/584: rmdir d13/d15/d1d/d23/d44 39 2024-04-24T23:13:43.827 INFO:tasks.workunit.client.1.smithi040.stdout:1/59: mknod da/c17 0 2024-04-24T23:13:43.828 INFO:tasks.workunit.client.1.smithi040.stdout:1/60: chown da/cb 11590 1 2024-04-24T23:13:43.829 INFO:tasks.workunit.client.0.smithi031.stdout:3/393: link d6/d14/f7d d6/df/d16/f7f 0 2024-04-24T23:13:43.830 INFO:tasks.workunit.client.0.smithi031.stdout:3/394: dread - d6/d72/d1a/d25/f5f zero size 2024-04-24T23:13:43.830 INFO:tasks.workunit.client.1.smithi040.stdout:6/79: symlink d7/de/l14 0 2024-04-24T23:13:43.831 INFO:tasks.workunit.client.0.smithi031.stdout:3/395: read - d6/df/d16/d53/f6e zero size 2024-04-24T23:13:43.831 INFO:tasks.workunit.client.1.smithi040.stdout:7/139: dwrite db/f24 [0,4194304] 0 2024-04-24T23:13:43.832 INFO:tasks.workunit.client.0.smithi031.stdout:3/396: write d6/d72/d1a/f22 [4341456,70664] 0 2024-04-24T23:13:43.833 INFO:tasks.workunit.client.1.smithi040.stdout:6/80: write d7/fb [554737,89942] 0 2024-04-24T23:13:43.833 INFO:tasks.workunit.client.1.smithi040.stdout:7/140: dread - db/d2f/f30 zero size 2024-04-24T23:13:43.836 INFO:tasks.workunit.client.1.smithi040.stdout:3/84: creat d1/d9/d18/f19 x:0 0 0 2024-04-24T23:13:43.841 INFO:tasks.workunit.client.0.smithi031.stdout:3/397: dwrite d6/df/d16/d53/f67 [0,4194304] 0 2024-04-24T23:13:43.841 INFO:tasks.workunit.client.1.smithi040.stdout:9/129: mknod d7/d1b/d2f/c32 0 2024-04-24T23:13:43.842 INFO:tasks.workunit.client.1.smithi040.stdout:7/141: dwrite db/d1b/f28 [0,4194304] 0 2024-04-24T23:13:43.844 INFO:tasks.workunit.client.0.smithi031.stdout:1/404: creat d1/da/d12/f83 x:0 0 0 2024-04-24T23:13:43.846 INFO:tasks.workunit.client.0.smithi031.stdout:1/405: write d1/d3e/d46/f59 [383305,109669] 0 2024-04-24T23:13:43.848 INFO:tasks.workunit.client.0.smithi031.stdout:1/406: write d1/d33/f48 [335379,43593] 0 2024-04-24T23:13:43.848 INFO:tasks.workunit.client.0.smithi031.stdout:9/538: mkdir d2/d43/d9d 0 2024-04-24T23:13:43.850 INFO:tasks.workunit.client.0.smithi031.stdout:9/539: dread - d2/d1b/d14/d78/d84/d4a/d4b/f87 zero size 2024-04-24T23:13:43.854 INFO:tasks.workunit.client.1.smithi040.stdout:1/61: mkdir da/d12/d18 0 2024-04-24T23:13:43.854 INFO:tasks.workunit.client.0.smithi031.stdout:5/401: creat d0/d7/d74/f82 x:0 0 0 2024-04-24T23:13:43.856 INFO:tasks.workunit.client.1.smithi040.stdout:5/88: getdents d2/d7/d8/d11/d14 0 2024-04-24T23:13:43.857 INFO:tasks.workunit.client.0.smithi031.stdout:4/549: rename d3/d23/d3b/f72 to d3/d20/d25/fb0 0 2024-04-24T23:13:43.858 INFO:tasks.workunit.client.1.smithi040.stdout:6/81: mkdir d7/de/d15 0 2024-04-24T23:13:43.859 INFO:tasks.workunit.client.0.smithi031.stdout:4/550: dread - d3/d20/d2d/d44/d5b/d5d/f77 zero size 2024-04-24T23:13:43.862 INFO:tasks.workunit.client.1.smithi040.stdout:6/82: dread f0 [0,4194304] 0 2024-04-24T23:13:43.862 INFO:tasks.workunit.client.1.smithi040.stdout:9/130: unlink d7/f2a 0 2024-04-24T23:13:43.864 INFO:tasks.workunit.client.1.smithi040.stdout:7/142: rename db/d1b/l1d to db/d2f/d37/l38 0 2024-04-24T23:13:43.866 INFO:tasks.workunit.client.0.smithi031.stdout:1/407: write d1/d1d/f35 [1734400,32732] 0 2024-04-24T23:13:43.866 INFO:tasks.workunit.client.0.smithi031.stdout:9/540: mknod d2/d1b/d14/d18/c9e 0 2024-04-24T23:13:43.868 INFO:tasks.workunit.client.1.smithi040.stdout:5/89: mkdir d2/d7/d8/db/d1e/d22/d27 0 2024-04-24T23:13:43.871 INFO:tasks.workunit.client.0.smithi031.stdout:7/418: rename d1/d2/dd/d33/l3d to d1/d45/d75/l8d 0 2024-04-24T23:13:43.872 INFO:tasks.workunit.client.1.smithi040.stdout:8/54: sync 2024-04-24T23:13:43.872 INFO:tasks.workunit.client.1.smithi040.stdout:2/130: sync 2024-04-24T23:13:43.873 INFO:tasks.workunit.client.0.smithi031.stdout:4/551: symlink d3/d20/d2d/d44/d5b/lb1 0 2024-04-24T23:13:43.873 INFO:tasks.workunit.client.1.smithi040.stdout:9/131: dwrite f4 [4194304,4194304] 0 2024-04-24T23:13:43.873 INFO:tasks.workunit.client.1.smithi040.stdout:8/55: write d0/f7 [2122212,42382] 0 2024-04-24T23:13:43.874 INFO:tasks.workunit.client.0.smithi031.stdout:2/333: sync 2024-04-24T23:13:43.874 INFO:tasks.workunit.client.0.smithi031.stdout:8/532: sync 2024-04-24T23:13:43.874 INFO:tasks.workunit.client.1.smithi040.stdout:9/132: write f4 [911955,79352] 0 2024-04-24T23:13:43.875 INFO:tasks.workunit.client.0.smithi031.stdout:6/585: write d13/d1b/f6b [1247001,66802] 0 2024-04-24T23:13:43.875 INFO:tasks.workunit.client.0.smithi031.stdout:8/533: chown d8/d1a/d1d/d50 5902 1 2024-04-24T23:13:43.881 INFO:tasks.workunit.client.1.smithi040.stdout:1/62: symlink da/l19 0 2024-04-24T23:13:43.883 INFO:tasks.workunit.client.0.smithi031.stdout:8/534: dwrite d8/d15/d37/d43/d7b/d86/fb6 [0,4194304] 0 2024-04-24T23:13:43.884 INFO:tasks.workunit.client.1.smithi040.stdout:1/63: dread da/ff [0,4194304] 0 2024-04-24T23:13:43.884 INFO:tasks.workunit.client.0.smithi031.stdout:8/535: fsync d8/d1a/d1d/d50/dc0/d9d/fa3 0 2024-04-24T23:13:43.889 INFO:tasks.workunit.client.1.smithi040.stdout:5/90: creat d2/d7/d8/db/d1e/f28 x:0 0 0 2024-04-24T23:13:43.890 INFO:tasks.workunit.client.0.smithi031.stdout:2/334: mknod d5/d8/d45/c73 0 2024-04-24T23:13:43.893 INFO:tasks.workunit.client.0.smithi031.stdout:1/408: creat d1/da/d61/d80/f84 x:0 0 0 2024-04-24T23:13:43.893 INFO:tasks.workunit.client.0.smithi031.stdout:3/398: dwrite d6/d14/d1d/d69/f6d [0,4194304] 0 2024-04-24T23:13:43.894 INFO:tasks.workunit.client.1.smithi040.stdout:7/143: rmdir db/d2f 39 2024-04-24T23:13:43.897 INFO:tasks.workunit.client.0.smithi031.stdout:3/399: truncate d6/df/d16/d53/f6e 97005 0 2024-04-24T23:13:43.897 INFO:tasks.workunit.client.0.smithi031.stdout:3/400: chown d6/df/d16/d53 0 1 2024-04-24T23:13:43.899 INFO:tasks.workunit.client.1.smithi040.stdout:6/83: mknod d7/de/d15/c16 0 2024-04-24T23:13:43.904 INFO:tasks.workunit.client.1.smithi040.stdout:8/56: rename d0/f7 to d0/d8/dc/df/f10 0 2024-04-24T23:13:43.905 INFO:tasks.workunit.client.1.smithi040.stdout:5/91: symlink d2/d7/d8/d11/l29 0 2024-04-24T23:13:43.905 INFO:tasks.workunit.client.0.smithi031.stdout:7/419: link d1/d6a/d66/f69 d1/d2/dd/d33/d5e/d64/f8e 0 2024-04-24T23:13:43.905 INFO:tasks.workunit.client.1.smithi040.stdout:1/64: rename da/d12 to da/d12/d14/d1a 22 2024-04-24T23:13:43.907 INFO:tasks.workunit.client.0.smithi031.stdout:9/541: write d2/d1b/d14/d1d/d45/f5e [1642722,68168] 0 2024-04-24T23:13:43.908 INFO:tasks.workunit.client.1.smithi040.stdout:2/131: creat da/d1c/f27 x:0 0 0 2024-04-24T23:13:43.908 INFO:tasks.workunit.client.1.smithi040.stdout:5/92: chown d2/d7/d8/d11/d14 7 1 2024-04-24T23:13:43.909 INFO:tasks.workunit.client.0.smithi031.stdout:9/542: truncate d2/d1b/f8f 49846 0 2024-04-24T23:13:43.909 INFO:tasks.workunit.client.1.smithi040.stdout:4/54: dwrite f4 [0,4194304] 0 2024-04-24T23:13:43.910 INFO:tasks.workunit.client.1.smithi040.stdout:8/57: write d0/d8/fd [1142942,93398] 0 2024-04-24T23:13:43.910 INFO:tasks.workunit.client.1.smithi040.stdout:7/144: dwrite db/f10 [4194304,4194304] 0 2024-04-24T23:13:43.912 INFO:tasks.workunit.client.0.smithi031.stdout:4/552: link d3/d5e/f8d d3/d20/d2d/d44/fb2 0 2024-04-24T23:13:43.915 INFO:tasks.workunit.client.0.smithi031.stdout:8/536: dwrite d8/d1a/d1d/d35/d58/f4c [0,4194304] 0 2024-04-24T23:13:43.916 INFO:tasks.workunit.client.0.smithi031.stdout:8/537: dread - d8/d15/f3d zero size 2024-04-24T23:13:43.917 INFO:tasks.workunit.client.0.smithi031.stdout:8/538: dread - d8/d15/d9a/fb1 zero size 2024-04-24T23:13:43.920 INFO:tasks.workunit.client.0.smithi031.stdout:1/409: read d1/da/d12/d7e/d7b/f6a [6490463,116414] 0 2024-04-24T23:13:43.920 INFO:tasks.workunit.client.0.smithi031.stdout:8/539: dread - d8/d15/d37/d43/d7b/d87/fa0 zero size 2024-04-24T23:13:43.923 INFO:tasks.workunit.client.1.smithi040.stdout:9/133: unlink d7/da/c18 0 2024-04-24T23:13:43.929 INFO:tasks.workunit.client.0.smithi031.stdout:1/410: read d1/da/d12/f23 [2377190,59837] 0 2024-04-24T23:13:43.929 INFO:tasks.workunit.client.0.smithi031.stdout:3/401: rmdir d6/d72 39 2024-04-24T23:13:43.930 INFO:tasks.workunit.client.0.smithi031.stdout:5/402: rename d0/d6/d3b/d4b/d1d/d2e/f50 to d0/d6/d3b/d4b/f83 0 2024-04-24T23:13:43.932 INFO:tasks.workunit.client.0.smithi031.stdout:5/403: write d0/d6/d32/f4 [3808061,129232] 0 2024-04-24T23:13:43.932 INFO:tasks.workunit.client.0.smithi031.stdout:3/402: write d6/d14/d1d/d59/f6c [228826,1901] 0 2024-04-24T23:13:43.933 INFO:tasks.workunit.client.1.smithi040.stdout:5/93: mknod d2/c2a 0 2024-04-24T23:13:43.934 INFO:tasks.workunit.client.0.smithi031.stdout:9/543: stat d2/d16/l3f 0 2024-04-24T23:13:43.935 INFO:tasks.workunit.client.0.smithi031.stdout:9/544: dread - d2/d1b/d14/d78/d84/d4a/d4b/f87 zero size 2024-04-24T23:13:43.936 INFO:tasks.workunit.client.0.smithi031.stdout:1/411: dread d1/f66 [0,4194304] 0 2024-04-24T23:13:43.937 INFO:tasks.workunit.client.1.smithi040.stdout:7/145: rename db/f17 to db/d1b/f39 0 2024-04-24T23:13:43.938 INFO:tasks.workunit.client.1.smithi040.stdout:1/65: dwrite da/d12/d14/f16 [0,4194304] 0 2024-04-24T23:13:43.940 INFO:tasks.workunit.client.1.smithi040.stdout:2/132: symlink da/l28 0 2024-04-24T23:13:43.943 INFO:tasks.workunit.client.1.smithi040.stdout:0/139: sync 2024-04-24T23:13:43.945 INFO:tasks.workunit.client.1.smithi040.stdout:5/94: dread d2/d7/d8/f13 [0,4194304] 0 2024-04-24T23:13:43.945 INFO:tasks.workunit.client.1.smithi040.stdout:7/146: chown db/d12/c2d 1 1 2024-04-24T23:13:43.946 INFO:tasks.workunit.client.0.smithi031.stdout:8/540: dwrite d8/d1a/d1d/d50/fb7 [0,4194304] 0 2024-04-24T23:13:43.947 INFO:tasks.workunit.client.1.smithi040.stdout:0/140: chown df/d14/f16 323 1 2024-04-24T23:13:43.950 INFO:tasks.workunit.client.0.smithi031.stdout:0/485: sync 2024-04-24T23:13:43.951 INFO:tasks.workunit.client.1.smithi040.stdout:4/55: dwrite da/f13 [4194304,4194304] 0 2024-04-24T23:13:43.951 INFO:tasks.workunit.client.0.smithi031.stdout:0/486: read da/f10 [165751,41001] 0 2024-04-24T23:13:43.952 INFO:tasks.workunit.client.0.smithi031.stdout:0/487: fdatasync da/d11/f33 0 2024-04-24T23:13:43.952 INFO:tasks.workunit.client.0.smithi031.stdout:2/335: write d5/d28/d43/f55 [1869239,20628] 0 2024-04-24T23:13:43.953 INFO:tasks.workunit.client.0.smithi031.stdout:0/488: chown da/d20/c6c 22 1 2024-04-24T23:13:43.954 INFO:tasks.workunit.client.1.smithi040.stdout:9/134: rename d7/c22 to d7/d1b/c33 0 2024-04-24T23:13:43.955 INFO:tasks.workunit.client.1.smithi040.stdout:6/84: write d7/de/ff [1395076,111346] 0 2024-04-24T23:13:43.957 INFO:tasks.workunit.client.0.smithi031.stdout:1/412: dread d1/da/d12/d7e/d7b/f6a [0,4194304] 0 2024-04-24T23:13:43.957 INFO:tasks.workunit.client.1.smithi040.stdout:2/133: rmdir da/d1c 39 2024-04-24T23:13:43.961 INFO:tasks.workunit.client.1.smithi040.stdout:9/135: write f4 [8173029,112130] 0 2024-04-24T23:13:43.961 INFO:tasks.workunit.client.1.smithi040.stdout:6/85: write d7/f9 [1403473,124946] 0 2024-04-24T23:13:43.962 INFO:tasks.workunit.client.1.smithi040.stdout:6/86: stat d7/de/c13 0 2024-04-24T23:13:43.963 INFO:tasks.workunit.client.1.smithi040.stdout:9/136: chown d7/da/l24 3671294 1 2024-04-24T23:13:43.964 INFO:tasks.workunit.client.1.smithi040.stdout:4/56: dwrite da/f13 [4194304,4194304] 0 2024-04-24T23:13:43.965 INFO:tasks.workunit.client.1.smithi040.stdout:9/137: read f2 [54549,14700] 0 2024-04-24T23:13:43.966 INFO:tasks.workunit.client.1.smithi040.stdout:6/87: dread d7/fd [0,4194304] 0 2024-04-24T23:13:43.966 INFO:tasks.workunit.client.1.smithi040.stdout:0/141: symlink df/l26 0 2024-04-24T23:13:43.968 INFO:tasks.workunit.client.1.smithi040.stdout:2/134: dread da/f19 [0,4194304] 0 2024-04-24T23:13:43.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/553: dwrite d3/f85 [4194304,4194304] 0 2024-04-24T23:13:43.974 INFO:tasks.workunit.client.1.smithi040.stdout:5/95: rename d2/d7/d8/l1d to d2/d7/d8/db/d1e/d22/l2b 0 2024-04-24T23:13:43.977 INFO:tasks.workunit.client.1.smithi040.stdout:3/85: sync 2024-04-24T23:13:43.977 INFO:tasks.workunit.client.1.smithi040.stdout:3/86: stat d1/d9/da 0 2024-04-24T23:13:43.979 INFO:tasks.workunit.client.1.smithi040.stdout:8/58: truncate d0/d8/dc/df/f10 4124130 0 2024-04-24T23:13:43.979 INFO:tasks.workunit.client.1.smithi040.stdout:7/147: readlink db/d2f/d37/l38 0 2024-04-24T23:13:43.985 INFO:tasks.workunit.client.1.smithi040.stdout:8/59: dwrite d0/d8/fd [0,4194304] 0 2024-04-24T23:13:43.988 INFO:tasks.workunit.client.1.smithi040.stdout:4/57: symlink da/l17 0 2024-04-24T23:13:43.990 INFO:tasks.workunit.client.1.smithi040.stdout:9/138: mkdir d7/d1b/d2f/d1f/d34 0 2024-04-24T23:13:43.991 INFO:tasks.workunit.client.1.smithi040.stdout:0/142: symlink df/d14/l27 0 2024-04-24T23:13:43.993 INFO:tasks.workunit.client.1.smithi040.stdout:6/88: creat d7/de/d15/f17 x:0 0 0 2024-04-24T23:13:43.994 INFO:tasks.workunit.client.1.smithi040.stdout:6/89: stat d7/de/d15/c16 0 2024-04-24T23:13:43.995 INFO:tasks.workunit.client.1.smithi040.stdout:5/96: symlink d2/d7/d8/l2c 0 2024-04-24T23:13:43.996 INFO:tasks.workunit.client.1.smithi040.stdout:3/87: chown d1/d9/l11 519919 1 2024-04-24T23:13:43.998 INFO:tasks.workunit.client.1.smithi040.stdout:5/97: truncate d2/d7/d8/f1b 673114 0 2024-04-24T23:13:43.998 INFO:tasks.workunit.client.1.smithi040.stdout:1/66: link da/d12/d14/l15 da/d12/l1b 0 2024-04-24T23:13:43.999 INFO:tasks.workunit.client.0.smithi031.stdout:6/586: chown d13/d15/d1d/d23/d44/d47/f4a 33613 1 2024-04-24T23:13:43.999 INFO:tasks.workunit.client.1.smithi040.stdout:0/143: dwrite df/d12/f17 [0,4194304] 0 2024-04-24T23:13:44.004 INFO:tasks.workunit.client.1.smithi040.stdout:4/58: rename da/f12 to da/f18 0 2024-04-24T23:13:44.010 INFO:tasks.workunit.client.0.smithi031.stdout:9/545: mknod d2/d1b/d14/d1d/d45/d99/c9f 0 2024-04-24T23:13:44.011 INFO:tasks.workunit.client.1.smithi040.stdout:2/135: mkdir da/d29 0 2024-04-24T23:13:44.015 INFO:tasks.workunit.client.1.smithi040.stdout:3/88: mknod d1/d9/d18/c1a 0 2024-04-24T23:13:44.016 INFO:tasks.workunit.client.1.smithi040.stdout:3/89: chown d1/f16 3550 1 2024-04-24T23:13:44.017 INFO:tasks.workunit.client.1.smithi040.stdout:5/98: mknod d2/c2d 0 2024-04-24T23:13:44.017 INFO:tasks.workunit.client.0.smithi031.stdout:9/546: dwrite d2/f12 [0,4194304] 0 2024-04-24T23:13:44.018 INFO:tasks.workunit.client.0.smithi031.stdout:9/547: stat d2/d1b/d14/d1d 0 2024-04-24T23:13:44.018 INFO:tasks.workunit.client.1.smithi040.stdout:5/99: read d2/d7/d8/d11/d14/f17 [2232121,77985] 0 2024-04-24T23:13:44.023 INFO:tasks.workunit.client.1.smithi040.stdout:1/67: rename da/f10 to da/d12/f1c 0 2024-04-24T23:13:44.023 INFO:tasks.workunit.client.1.smithi040.stdout:1/68: read da/ff [2019336,55244] 0 2024-04-24T23:13:44.024 INFO:tasks.workunit.client.0.smithi031.stdout:2/336: symlink d5/d6/d29/d26/l74 0 2024-04-24T23:13:44.024 INFO:tasks.workunit.client.1.smithi040.stdout:8/60: fdatasync d0/d8/dc/df/f10 0 2024-04-24T23:13:44.025 INFO:tasks.workunit.client.0.smithi031.stdout:2/337: write d5/d28/d43/f53 [1886520,53276] 0 2024-04-24T23:13:44.027 INFO:tasks.workunit.client.1.smithi040.stdout:4/59: creat da/f19 x:0 0 0 2024-04-24T23:13:44.029 INFO:tasks.workunit.client.1.smithi040.stdout:9/139: creat d7/d1b/d2f/d1f/d31/f35 x:0 0 0 2024-04-24T23:13:44.030 INFO:tasks.workunit.client.0.smithi031.stdout:4/554: symlink d3/d23/d3b/lb3 0 2024-04-24T23:13:44.031 INFO:tasks.workunit.client.1.smithi040.stdout:9/140: write d7/d1b/d2f/d1f/d31/f35 [223665,117833] 0 2024-04-24T23:13:44.032 INFO:tasks.workunit.client.1.smithi040.stdout:6/90: symlink d7/l18 0 2024-04-24T23:13:44.038 INFO:tasks.workunit.client.1.smithi040.stdout:5/100: creat d2/d7/d8/db/d1e/f2e x:0 0 0 2024-04-24T23:13:44.040 INFO:tasks.workunit.client.0.smithi031.stdout:7/420: dwrite d1/d6a/f3c [0,4194304] 0 2024-04-24T23:13:44.041 INFO:tasks.workunit.client.0.smithi031.stdout:7/421: chown d1/f9 678 1 2024-04-24T23:13:44.043 INFO:tasks.workunit.client.0.smithi031.stdout:5/404: dwrite d0/d6/d3b/d4b/f7a [0,4194304] 0 2024-04-24T23:13:44.044 INFO:tasks.workunit.client.0.smithi031.stdout:8/541: mkdir d8/d15/d37/d43/dbd/dc4 0 2024-04-24T23:13:44.047 INFO:tasks.workunit.client.0.smithi031.stdout:9/548: rename d2/d1b/f4f to d2/d1b/d14/d1d/d45/d56/fa0 0 2024-04-24T23:13:44.048 INFO:tasks.workunit.client.0.smithi031.stdout:5/405: dread d0/d6/d32/f46 [0,4194304] 0 2024-04-24T23:13:44.050 INFO:tasks.workunit.client.1.smithi040.stdout:8/61: write d0/d8/dc/df/f10 [3748867,54634] 0 2024-04-24T23:13:44.055 INFO:tasks.workunit.client.1.smithi040.stdout:0/144: dwrite f2 [8388608,4194304] 0 2024-04-24T23:13:44.055 INFO:tasks.workunit.client.1.smithi040.stdout:2/136: rename da/l28 to da/d1c/l2a 0 2024-04-24T23:13:44.058 INFO:tasks.workunit.client.1.smithi040.stdout:2/137: write da/d1c/f27 [687240,120328] 0 2024-04-24T23:13:44.060 INFO:tasks.workunit.client.1.smithi040.stdout:6/91: symlink d7/de/l19 0 2024-04-24T23:13:44.062 INFO:tasks.workunit.client.1.smithi040.stdout:3/90: write d1/f5 [1546816,71962] 0 2024-04-24T23:13:44.062 INFO:tasks.workunit.client.0.smithi031.stdout:6/587: symlink d13/d15/d1d/d23/d44/d47/lce 0 2024-04-24T23:13:44.062 INFO:tasks.workunit.client.1.smithi040.stdout:6/92: chown f2 127247544 1 2024-04-24T23:13:44.065 INFO:tasks.workunit.client.1.smithi040.stdout:1/69: creat da/d12/d18/f1d x:0 0 0 2024-04-24T23:13:44.067 INFO:tasks.workunit.client.1.smithi040.stdout:3/91: truncate d1/d9/f17 263116 0 2024-04-24T23:13:44.068 INFO:tasks.workunit.client.1.smithi040.stdout:5/101: rename d2/d7/d8/d11 to d2/d2f 0 2024-04-24T23:13:44.071 INFO:tasks.workunit.client.0.smithi031.stdout:9/549: creat d2/d1b/d14/d18/d7d/fa1 x:0 0 0 2024-04-24T23:13:44.072 INFO:tasks.workunit.client.1.smithi040.stdout:9/141: dwrite d7/d1b/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:13:44.073 INFO:tasks.workunit.client.1.smithi040.stdout:6/93: fsync d7/fd 0 2024-04-24T23:13:44.073 INFO:tasks.workunit.client.1.smithi040.stdout:2/138: dwrite da/f20 [0,4194304] 0 2024-04-24T23:13:44.074 INFO:tasks.workunit.client.0.smithi031.stdout:0/489: creat da/f9e x:0 0 0 2024-04-24T23:13:44.075 INFO:tasks.workunit.client.1.smithi040.stdout:7/148: sync 2024-04-24T23:13:44.078 INFO:tasks.workunit.client.1.smithi040.stdout:4/60: write f5 [577547,25129] 0 2024-04-24T23:13:44.079 INFO:tasks.workunit.client.1.smithi040.stdout:7/149: write db/f24 [5027406,125312] 0 2024-04-24T23:13:44.080 INFO:tasks.workunit.client.1.smithi040.stdout:5/102: chown d2/cd 1238 1 2024-04-24T23:13:44.082 INFO:tasks.workunit.client.1.smithi040.stdout:7/150: read db/d1b/f28 [994324,12207] 0 2024-04-24T23:13:44.082 INFO:tasks.workunit.client.1.smithi040.stdout:6/94: dread d7/f9 [0,4194304] 0 2024-04-24T23:13:44.084 INFO:tasks.workunit.client.1.smithi040.stdout:5/103: dread - d2/d7/d8/db/d1e/f28 zero size 2024-04-24T23:13:44.085 INFO:tasks.workunit.client.1.smithi040.stdout:3/92: dwrite d1/d9/da/de/f14 [0,4194304] 0 2024-04-24T23:13:44.086 INFO:tasks.workunit.client.1.smithi040.stdout:5/104: readlink d2/d7/d8/db/l1a 0 2024-04-24T23:13:44.088 INFO:tasks.workunit.client.0.smithi031.stdout:3/403: link d6/d72/d1a/d25/f74 d6/d72/d1a/f80 0 2024-04-24T23:13:44.089 INFO:tasks.workunit.client.1.smithi040.stdout:7/151: dread db/f24 [0,4194304] 0 2024-04-24T23:13:44.090 INFO:tasks.workunit.client.0.smithi031.stdout:2/338: dwrite d5/d28/f3a [0,4194304] 0 2024-04-24T23:13:44.090 INFO:tasks.workunit.client.0.smithi031.stdout:3/404: write d6/df/d16/d2a/f5a [1099908,116383] 0 2024-04-24T23:13:44.091 INFO:tasks.workunit.client.1.smithi040.stdout:7/152: write db/d12/f33 [1100723,93353] 0 2024-04-24T23:13:44.094 INFO:tasks.workunit.client.1.smithi040.stdout:2/139: fsync da/fe 0 2024-04-24T23:13:44.094 INFO:tasks.workunit.client.1.smithi040.stdout:0/145: write df/d14/f16 [1328242,119954] 0 2024-04-24T23:13:44.096 INFO:tasks.workunit.client.0.smithi031.stdout:9/550: rmdir d2/d1b/d14/d1d 39 2024-04-24T23:13:44.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/555: dread d3/f7 [0,4194304] 0 2024-04-24T23:13:44.097 INFO:tasks.workunit.client.0.smithi031.stdout:9/551: read - d2/d1b/d14/d78/d84/d4a/f76 zero size 2024-04-24T23:13:44.098 INFO:tasks.workunit.client.1.smithi040.stdout:8/62: link d0/l4 d0/d8/l11 0 2024-04-24T23:13:44.099 INFO:tasks.workunit.client.0.smithi031.stdout:0/490: creat da/d18/f9f x:0 0 0 2024-04-24T23:13:44.102 INFO:tasks.workunit.client.1.smithi040.stdout:8/63: read d0/d8/fd [2949282,22727] 0 2024-04-24T23:13:44.104 INFO:tasks.workunit.client.1.smithi040.stdout:8/64: read d0/d8/fd [599124,79932] 0 2024-04-24T23:13:44.104 INFO:tasks.workunit.client.1.smithi040.stdout:7/153: dwrite db/d12/d20/f21 [0,4194304] 0 2024-04-24T23:13:44.112 INFO:tasks.workunit.client.1.smithi040.stdout:8/65: dwrite d0/d8/dc/df/f10 [4194304,4194304] 0 2024-04-24T23:13:44.161 INFO:tasks.workunit.client.0.smithi031.stdout:1/413: getdents d1/da/d12 0 2024-04-24T23:13:44.162 INFO:tasks.workunit.client.1.smithi040.stdout:4/61: creat da/f1a x:0 0 0 2024-04-24T23:13:44.169 INFO:tasks.workunit.client.0.smithi031.stdout:7/422: rmdir d1/d2/d58 0 2024-04-24T23:13:44.176 INFO:tasks.workunit.client.0.smithi031.stdout:7/423: dwrite d1/d6a/f7c [0,4194304] 0 2024-04-24T23:13:44.177 INFO:tasks.workunit.client.0.smithi031.stdout:0/491: creat da/d11/fa0 x:0 0 0 2024-04-24T23:13:44.179 INFO:tasks.workunit.client.1.smithi040.stdout:3/93: dwrite d1/fd [0,4194304] 0 2024-04-24T23:13:44.180 INFO:tasks.workunit.client.0.smithi031.stdout:5/406: link d0/d6/d3b/d4b/dc/f24 d0/d6/d3b/d4b/dc/f84 0 2024-04-24T23:13:44.180 INFO:tasks.workunit.client.1.smithi040.stdout:2/140: creat da/d10/f2b x:0 0 0 2024-04-24T23:13:44.182 INFO:tasks.workunit.client.1.smithi040.stdout:0/146: creat df/d14/d1b/f28 x:0 0 0 2024-04-24T23:13:44.182 INFO:tasks.workunit.client.0.smithi031.stdout:6/588: creat d13/d15/fcf x:0 0 0 2024-04-24T23:13:44.184 INFO:tasks.workunit.client.0.smithi031.stdout:3/405: mknod d6/d14/d1d/d69/c81 0 2024-04-24T23:13:44.184 INFO:tasks.workunit.client.0.smithi031.stdout:6/589: read d13/d1b/f46 [2736565,56090] 0 2024-04-24T23:13:44.185 INFO:tasks.workunit.client.1.smithi040.stdout:7/154: rmdir db/d2f 39 2024-04-24T23:13:44.185 INFO:tasks.workunit.client.0.smithi031.stdout:8/542: getdents d8/d15/d37/d43/daf 0 2024-04-24T23:13:44.187 INFO:tasks.workunit.client.1.smithi040.stdout:2/141: dread da/d1c/f27 [0,4194304] 0 2024-04-24T23:13:44.192 INFO:tasks.workunit.client.1.smithi040.stdout:9/142: link d7/ff d7/d1b/d2f/d1f/d31/f36 0 2024-04-24T23:13:44.193 INFO:tasks.workunit.client.0.smithi031.stdout:7/424: symlink d1/d45/d84/l8f 0 2024-04-24T23:13:44.194 INFO:tasks.workunit.client.1.smithi040.stdout:1/70: link da/l13 da/d12/d18/l1e 0 2024-04-24T23:13:44.195 INFO:tasks.workunit.client.0.smithi031.stdout:9/552: truncate d2/d1b/d14/d78/d84/d4a/d4b/f57 413360 0 2024-04-24T23:13:44.197 INFO:tasks.workunit.client.1.smithi040.stdout:9/143: dwrite d7/f15 [0,4194304] 0 2024-04-24T23:13:44.197 INFO:tasks.workunit.client.1.smithi040.stdout:4/62: mknod da/c1b 0 2024-04-24T23:13:44.202 INFO:tasks.workunit.client.1.smithi040.stdout:6/95: mkdir d7/d1a 0 2024-04-24T23:13:44.202 INFO:tasks.workunit.client.1.smithi040.stdout:6/96: truncate f2 4499331 0 2024-04-24T23:13:44.203 INFO:tasks.workunit.client.0.smithi031.stdout:6/590: fsync d13/d1b/f8f 0 2024-04-24T23:13:44.206 INFO:tasks.workunit.client.0.smithi031.stdout:8/543: rename d8/c83 to d8/d15/d9a/cc5 0 2024-04-24T23:13:44.210 INFO:tasks.workunit.client.0.smithi031.stdout:9/553: symlink d2/d1b/d14/d78/d84/d4a/la2 0 2024-04-24T23:13:44.214 INFO:tasks.workunit.client.0.smithi031.stdout:9/554: dread d2/fc [0,4194304] 0 2024-04-24T23:13:44.215 INFO:tasks.workunit.client.0.smithi031.stdout:5/407: symlink d0/d6/d3b/d4b/d1d/d2e/l85 0 2024-04-24T23:13:44.217 INFO:tasks.workunit.client.0.smithi031.stdout:1/414: creat d1/f85 x:0 0 0 2024-04-24T23:13:44.218 INFO:tasks.workunit.client.0.smithi031.stdout:9/555: dread d2/d1b/d14/f46 [0,4194304] 0 2024-04-24T23:13:44.219 INFO:tasks.workunit.client.1.smithi040.stdout:7/155: mknod db/d12/d20/c3a 0 2024-04-24T23:13:44.220 INFO:tasks.workunit.client.0.smithi031.stdout:6/591: fdatasync d13/d15/d1d/d23/d44/da0/da6/fb6 0 2024-04-24T23:13:44.223 INFO:tasks.workunit.client.0.smithi031.stdout:1/415: dread d1/d33/f48 [0,4194304] 0 2024-04-24T23:13:44.224 INFO:tasks.workunit.client.1.smithi040.stdout:1/71: unlink da/d12/d18/l1e 0 2024-04-24T23:13:44.225 INFO:tasks.workunit.client.1.smithi040.stdout:1/72: chown da/cb 176 1 2024-04-24T23:13:44.225 INFO:tasks.workunit.client.0.smithi031.stdout:4/556: getdents d3/d23/d3b 0 2024-04-24T23:13:44.226 INFO:tasks.workunit.client.1.smithi040.stdout:7/156: dwrite db/f2b [0,4194304] 0 2024-04-24T23:13:44.226 INFO:tasks.workunit.client.0.smithi031.stdout:4/557: readlink d3/d20/d2d/l55 0 2024-04-24T23:13:44.231 INFO:tasks.workunit.client.1.smithi040.stdout:5/105: rmdir d2/d2f/d21 0 2024-04-24T23:13:44.238 INFO:tasks.workunit.client.0.smithi031.stdout:8/544: dread d8/d1a/d1d/d35/d58/d5b/f91 [0,4194304] 0 2024-04-24T23:13:44.239 INFO:tasks.workunit.client.1.smithi040.stdout:0/147: creat df/d12/d19/f29 x:0 0 0 2024-04-24T23:13:44.239 INFO:tasks.workunit.client.1.smithi040.stdout:0/148: chown df 12152035 1 2024-04-24T23:13:44.240 INFO:tasks.workunit.client.1.smithi040.stdout:0/149: chown fb 0 1 2024-04-24T23:13:44.243 INFO:tasks.workunit.client.0.smithi031.stdout:3/406: creat d6/df/f82 x:0 0 0 2024-04-24T23:13:44.245 INFO:tasks.workunit.client.1.smithi040.stdout:8/66: link d0/l3 d0/l12 0 2024-04-24T23:13:44.245 INFO:tasks.workunit.client.0.smithi031.stdout:3/407: write d6/df/d16/d53/f67 [3084902,18380] 0 2024-04-24T23:13:44.246 INFO:tasks.workunit.client.0.smithi031.stdout:5/408: write d0/d6/f1a [727459,72879] 0 2024-04-24T23:13:44.247 INFO:tasks.workunit.client.1.smithi040.stdout:3/94: dwrite d1/f8 [0,4194304] 0 2024-04-24T23:13:44.249 INFO:tasks.workunit.client.1.smithi040.stdout:1/73: rmdir da/d12 39 2024-04-24T23:13:44.250 INFO:tasks.workunit.client.1.smithi040.stdout:2/142: truncate da/f20 216347 0 2024-04-24T23:13:44.251 INFO:tasks.workunit.client.0.smithi031.stdout:6/592: mkdir d13/d1b/dd0 0 2024-04-24T23:13:44.252 INFO:tasks.workunit.client.0.smithi031.stdout:6/593: write d13/f6c [4630605,50479] 0 2024-04-24T23:13:44.253 INFO:tasks.workunit.client.1.smithi040.stdout:7/157: rename ca to db/d12/c3b 0 2024-04-24T23:13:44.255 INFO:tasks.workunit.client.1.smithi040.stdout:7/158: truncate db/d12/f29 848318 0 2024-04-24T23:13:44.255 INFO:tasks.workunit.client.1.smithi040.stdout:9/144: truncate d7/fe 190903 0 2024-04-24T23:13:44.255 INFO:tasks.workunit.client.0.smithi031.stdout:1/416: fdatasync d1/da/d12/d7e/d7b/f6a 0 2024-04-24T23:13:44.258 INFO:tasks.workunit.client.0.smithi031.stdout:1/417: dread d1/da/d12/f82 [0,4194304] 0 2024-04-24T23:13:44.262 INFO:tasks.workunit.client.1.smithi040.stdout:5/106: fdatasync d2/d7/fe 0 2024-04-24T23:13:44.263 INFO:tasks.workunit.client.0.smithi031.stdout:0/492: getdents da/d11 0 2024-04-24T23:13:44.267 INFO:tasks.workunit.client.1.smithi040.stdout:4/63: truncate f6 1055055 0 2024-04-24T23:13:44.267 INFO:tasks.workunit.client.1.smithi040.stdout:4/64: stat l0 0 2024-04-24T23:13:44.271 INFO:tasks.workunit.client.1.smithi040.stdout:4/65: dread f4 [0,4194304] 0 2024-04-24T23:13:44.271 INFO:tasks.workunit.client.1.smithi040.stdout:4/66: dread - da/f19 zero size 2024-04-24T23:13:44.274 INFO:tasks.workunit.client.0.smithi031.stdout:4/558: dread d3/d20/d25/d60/d63/f81 [0,4194304] 0 2024-04-24T23:13:44.278 INFO:tasks.workunit.client.0.smithi031.stdout:4/559: dread d3/d23/d69/f13 [0,4194304] 0 2024-04-24T23:13:44.278 INFO:tasks.workunit.client.0.smithi031.stdout:9/556: mknod d2/d1b/d14/ca3 0 2024-04-24T23:13:44.279 INFO:tasks.workunit.client.1.smithi040.stdout:2/143: creat da/d10/d15/f2c x:0 0 0 2024-04-24T23:13:44.280 INFO:tasks.workunit.client.1.smithi040.stdout:2/144: dread - da/d10/d15/f2c zero size 2024-04-24T23:13:44.280 INFO:tasks.workunit.client.0.smithi031.stdout:3/408: mkdir d6/d72/d1a/d25/d83 0 2024-04-24T23:13:44.281 INFO:tasks.workunit.client.0.smithi031.stdout:8/545: write d8/d1a/d73/f80 [355426,37819] 0 2024-04-24T23:13:44.281 INFO:tasks.workunit.client.1.smithi040.stdout:2/145: chown da/d10/l1f 39 1 2024-04-24T23:13:44.281 INFO:tasks.workunit.client.0.smithi031.stdout:3/409: dread - d6/df/f82 zero size 2024-04-24T23:13:44.281 INFO:tasks.workunit.client.0.smithi031.stdout:8/546: readlink d8/d1a/d74/l93 0 2024-04-24T23:13:44.282 INFO:tasks.workunit.client.1.smithi040.stdout:7/159: write db/d2f/f30 [851041,90993] 0 2024-04-24T23:13:44.284 INFO:tasks.workunit.client.1.smithi040.stdout:9/145: unlink d7/d1b/d2f/f30 0 2024-04-24T23:13:44.285 INFO:tasks.workunit.client.0.smithi031.stdout:3/410: dread d6/d72/d1a/f22 [4194304,4194304] 0 2024-04-24T23:13:44.287 INFO:tasks.workunit.client.1.smithi040.stdout:5/107: symlink d2/d7/d8/db/d1e/l30 0 2024-04-24T23:13:44.290 INFO:tasks.workunit.client.0.smithi031.stdout:7/425: rename d1/d2/dd/d33/l73 to d1/d2/l90 0 2024-04-24T23:13:44.293 INFO:tasks.workunit.client.0.smithi031.stdout:1/418: mknod d1/dc/d11/d21/c86 0 2024-04-24T23:13:44.294 INFO:tasks.workunit.client.1.smithi040.stdout:5/108: dwrite d2/d7/d8/db/d1e/f2e [0,4194304] 0 2024-04-24T23:13:44.294 INFO:tasks.workunit.client.0.smithi031.stdout:1/419: stat d1/dc/l43 0 2024-04-24T23:13:44.295 INFO:tasks.workunit.client.1.smithi040.stdout:8/67: unlink d0/l4 0 2024-04-24T23:13:44.297 INFO:tasks.workunit.client.1.smithi040.stdout:0/150: truncate fb 1987482 0 2024-04-24T23:13:44.300 INFO:tasks.workunit.client.1.smithi040.stdout:3/95: creat d1/d9/d15/f1b x:0 0 0 2024-04-24T23:13:44.301 INFO:tasks.workunit.client.0.smithi031.stdout:1/420: dwrite d1/da/d61/d80/f84 [0,4194304] 0 2024-04-24T23:13:44.303 INFO:tasks.workunit.client.1.smithi040.stdout:0/151: dwrite df/d12/f1f [8388608,4194304] 0 2024-04-24T23:13:44.304 INFO:tasks.workunit.client.1.smithi040.stdout:0/152: dread - df/d12/f25 zero size 2024-04-24T23:13:44.305 INFO:tasks.workunit.client.1.smithi040.stdout:0/153: chown df/d12/f1f 377380 1 2024-04-24T23:13:44.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/409: mkdir d0/d6/d3b/d4b/dc/d86 0 2024-04-24T23:13:44.306 INFO:tasks.workunit.client.1.smithi040.stdout:0/154: dread - df/d12/f22 zero size 2024-04-24T23:13:44.307 INFO:tasks.workunit.client.1.smithi040.stdout:2/146: mkdir da/d10/d15/d2d 0 2024-04-24T23:13:44.312 INFO:tasks.workunit.client.0.smithi031.stdout:0/493: rename da/d18/d1a/d37/f3c to da/d18/fa1 0 2024-04-24T23:13:44.314 INFO:tasks.workunit.client.0.smithi031.stdout:7/426: creat d1/d2/dd/d33/d5e/d64/f91 x:0 0 0 2024-04-24T23:13:44.319 INFO:tasks.workunit.client.0.smithi031.stdout:1/421: chown d1/da/d12/d7e/d7b/d4e/l5f 23 1 2024-04-24T23:13:44.319 INFO:tasks.workunit.client.1.smithi040.stdout:3/96: mkdir d1/d9/d15/d1c 0 2024-04-24T23:13:44.321 INFO:tasks.workunit.client.0.smithi031.stdout:4/560: link d3/d23/d3b/f43 d3/d23/d3b/fb4 0 2024-04-24T23:13:44.322 INFO:tasks.workunit.client.1.smithi040.stdout:1/74: creat da/f1f x:0 0 0 2024-04-24T23:13:44.324 INFO:tasks.workunit.client.1.smithi040.stdout:6/97: sync 2024-04-24T23:13:44.324 INFO:tasks.workunit.client.0.smithi031.stdout:5/410: rmdir d0/d7/d56/d7c 39 2024-04-24T23:13:44.324 INFO:tasks.workunit.client.1.smithi040.stdout:6/98: chown d7/fd 20140689 1 2024-04-24T23:13:44.325 INFO:tasks.workunit.client.0.smithi031.stdout:1/422: dwrite d1/da/d10/f51 [0,4194304] 0 2024-04-24T23:13:44.332 INFO:tasks.workunit.client.1.smithi040.stdout:2/147: creat da/d10/d15/f2e x:0 0 0 2024-04-24T23:13:44.333 INFO:tasks.workunit.client.1.smithi040.stdout:2/148: truncate da/d10/d15/f2e 814851 0 2024-04-24T23:13:44.333 INFO:tasks.workunit.client.1.smithi040.stdout:2/149: dread - da/d10/f18 zero size 2024-04-24T23:13:44.334 INFO:tasks.workunit.client.0.smithi031.stdout:8/547: link d8/d1a/d73/f80 d8/d1a/d1d/d50/dc0/d9d/fc6 0 2024-04-24T23:13:44.335 INFO:tasks.workunit.client.0.smithi031.stdout:8/548: fdatasync d8/d15/f3d 0 2024-04-24T23:13:44.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/411: creat d6/d72/d60/f84 x:0 0 0 2024-04-24T23:13:44.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/412: read - d6/f6b zero size 2024-04-24T23:13:44.339 INFO:tasks.workunit.client.0.smithi031.stdout:6/594: creat d13/d15/d1d/d23/d24/d43/d68/db8/fd1 x:0 0 0 2024-04-24T23:13:44.340 INFO:tasks.workunit.client.1.smithi040.stdout:5/109: mkdir d2/d7/d8/d1f/d31 0 2024-04-24T23:13:44.341 INFO:tasks.workunit.client.1.smithi040.stdout:5/110: chown d2/d7/d8/d1f 629244 1 2024-04-24T23:13:44.341 INFO:tasks.workunit.client.0.smithi031.stdout:0/494: mkdir da/d11/d6a/da2 0 2024-04-24T23:13:44.343 INFO:tasks.workunit.client.0.smithi031.stdout:7/427: write d1/d6a/f35 [3910458,124240] 0 2024-04-24T23:13:44.346 INFO:tasks.workunit.client.0.smithi031.stdout:9/557: dread d2/d1b/d14/d78/d84/d4a/d4b/f57 [0,4194304] 0 2024-04-24T23:13:44.346 INFO:tasks.workunit.client.1.smithi040.stdout:3/97: creat d1/d9/d15/f1d x:0 0 0 2024-04-24T23:13:44.346 INFO:tasks.workunit.client.1.smithi040.stdout:3/98: write d1/fd [1863125,50431] 0 2024-04-24T23:13:44.348 INFO:tasks.workunit.client.0.smithi031.stdout:4/561: truncate d3/d23/d69/d3f/f6d 1386787 0 2024-04-24T23:13:44.350 INFO:tasks.workunit.client.0.smithi031.stdout:5/411: creat d0/d6/d3b/f87 x:0 0 0 2024-04-24T23:13:44.352 INFO:tasks.workunit.client.1.smithi040.stdout:0/155: link df/d12/f25 df/d12/f2a 0 2024-04-24T23:13:44.352 INFO:tasks.workunit.client.0.smithi031.stdout:1/423: mknod d1/da/d12/c87 0 2024-04-24T23:13:44.353 INFO:tasks.workunit.client.1.smithi040.stdout:1/75: dwrite da/d12/d18/f1d [0,4194304] 0 2024-04-24T23:13:44.353 INFO:tasks.workunit.client.1.smithi040.stdout:4/67: sync 2024-04-24T23:13:44.354 INFO:tasks.workunit.client.1.smithi040.stdout:4/68: fdatasync da/fb 0 2024-04-24T23:13:44.354 INFO:tasks.workunit.client.1.smithi040.stdout:4/69: fdatasync da/f13 0 2024-04-24T23:13:44.359 INFO:tasks.workunit.client.1.smithi040.stdout:2/150: dread f7 [0,4194304] 0 2024-04-24T23:13:44.360 INFO:tasks.workunit.client.0.smithi031.stdout:3/413: symlink d6/d72/d29/l85 0 2024-04-24T23:13:44.362 INFO:tasks.workunit.client.1.smithi040.stdout:7/160: rename db/d1b/f39 to db/f3c 0 2024-04-24T23:13:44.365 INFO:tasks.workunit.client.0.smithi031.stdout:3/414: dwrite d6/df/d16/d53/f6e [0,4194304] 0 2024-04-24T23:13:44.366 INFO:tasks.workunit.client.1.smithi040.stdout:8/68: truncate d0/d8/dc/df/f10 1827699 0 2024-04-24T23:13:44.367 INFO:tasks.workunit.client.0.smithi031.stdout:0/495: mkdir da/d86/da3 0 2024-04-24T23:13:44.368 INFO:tasks.workunit.client.1.smithi040.stdout:5/111: stat d2/d7/d8/c12 0 2024-04-24T23:13:44.374 INFO:tasks.workunit.client.1.smithi040.stdout:6/99: mknod d7/c1b 0 2024-04-24T23:13:44.390 INFO:tasks.workunit.client.1.smithi040.stdout:2/151: rmdir da/d1c 39 2024-04-24T23:13:44.392 INFO:tasks.workunit.client.1.smithi040.stdout:2/152: fsync da/d10/d15/f2e 0 2024-04-24T23:13:44.392 INFO:tasks.workunit.client.1.smithi040.stdout:0/156: rename df/d12/d19 to df/d1e/d2b 0 2024-04-24T23:13:44.394 INFO:tasks.workunit.client.1.smithi040.stdout:7/161: write db/d1b/f28 [1080707,130383] 0 2024-04-24T23:13:44.394 INFO:tasks.workunit.client.1.smithi040.stdout:8/69: symlink d0/l13 0 2024-04-24T23:13:44.394 INFO:tasks.workunit.client.1.smithi040.stdout:9/146: getdents d7/da 0 2024-04-24T23:13:44.394 INFO:tasks.workunit.client.1.smithi040.stdout:0/157: fdatasync df/d14/f16 0 2024-04-24T23:13:44.395 INFO:tasks.workunit.client.1.smithi040.stdout:3/99: symlink d1/d9/l1e 0 2024-04-24T23:13:44.396 INFO:tasks.workunit.client.1.smithi040.stdout:6/100: fsync d7/fd 0 2024-04-24T23:13:44.397 INFO:tasks.workunit.client.1.smithi040.stdout:2/153: write da/d10/d15/f2e [800594,14953] 0 2024-04-24T23:13:44.398 INFO:tasks.workunit.client.1.smithi040.stdout:3/100: write d1/fd [4205283,31011] 0 2024-04-24T23:13:44.400 INFO:tasks.workunit.client.1.smithi040.stdout:4/70: dwrite f4 [4194304,4194304] 0 2024-04-24T23:13:44.400 INFO:tasks.workunit.client.1.smithi040.stdout:2/154: write da/d10/d15/f2c [220275,62272] 0 2024-04-24T23:13:44.402 INFO:tasks.workunit.client.1.smithi040.stdout:2/155: write da/f11 [3843958,44372] 0 2024-04-24T23:13:44.459 INFO:tasks.workunit.client.1.smithi040.stdout:1/76: mknod da/c20 0 2024-04-24T23:13:44.459 INFO:tasks.workunit.client.1.smithi040.stdout:1/77: fsync f8 0 2024-04-24T23:13:44.460 INFO:tasks.workunit.client.1.smithi040.stdout:1/78: chown da/cb 105187036 1 2024-04-24T23:13:44.460 INFO:tasks.workunit.client.1.smithi040.stdout:5/112: creat d2/d7/d8/db/d1e/d22/d27/f32 x:0 0 0 2024-04-24T23:13:44.462 INFO:tasks.workunit.client.1.smithi040.stdout:0/158: rmdir df/d12 39 2024-04-24T23:13:44.465 INFO:tasks.workunit.client.1.smithi040.stdout:3/101: creat d1/d9/d18/f1f x:0 0 0 2024-04-24T23:13:44.470 INFO:tasks.workunit.client.1.smithi040.stdout:1/79: write da/ff [4850898,7388] 0 2024-04-24T23:13:44.471 INFO:tasks.workunit.client.1.smithi040.stdout:1/80: fdatasync f7 0 2024-04-24T23:13:44.472 INFO:tasks.workunit.client.1.smithi040.stdout:0/159: creat df/d1e/f2c x:0 0 0 2024-04-24T23:13:44.473 INFO:tasks.workunit.client.1.smithi040.stdout:4/71: link da/f1a da/f1c 0 2024-04-24T23:13:44.474 INFO:tasks.workunit.client.1.smithi040.stdout:7/162: dwrite db/f24 [0,4194304] 0 2024-04-24T23:13:44.475 INFO:tasks.workunit.client.1.smithi040.stdout:2/156: mknod da/d1c/c2f 0 2024-04-24T23:13:44.476 INFO:tasks.workunit.client.1.smithi040.stdout:9/147: creat d7/f37 x:0 0 0 2024-04-24T23:13:44.478 INFO:tasks.workunit.client.1.smithi040.stdout:3/102: dwrite d1/f8 [0,4194304] 0 2024-04-24T23:13:44.479 INFO:tasks.workunit.client.1.smithi040.stdout:2/157: write da/f19 [267919,114053] 0 2024-04-24T23:13:44.479 INFO:tasks.workunit.client.1.smithi040.stdout:1/81: creat da/d12/f21 x:0 0 0 2024-04-24T23:13:44.483 INFO:tasks.workunit.client.1.smithi040.stdout:9/148: mknod d7/d1b/d2f/d1f/d31/c38 0 2024-04-24T23:13:44.484 INFO:tasks.workunit.client.0.smithi031.stdout:0/496: chown da/d20/f44 9140076 1 2024-04-24T23:13:44.486 INFO:tasks.workunit.client.0.smithi031.stdout:8/549: creat d8/d15/d37/d43/dbd/dc4/fc7 x:0 0 0 2024-04-24T23:13:44.486 INFO:tasks.workunit.client.1.smithi040.stdout:0/160: dwrite f2 [8388608,4194304] 0 2024-04-24T23:13:44.487 INFO:tasks.workunit.client.0.smithi031.stdout:1/424: write d1/da/d12/d7e/d7b/f6a [7855240,63032] 0 2024-04-24T23:13:44.488 INFO:tasks.workunit.client.0.smithi031.stdout:7/428: symlink d1/d45/d75/d81/l92 0 2024-04-24T23:13:44.489 INFO:tasks.workunit.client.1.smithi040.stdout:9/149: stat d7/da/d17/l2d 0 2024-04-24T23:13:44.492 INFO:tasks.workunit.client.1.smithi040.stdout:1/82: readlink da/d12/d14/l15 0 2024-04-24T23:13:44.494 INFO:tasks.workunit.client.0.smithi031.stdout:3/415: dwrite d6/df/d16/d2a/f31 [0,4194304] 0 2024-04-24T23:13:44.494 INFO:tasks.workunit.client.1.smithi040.stdout:4/72: dwrite da/f18 [0,4194304] 0 2024-04-24T23:13:44.497 INFO:tasks.workunit.client.0.smithi031.stdout:8/550: dread d8/d1a/d1d/d50/fb7 [0,4194304] 0 2024-04-24T23:13:44.498 INFO:tasks.workunit.client.0.smithi031.stdout:7/429: dread d1/d2/dd/d33/f77 [0,4194304] 0 2024-04-24T23:13:44.500 INFO:tasks.workunit.client.0.smithi031.stdout:7/430: write d1/d2/dd/d33/f62 [1037386,113480] 0 2024-04-24T23:13:44.502 INFO:tasks.workunit.client.1.smithi040.stdout:0/161: dread - df/d12/f2a zero size 2024-04-24T23:13:44.508 INFO:tasks.workunit.client.1.smithi040.stdout:1/83: creat da/d12/d18/f22 x:0 0 0 2024-04-24T23:13:44.512 INFO:tasks.workunit.client.1.smithi040.stdout:2/158: rmdir da/d10/d15/d2d 0 2024-04-24T23:13:44.513 INFO:tasks.workunit.client.0.smithi031.stdout:1/425: rename d1/d1d/l73 to d1/dc/l88 0 2024-04-24T23:13:44.518 INFO:tasks.workunit.client.0.smithi031.stdout:5/412: creat d0/f88 x:0 0 0 2024-04-24T23:13:44.519 INFO:tasks.workunit.client.1.smithi040.stdout:0/162: rmdir df/d12 39 2024-04-24T23:13:44.520 INFO:tasks.workunit.client.0.smithi031.stdout:8/551: mkdir d8/d15/d37/d43/d7b/dc8 0 2024-04-24T23:13:44.521 INFO:tasks.workunit.client.0.smithi031.stdout:6/595: link d13/f87 d13/d15/d1d/d23/d2d/fd2 0 2024-04-24T23:13:44.522 INFO:tasks.workunit.client.0.smithi031.stdout:6/596: readlink d13/d1b/d32/db4/l4f 0 2024-04-24T23:13:44.525 INFO:tasks.workunit.client.1.smithi040.stdout:4/73: truncate f6 37604 0 2024-04-24T23:13:44.525 INFO:tasks.workunit.client.1.smithi040.stdout:0/163: dwrite df/d1e/d2b/f29 [0,4194304] 0 2024-04-24T23:13:44.525 INFO:tasks.workunit.client.1.smithi040.stdout:4/74: readlink da/l10 0 2024-04-24T23:13:44.531 INFO:tasks.workunit.client.1.smithi040.stdout:4/75: dread f5 [0,4194304] 0 2024-04-24T23:13:44.532 INFO:tasks.workunit.client.1.smithi040.stdout:4/76: write da/fb [4029324,97591] 0 2024-04-24T23:13:44.536 INFO:tasks.workunit.client.0.smithi031.stdout:3/416: rename d6/df/d16/c3b to d6/d72/d1a/d25/c86 0 2024-04-24T23:13:44.536 INFO:tasks.workunit.client.0.smithi031.stdout:3/417: stat d6/df/d16/d2a/c66 0 2024-04-24T23:13:44.537 INFO:tasks.workunit.client.1.smithi040.stdout:0/164: symlink df/d1e/l2d 0 2024-04-24T23:13:44.538 INFO:tasks.workunit.client.1.smithi040.stdout:0/165: write df/d14/d1b/f28 [382287,57524] 0 2024-04-24T23:13:44.540 INFO:tasks.workunit.client.0.smithi031.stdout:9/558: getdents d2/d1b/d85 0 2024-04-24T23:13:44.543 INFO:tasks.workunit.client.0.smithi031.stdout:9/559: write d2/f92 [512048,41048] 0 2024-04-24T23:13:44.546 INFO:tasks.workunit.client.0.smithi031.stdout:3/418: dwrite d6/df/d16/d53/f6e [0,4194304] 0 2024-04-24T23:13:44.546 INFO:tasks.workunit.client.0.smithi031.stdout:1/426: dwrite d1/dc/d11/d21/f28 [0,4194304] 0 2024-04-24T23:13:44.549 INFO:tasks.workunit.client.0.smithi031.stdout:7/431: mkdir d1/d6a/d93 0 2024-04-24T23:13:44.550 INFO:tasks.workunit.client.0.smithi031.stdout:7/432: chown d1/d2/f2a 709 1 2024-04-24T23:13:44.554 INFO:tasks.workunit.client.0.smithi031.stdout:0/497: link da/d11/c49 da/d18/ca4 0 2024-04-24T23:13:44.555 INFO:tasks.workunit.client.0.smithi031.stdout:0/498: chown f3 0 1 2024-04-24T23:13:44.558 INFO:tasks.workunit.client.1.smithi040.stdout:0/166: fdatasync df/d12/f2a 0 2024-04-24T23:13:44.562 INFO:tasks.workunit.client.1.smithi040.stdout:0/167: rmdir df/d1e/d20 39 2024-04-24T23:13:44.565 INFO:tasks.workunit.client.1.smithi040.stdout:0/168: dread f6 [0,4194304] 0 2024-04-24T23:13:44.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/419: creat d6/d72/d1a/d25/f87 x:0 0 0 2024-04-24T23:13:44.566 INFO:tasks.workunit.client.1.smithi040.stdout:0/169: chown df/d14/d1b/f28 66 1 2024-04-24T23:13:44.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/433: creat d1/d6a/d66/f94 x:0 0 0 2024-04-24T23:13:44.574 INFO:tasks.workunit.client.0.smithi031.stdout:5/413: creat d0/f89 x:0 0 0 2024-04-24T23:13:44.575 INFO:tasks.workunit.client.0.smithi031.stdout:5/414: write d0/d6/d12/f6f [790756,20973] 0 2024-04-24T23:13:44.577 INFO:tasks.workunit.client.0.smithi031.stdout:0/499: dread da/d18/d1a/d37/f3d [4194304,4194304] 0 2024-04-24T23:13:44.578 INFO:tasks.workunit.client.0.smithi031.stdout:9/560: symlink d2/d1b/la4 0 2024-04-24T23:13:44.580 INFO:tasks.workunit.client.0.smithi031.stdout:3/420: fsync d6/d14/f7d 0 2024-04-24T23:13:44.580 INFO:tasks.workunit.client.0.smithi031.stdout:2/339: sync 2024-04-24T23:13:44.581 INFO:tasks.workunit.client.0.smithi031.stdout:7/434: creat d1/d2/dd/f95 x:0 0 0 2024-04-24T23:13:44.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/597: link d13/d15/d1d/d23/d24/d8b/l92 d13/d15/d1d/d23/d2d/d9e/ld3 0 2024-04-24T23:13:44.583 INFO:tasks.workunit.client.0.smithi031.stdout:2/340: write d5/d8/d45/d5b/d49/f6d [425346,89899] 0 2024-04-24T23:13:44.584 INFO:tasks.workunit.client.0.smithi031.stdout:8/552: getdents d8/d15/d37/d43/d7b/d87 0 2024-04-24T23:13:44.586 INFO:tasks.workunit.client.0.smithi031.stdout:0/500: creat da/d18/d1a/d37/fa5 x:0 0 0 2024-04-24T23:13:44.590 INFO:tasks.workunit.client.0.smithi031.stdout:7/435: creat d1/d45/d84/f96 x:0 0 0 2024-04-24T23:13:44.590 INFO:tasks.workunit.client.0.smithi031.stdout:7/436: chown d1/d6a/d66 242402543 1 2024-04-24T23:13:44.596 INFO:tasks.workunit.client.0.smithi031.stdout:0/501: symlink da/d22/la6 0 2024-04-24T23:13:44.599 INFO:tasks.workunit.client.0.smithi031.stdout:8/553: dread d8/d1a/d1d/d35/f8f [0,4194304] 0 2024-04-24T23:13:44.600 INFO:tasks.workunit.client.0.smithi031.stdout:7/437: creat d1/d2/dd/f97 x:0 0 0 2024-04-24T23:13:44.602 INFO:tasks.workunit.client.0.smithi031.stdout:5/415: rename d0/d6/d3b/d4f/c6d to d0/d6/d3b/d4b/d1d/c8a 0 2024-04-24T23:13:44.604 INFO:tasks.workunit.client.0.smithi031.stdout:9/561: creat d2/d1b/d14/d78/d84/d4a/fa5 x:0 0 0 2024-04-24T23:13:44.607 INFO:tasks.workunit.client.0.smithi031.stdout:7/438: creat d1/d6a/d65/f98 x:0 0 0 2024-04-24T23:13:44.611 INFO:tasks.workunit.client.0.smithi031.stdout:9/562: unlink d2/d1b/d14/d78/d84/d4a/d4b/l83 0 2024-04-24T23:13:44.612 INFO:tasks.workunit.client.0.smithi031.stdout:9/563: fsync d2/d1b/d14/d18/d7d/fa1 0 2024-04-24T23:13:44.614 INFO:tasks.workunit.client.0.smithi031.stdout:8/554: mknod d8/d15/d37/d43/d7b/d87/dad/cc9 0 2024-04-24T23:13:44.615 INFO:tasks.workunit.client.0.smithi031.stdout:7/439: creat d1/d2/dd/d33/d5e/d64/f99 x:0 0 0 2024-04-24T23:13:44.615 INFO:tasks.workunit.client.0.smithi031.stdout:8/555: truncate d8/d15/d37/d62/f7f 856443 0 2024-04-24T23:13:44.616 INFO:tasks.workunit.client.0.smithi031.stdout:5/416: mkdir d0/d6/d3b/d4b/dc/d8b 0 2024-04-24T23:13:44.617 INFO:tasks.workunit.client.0.smithi031.stdout:8/556: write d8/d15/d37/f85 [2854874,58528] 0 2024-04-24T23:13:44.618 INFO:tasks.workunit.client.1.smithi040.stdout:1/84: getdents da 0 2024-04-24T23:13:44.620 INFO:tasks.workunit.client.0.smithi031.stdout:0/502: link da/d11/c49 da/d18/ca7 0 2024-04-24T23:13:44.622 INFO:tasks.workunit.client.1.smithi040.stdout:1/85: unlink c5 0 2024-04-24T23:13:44.623 INFO:tasks.workunit.client.0.smithi031.stdout:8/557: dread d8/d15/d37/d43/f4b [0,4194304] 0 2024-04-24T23:13:44.624 INFO:tasks.workunit.client.0.smithi031.stdout:7/440: symlink d1/d2/dd/d33/d5e/l9a 0 2024-04-24T23:13:44.625 INFO:tasks.workunit.client.0.smithi031.stdout:8/558: write d8/fb [4589031,74174] 0 2024-04-24T23:13:44.626 INFO:tasks.workunit.client.1.smithi040.stdout:1/86: unlink c4 0 2024-04-24T23:13:44.627 INFO:tasks.workunit.client.1.smithi040.stdout:1/87: chown da/l19 136508 1 2024-04-24T23:13:44.629 INFO:tasks.workunit.client.1.smithi040.stdout:6/101: dwrite d7/f9 [0,4194304] 0 2024-04-24T23:13:44.632 INFO:tasks.workunit.client.1.smithi040.stdout:1/88: mkdir da/d12/d23 0 2024-04-24T23:13:44.633 INFO:tasks.workunit.client.1.smithi040.stdout:1/89: write da/f1f [881867,2128] 0 2024-04-24T23:13:44.635 INFO:tasks.workunit.client.1.smithi040.stdout:8/70: rename d0/d8/dc/df/f10 to d0/f14 0 2024-04-24T23:13:44.635 INFO:tasks.workunit.client.1.smithi040.stdout:4/77: rmdir da 39 2024-04-24T23:13:44.636 INFO:tasks.workunit.client.0.smithi031.stdout:7/441: creat d1/d2/d1e/d4f/f9b x:0 0 0 2024-04-24T23:13:44.639 INFO:tasks.workunit.client.1.smithi040.stdout:1/90: creat da/d12/d14/f24 x:0 0 0 2024-04-24T23:13:44.639 INFO:tasks.workunit.client.1.smithi040.stdout:5/113: truncate d2/d7/d8/db/d1e/f2e 3042849 0 2024-04-24T23:13:44.639 INFO:tasks.workunit.client.1.smithi040.stdout:1/91: write f7 [2897583,10530] 0 2024-04-24T23:13:44.641 INFO:tasks.workunit.client.1.smithi040.stdout:1/92: chown da/l19 93765142 1 2024-04-24T23:13:44.642 INFO:tasks.workunit.client.0.smithi031.stdout:0/503: creat da/d18/d1a/d37/d41/d92/fa8 x:0 0 0 2024-04-24T23:13:44.642 INFO:tasks.workunit.client.1.smithi040.stdout:1/93: write da/d12/f21 [223322,35134] 0 2024-04-24T23:13:44.643 INFO:tasks.workunit.client.1.smithi040.stdout:7/163: truncate db/f1e 769876 0 2024-04-24T23:13:44.644 INFO:tasks.workunit.client.0.smithi031.stdout:7/442: chown d1/l48 208465848 1 2024-04-24T23:13:44.645 INFO:tasks.workunit.client.1.smithi040.stdout:8/71: dread d0/d8/fd [0,4194304] 0 2024-04-24T23:13:44.645 INFO:tasks.workunit.client.1.smithi040.stdout:8/72: readlink d0/l13 0 2024-04-24T23:13:44.652 INFO:tasks.workunit.client.0.smithi031.stdout:7/443: mknod d1/d2/dd/d33/c9c 0 2024-04-24T23:13:44.652 INFO:tasks.workunit.client.1.smithi040.stdout:4/78: dwrite da/fb [0,4194304] 0 2024-04-24T23:13:44.654 INFO:tasks.workunit.client.1.smithi040.stdout:7/164: mknod db/d1b/c3d 0 2024-04-24T23:13:44.654 INFO:tasks.workunit.client.1.smithi040.stdout:9/150: dwrite f2 [0,4194304] 0 2024-04-24T23:13:44.656 INFO:tasks.workunit.client.1.smithi040.stdout:7/165: write db/d12/f16 [2595753,50598] 0 2024-04-24T23:13:44.656 INFO:tasks.workunit.client.0.smithi031.stdout:7/444: creat d1/d2/dd/d33/d5e/d64/f9d x:0 0 0 2024-04-24T23:13:44.657 INFO:tasks.workunit.client.0.smithi031.stdout:7/445: read - d1/d2/dd/f97 zero size 2024-04-24T23:13:44.658 INFO:tasks.workunit.client.0.smithi031.stdout:7/446: chown d1/d2/d1e/f36 1684097 1 2024-04-24T23:13:44.659 INFO:tasks.workunit.client.0.smithi031.stdout:7/447: write d1/d2/dd/f7d [379823,97603] 0 2024-04-24T23:13:44.662 INFO:tasks.workunit.client.1.smithi040.stdout:3/103: rename d1/d9/da/de/f14 to d1/d9/f20 0 2024-04-24T23:13:44.664 INFO:tasks.workunit.client.0.smithi031.stdout:7/448: truncate d1/d2/dd/d33/f77 4371884 0 2024-04-24T23:13:44.665 INFO:tasks.workunit.client.1.smithi040.stdout:2/159: dwrite f7 [0,4194304] 0 2024-04-24T23:13:44.668 INFO:tasks.workunit.client.1.smithi040.stdout:4/79: mkdir da/d1d 0 2024-04-24T23:13:44.670 INFO:tasks.workunit.client.1.smithi040.stdout:9/151: fsync d7/d1b/d2f/d1f/d31/f36 0 2024-04-24T23:13:44.673 INFO:tasks.workunit.client.1.smithi040.stdout:7/166: creat db/d12/f3e x:0 0 0 2024-04-24T23:13:44.674 INFO:tasks.workunit.client.0.smithi031.stdout:1/427: write d1/d33/f48 [804783,63639] 0 2024-04-24T23:13:44.679 INFO:tasks.workunit.client.0.smithi031.stdout:1/428: symlink d1/dc/d11/d21/d5a/l89 0 2024-04-24T23:13:44.680 INFO:tasks.workunit.client.1.smithi040.stdout:2/160: truncate da/d1c/f27 1267054 0 2024-04-24T23:13:44.682 INFO:tasks.workunit.client.1.smithi040.stdout:0/170: write fb [2318501,88775] 0 2024-04-24T23:13:44.683 INFO:tasks.workunit.client.0.smithi031.stdout:1/429: symlink d1/da/d12/d65/l8a 0 2024-04-24T23:13:44.684 INFO:tasks.workunit.client.1.smithi040.stdout:9/152: creat d7/da/f39 x:0 0 0 2024-04-24T23:13:44.686 INFO:tasks.workunit.client.1.smithi040.stdout:7/167: mkdir db/d12/d3f 0 2024-04-24T23:13:44.688 INFO:tasks.workunit.client.0.smithi031.stdout:6/598: write d13/d15/d1d/d23/d2d/d9e/dab/fbc [3435247,23406] 0 2024-04-24T23:13:44.689 INFO:tasks.workunit.client.1.smithi040.stdout:4/80: mkdir da/d1d/d1e 0 2024-04-24T23:13:44.689 INFO:tasks.workunit.client.0.smithi031.stdout:6/599: stat d13/d1b/f8f 0 2024-04-24T23:13:44.691 INFO:tasks.workunit.client.1.smithi040.stdout:2/161: creat da/d10/d15/f30 x:0 0 0 2024-04-24T23:13:44.692 INFO:tasks.workunit.client.1.smithi040.stdout:0/171: rename df/d1e/f2c to df/d12/d1a/f2e 0 2024-04-24T23:13:44.692 INFO:tasks.workunit.client.0.smithi031.stdout:2/341: write d5/f36 [1499817,69476] 0 2024-04-24T23:13:44.692 INFO:tasks.workunit.client.1.smithi040.stdout:7/168: dwrite f2 [0,4194304] 0 2024-04-24T23:13:44.693 INFO:tasks.workunit.client.0.smithi031.stdout:2/342: stat d5/d8/d15/f2b 0 2024-04-24T23:13:44.693 INFO:tasks.workunit.client.1.smithi040.stdout:7/169: dread - db/d1b/f23 zero size 2024-04-24T23:13:44.695 INFO:tasks.workunit.client.1.smithi040.stdout:7/170: write db/d12/f29 [120193,77484] 0 2024-04-24T23:13:44.697 INFO:tasks.workunit.client.1.smithi040.stdout:3/104: creat d1/d9/f21 x:0 0 0 2024-04-24T23:13:44.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/600: link d13/d1b/d32/fc3 d13/d15/d1d/d23/d44/d47/d69/fd4 0 2024-04-24T23:13:44.701 INFO:tasks.workunit.client.1.smithi040.stdout:4/81: mkdir da/d1f 0 2024-04-24T23:13:44.706 INFO:tasks.workunit.client.0.smithi031.stdout:6/601: read d13/d15/d1d/d23/d2d/f4c [3751894,51090] 0 2024-04-24T23:13:44.707 INFO:tasks.workunit.client.1.smithi040.stdout:2/162: creat da/d10/f31 x:0 0 0 2024-04-24T23:13:44.708 INFO:tasks.workunit.client.1.smithi040.stdout:2/163: write da/d10/d15/f30 [548931,44026] 0 2024-04-24T23:13:44.711 INFO:tasks.workunit.client.0.smithi031.stdout:5/417: write d0/d6/d3b/d4b/dc/f47 [579486,2356] 0 2024-04-24T23:13:44.712 INFO:tasks.workunit.client.0.smithi031.stdout:5/418: fsync d0/d6/d3b/d4b/dc/f6e 0 2024-04-24T23:13:44.713 INFO:tasks.workunit.client.0.smithi031.stdout:2/343: rmdir d5/d28/d4e/d64 0 2024-04-24T23:13:44.713 INFO:tasks.workunit.client.0.smithi031.stdout:2/344: readlink d5/d6/d29/l2e 0 2024-04-24T23:13:44.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/564: write d2/f42 [2103448,34413] 0 2024-04-24T23:13:44.714 INFO:tasks.workunit.client.1.smithi040.stdout:2/164: dwrite f7 [0,4194304] 0 2024-04-24T23:13:44.715 INFO:tasks.workunit.client.0.smithi031.stdout:2/345: write d5/f25 [116088,60583] 0 2024-04-24T23:13:44.715 INFO:tasks.workunit.client.0.smithi031.stdout:6/602: write d13/d15/d1d/d23/d4d/d7b/f82 [1155809,114131] 0 2024-04-24T23:13:44.716 INFO:tasks.workunit.client.1.smithi040.stdout:7/171: mknod db/d2f/c40 0 2024-04-24T23:13:44.717 INFO:tasks.workunit.client.1.smithi040.stdout:7/172: read db/d2f/f35 [301060,96262] 0 2024-04-24T23:13:44.718 INFO:tasks.workunit.client.0.smithi031.stdout:2/346: truncate d5/d6/d29/f72 487327 0 2024-04-24T23:13:44.718 INFO:tasks.workunit.client.1.smithi040.stdout:7/173: truncate db/f31 275035 0 2024-04-24T23:13:44.720 INFO:tasks.workunit.client.1.smithi040.stdout:3/105: symlink d1/d9/da/de/l22 0 2024-04-24T23:13:44.720 INFO:tasks.workunit.client.1.smithi040.stdout:8/73: dread d0/f14 [0,4194304] 0 2024-04-24T23:13:44.721 INFO:tasks.workunit.client.1.smithi040.stdout:3/106: write d1/d9/d15/f1b [286251,119633] 0 2024-04-24T23:13:44.723 INFO:tasks.workunit.client.1.smithi040.stdout:4/82: truncate f5 1368008 0 2024-04-24T23:13:44.724 INFO:tasks.workunit.client.1.smithi040.stdout:1/94: getdents da/d12/d14 0 2024-04-24T23:13:44.725 INFO:tasks.workunit.client.1.smithi040.stdout:1/95: fsync da/d12/d14/f24 0 2024-04-24T23:13:44.726 INFO:tasks.workunit.client.1.smithi040.stdout:1/96: read da/d12/f1c [3853320,77245] 0 2024-04-24T23:13:44.727 INFO:tasks.workunit.client.0.smithi031.stdout:9/565: mkdir d2/d1b/d14/d78/d84/da6 0 2024-04-24T23:13:44.727 INFO:tasks.workunit.client.0.smithi031.stdout:9/566: fsync d2/f75 0 2024-04-24T23:13:44.728 INFO:tasks.workunit.client.1.smithi040.stdout:9/153: creat d7/f3a x:0 0 0 2024-04-24T23:13:44.730 INFO:tasks.workunit.client.0.smithi031.stdout:6/603: mkdir d13/d15/d1d/d23/d44/d47/d69/dd5 0 2024-04-24T23:13:44.731 INFO:tasks.workunit.client.1.smithi040.stdout:6/102: truncate f2 3194258 0 2024-04-24T23:13:44.732 INFO:tasks.workunit.client.0.smithi031.stdout:8/559: truncate d8/f70 6970939 0 2024-04-24T23:13:44.733 INFO:tasks.workunit.client.0.smithi031.stdout:0/504: write da/d20/d4e/f51 [1971510,11562] 0 2024-04-24T23:13:44.735 INFO:tasks.workunit.client.1.smithi040.stdout:2/165: creat da/d10/d15/f32 x:0 0 0 2024-04-24T23:13:44.736 INFO:tasks.workunit.client.1.smithi040.stdout:2/166: fsync da/d10/d15/f2c 0 2024-04-24T23:13:44.740 INFO:tasks.workunit.client.1.smithi040.stdout:2/167: dread da/d10/d15/f2e [0,4194304] 0 2024-04-24T23:13:44.741 INFO:tasks.workunit.client.0.smithi031.stdout:0/505: dwrite da/d18/d1a/d37/fa5 [0,4194304] 0 2024-04-24T23:13:44.742 INFO:tasks.workunit.client.0.smithi031.stdout:0/506: truncate da/d18/f9a 316011 0 2024-04-24T23:13:44.743 INFO:tasks.workunit.client.0.smithi031.stdout:2/347: mkdir d5/d8/d6e/d70/d75 0 2024-04-24T23:13:44.745 INFO:tasks.workunit.client.1.smithi040.stdout:8/74: symlink d0/d8/dc/df/l15 0 2024-04-24T23:13:44.746 INFO:tasks.workunit.client.0.smithi031.stdout:8/560: mkdir d8/d15/d37/d43/daf/dca 0 2024-04-24T23:13:44.747 INFO:tasks.workunit.client.0.smithi031.stdout:7/449: truncate d1/d2/dd/d33/f62 382371 0 2024-04-24T23:13:44.748 INFO:tasks.workunit.client.0.smithi031.stdout:8/561: truncate d8/fb4 677119 0 2024-04-24T23:13:44.748 INFO:tasks.workunit.client.1.smithi040.stdout:3/107: rmdir d1 39 2024-04-24T23:13:44.749 INFO:tasks.workunit.client.0.smithi031.stdout:8/562: write d8/fb [1644144,99421] 0 2024-04-24T23:13:44.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/419: creat d0/d7/d56/f8c x:0 0 0 2024-04-24T23:13:44.752 INFO:tasks.workunit.client.1.smithi040.stdout:4/83: creat da/f20 x:0 0 0 2024-04-24T23:13:44.754 INFO:tasks.workunit.client.0.smithi031.stdout:9/567: creat d2/d1b/d14/d78/d84/da6/fa7 x:0 0 0 2024-04-24T23:13:44.756 INFO:tasks.workunit.client.1.smithi040.stdout:1/97: unlink f8 0 2024-04-24T23:13:44.757 INFO:tasks.workunit.client.0.smithi031.stdout:6/604: dread d13/d15/f27 [0,4194304] 0 2024-04-24T23:13:44.757 INFO:tasks.workunit.client.0.smithi031.stdout:6/605: readlink d13/d15/l19 0 2024-04-24T23:13:44.759 INFO:tasks.workunit.client.0.smithi031.stdout:6/606: write d13/d15/d1d/d23/d24/d8b/fcd [16143,3529] 0 2024-04-24T23:13:44.760 INFO:tasks.workunit.client.1.smithi040.stdout:1/98: dread da/d12/d14/f16 [0,4194304] 0 2024-04-24T23:13:44.766 INFO:tasks.workunit.client.1.smithi040.stdout:9/154: dwrite d7/da/f39 [0,4194304] 0 2024-04-24T23:13:44.767 INFO:tasks.workunit.client.1.smithi040.stdout:1/99: dread da/d12/d14/f16 [0,4194304] 0 2024-04-24T23:13:44.769 INFO:tasks.workunit.client.0.smithi031.stdout:7/450: rename d1/d2/dd/d33/d5e/d64/f9d to d1/d45/d84/f9e 0 2024-04-24T23:13:44.772 INFO:tasks.workunit.client.1.smithi040.stdout:7/174: link db/d12/d20/f22 db/d12/d3f/f41 0 2024-04-24T23:13:44.772 INFO:tasks.workunit.client.1.smithi040.stdout:9/155: dwrite d7/f15 [0,4194304] 0 2024-04-24T23:13:44.772 INFO:tasks.workunit.client.1.smithi040.stdout:1/100: dread da/d12/d14/f16 [0,4194304] 0 2024-04-24T23:13:44.773 INFO:tasks.workunit.client.0.smithi031.stdout:8/563: creat d8/d1a/d74/fcb x:0 0 0 2024-04-24T23:13:44.773 INFO:tasks.workunit.client.1.smithi040.stdout:7/175: chown db/d12/d20/f21 3 1 2024-04-24T23:13:44.774 INFO:tasks.workunit.client.0.smithi031.stdout:8/564: stat d8/d1a/d1d/d35/d58/d47/c8e 0 2024-04-24T23:13:44.775 INFO:tasks.workunit.client.1.smithi040.stdout:2/168: mknod da/d1c/c33 0 2024-04-24T23:13:44.778 INFO:tasks.workunit.client.0.smithi031.stdout:8/565: stat d8/d1a/d1d/d35/d58/d47/c94 0 2024-04-24T23:13:44.778 INFO:tasks.workunit.client.1.smithi040.stdout:8/75: write d0/d8/fd [1719074,88637] 0 2024-04-24T23:13:44.778 INFO:tasks.workunit.client.1.smithi040.stdout:0/172: write df/d12/f2a [655166,65151] 0 2024-04-24T23:13:44.782 INFO:tasks.workunit.client.1.smithi040.stdout:3/108: rename d1 to d1/d9/da/d23 22 2024-04-24T23:13:44.783 INFO:tasks.workunit.client.1.smithi040.stdout:8/76: dread d0/d8/fd [0,4194304] 0 2024-04-24T23:13:44.783 INFO:tasks.workunit.client.1.smithi040.stdout:8/77: chown d0/d8/dc/df 287755083 1 2024-04-24T23:13:44.784 INFO:tasks.workunit.client.1.smithi040.stdout:4/84: creat da/d1d/f21 x:0 0 0 2024-04-24T23:13:44.785 INFO:tasks.workunit.client.1.smithi040.stdout:4/85: read - da/f20 zero size 2024-04-24T23:13:44.786 INFO:tasks.workunit.client.1.smithi040.stdout:4/86: write da/f18 [2144992,64660] 0 2024-04-24T23:13:44.786 INFO:tasks.workunit.client.1.smithi040.stdout:4/87: chown da 1 1 2024-04-24T23:13:44.788 INFO:tasks.workunit.client.0.smithi031.stdout:7/451: rename d1/d6a to d1/d45/d75/d9f 0 2024-04-24T23:13:44.789 INFO:tasks.workunit.client.1.smithi040.stdout:5/114: truncate d2/d7/d8/db/d1e/f2e 3557458 0 2024-04-24T23:13:44.790 INFO:tasks.workunit.client.1.smithi040.stdout:5/115: fsync d2/d7/d8/f1b 0 2024-04-24T23:13:44.792 INFO:tasks.workunit.client.0.smithi031.stdout:5/420: link d0/d6/d3b/d4b/dc/d25/f69 d0/d6/d3b/f8d 0 2024-04-24T23:13:44.794 INFO:tasks.workunit.client.0.smithi031.stdout:4/562: sync 2024-04-24T23:13:44.796 INFO:tasks.workunit.client.0.smithi031.stdout:4/563: truncate d3/d20/d25/d7e/fa1 307223 0 2024-04-24T23:13:44.796 INFO:tasks.workunit.client.1.smithi040.stdout:6/103: symlink d7/l1c 0 2024-04-24T23:13:44.797 INFO:tasks.workunit.client.0.smithi031.stdout:4/564: chown d3/d20/d25/d7e 208292378 1 2024-04-24T23:13:44.803 INFO:tasks.workunit.client.0.smithi031.stdout:5/421: symlink d0/d6/d3b/d4f/l8e 0 2024-04-24T23:13:44.805 INFO:tasks.workunit.client.0.smithi031.stdout:8/566: mkdir d8/d15/d37/d43/d7b/dbf/dcc 0 2024-04-24T23:13:44.806 INFO:tasks.workunit.client.1.smithi040.stdout:9/156: mkdir d7/d1b/d2f/d1f/d31/d3b 0 2024-04-24T23:13:44.807 INFO:tasks.workunit.client.0.smithi031.stdout:4/565: mknod d3/d20/d25/d95/cb5 0 2024-04-24T23:13:44.808 INFO:tasks.workunit.client.0.smithi031.stdout:4/566: read d3/d20/d2d/d44/d5b/d5d/f97 [462283,67873] 0 2024-04-24T23:13:44.809 INFO:tasks.workunit.client.1.smithi040.stdout:1/101: mkdir da/d12/d14/d25 0 2024-04-24T23:13:44.810 INFO:tasks.workunit.client.1.smithi040.stdout:1/102: write f2 [5067942,102638] 0 2024-04-24T23:13:44.811 INFO:tasks.workunit.client.0.smithi031.stdout:9/568: link d2/f75 d2/d1b/d14/d78/d84/d4a/d4b/d96/fa8 0 2024-04-24T23:13:44.812 INFO:tasks.workunit.client.1.smithi040.stdout:7/176: rmdir db/d2f 39 2024-04-24T23:13:44.813 INFO:tasks.workunit.client.0.smithi031.stdout:4/567: dread d3/d23/d3b/f56 [0,4194304] 0 2024-04-24T23:13:44.814 INFO:tasks.workunit.client.1.smithi040.stdout:2/169: creat da/d10/f34 x:0 0 0 2024-04-24T23:13:44.815 INFO:tasks.workunit.client.0.smithi031.stdout:5/422: mknod d0/d7/c8f 0 2024-04-24T23:13:44.815 INFO:tasks.workunit.client.0.smithi031.stdout:4/568: write d3/d20/d2d/f52 [1015527,130765] 0 2024-04-24T23:13:44.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/567: write d8/d1a/d1d/d35/d58/f44 [5049970,121718] 0 2024-04-24T23:13:44.867 INFO:tasks.workunit.client.1.smithi040.stdout:8/78: fdatasync d0/f14 0 2024-04-24T23:13:44.868 INFO:tasks.workunit.client.0.smithi031.stdout:9/569: creat d2/d43/fa9 x:0 0 0 2024-04-24T23:13:44.869 INFO:tasks.workunit.client.0.smithi031.stdout:0/507: dwrite da/d11/f5d [0,4194304] 0 2024-04-24T23:13:44.870 INFO:tasks.workunit.client.0.smithi031.stdout:0/508: stat da/d11/d6a/c93 0 2024-04-24T23:13:44.871 INFO:tasks.workunit.client.0.smithi031.stdout:4/569: creat d3/d5e/fb6 x:0 0 0 2024-04-24T23:13:44.875 INFO:tasks.workunit.client.0.smithi031.stdout:5/423: write d0/d6/d3b/f8d [716617,68011] 0 2024-04-24T23:13:44.879 INFO:tasks.workunit.client.0.smithi031.stdout:0/509: dwrite da/d18/d1a/d37/f75 [0,4194304] 0 2024-04-24T23:13:44.880 INFO:tasks.workunit.client.0.smithi031.stdout:0/510: write da/d22/f4b [859374,44308] 0 2024-04-24T23:13:44.890 INFO:tasks.workunit.client.0.smithi031.stdout:4/570: dread d3/d20/d2d/f40 [0,4194304] 0 2024-04-24T23:13:44.891 INFO:tasks.workunit.client.1.smithi040.stdout:6/104: fdatasync d7/fd 0 2024-04-24T23:13:44.892 INFO:tasks.workunit.client.1.smithi040.stdout:6/105: chown d7/cc 5584 1 2024-04-24T23:13:44.893 INFO:tasks.workunit.client.0.smithi031.stdout:5/424: mknod d0/d7/c90 0 2024-04-24T23:13:44.894 INFO:tasks.workunit.client.0.smithi031.stdout:7/452: write d1/d2/f11 [1706540,7132] 0 2024-04-24T23:13:44.896 INFO:tasks.workunit.client.0.smithi031.stdout:7/453: write d1/d2/dd/f97 [684735,50459] 0 2024-04-24T23:13:44.897 INFO:tasks.workunit.client.0.smithi031.stdout:7/454: chown d1/f53 8122 1 2024-04-24T23:13:44.897 INFO:tasks.workunit.client.0.smithi031.stdout:7/455: stat d1/d2/dd/c6b 0 2024-04-24T23:13:44.898 INFO:tasks.workunit.client.0.smithi031.stdout:7/456: stat d1/d2/d1e/d4f/l68 0 2024-04-24T23:13:44.898 INFO:tasks.workunit.client.1.smithi040.stdout:1/103: rename da/c17 to da/d12/d23/c26 0 2024-04-24T23:13:44.900 INFO:tasks.workunit.client.1.smithi040.stdout:6/106: dwrite d7/de/ff [0,4194304] 0 2024-04-24T23:13:44.900 INFO:tasks.workunit.client.0.smithi031.stdout:9/570: unlink d2/d1b/d14/d1d/d45/d56/fa0 0 2024-04-24T23:13:44.904 INFO:tasks.workunit.client.1.smithi040.stdout:2/170: symlink da/d1c/l35 0 2024-04-24T23:13:44.904 INFO:tasks.workunit.client.0.smithi031.stdout:4/571: rmdir d3/d20/d25/d95 39 2024-04-24T23:13:44.904 INFO:tasks.workunit.client.1.smithi040.stdout:6/107: dread d7/f9 [0,4194304] 0 2024-04-24T23:13:44.911 INFO:tasks.workunit.client.0.smithi031.stdout:7/457: rename d1/d2/d1e to d1/d45/da0 0 2024-04-24T23:13:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:44 smithi040 ceph-mon[33128]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:44 smithi040 ceph-mon[33128]: pgmap v25: 65 pgs: 65 active+clean; 1.0 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 18 MiB/s rd, 122 MiB/s wr, 298 op/s 2024-04-24T23:13:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:44 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:13:44.915 INFO:tasks.workunit.client.1.smithi040.stdout:4/88: mknod da/d1d/d1e/c22 0 2024-04-24T23:13:44.916 INFO:tasks.workunit.client.1.smithi040.stdout:4/89: readlink l1 0 2024-04-24T23:13:44.917 INFO:tasks.workunit.client.1.smithi040.stdout:4/90: write f4 [8509043,19775] 0 2024-04-24T23:13:44.917 INFO:tasks.workunit.client.1.smithi040.stdout:5/116: getdents d2/d2f/d14 0 2024-04-24T23:13:44.918 INFO:tasks.workunit.client.0.smithi031.stdout:5/425: mkdir d0/d6/d3b/d4b/dc/d86/d91 0 2024-04-24T23:13:44.918 INFO:tasks.workunit.client.0.smithi031.stdout:8/568: write d8/d15/d37/d43/f6f [1377700,4500] 0 2024-04-24T23:13:44.919 INFO:tasks.workunit.client.0.smithi031.stdout:5/426: chown d0/d6/d3b/d4b 400473 1 2024-04-24T23:13:44.921 INFO:tasks.workunit.client.0.smithi031.stdout:7/458: mkdir d1/d2/dd/d33/d5e/d64/d6d/da1 0 2024-04-24T23:13:44.921 INFO:tasks.workunit.client.0.smithi031.stdout:7/459: dread - d1/d45/d84/f9e zero size 2024-04-24T23:13:44.923 INFO:tasks.workunit.client.0.smithi031.stdout:7/460: write d1/d2/dd/f97 [10004,125830] 0 2024-04-24T23:13:44.931 INFO:tasks.workunit.client.1.smithi040.stdout:9/157: dwrite d7/ff [0,4194304] 0 2024-04-24T23:13:44.932 INFO:tasks.workunit.client.1.smithi040.stdout:8/79: rename d0/d8/dc/df to d0/d16 0 2024-04-24T23:13:44.932 INFO:tasks.workunit.client.1.smithi040.stdout:8/80: stat d0/l13 0 2024-04-24T23:13:44.935 INFO:tasks.workunit.client.0.smithi031.stdout:8/569: unlink d8/d15/d37/c48 0 2024-04-24T23:13:44.937 INFO:tasks.workunit.client.1.smithi040.stdout:7/177: mknod db/c42 0 2024-04-24T23:13:44.937 INFO:tasks.workunit.client.1.smithi040.stdout:3/109: truncate d1/f8 4072558 0 2024-04-24T23:13:44.942 INFO:tasks.workunit.client.1.smithi040.stdout:0/173: link df/d12/f17 df/d1e/f2f 0 2024-04-24T23:13:44.943 INFO:tasks.workunit.client.1.smithi040.stdout:0/174: write df/d14/f16 [862023,47797] 0 2024-04-24T23:13:44.946 INFO:tasks.workunit.client.0.smithi031.stdout:4/572: dwrite d3/d20/d2d/d44/f84 [0,4194304] 0 2024-04-24T23:13:44.947 INFO:tasks.workunit.client.1.smithi040.stdout:4/91: creat da/d1d/d1e/f23 x:0 0 0 2024-04-24T23:13:44.947 INFO:tasks.workunit.client.0.smithi031.stdout:4/573: read - d3/d23/d3b/f9c zero size 2024-04-24T23:13:44.948 INFO:tasks.workunit.client.1.smithi040.stdout:4/92: truncate da/f1c 84080 0 2024-04-24T23:13:44.949 INFO:tasks.workunit.client.1.smithi040.stdout:5/117: write d2/d2f/d14/f17 [143205,117979] 0 2024-04-24T23:13:44.950 INFO:tasks.workunit.client.0.smithi031.stdout:9/571: truncate d2/d1b/d14/d78/d84/d4a/f94 2642780 0 2024-04-24T23:13:44.952 INFO:tasks.workunit.client.1.smithi040.stdout:1/104: chown da/d12/d18 55013451 1 2024-04-24T23:13:44.958 INFO:tasks.workunit.client.0.smithi031.stdout:9/572: dwrite d2/d43/fa9 [0,4194304] 0 2024-04-24T23:13:44.961 INFO:tasks.workunit.client.1.smithi040.stdout:3/110: creat d1/d9/da/f24 x:0 0 0 2024-04-24T23:13:44.962 INFO:tasks.workunit.client.1.smithi040.stdout:3/111: write d1/f3 [2226519,37111] 0 2024-04-24T23:13:44.963 INFO:tasks.workunit.client.1.smithi040.stdout:3/112: readlink d1/d9/da/de/l22 0 2024-04-24T23:13:44.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:44 smithi031 ceph-mon[26411]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:13:44.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:44 smithi031 ceph-mon[26411]: pgmap v25: 65 pgs: 65 active+clean; 1.0 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 18 MiB/s rd, 122 MiB/s wr, 298 op/s 2024-04-24T23:13:44.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:44 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:13:45.006 INFO:tasks.workunit.client.1.smithi040.stdout:0/175: mknod df/d14/d1b/c30 0 2024-04-24T23:13:45.007 INFO:tasks.workunit.client.1.smithi040.stdout:0/176: read df/d1e/d2b/f29 [1174130,54161] 0 2024-04-24T23:13:45.007 INFO:tasks.workunit.client.1.smithi040.stdout:6/108: fsync f2 0 2024-04-24T23:13:45.008 INFO:tasks.workunit.client.0.smithi031.stdout:3/421: sync 2024-04-24T23:13:45.019 INFO:tasks.workunit.client.1.smithi040.stdout:4/93: creat da/d1d/d1e/f24 x:0 0 0 2024-04-24T23:13:45.020 INFO:tasks.workunit.client.1.smithi040.stdout:5/118: creat d2/d2f/f33 x:0 0 0 2024-04-24T23:13:45.024 INFO:tasks.workunit.client.1.smithi040.stdout:9/158: mknod d7/da/d17/c3c 0 2024-04-24T23:13:45.027 INFO:tasks.workunit.client.1.smithi040.stdout:7/178: mkdir db/d43 0 2024-04-24T23:13:45.030 INFO:tasks.workunit.client.1.smithi040.stdout:1/105: dwrite da/ff [4194304,4194304] 0 2024-04-24T23:13:45.030 INFO:tasks.workunit.client.1.smithi040.stdout:2/171: rename da/l25 to da/l36 0 2024-04-24T23:13:45.031 INFO:tasks.workunit.client.1.smithi040.stdout:3/113: fsync d1/d9/f20 0 2024-04-24T23:13:45.032 INFO:tasks.workunit.client.1.smithi040.stdout:8/81: dwrite d0/f14 [0,4194304] 0 2024-04-24T23:13:45.035 INFO:tasks.workunit.client.1.smithi040.stdout:0/177: fdatasync df/d1e/f2f 0 2024-04-24T23:13:45.039 INFO:tasks.workunit.client.1.smithi040.stdout:7/179: dread db/f2b [0,4194304] 0 2024-04-24T23:13:45.041 INFO:tasks.workunit.client.1.smithi040.stdout:4/94: creat da/d1d/f25 x:0 0 0 2024-04-24T23:13:45.041 INFO:tasks.workunit.client.1.smithi040.stdout:6/109: write d7/f9 [3075668,22447] 0 2024-04-24T23:13:45.042 INFO:tasks.workunit.client.1.smithi040.stdout:5/119: creat d2/d7/d8/f34 x:0 0 0 2024-04-24T23:13:45.045 INFO:tasks.workunit.client.1.smithi040.stdout:9/159: chown d7/fe 1940 1 2024-04-24T23:13:45.045 INFO:tasks.workunit.client.1.smithi040.stdout:7/180: truncate db/d1b/f23 271354 0 2024-04-24T23:13:45.046 INFO:tasks.workunit.client.1.smithi040.stdout:1/106: dwrite da/d12/f21 [0,4194304] 0 2024-04-24T23:13:45.050 INFO:tasks.workunit.client.1.smithi040.stdout:4/95: dread da/f18 [0,4194304] 0 2024-04-24T23:13:45.051 INFO:tasks.workunit.client.0.smithi031.stdout:4/574: symlink d3/d5e/lb7 0 2024-04-24T23:13:45.052 INFO:tasks.workunit.client.0.smithi031.stdout:4/575: chown d3/d23/c35 0 1 2024-04-24T23:13:45.052 INFO:tasks.workunit.client.1.smithi040.stdout:2/172: mkdir da/d1c/d37 0 2024-04-24T23:13:45.056 INFO:tasks.workunit.client.1.smithi040.stdout:4/96: dread da/f13 [0,4194304] 0 2024-04-24T23:13:45.058 INFO:tasks.workunit.client.1.smithi040.stdout:3/114: unlink d1/d9/da/de/cf 0 2024-04-24T23:13:45.060 INFO:tasks.workunit.client.0.smithi031.stdout:7/461: link d1/d45/d75/d9f/d2c/c3a d1/d45/d75/d9f/d65/ca2 0 2024-04-24T23:13:45.061 INFO:tasks.workunit.client.0.smithi031.stdout:7/462: dread - d1/d45/d84/f96 zero size 2024-04-24T23:13:45.062 INFO:tasks.workunit.client.1.smithi040.stdout:0/178: dread fb [0,4194304] 0 2024-04-24T23:13:45.063 INFO:tasks.workunit.client.0.smithi031.stdout:5/427: getdents d0/d6/d3b/d4b/dc 0 2024-04-24T23:13:45.065 INFO:tasks.workunit.client.1.smithi040.stdout:8/82: creat d0/d8/dc/f17 x:0 0 0 2024-04-24T23:13:45.065 INFO:tasks.workunit.client.1.smithi040.stdout:8/83: chown d0/l2 5701 1 2024-04-24T23:13:45.066 INFO:tasks.workunit.client.1.smithi040.stdout:0/179: dread df/d12/f2a [0,4194304] 0 2024-04-24T23:13:45.069 INFO:tasks.workunit.client.1.smithi040.stdout:5/120: symlink d2/d7/d8/db/d1e/l35 0 2024-04-24T23:13:45.069 INFO:tasks.workunit.client.1.smithi040.stdout:0/180: dread df/d14/f16 [0,4194304] 0 2024-04-24T23:13:45.070 INFO:tasks.workunit.client.1.smithi040.stdout:0/181: chown df/d14/l27 965585 1 2024-04-24T23:13:45.071 INFO:tasks.workunit.client.0.smithi031.stdout:7/463: symlink d1/d2/dd/d33/d5e/d64/d6d/la3 0 2024-04-24T23:13:45.071 INFO:tasks.workunit.client.1.smithi040.stdout:0/182: stat df/d1e/d2b 0 2024-04-24T23:13:45.071 INFO:tasks.workunit.client.0.smithi031.stdout:5/428: dread - d0/d6/d32/f4a zero size 2024-04-24T23:13:45.073 INFO:tasks.workunit.client.1.smithi040.stdout:7/181: mkdir db/d1b/d44 0 2024-04-24T23:13:45.074 INFO:tasks.workunit.client.0.smithi031.stdout:8/570: write d8/d15/d37/f95 [822461,64778] 0 2024-04-24T23:13:45.076 INFO:tasks.workunit.client.0.smithi031.stdout:3/422: link d6/df/d16/f7f d6/df/d16/d53/f88 0 2024-04-24T23:13:45.077 INFO:tasks.workunit.client.0.smithi031.stdout:7/464: dwrite d1/f2b [0,4194304] 0 2024-04-24T23:13:45.079 INFO:tasks.workunit.client.1.smithi040.stdout:1/107: rename f7 to da/d12/d18/f27 0 2024-04-24T23:13:45.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/429: chown d0/d6/d3b/d4b/d1d/d2e/d28/c59 96 1 2024-04-24T23:13:45.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/571: dwrite d8/d15/d37/d43/d7b/d86/fb6 [0,4194304] 0 2024-04-24T23:13:45.088 INFO:tasks.workunit.client.1.smithi040.stdout:1/108: dwrite da/d12/d14/f24 [0,4194304] 0 2024-04-24T23:13:45.089 INFO:tasks.workunit.client.0.smithi031.stdout:7/465: dread d1/d45/da0/f36 [0,4194304] 0 2024-04-24T23:13:45.092 INFO:tasks.workunit.client.0.smithi031.stdout:7/466: chown d1/d45/da0/d4f/f5d 144 1 2024-04-24T23:13:45.092 INFO:tasks.workunit.client.0.smithi031.stdout:9/573: dwrite d2/d1b/d14/f62 [0,4194304] 0 2024-04-24T23:13:45.093 INFO:tasks.workunit.client.1.smithi040.stdout:6/110: mknod d7/c1d 0 2024-04-24T23:13:45.095 INFO:tasks.workunit.client.1.smithi040.stdout:8/84: symlink d0/d16/l18 0 2024-04-24T23:13:45.096 INFO:tasks.workunit.client.1.smithi040.stdout:8/85: chown d0/f14 4 1 2024-04-24T23:13:45.102 INFO:tasks.workunit.client.1.smithi040.stdout:0/183: rmdir df/d12/d1a 39 2024-04-24T23:13:45.103 INFO:tasks.workunit.client.1.smithi040.stdout:7/182: fsync db/f2e 0 2024-04-24T23:13:45.104 INFO:tasks.workunit.client.1.smithi040.stdout:2/173: mkdir da/d1c/d37/d38 0 2024-04-24T23:13:45.105 INFO:tasks.workunit.client.1.smithi040.stdout:2/174: chown da/f19 374866425 1 2024-04-24T23:13:45.105 INFO:tasks.workunit.client.1.smithi040.stdout:4/97: mknod da/d1f/c26 0 2024-04-24T23:13:45.108 INFO:tasks.workunit.client.1.smithi040.stdout:1/109: write f2 [4073459,47235] 0 2024-04-24T23:13:45.110 INFO:tasks.workunit.client.1.smithi040.stdout:6/111: mkdir d7/de/d15/d1e 0 2024-04-24T23:13:45.110 INFO:tasks.workunit.client.1.smithi040.stdout:8/86: mkdir d0/d16/d19 0 2024-04-24T23:13:45.115 INFO:tasks.workunit.client.1.smithi040.stdout:0/184: rmdir df 39 2024-04-24T23:13:45.115 INFO:tasks.workunit.client.1.smithi040.stdout:1/110: dwrite da/d12/d18/f27 [0,4194304] 0 2024-04-24T23:13:45.118 INFO:tasks.workunit.client.1.smithi040.stdout:6/112: dwrite d7/fb [0,4194304] 0 2024-04-24T23:13:45.118 INFO:tasks.workunit.client.1.smithi040.stdout:2/175: mkdir da/d1c/d37/d39 0 2024-04-24T23:13:45.120 INFO:tasks.workunit.client.1.smithi040.stdout:4/98: rename da/d1d/d1e/c22 to da/c27 0 2024-04-24T23:13:45.121 INFO:tasks.workunit.client.0.smithi031.stdout:3/423: mkdir d6/d72/d89 0 2024-04-24T23:13:45.121 INFO:tasks.workunit.client.0.smithi031.stdout:7/467: write d1/d45/d75/d9f/d2c/f4c [784972,21931] 0 2024-04-24T23:13:45.122 INFO:tasks.workunit.client.1.smithi040.stdout:4/99: write da/f20 [490797,45840] 0 2024-04-24T23:13:45.123 INFO:tasks.workunit.client.1.smithi040.stdout:4/100: write da/fb [2723816,123758] 0 2024-04-24T23:13:45.125 INFO:tasks.workunit.client.1.smithi040.stdout:3/115: rmdir d1/d9/da/de/d13 0 2024-04-24T23:13:45.128 INFO:tasks.workunit.client.0.smithi031.stdout:9/574: read d2/d1b/d14/f39 [711555,102817] 0 2024-04-24T23:13:45.129 INFO:tasks.workunit.client.0.smithi031.stdout:4/576: dwrite d3/d23/d69/f30 [0,4194304] 0 2024-04-24T23:13:45.132 INFO:tasks.workunit.client.0.smithi031.stdout:3/424: symlink d6/d14/d1d/d69/l8a 0 2024-04-24T23:13:45.132 INFO:tasks.workunit.client.0.smithi031.stdout:3/425: chown d6/d72/d1a/d25 255 1 2024-04-24T23:13:45.133 INFO:tasks.workunit.client.1.smithi040.stdout:5/121: dwrite d2/f6 [0,4194304] 0 2024-04-24T23:13:45.133 INFO:tasks.workunit.client.0.smithi031.stdout:8/572: creat d8/d1a/d1d/d35/fcd x:0 0 0 2024-04-24T23:13:45.137 INFO:tasks.workunit.client.1.smithi040.stdout:8/87: fsync d0/d8/fd 0 2024-04-24T23:13:45.138 INFO:tasks.workunit.client.1.smithi040.stdout:7/183: dwrite db/d12/d20/f22 [0,4194304] 0 2024-04-24T23:13:45.140 INFO:tasks.workunit.client.1.smithi040.stdout:8/88: write d0/f14 [2769417,32334] 0 2024-04-24T23:13:45.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/426: dwrite d6/df/f82 [0,4194304] 0 2024-04-24T23:13:45.142 INFO:tasks.workunit.client.0.smithi031.stdout:4/577: rename d3/d20/c27 to d3/d20/d2d/d44/cb8 0 2024-04-24T23:13:45.142 INFO:tasks.workunit.client.1.smithi040.stdout:9/160: getdents d7 0 2024-04-24T23:13:45.147 INFO:tasks.workunit.client.1.smithi040.stdout:6/113: creat d7/de/d15/f1f x:0 0 0 2024-04-24T23:13:45.147 INFO:tasks.workunit.client.1.smithi040.stdout:6/114: stat d7/de/l14 0 2024-04-24T23:13:45.148 INFO:tasks.workunit.client.0.smithi031.stdout:7/468: unlink d1/d45/da0/c4d 0 2024-04-24T23:13:45.148 INFO:tasks.workunit.client.1.smithi040.stdout:6/115: write d7/f9 [2526146,106369] 0 2024-04-24T23:13:45.149 INFO:tasks.workunit.client.0.smithi031.stdout:8/573: symlink d8/d15/d37/d43/daf/lce 0 2024-04-24T23:13:45.149 INFO:tasks.workunit.client.0.smithi031.stdout:3/427: creat d6/d14/d1d/f8b x:0 0 0 2024-04-24T23:13:45.153 INFO:tasks.workunit.client.0.smithi031.stdout:4/578: mkdir d3/d20/d25/d60/d63/db9 0 2024-04-24T23:13:45.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/574: mkdir d8/d15/dcf 0 2024-04-24T23:13:45.158 INFO:tasks.workunit.client.0.smithi031.stdout:8/575: chown d8/d15/d37/d43/c75 2 1 2024-04-24T23:13:45.159 INFO:tasks.workunit.client.0.smithi031.stdout:4/579: chown d3/d20/d2d/c42 281295 1 2024-04-24T23:13:45.159 INFO:tasks.workunit.client.0.smithi031.stdout:4/580: chown d3/f9 9 1 2024-04-24T23:13:45.162 INFO:tasks.workunit.client.0.smithi031.stdout:7/469: chown d1/d2/dd/d33/c5c 472743 1 2024-04-24T23:13:45.162 INFO:tasks.workunit.client.0.smithi031.stdout:7/470: fdatasync d1/f2b 0 2024-04-24T23:13:45.170 INFO:tasks.workunit.client.0.smithi031.stdout:7/471: mkdir d1/d45/d75/d9f/d65/da4 0 2024-04-24T23:13:45.170 INFO:tasks.workunit.client.0.smithi031.stdout:9/575: write d2/f12 [5187166,9995] 0 2024-04-24T23:13:45.175 INFO:tasks.workunit.client.0.smithi031.stdout:4/581: mknod d3/cba 0 2024-04-24T23:13:45.177 INFO:tasks.workunit.client.0.smithi031.stdout:4/582: chown d3/f9 2074550123 1 2024-04-24T23:13:45.178 INFO:tasks.workunit.client.0.smithi031.stdout:9/576: dwrite d2/d1b/d14/f62 [4194304,4194304] 0 2024-04-24T23:13:45.179 INFO:tasks.workunit.client.0.smithi031.stdout:4/583: dread - d3/d23/d37/fa9 zero size 2024-04-24T23:13:45.181 INFO:tasks.workunit.client.0.smithi031.stdout:4/584: chown d3/d20/d2d/f52 64 1 2024-04-24T23:13:45.181 INFO:tasks.workunit.client.1.smithi040.stdout:4/101: link da/f19 da/f28 0 2024-04-24T23:13:45.182 INFO:tasks.workunit.client.1.smithi040.stdout:4/102: chown da/cd 1 1 2024-04-24T23:13:45.183 INFO:tasks.workunit.client.0.smithi031.stdout:4/585: dread - d3/d23/d3b/f9c zero size 2024-04-24T23:13:45.184 INFO:tasks.workunit.client.0.smithi031.stdout:4/586: chown d3/d5e/fb6 250 1 2024-04-24T23:13:45.185 INFO:tasks.workunit.client.1.smithi040.stdout:3/116: mkdir d1/d9/d25 0 2024-04-24T23:13:45.189 INFO:tasks.workunit.client.0.smithi031.stdout:9/577: dwrite d2/d1b/d14/d1d/d45/f93 [0,4194304] 0 2024-04-24T23:13:45.189 INFO:tasks.workunit.client.1.smithi040.stdout:4/103: dread f9 [0,4194304] 0 2024-04-24T23:13:45.192 INFO:tasks.workunit.client.1.smithi040.stdout:0/185: write df/d14/f16 [1323929,100668] 0 2024-04-24T23:13:45.192 INFO:tasks.workunit.client.0.smithi031.stdout:3/428: dwrite d6/df/d16/d2a/f2e [0,4194304] 0 2024-04-24T23:13:45.194 INFO:tasks.workunit.client.0.smithi031.stdout:3/429: fdatasync d6/d72/d29/f79 0 2024-04-24T23:13:45.194 INFO:tasks.workunit.client.1.smithi040.stdout:3/117: dwrite d1/d9/d15/f1b [0,4194304] 0 2024-04-24T23:13:45.195 INFO:tasks.workunit.client.1.smithi040.stdout:1/111: truncate da/d12/d14/f24 921385 0 2024-04-24T23:13:45.196 INFO:tasks.workunit.client.0.smithi031.stdout:8/576: getdents d8/d15/d37/d62 0 2024-04-24T23:13:45.198 INFO:tasks.workunit.client.0.smithi031.stdout:8/577: dread - d8/d1a/d1d/d50/dc0/d9d/fa3 zero size 2024-04-24T23:13:45.200 INFO:tasks.workunit.client.1.smithi040.stdout:9/161: mkdir d7/da/d3d 0 2024-04-24T23:13:45.202 INFO:tasks.workunit.client.0.smithi031.stdout:4/587: rename d3/d23/l47 to d3/d20/d2d/d44/lbb 0 2024-04-24T23:13:45.203 INFO:tasks.workunit.client.1.smithi040.stdout:0/186: dwrite df/d14/d1b/f28 [0,4194304] 0 2024-04-24T23:13:45.204 INFO:tasks.workunit.client.1.smithi040.stdout:3/118: dread d1/d9/f20 [0,4194304] 0 2024-04-24T23:13:45.208 INFO:tasks.workunit.client.1.smithi040.stdout:6/116: mkdir d7/d1a/d20 0 2024-04-24T23:13:45.208 INFO:tasks.workunit.client.1.smithi040.stdout:6/117: chown d7/fb 113001186 1 2024-04-24T23:13:45.209 INFO:tasks.workunit.client.0.smithi031.stdout:7/472: link d1/d45/d75/d9f/f80 d1/d45/d75/d9f/fa5 0 2024-04-24T23:13:45.210 INFO:tasks.workunit.client.1.smithi040.stdout:7/184: truncate db/f10 3287956 0 2024-04-24T23:13:45.210 INFO:tasks.workunit.client.0.smithi031.stdout:8/578: dwrite d8/d1a/d1d/d35/d58/d47/f77 [0,4194304] 0 2024-04-24T23:13:45.211 INFO:tasks.workunit.client.0.smithi031.stdout:4/588: dread d3/d20/d2d/f52 [0,4194304] 0 2024-04-24T23:13:45.211 INFO:tasks.workunit.client.0.smithi031.stdout:3/430: mknod d6/df/d16/d53/c8c 0 2024-04-24T23:13:45.214 INFO:tasks.workunit.client.1.smithi040.stdout:5/122: creat d2/d7/f36 x:0 0 0 2024-04-24T23:13:45.217 INFO:tasks.workunit.client.1.smithi040.stdout:4/104: mknod da/d1d/c29 0 2024-04-24T23:13:45.218 INFO:tasks.workunit.client.0.smithi031.stdout:7/473: chown d1/l55 109634 1 2024-04-24T23:13:45.221 INFO:tasks.workunit.client.0.smithi031.stdout:7/474: write d1/d45/d75/f87 [191264,48333] 0 2024-04-24T23:13:45.222 INFO:tasks.workunit.client.0.smithi031.stdout:3/431: unlink d6/f7e 0 2024-04-24T23:13:45.223 INFO:tasks.workunit.client.1.smithi040.stdout:0/187: symlink df/d14/d1b/l31 0 2024-04-24T23:13:45.225 INFO:tasks.workunit.client.1.smithi040.stdout:4/105: dwrite da/d1d/f21 [0,4194304] 0 2024-04-24T23:13:45.225 INFO:tasks.workunit.client.1.smithi040.stdout:3/119: creat d1/d9/da/de/f26 x:0 0 0 2024-04-24T23:13:45.226 INFO:tasks.workunit.client.1.smithi040.stdout:6/118: fdatasync d7/fd 0 2024-04-24T23:13:45.228 INFO:tasks.workunit.client.1.smithi040.stdout:9/162: dread d7/da/fd [0,4194304] 0 2024-04-24T23:13:45.228 INFO:tasks.workunit.client.1.smithi040.stdout:7/185: fdatasync db/f3c 0 2024-04-24T23:13:45.229 INFO:tasks.workunit.client.0.smithi031.stdout:7/475: mknod d1/d45/da0/d4f/ca6 0 2024-04-24T23:13:45.230 INFO:tasks.workunit.client.0.smithi031.stdout:4/589: creat d3/fbc x:0 0 0 2024-04-24T23:13:45.232 INFO:tasks.workunit.client.0.smithi031.stdout:4/590: read d3/f7 [2151513,70919] 0 2024-04-24T23:13:45.233 INFO:tasks.workunit.client.0.smithi031.stdout:4/591: stat d3/f85 0 2024-04-24T23:13:45.233 INFO:tasks.workunit.client.1.smithi040.stdout:5/123: rmdir d2/d7/d8/db/d1e/d22/d27 39 2024-04-24T23:13:45.233 INFO:tasks.workunit.client.1.smithi040.stdout:3/120: dwrite d1/d9/da/f24 [0,4194304] 0 2024-04-24T23:13:45.234 INFO:tasks.workunit.client.1.smithi040.stdout:5/124: dread - d2/d7/d8/f34 zero size 2024-04-24T23:13:45.235 INFO:tasks.workunit.client.0.smithi031.stdout:7/476: chown d1/d45/d75/d9f/d65/c71 456651 1 2024-04-24T23:13:45.236 INFO:tasks.workunit.client.1.smithi040.stdout:3/121: fdatasync d1/d9/da/de/f26 0 2024-04-24T23:13:45.238 INFO:tasks.workunit.client.1.smithi040.stdout:8/89: getdents d0 0 2024-04-24T23:13:45.239 INFO:tasks.workunit.client.0.smithi031.stdout:4/592: symlink d3/d20/d2d/lbd 0 2024-04-24T23:13:45.239 INFO:tasks.workunit.client.1.smithi040.stdout:8/90: chown d0/l3 19616133 1 2024-04-24T23:13:45.240 INFO:tasks.workunit.client.0.smithi031.stdout:9/578: write d2/d1b/d14/d78/d84/d4a/d4b/d96/fa8 [1161701,35632] 0 2024-04-24T23:13:45.241 INFO:tasks.workunit.client.0.smithi031.stdout:7/477: symlink d1/d2/d60/la7 0 2024-04-24T23:13:45.245 INFO:tasks.workunit.client.0.smithi031.stdout:4/593: truncate d3/fb 135812 0 2024-04-24T23:13:45.246 INFO:tasks.workunit.client.0.smithi031.stdout:1/430: sync 2024-04-24T23:13:45.250 INFO:tasks.workunit.client.0.smithi031.stdout:8/579: dread d8/d15/d37/f2d [0,4194304] 0 2024-04-24T23:13:45.250 INFO:tasks.workunit.client.0.smithi031.stdout:9/579: dwrite d2/d1b/d14/d1d/d45/f5e [0,4194304] 0 2024-04-24T23:13:45.251 INFO:tasks.workunit.client.0.smithi031.stdout:8/580: chown c7 2775 1 2024-04-24T23:13:45.252 INFO:tasks.workunit.client.0.smithi031.stdout:8/581: readlink d8/d15/l5f 0 2024-04-24T23:13:45.253 INFO:tasks.workunit.client.0.smithi031.stdout:3/432: dread d6/f8 [4194304,4194304] 0 2024-04-24T23:13:45.258 INFO:tasks.workunit.client.0.smithi031.stdout:4/594: truncate d3/d5e/f98 51627 0 2024-04-24T23:13:45.261 INFO:tasks.workunit.client.0.smithi031.stdout:4/595: fdatasync d3/da3/faf 0 2024-04-24T23:13:45.261 INFO:tasks.workunit.client.0.smithi031.stdout:1/431: symlink d1/da/d61/l8b 0 2024-04-24T23:13:45.261 INFO:tasks.workunit.client.0.smithi031.stdout:1/432: chown d1/d1d/c56 423 1 2024-04-24T23:13:45.265 INFO:tasks.workunit.client.0.smithi031.stdout:1/433: dread d1/dc/f70 [8388608,4194304] 0 2024-04-24T23:13:45.266 INFO:tasks.workunit.client.1.smithi040.stdout:5/125: mknod d2/d7/d8/db/c37 0 2024-04-24T23:13:45.270 INFO:tasks.workunit.client.0.smithi031.stdout:4/596: creat d3/d20/d2d/fbe x:0 0 0 2024-04-24T23:13:45.271 INFO:tasks.workunit.client.0.smithi031.stdout:4/597: write d3/d23/d37/d86/f9a [594654,4758] 0 2024-04-24T23:13:45.272 INFO:tasks.workunit.client.1.smithi040.stdout:8/91: truncate d0/d8/fd 3181230 0 2024-04-24T23:13:45.276 INFO:tasks.workunit.client.1.smithi040.stdout:6/119: mknod d7/c21 0 2024-04-24T23:13:45.279 INFO:tasks.workunit.client.0.smithi031.stdout:1/434: rename d1/dc/d11/c1b to d1/d33/d41/c8c 0 2024-04-24T23:13:45.280 INFO:tasks.workunit.client.0.smithi031.stdout:1/435: readlink d1/d33/l53 0 2024-04-24T23:13:45.280 INFO:tasks.workunit.client.1.smithi040.stdout:9/163: symlink d7/da/l3e 0 2024-04-24T23:13:45.282 INFO:tasks.workunit.client.0.smithi031.stdout:9/580: write d2/d1b/d14/d1d/f24 [3883709,67705] 0 2024-04-24T23:13:45.285 INFO:tasks.workunit.client.0.smithi031.stdout:4/598: rename d3/d20/fa2 to d3/d20/d25/d60/d63/fbf 0 2024-04-24T23:13:45.286 INFO:tasks.workunit.client.1.smithi040.stdout:9/164: dwrite d7/d1b/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:13:45.287 INFO:tasks.workunit.client.1.smithi040.stdout:5/126: creat d2/d7/d8/d1f/f38 x:0 0 0 2024-04-24T23:13:45.290 INFO:tasks.workunit.client.0.smithi031.stdout:8/582: dwrite d8/d15/d37/d43/d7b/f8c [0,4194304] 0 2024-04-24T23:13:45.300 INFO:tasks.workunit.client.0.smithi031.stdout:8/583: dwrite d8/d15/d37/d62/fb3 [0,4194304] 0 2024-04-24T23:13:45.301 INFO:tasks.workunit.client.0.smithi031.stdout:8/584: readlink l4 0 2024-04-24T23:13:45.326 INFO:tasks.workunit.client.1.smithi040.stdout:2/176: sync 2024-04-24T23:13:45.349 INFO:tasks.workunit.client.0.smithi031.stdout:2/348: sync 2024-04-24T23:13:45.350 INFO:tasks.workunit.client.0.smithi031.stdout:5/430: sync 2024-04-24T23:13:45.350 INFO:tasks.workunit.client.0.smithi031.stdout:6/607: sync 2024-04-24T23:13:45.350 INFO:tasks.workunit.client.0.smithi031.stdout:0/511: sync 2024-04-24T23:13:45.350 INFO:tasks.workunit.client.0.smithi031.stdout:9/581: mknod d2/d1b/d14/d78/d84/da6/caa 0 2024-04-24T23:13:45.351 INFO:tasks.workunit.client.1.smithi040.stdout:1/112: rename da/c20 to da/c28 0 2024-04-24T23:13:45.356 INFO:tasks.workunit.client.0.smithi031.stdout:7/478: dwrite d1/d2/dd/d33/f62 [0,4194304] 0 2024-04-24T23:13:45.358 INFO:tasks.workunit.client.1.smithi040.stdout:8/92: unlink d0/f14 0 2024-04-24T23:13:45.359 INFO:tasks.workunit.client.1.smithi040.stdout:8/93: chown d0/l3 753549691 1 2024-04-24T23:13:45.362 INFO:tasks.workunit.client.1.smithi040.stdout:0/188: creat df/d1e/d20/d21/f32 x:0 0 0 2024-04-24T23:13:45.364 INFO:tasks.workunit.client.0.smithi031.stdout:7/479: dwrite d1/f9 [0,4194304] 0 2024-04-24T23:13:45.364 INFO:tasks.workunit.client.1.smithi040.stdout:6/120: rename f0 to d7/de/d15/d1e/f22 0 2024-04-24T23:13:45.365 INFO:tasks.workunit.client.1.smithi040.stdout:6/121: write d7/f9 [4767283,50178] 0 2024-04-24T23:13:45.367 INFO:tasks.workunit.client.1.smithi040.stdout:6/122: write d7/f9 [774315,24200] 0 2024-04-24T23:13:45.368 INFO:tasks.workunit.client.1.smithi040.stdout:4/106: link da/c14 da/d1f/c2a 0 2024-04-24T23:13:45.371 INFO:tasks.workunit.client.0.smithi031.stdout:8/585: symlink d8/d15/d37/d43/d7b/d87/dad/ld0 0 2024-04-24T23:13:45.385 INFO:tasks.workunit.client.1.smithi040.stdout:9/165: fdatasync d7/fe 0 2024-04-24T23:13:45.388 INFO:tasks.workunit.client.1.smithi040.stdout:5/127: creat d2/d7/d8/f39 x:0 0 0 2024-04-24T23:13:45.393 INFO:tasks.workunit.client.1.smithi040.stdout:3/122: link d1/d9/da/f24 d1/d9/f27 0 2024-04-24T23:13:45.394 INFO:tasks.workunit.client.1.smithi040.stdout:2/177: mknod da/d1c/c3a 0 2024-04-24T23:13:45.395 INFO:tasks.workunit.client.1.smithi040.stdout:2/178: fdatasync da/fe 0 2024-04-24T23:13:45.396 INFO:tasks.workunit.client.1.smithi040.stdout:2/179: chown da/d10/f34 0 1 2024-04-24T23:13:45.398 INFO:tasks.workunit.client.1.smithi040.stdout:3/123: dwrite d1/d9/da/de/f26 [0,4194304] 0 2024-04-24T23:13:45.399 INFO:tasks.workunit.client.1.smithi040.stdout:3/124: stat d1/d9/d18/l12 0 2024-04-24T23:13:45.399 INFO:tasks.workunit.client.1.smithi040.stdout:8/94: symlink d0/l1a 0 2024-04-24T23:13:45.400 INFO:tasks.workunit.client.1.smithi040.stdout:8/95: write d0/d8/dc/f17 [866852,42331] 0 2024-04-24T23:13:45.400 INFO:tasks.workunit.client.0.smithi031.stdout:8/586: symlink d8/d1a/d73/ld1 0 2024-04-24T23:13:45.403 INFO:tasks.workunit.client.0.smithi031.stdout:8/587: dread d8/d15/d9a/fc2 [0,4194304] 0 2024-04-24T23:13:45.404 INFO:tasks.workunit.client.1.smithi040.stdout:0/189: creat df/d1e/d20/f33 x:0 0 0 2024-04-24T23:13:45.404 INFO:tasks.workunit.client.0.smithi031.stdout:8/588: readlink d8/d15/d37/d43/daf/lce 0 2024-04-24T23:13:45.405 INFO:tasks.workunit.client.1.smithi040.stdout:3/125: dwrite d1/fd [0,4194304] 0 2024-04-24T23:13:45.406 INFO:tasks.workunit.client.0.smithi031.stdout:0/512: symlink da/d86/da3/la9 0 2024-04-24T23:13:45.407 INFO:tasks.workunit.client.0.smithi031.stdout:8/589: read d8/d1a/d1d/d35/d58/d47/f49 [3551223,58816] 0 2024-04-24T23:13:45.411 INFO:tasks.workunit.client.0.smithi031.stdout:5/431: mknod d0/d6/d3b/d4b/d1d/d2e/c92 0 2024-04-24T23:13:45.412 INFO:tasks.workunit.client.0.smithi031.stdout:5/432: dread - d0/d6/d3b/d4b/dc/f81 zero size 2024-04-24T23:13:45.414 INFO:tasks.workunit.client.0.smithi031.stdout:9/582: mkdir d2/d1b/d14/d78/d84/d4a/dab 0 2024-04-24T23:13:45.418 INFO:tasks.workunit.client.1.smithi040.stdout:4/107: mkdir da/d1d/d2b 0 2024-04-24T23:13:45.422 INFO:tasks.workunit.client.1.smithi040.stdout:9/166: rename d7/da/d3d to d7/d1b/d2f/d1f/d31/d3b/d3f 0 2024-04-24T23:13:45.424 INFO:tasks.workunit.client.0.smithi031.stdout:0/513: creat da/d86/faa x:0 0 0 2024-04-24T23:13:45.427 INFO:tasks.workunit.client.0.smithi031.stdout:8/590: unlink d8/d1a/d1d/d35/d58/f44 0 2024-04-24T23:13:45.428 INFO:tasks.workunit.client.1.smithi040.stdout:9/167: dwrite d7/f3a [0,4194304] 0 2024-04-24T23:13:45.428 INFO:tasks.workunit.client.1.smithi040.stdout:9/168: chown d7/c9 3427 1 2024-04-24T23:13:45.429 INFO:tasks.workunit.client.1.smithi040.stdout:9/169: stat d7/d1b/d2f/c2e 0 2024-04-24T23:13:45.429 INFO:tasks.workunit.client.1.smithi040.stdout:9/170: chown d7/da/f39 24045738 1 2024-04-24T23:13:45.432 INFO:tasks.workunit.client.0.smithi031.stdout:9/583: creat d2/d1b/d14/d18/d7d/fac x:0 0 0 2024-04-24T23:13:45.435 INFO:tasks.workunit.client.1.smithi040.stdout:9/171: dwrite d7/d1b/d2f/d1f/d31/f36 [4194304,4194304] 0 2024-04-24T23:13:45.438 INFO:tasks.workunit.client.0.smithi031.stdout:6/608: rename d13/d15/d1d/d23/d24/d43/f4b to d13/d15/d1d/d23/d44/d47/fd6 0 2024-04-24T23:13:45.441 INFO:tasks.workunit.client.1.smithi040.stdout:2/180: symlink da/d10/d15/l3b 0 2024-04-24T23:13:45.442 INFO:tasks.workunit.client.1.smithi040.stdout:9/172: dwrite f2 [0,4194304] 0 2024-04-24T23:13:45.447 INFO:tasks.workunit.client.0.smithi031.stdout:9/584: truncate d2/f1c 1724170 0 2024-04-24T23:13:45.453 INFO:tasks.workunit.client.0.smithi031.stdout:6/609: rmdir d13/d15/d1d/d23/d44/da0/da6 39 2024-04-24T23:13:45.454 INFO:tasks.workunit.client.1.smithi040.stdout:6/123: creat d7/d1a/d20/f23 x:0 0 0 2024-04-24T23:13:45.457 INFO:tasks.workunit.client.0.smithi031.stdout:7/480: rename d1/d45/d75/d9f/l51 to d1/d45/d84/la8 0 2024-04-24T23:13:45.459 INFO:tasks.workunit.client.0.smithi031.stdout:7/481: fsync d1/d2/dd/d33/d5e/d64/f99 0 2024-04-24T23:13:45.463 INFO:tasks.workunit.client.0.smithi031.stdout:6/610: dread - d13/f87 zero size 2024-04-24T23:13:45.465 INFO:tasks.workunit.client.0.smithi031.stdout:6/611: write d13/d15/d1d/d23/d24/f9f [3865981,14239] 0 2024-04-24T23:13:45.467 INFO:tasks.workunit.client.0.smithi031.stdout:0/514: rename da/f53 to da/d11/d6a/da2/fab 0 2024-04-24T23:13:45.470 INFO:tasks.workunit.client.0.smithi031.stdout:9/585: dread d2/f64 [0,4194304] 0 2024-04-24T23:13:45.475 INFO:tasks.workunit.client.0.smithi031.stdout:7/482: symlink d1/d45/d75/d9f/d66/la9 0 2024-04-24T23:13:45.475 INFO:tasks.workunit.client.0.smithi031.stdout:9/586: write d2/d1b/d14/d1d/f24 [4872904,38615] 0 2024-04-24T23:13:45.476 INFO:tasks.workunit.client.0.smithi031.stdout:0/515: dread da/d18/d1a/d37/f75 [0,4194304] 0 2024-04-24T23:13:45.478 INFO:tasks.workunit.client.0.smithi031.stdout:6/612: dread d13/d15/fb9 [0,4194304] 0 2024-04-24T23:13:45.479 INFO:tasks.workunit.client.0.smithi031.stdout:0/516: dread da/d11/d6a/f8a [0,4194304] 0 2024-04-24T23:13:45.480 INFO:tasks.workunit.client.0.smithi031.stdout:1/436: truncate d1/dc/d11/d21/f28 2568748 0 2024-04-24T23:13:45.484 INFO:tasks.workunit.client.0.smithi031.stdout:1/437: write d1/da/d42/f77 [575090,28972] 0 2024-04-24T23:13:45.484 INFO:tasks.workunit.client.0.smithi031.stdout:4/599: dwrite d3/d20/d25/d60/d63/f81 [4194304,4194304] 0 2024-04-24T23:13:45.484 INFO:tasks.workunit.client.0.smithi031.stdout:1/438: fdatasync d1/d3e/d46/f59 0 2024-04-24T23:13:45.492 INFO:tasks.workunit.client.0.smithi031.stdout:6/613: dread - d13/d1b/d32/f8a zero size 2024-04-24T23:13:45.493 INFO:tasks.workunit.client.1.smithi040.stdout:6/124: mkdir d7/de/d24 0 2024-04-24T23:13:45.494 INFO:tasks.workunit.client.1.smithi040.stdout:6/125: dread d7/fd [0,4194304] 0 2024-04-24T23:13:45.495 INFO:tasks.workunit.client.1.smithi040.stdout:4/108: truncate f6 142735 0 2024-04-24T23:13:45.496 INFO:tasks.workunit.client.1.smithi040.stdout:7/186: chown db/d12/d20/f21 61 1 2024-04-24T23:13:45.496 INFO:tasks.workunit.client.1.smithi040.stdout:7/187: chown db/d12/l26 0 1 2024-04-24T23:13:45.496 INFO:tasks.workunit.client.1.smithi040.stdout:5/128: link d2/cd d2/d7/d8/d1f/c3a 0 2024-04-24T23:13:45.497 INFO:tasks.workunit.client.0.smithi031.stdout:2/349: dwrite d5/d8/d15/f50 [0,4194304] 0 2024-04-24T23:13:45.498 INFO:tasks.workunit.client.0.smithi031.stdout:2/350: chown d5/d8/d15/f50 63532324 1 2024-04-24T23:13:45.502 INFO:tasks.workunit.client.1.smithi040.stdout:2/181: mknod da/d1c/d37/d39/c3c 0 2024-04-24T23:13:45.505 INFO:tasks.workunit.client.1.smithi040.stdout:9/173: mkdir d7/da/d17/d40 0 2024-04-24T23:13:45.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/587: mkdir d2/d1b/d14/d78/d84/d4a/dab/dad 0 2024-04-24T23:13:45.506 INFO:tasks.workunit.client.0.smithi031.stdout:2/351: dwrite d5/d6/df/f44 [0,4194304] 0 2024-04-24T23:13:45.508 INFO:tasks.workunit.client.0.smithi031.stdout:2/352: write d5/d28/d43/f55 [349068,86202] 0 2024-04-24T23:13:45.511 INFO:tasks.workunit.client.0.smithi031.stdout:7/483: rename d1/d45/d75/d9f/d65/f8a to d1/d2/dd/d33/d5e/d64/faa 0 2024-04-24T23:13:45.515 INFO:tasks.workunit.client.1.smithi040.stdout:0/190: creat df/f34 x:0 0 0 2024-04-24T23:13:45.517 INFO:tasks.workunit.client.0.smithi031.stdout:0/517: mknod da/d22/d62/d9c/cac 0 2024-04-24T23:13:45.518 INFO:tasks.workunit.client.1.smithi040.stdout:3/126: link d1/d9/l11 d1/d9/d25/l28 0 2024-04-24T23:13:45.521 INFO:tasks.workunit.client.1.smithi040.stdout:4/109: creat da/d1f/f2c x:0 0 0 2024-04-24T23:13:45.521 INFO:tasks.workunit.client.1.smithi040.stdout:4/110: chown da/d1f/c2a 240788301 1 2024-04-24T23:13:45.523 INFO:tasks.workunit.client.1.smithi040.stdout:6/126: mknod d7/de/c25 0 2024-04-24T23:13:45.526 INFO:tasks.workunit.client.1.smithi040.stdout:5/129: fsync d2/d7/fe 0 2024-04-24T23:13:45.527 INFO:tasks.workunit.client.1.smithi040.stdout:5/130: write d2/d7/d8/db/d1e/f28 [646057,40535] 0 2024-04-24T23:13:45.528 INFO:tasks.workunit.client.1.smithi040.stdout:1/113: symlink da/d12/d18/l29 0 2024-04-24T23:13:45.530 INFO:tasks.workunit.client.1.smithi040.stdout:2/182: symlink da/d1c/d37/d39/l3d 0 2024-04-24T23:13:45.531 INFO:tasks.workunit.client.0.smithi031.stdout:0/518: creat da/d18/d1a/d76/fad x:0 0 0 2024-04-24T23:13:45.533 INFO:tasks.workunit.client.1.smithi040.stdout:9/174: mknod d7/d1b/d2f/d1f/d31/d3b/c41 0 2024-04-24T23:13:45.534 INFO:tasks.workunit.client.0.smithi031.stdout:9/588: truncate d2/d1b/d14/d78/d84/d4a/f94 334637 0 2024-04-24T23:13:45.536 INFO:tasks.workunit.client.0.smithi031.stdout:2/353: creat d5/d8/d6e/d70/d75/f76 x:0 0 0 2024-04-24T23:13:45.538 INFO:tasks.workunit.client.0.smithi031.stdout:0/519: creat da/d18/d1a/d37/d41/fae x:0 0 0 2024-04-24T23:13:45.540 INFO:tasks.workunit.client.0.smithi031.stdout:0/520: chown da/d86/c9d 745999130 1 2024-04-24T23:13:45.542 INFO:tasks.workunit.client.0.smithi031.stdout:2/354: dwrite d5/d6/d29/f61 [0,4194304] 0 2024-04-24T23:13:45.545 INFO:tasks.workunit.client.1.smithi040.stdout:7/188: symlink db/d43/l45 0 2024-04-24T23:13:45.546 INFO:tasks.workunit.client.1.smithi040.stdout:7/189: write db/d12/d20/f22 [4463778,122088] 0 2024-04-24T23:13:45.548 INFO:tasks.workunit.client.0.smithi031.stdout:5/433: link d0/d6/d3b/d4b/f51 d0/f93 0 2024-04-24T23:13:45.549 INFO:tasks.workunit.client.0.smithi031.stdout:2/355: dwrite d5/d28/f40 [0,4194304] 0 2024-04-24T23:13:45.551 INFO:tasks.workunit.client.0.smithi031.stdout:2/356: write d5/d8/d15/f22 [239932,34562] 0 2024-04-24T23:13:45.553 INFO:tasks.workunit.client.1.smithi040.stdout:1/114: mkdir da/d12/d18/d2a 0 2024-04-24T23:13:45.555 INFO:tasks.workunit.client.0.smithi031.stdout:0/521: symlink da/d18/d1a/d37/laf 0 2024-04-24T23:13:45.555 INFO:tasks.workunit.client.1.smithi040.stdout:1/115: chown da/d12/d18/f1d 19170126 1 2024-04-24T23:13:45.568 INFO:tasks.workunit.client.0.smithi031.stdout:5/434: dwrite d0/d6/d3b/d4b/dc/d25/f62 [0,4194304] 0 2024-04-24T23:13:45.569 INFO:tasks.workunit.client.0.smithi031.stdout:5/435: write d0/d6/d3b/d4b/f72 [1026970,101362] 0 2024-04-24T23:13:45.572 INFO:tasks.workunit.client.0.smithi031.stdout:0/522: mkdir da/d22/db0 0 2024-04-24T23:13:45.575 INFO:tasks.workunit.client.1.smithi040.stdout:7/190: mknod db/d43/c46 0 2024-04-24T23:13:45.576 INFO:tasks.workunit.client.0.smithi031.stdout:0/523: read da/d18/d1a/d37/d41/f83 [1014905,33144] 0 2024-04-24T23:13:45.577 INFO:tasks.workunit.client.1.smithi040.stdout:1/116: mknod da/d12/d18/c2b 0 2024-04-24T23:13:45.578 INFO:tasks.workunit.client.1.smithi040.stdout:2/183: creat da/d1c/d37/d38/f3e x:0 0 0 2024-04-24T23:13:45.579 INFO:tasks.workunit.client.1.smithi040.stdout:2/184: write da/f19 [806511,64097] 0 2024-04-24T23:13:45.580 INFO:tasks.workunit.client.1.smithi040.stdout:2/185: chown da/d1c/d37/d39 89 1 2024-04-24T23:13:45.581 INFO:tasks.workunit.client.1.smithi040.stdout:2/186: write da/d10/d15/f2c [729042,116179] 0 2024-04-24T23:13:45.639 INFO:tasks.workunit.client.1.smithi040.stdout:9/175: rmdir d7/d1b/d2f/d1f/d31/d3b/d3f 0 2024-04-24T23:13:45.639 INFO:tasks.workunit.client.1.smithi040.stdout:6/127: getdents d7/d1a 0 2024-04-24T23:13:45.640 INFO:tasks.workunit.client.1.smithi040.stdout:2/187: mkdir da/d1c/d37/d39/d3f 0 2024-04-24T23:13:45.641 INFO:tasks.workunit.client.1.smithi040.stdout:2/188: dread - da/d10/f31 zero size 2024-04-24T23:13:45.643 INFO:tasks.workunit.client.1.smithi040.stdout:7/191: creat db/d1b/d44/f47 x:0 0 0 2024-04-24T23:13:45.643 INFO:tasks.workunit.client.1.smithi040.stdout:2/189: write f8 [395216,61721] 0 2024-04-24T23:13:45.644 INFO:tasks.workunit.client.1.smithi040.stdout:9/176: dwrite d7/f15 [0,4194304] 0 2024-04-24T23:13:45.648 INFO:tasks.workunit.client.0.smithi031.stdout:4/600: creat d3/d20/d25/d60/fc0 x:0 0 0 2024-04-24T23:13:45.649 INFO:tasks.workunit.client.0.smithi031.stdout:4/601: stat d3/d23/d69/d3f/l58 0 2024-04-24T23:13:45.656 INFO:tasks.workunit.client.1.smithi040.stdout:9/177: rename d7/da/fd to d7/da/d17/d40/f42 0 2024-04-24T23:13:45.660 INFO:tasks.workunit.client.1.smithi040.stdout:9/178: dread d7/d1b/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:13:45.661 INFO:tasks.workunit.client.1.smithi040.stdout:9/179: write f4 [3478610,58696] 0 2024-04-24T23:13:45.662 INFO:tasks.workunit.client.1.smithi040.stdout:2/190: creat da/f40 x:0 0 0 2024-04-24T23:13:45.662 INFO:tasks.workunit.client.0.smithi031.stdout:6/614: write d13/d1b/f8f [776931,54866] 0 2024-04-24T23:13:45.666 INFO:tasks.workunit.client.1.smithi040.stdout:2/191: chown da/f20 117 1 2024-04-24T23:13:45.668 INFO:tasks.workunit.client.0.smithi031.stdout:6/615: rename d13/d15/d1d/d23/d2d/d9e/dab to d13/d1b/d32/db4/d79/dd7 0 2024-04-24T23:13:45.669 INFO:tasks.workunit.client.0.smithi031.stdout:6/616: readlink d13/d15/d1d/d23/d44/d47/lce 0 2024-04-24T23:13:45.671 INFO:tasks.workunit.client.1.smithi040.stdout:2/192: link da/fe da/d1c/d37/f41 0 2024-04-24T23:13:45.672 INFO:tasks.workunit.client.0.smithi031.stdout:6/617: dread d13/d15/d1d/d23/d24/d8b/fcd [0,4194304] 0 2024-04-24T23:13:45.679 INFO:tasks.workunit.client.0.smithi031.stdout:9/589: rename d2/d1b to d2/dae 0 2024-04-24T23:13:45.680 INFO:tasks.workunit.client.1.smithi040.stdout:0/191: write df/d12/f17 [4599974,4421] 0 2024-04-24T23:13:45.681 INFO:tasks.workunit.client.0.smithi031.stdout:0/524: rmdir da/d18/d1a/d37/d41 39 2024-04-24T23:13:45.683 INFO:tasks.workunit.client.1.smithi040.stdout:0/192: rmdir df/d14/d1b 39 2024-04-24T23:13:45.684 INFO:tasks.workunit.client.1.smithi040.stdout:0/193: fsync df/d1e/f2f 0 2024-04-24T23:13:45.684 INFO:tasks.workunit.client.0.smithi031.stdout:9/590: getdents d2/dae/d14/d78/d84/d4a/dab/dad 0 2024-04-24T23:13:45.685 INFO:tasks.workunit.client.1.smithi040.stdout:0/194: chown df/d1e/l2d 8 1 2024-04-24T23:13:45.686 INFO:tasks.workunit.client.1.smithi040.stdout:0/195: dread - df/d12/f22 zero size 2024-04-24T23:13:45.686 INFO:tasks.workunit.client.1.smithi040.stdout:4/111: dwrite f9 [0,4194304] 0 2024-04-24T23:13:45.687 INFO:tasks.workunit.client.1.smithi040.stdout:4/112: read da/fb [3828578,1112] 0 2024-04-24T23:13:45.688 INFO:tasks.workunit.client.1.smithi040.stdout:4/113: read - da/f28 zero size 2024-04-24T23:13:45.691 INFO:tasks.workunit.client.0.smithi031.stdout:9/591: mkdir d2/dae/d14/d78/d84/d4a/d4b/daf 0 2024-04-24T23:13:45.693 INFO:tasks.workunit.client.0.smithi031.stdout:9/592: chown d2/dae/d14/d18/f31 62176 1 2024-04-24T23:13:45.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/525: getdents da/d18/d1a/d76 0 2024-04-24T23:13:45.697 INFO:tasks.workunit.client.0.smithi031.stdout:2/357: dwrite d5/d6/f35 [0,4194304] 0 2024-04-24T23:13:45.697 INFO:tasks.workunit.client.1.smithi040.stdout:3/127: sync 2024-04-24T23:13:45.702 INFO:tasks.workunit.client.0.smithi031.stdout:9/593: mknod d2/dae/d14/d78/d84/da6/cb0 0 2024-04-24T23:13:45.704 INFO:tasks.workunit.client.1.smithi040.stdout:4/114: read f6 [1170,109429] 0 2024-04-24T23:13:45.704 INFO:tasks.workunit.client.1.smithi040.stdout:0/196: rename df/d1e/l2d to df/d12/d1a/l35 0 2024-04-24T23:13:45.706 INFO:tasks.workunit.client.0.smithi031.stdout:0/526: dwrite da/f99 [0,4194304] 0 2024-04-24T23:13:45.708 INFO:tasks.workunit.client.0.smithi031.stdout:0/527: dread - da/d86/faa zero size 2024-04-24T23:13:45.709 INFO:tasks.workunit.client.1.smithi040.stdout:3/128: mknod d1/d9/da/de/c29 0 2024-04-24T23:13:45.713 INFO:tasks.workunit.client.0.smithi031.stdout:2/358: dwrite d5/f24 [0,4194304] 0 2024-04-24T23:13:45.717 INFO:tasks.workunit.client.1.smithi040.stdout:0/197: dread df/d14/d1b/f28 [0,4194304] 0 2024-04-24T23:13:45.719 INFO:tasks.workunit.client.1.smithi040.stdout:1/117: write da/d12/d14/f24 [956369,104740] 0 2024-04-24T23:13:45.721 INFO:tasks.workunit.client.1.smithi040.stdout:6/128: write d7/fd [244409,114648] 0 2024-04-24T23:13:45.726 INFO:tasks.workunit.client.0.smithi031.stdout:9/594: creat d2/dae/d14/d78/d84/d4a/fb1 x:0 0 0 2024-04-24T23:13:45.728 INFO:tasks.workunit.client.1.smithi040.stdout:6/129: dwrite d7/de/d15/f1f [0,4194304] 0 2024-04-24T23:13:45.730 INFO:tasks.workunit.client.1.smithi040.stdout:0/198: rename df/d14/f16 to df/d14/d1b/f36 0 2024-04-24T23:13:45.730 INFO:tasks.workunit.client.0.smithi031.stdout:4/602: dwrite d3/f2b [0,4194304] 0 2024-04-24T23:13:45.731 INFO:tasks.workunit.client.1.smithi040.stdout:7/192: dwrite f2 [0,4194304] 0 2024-04-24T23:13:45.731 INFO:tasks.workunit.client.0.smithi031.stdout:2/359: creat d5/d6/d29/d4f/f77 x:0 0 0 2024-04-24T23:13:45.732 INFO:tasks.workunit.client.0.smithi031.stdout:2/360: stat d5/d6/d68 0 2024-04-24T23:13:45.732 INFO:tasks.workunit.client.0.smithi031.stdout:4/603: stat d3/d20/d2d/d44/d5b/l82 0 2024-04-24T23:13:45.733 INFO:tasks.workunit.client.1.smithi040.stdout:1/118: symlink da/d12/l2c 0 2024-04-24T23:13:45.735 INFO:tasks.workunit.client.0.smithi031.stdout:9/595: creat d2/dae/d14/d18/fb2 x:0 0 0 2024-04-24T23:13:45.739 INFO:tasks.workunit.client.1.smithi040.stdout:4/115: link da/l16 da/d1d/d2b/l2d 0 2024-04-24T23:13:45.739 INFO:tasks.workunit.client.1.smithi040.stdout:2/193: truncate f7 2107095 0 2024-04-24T23:13:45.743 INFO:tasks.workunit.client.1.smithi040.stdout:8/96: dwrite d0/d8/fd [0,4194304] 0 2024-04-24T23:13:45.743 INFO:tasks.workunit.client.1.smithi040.stdout:2/194: chown da/d1c/d37/d39/d3f 2719 1 2024-04-24T23:13:45.744 INFO:tasks.workunit.client.0.smithi031.stdout:2/361: symlink d5/d6/d29/l78 0 2024-04-24T23:13:45.745 INFO:tasks.workunit.client.1.smithi040.stdout:9/180: dwrite d7/fe [0,4194304] 0 2024-04-24T23:13:45.748 INFO:tasks.workunit.client.1.smithi040.stdout:4/116: dread f9 [0,4194304] 0 2024-04-24T23:13:45.748 INFO:tasks.workunit.client.1.smithi040.stdout:4/117: read f6 [78389,22434] 0 2024-04-24T23:13:45.749 INFO:tasks.workunit.client.0.smithi031.stdout:2/362: dread d5/d8/d45/d5b/d49/f6d [0,4194304] 0 2024-04-24T23:13:45.754 INFO:tasks.workunit.client.0.smithi031.stdout:6/618: dwrite d13/d15/d1d/d23/d24/f40 [0,4194304] 0 2024-04-24T23:13:45.757 INFO:tasks.workunit.client.0.smithi031.stdout:9/596: mknod d2/dae/d14/d78/d84/d4a/d4b/cb3 0 2024-04-24T23:13:45.762 INFO:tasks.workunit.client.1.smithi040.stdout:0/199: dread df/d14/d1b/f28 [0,4194304] 0 2024-04-24T23:13:45.763 INFO:tasks.workunit.client.1.smithi040.stdout:0/200: chown df/d12/f1f 24267 1 2024-04-24T23:13:45.763 INFO:tasks.workunit.client.1.smithi040.stdout:1/119: creat da/d12/d14/f2d x:0 0 0 2024-04-24T23:13:45.767 INFO:tasks.workunit.client.0.smithi031.stdout:2/363: creat d5/d6/df/f79 x:0 0 0 2024-04-24T23:13:45.776 INFO:tasks.workunit.client.0.smithi031.stdout:0/528: write da/d18/d1a/d37/d41/f83 [5043559,20100] 0 2024-04-24T23:13:45.776 INFO:tasks.workunit.client.1.smithi040.stdout:9/181: creat d7/d16/f43 x:0 0 0 2024-04-24T23:13:45.777 INFO:tasks.workunit.client.1.smithi040.stdout:9/182: chown d7/da/d17 32674123 1 2024-04-24T23:13:45.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/364: mknod d5/d8/d6e/d70/c7a 0 2024-04-24T23:13:45.778 INFO:tasks.workunit.client.1.smithi040.stdout:3/129: getdents d1/d9/da/de 0 2024-04-24T23:13:45.778 INFO:tasks.workunit.client.1.smithi040.stdout:3/130: dread - d1/d9/f21 zero size 2024-04-24T23:13:45.778 INFO:tasks.workunit.client.0.smithi031.stdout:6/619: symlink d13/d15/d1d/d23/d99/ld8 0 2024-04-24T23:13:45.779 INFO:tasks.workunit.client.0.smithi031.stdout:6/620: dread - d13/d15/d1d/d23/d44/d47/fca zero size 2024-04-24T23:13:45.781 INFO:tasks.workunit.client.0.smithi031.stdout:3/433: sync 2024-04-24T23:13:45.784 INFO:tasks.workunit.client.1.smithi040.stdout:4/118: mknod da/d1f/c2e 0 2024-04-24T23:13:45.785 INFO:tasks.workunit.client.0.smithi031.stdout:9/597: symlink d2/dae/d14/d1d/d45/d99/d9c/lb4 0 2024-04-24T23:13:45.786 INFO:tasks.workunit.client.1.smithi040.stdout:6/130: truncate d7/de/d15/d1e/f22 3494508 0 2024-04-24T23:13:45.787 INFO:tasks.workunit.client.0.smithi031.stdout:9/598: chown d2/dae/d14/d1d/c4c 922 1 2024-04-24T23:13:45.787 INFO:tasks.workunit.client.1.smithi040.stdout:6/131: write d7/f9 [3539865,44306] 0 2024-04-24T23:13:45.787 INFO:tasks.workunit.client.0.smithi031.stdout:0/529: creat da/d86/fb1 x:0 0 0 2024-04-24T23:13:45.790 INFO:tasks.workunit.client.0.smithi031.stdout:6/621: rename d13/d15/fa5 to d13/d1b/dd0/fd9 0 2024-04-24T23:13:45.791 INFO:tasks.workunit.client.0.smithi031.stdout:6/622: chown d13/d15/d1d/d23/d24/d43/fb3 14958 1 2024-04-24T23:13:45.792 INFO:tasks.workunit.client.0.smithi031.stdout:6/623: chown d13/d15/d1d/d23/d2d 0 1 2024-04-24T23:13:45.793 INFO:tasks.workunit.client.0.smithi031.stdout:3/434: dread - d6/f48 zero size 2024-04-24T23:13:45.797 INFO:tasks.workunit.client.0.smithi031.stdout:0/530: creat da/d18/d5a/fb2 x:0 0 0 2024-04-24T23:13:45.797 INFO:tasks.workunit.client.0.smithi031.stdout:0/531: readlink da/d18/d1a/d37/l89 0 2024-04-24T23:13:45.802 INFO:tasks.workunit.client.0.smithi031.stdout:3/435: unlink d6/df/d16/d53/c8c 0 2024-04-24T23:13:45.803 INFO:tasks.workunit.client.0.smithi031.stdout:3/436: chown d6/d72/d1a/d25/f74 12 1 2024-04-24T23:13:45.803 INFO:tasks.workunit.client.1.smithi040.stdout:9/183: chown d7/da/l21 7156667 1 2024-04-24T23:13:45.804 INFO:tasks.workunit.client.0.smithi031.stdout:9/599: mkdir d2/dae/d14/d78/d84/d4a/dab/dad/db5 0 2024-04-24T23:13:45.807 INFO:tasks.workunit.client.1.smithi040.stdout:4/119: creat da/f2f x:0 0 0 2024-04-24T23:13:45.808 INFO:tasks.workunit.client.1.smithi040.stdout:4/120: chown da/d1d/c29 26212816 1 2024-04-24T23:13:45.808 INFO:tasks.workunit.client.1.smithi040.stdout:4/121: stat f4 0 2024-04-24T23:13:45.810 INFO:tasks.workunit.client.0.smithi031.stdout:0/532: dwrite f6 [0,4194304] 0 2024-04-24T23:13:45.811 INFO:tasks.workunit.client.0.smithi031.stdout:3/437: symlink d6/d14/l8d 0 2024-04-24T23:13:45.811 INFO:tasks.workunit.client.1.smithi040.stdout:5/131: sync 2024-04-24T23:13:45.813 INFO:tasks.workunit.client.1.smithi040.stdout:6/132: unlink d7/c1d 0 2024-04-24T23:13:45.815 INFO:tasks.workunit.client.0.smithi031.stdout:9/600: write d2/dae/d14/d78/d84/d4a/f71 [3107192,51341] 0 2024-04-24T23:13:45.819 INFO:tasks.workunit.client.1.smithi040.stdout:1/120: write da/d12/f21 [4539552,119895] 0 2024-04-24T23:13:45.820 INFO:tasks.workunit.client.0.smithi031.stdout:3/438: mknod d6/df/d16/d2a/c8e 0 2024-04-24T23:13:45.821 INFO:tasks.workunit.client.1.smithi040.stdout:3/131: mkdir d1/d2a 0 2024-04-24T23:13:45.826 INFO:tasks.workunit.client.1.smithi040.stdout:6/133: creat d7/de/d15/f26 x:0 0 0 2024-04-24T23:13:45.827 INFO:tasks.workunit.client.1.smithi040.stdout:3/132: dread d1/f5 [0,4194304] 0 2024-04-24T23:13:45.831 INFO:tasks.workunit.client.1.smithi040.stdout:3/133: dread d1/d9/d15/f1b [0,4194304] 0 2024-04-24T23:13:45.834 INFO:tasks.workunit.client.0.smithi031.stdout:0/533: getdents da/d20 0 2024-04-24T23:13:45.834 INFO:tasks.workunit.client.1.smithi040.stdout:3/134: dread d1/f16 [0,4194304] 0 2024-04-24T23:13:45.835 INFO:tasks.workunit.client.0.smithi031.stdout:0/534: dread - da/d11/f7f zero size 2024-04-24T23:13:45.835 INFO:tasks.workunit.client.1.smithi040.stdout:3/135: stat d1/d9/d18/l12 0 2024-04-24T23:13:45.836 INFO:tasks.workunit.client.1.smithi040.stdout:9/184: link d7/d1b/d2f/c32 d7/da/d17/c44 0 2024-04-24T23:13:45.836 INFO:tasks.workunit.client.1.smithi040.stdout:1/121: symlink da/d12/d18/d2a/l2e 0 2024-04-24T23:13:45.839 INFO:tasks.workunit.client.1.smithi040.stdout:3/136: mkdir d1/d9/da/d2b 0 2024-04-24T23:13:45.841 INFO:tasks.workunit.client.1.smithi040.stdout:6/134: rmdir d7/de/d24 0 2024-04-24T23:13:45.843 INFO:tasks.workunit.client.1.smithi040.stdout:1/122: dread da/d12/d14/f16 [0,4194304] 0 2024-04-24T23:13:45.845 INFO:tasks.workunit.client.1.smithi040.stdout:3/137: dread d1/f16 [0,4194304] 0 2024-04-24T23:13:45.845 INFO:tasks.workunit.client.1.smithi040.stdout:3/138: readlink d1/d9/l1e 0 2024-04-24T23:13:45.845 INFO:tasks.workunit.client.1.smithi040.stdout:1/123: unlink da/d12/d14/f16 0 2024-04-24T23:13:45.846 INFO:tasks.workunit.client.1.smithi040.stdout:3/139: fsync d1/f3 0 2024-04-24T23:13:45.846 INFO:tasks.workunit.client.1.smithi040.stdout:1/124: read da/d12/d18/f27 [1830700,5796] 0 2024-04-24T23:13:45.848 INFO:tasks.workunit.client.1.smithi040.stdout:3/140: creat d1/d9/d18/f2c x:0 0 0 2024-04-24T23:13:45.848 INFO:tasks.workunit.client.1.smithi040.stdout:1/125: write da/ff [8831137,77020] 0 2024-04-24T23:13:45.850 INFO:tasks.workunit.client.1.smithi040.stdout:1/126: creat da/d12/d18/f2f x:0 0 0 2024-04-24T23:13:45.851 INFO:tasks.workunit.client.1.smithi040.stdout:1/127: dread - da/d12/d14/f2d zero size 2024-04-24T23:13:45.852 INFO:tasks.workunit.client.1.smithi040.stdout:1/128: dread - da/d12/d14/f2d zero size 2024-04-24T23:13:45.852 INFO:tasks.workunit.client.1.smithi040.stdout:1/129: chown da/d12/f21 112828084 1 2024-04-24T23:13:45.854 INFO:tasks.workunit.client.1.smithi040.stdout:8/97: sync 2024-04-24T23:13:45.858 INFO:tasks.workunit.client.1.smithi040.stdout:8/98: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:45.858 INFO:tasks.workunit.client.1.smithi040.stdout:8/99: stat d0/d8/fd 0 2024-04-24T23:13:45.859 INFO:tasks.workunit.client.1.smithi040.stdout:8/100: mkdir d0/d16/d1b 0 2024-04-24T23:13:45.860 INFO:tasks.workunit.client.1.smithi040.stdout:8/101: read d0/d8/dc/f17 [738356,54172] 0 2024-04-24T23:13:45.861 INFO:tasks.workunit.client.1.smithi040.stdout:8/102: symlink d0/l1c 0 2024-04-24T23:13:45.862 INFO:tasks.workunit.client.1.smithi040.stdout:8/103: chown d0/d16 767297 1 2024-04-24T23:13:45.862 INFO:tasks.workunit.client.1.smithi040.stdout:8/104: readlink d0/lb 0 2024-04-24T23:13:45.868 INFO:tasks.workunit.client.1.smithi040.stdout:8/105: dwrite d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:45.871 INFO:tasks.workunit.client.1.smithi040.stdout:8/106: unlink d0/d8/fd 0 2024-04-24T23:13:45.875 INFO:tasks.workunit.client.1.smithi040.stdout:8/107: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:45.878 INFO:tasks.workunit.client.1.smithi040.stdout:8/108: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:45.885 INFO:tasks.workunit.client.1.smithi040.stdout:8/109: dwrite d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:45.887 INFO:tasks.workunit.client.1.smithi040.stdout:8/110: mknod d0/d16/d19/c1d 0 2024-04-24T23:13:45.890 INFO:tasks.workunit.client.1.smithi040.stdout:8/111: link d0/l1a d0/d16/l1e 0 2024-04-24T23:13:45.893 INFO:tasks.workunit.client.1.smithi040.stdout:8/112: mkdir d0/d1f 0 2024-04-24T23:13:45.894 INFO:tasks.workunit.client.1.smithi040.stdout:8/113: write d0/d8/dc/f17 [2952709,53754] 0 2024-04-24T23:13:45.897 INFO:tasks.workunit.client.1.smithi040.stdout:8/114: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:45.899 INFO:tasks.workunit.client.1.smithi040.stdout:8/115: mknod d0/d16/c20 0 2024-04-24T23:13:45.904 INFO:tasks.workunit.client.1.smithi040.stdout:0/201: sync 2024-04-24T23:13:45.906 INFO:tasks.workunit.client.0.smithi031.stdout:0/535: dread da/d18/f9a [0,4194304] 0 2024-04-24T23:13:45.907 INFO:tasks.workunit.client.1.smithi040.stdout:0/202: unlink df/d12/f17 0 2024-04-24T23:13:45.909 INFO:tasks.workunit.client.1.smithi040.stdout:7/193: truncate db/d1b/f23 246147 0 2024-04-24T23:13:45.910 INFO:tasks.workunit.client.1.smithi040.stdout:7/194: readlink db/d12/l26 0 2024-04-24T23:13:45.912 INFO:tasks.workunit.client.0.smithi031.stdout:0/536: link da/d86/faa da/d18/fb3 0 2024-04-24T23:13:45.913 INFO:tasks.workunit.client.1.smithi040.stdout:2/195: write da/d1c/f27 [857892,16948] 0 2024-04-24T23:13:45.914 INFO:tasks.workunit.client.1.smithi040.stdout:2/196: write da/d10/d15/f30 [378198,71977] 0 2024-04-24T23:13:45.917 INFO:tasks.workunit.client.0.smithi031.stdout:0/537: unlink da/d11/l16 0 2024-04-24T23:13:45.917 INFO:tasks.workunit.client.0.smithi031.stdout:0/538: dread - da/d18/f85 zero size 2024-04-24T23:13:45.918 INFO:tasks.workunit.client.0.smithi031.stdout:0/539: write da/d18/f9f [673214,95472] 0 2024-04-24T23:13:45.922 INFO:tasks.workunit.client.1.smithi040.stdout:7/195: link f5 db/d1b/d44/f48 0 2024-04-24T23:13:45.923 INFO:tasks.workunit.client.0.smithi031.stdout:6/624: dwrite d13/f3f [0,4194304] 0 2024-04-24T23:13:45.924 INFO:tasks.workunit.client.0.smithi031.stdout:0/540: creat da/d22/d62/fb4 x:0 0 0 2024-04-24T23:13:45.925 INFO:tasks.workunit.client.1.smithi040.stdout:0/203: getdents df/d14/d1b 0 2024-04-24T23:13:45.925 INFO:tasks.workunit.client.0.smithi031.stdout:0/541: dread - da/d86/faa zero size 2024-04-24T23:13:45.925 INFO:tasks.workunit.client.0.smithi031.stdout:6/625: chown d13/d15/d1d/d23/d2d/f3d 994537 1 2024-04-24T23:13:45.926 INFO:tasks.workunit.client.0.smithi031.stdout:0/542: dread - da/d11/fa0 zero size 2024-04-24T23:13:45.927 INFO:tasks.workunit.client.0.smithi031.stdout:6/626: truncate d13/d1b/d32/fc3 397433 0 2024-04-24T23:13:45.928 INFO:tasks.workunit.client.1.smithi040.stdout:2/197: creat da/d29/f42 x:0 0 0 2024-04-24T23:13:45.931 INFO:tasks.workunit.client.1.smithi040.stdout:4/122: write da/f18 [4449365,77717] 0 2024-04-24T23:13:45.933 INFO:tasks.workunit.client.1.smithi040.stdout:0/204: symlink df/d1e/d2b/l37 0 2024-04-24T23:13:45.934 INFO:tasks.workunit.client.0.smithi031.stdout:0/543: dwrite da/d20/f44 [0,4194304] 0 2024-04-24T23:13:45.934 INFO:tasks.workunit.client.1.smithi040.stdout:6/135: getdents d7/de/d15 0 2024-04-24T23:13:45.935 INFO:tasks.workunit.client.1.smithi040.stdout:6/136: write d7/fd [805375,41] 0 2024-04-24T23:13:45.935 INFO:tasks.workunit.client.1.smithi040.stdout:6/137: readlink d7/l1c 0 2024-04-24T23:13:45.936 INFO:tasks.workunit.client.1.smithi040.stdout:6/138: chown d7/de/d15/f26 1975 1 2024-04-24T23:13:45.938 INFO:tasks.workunit.client.1.smithi040.stdout:2/198: symlink da/d1c/d37/l43 0 2024-04-24T23:13:45.939 INFO:tasks.workunit.client.0.smithi031.stdout:9/601: truncate d2/dae/d14/d1d/d45/f5e 2530253 0 2024-04-24T23:13:45.940 INFO:tasks.workunit.client.0.smithi031.stdout:6/627: unlink d13/d15/d1d/d23/d2d/f62 0 2024-04-24T23:13:45.941 INFO:tasks.workunit.client.0.smithi031.stdout:3/439: write d6/df/d16/f7f [763210,124711] 0 2024-04-24T23:13:45.944 INFO:tasks.workunit.client.1.smithi040.stdout:0/205: symlink df/d1e/d2b/l38 0 2024-04-24T23:13:45.947 INFO:tasks.workunit.client.1.smithi040.stdout:5/132: dwrite d2/d7/d8/db/d1e/f2e [0,4194304] 0 2024-04-24T23:13:45.949 INFO:tasks.workunit.client.1.smithi040.stdout:6/139: creat d7/de/d15/d1e/f27 x:0 0 0 2024-04-24T23:13:45.949 INFO:tasks.workunit.client.1.smithi040.stdout:3/141: rename d1/d9/da to d1/d2d 0 2024-04-24T23:13:45.952 INFO:tasks.workunit.client.1.smithi040.stdout:2/199: rmdir da/d10 39 2024-04-24T23:13:45.953 INFO:tasks.workunit.client.1.smithi040.stdout:6/140: dread d7/de/d15/f1f [0,4194304] 0 2024-04-24T23:13:45.955 INFO:tasks.workunit.client.1.smithi040.stdout:4/123: creat da/d1d/d2b/f30 x:0 0 0 2024-04-24T23:13:45.955 INFO:tasks.workunit.client.1.smithi040.stdout:9/185: truncate d7/d1b/d2f/d1f/d31/f36 5834271 0 2024-04-24T23:13:45.970 INFO:tasks.workunit.client.1.smithi040.stdout:2/200: fdatasync da/d10/d15/f30 0 2024-04-24T23:13:45.973 INFO:tasks.workunit.client.1.smithi040.stdout:4/124: creat da/d1d/d2b/f31 x:0 0 0 2024-04-24T23:13:45.975 INFO:tasks.workunit.client.1.smithi040.stdout:8/116: dwrite d0/d8/dc/f17 [4194304,4194304] 0 2024-04-24T23:13:45.981 INFO:tasks.workunit.client.1.smithi040.stdout:4/125: dwrite da/d1d/d2b/f30 [0,4194304] 0 2024-04-24T23:13:45.982 INFO:tasks.workunit.client.1.smithi040.stdout:4/126: write da/d1d/d1e/f23 [682864,2102] 0 2024-04-24T23:13:45.985 INFO:tasks.workunit.client.1.smithi040.stdout:1/130: rename da/d12/d18/f2f to da/d12/d23/f30 0 2024-04-24T23:13:45.986 INFO:tasks.workunit.client.1.smithi040.stdout:4/127: dread da/f1c [0,4194304] 0 2024-04-24T23:13:45.987 INFO:tasks.workunit.client.1.smithi040.stdout:3/142: mkdir d1/d9/d2e 0 2024-04-24T23:13:45.990 INFO:tasks.workunit.client.1.smithi040.stdout:1/131: dread da/d12/f1c [0,4194304] 0 2024-04-24T23:13:45.992 INFO:tasks.workunit.client.1.smithi040.stdout:2/201: symlink da/d1c/l44 0 2024-04-24T23:13:45.993 INFO:tasks.workunit.client.1.smithi040.stdout:2/202: dread - da/f40 zero size 2024-04-24T23:13:45.993 INFO:tasks.workunit.client.1.smithi040.stdout:3/143: dwrite d1/f3 [0,4194304] 0 2024-04-24T23:13:45.994 INFO:tasks.workunit.client.1.smithi040.stdout:3/144: dread - d1/d9/f21 zero size 2024-04-24T23:13:45.995 INFO:tasks.workunit.client.1.smithi040.stdout:3/145: chown d1/d2d/de/c29 1 1 2024-04-24T23:13:45.996 INFO:tasks.workunit.client.1.smithi040.stdout:3/146: chown d1/d2d/f24 1729155493 1 2024-04-24T23:13:46.002 INFO:tasks.workunit.client.1.smithi040.stdout:5/133: creat d2/d7/f3b x:0 0 0 2024-04-24T23:13:46.005 INFO:tasks.workunit.client.1.smithi040.stdout:8/117: mknod d0/d16/c21 0 2024-04-24T23:13:46.005 INFO:tasks.workunit.client.1.smithi040.stdout:3/147: dwrite d1/d9/f17 [0,4194304] 0 2024-04-24T23:13:46.005 INFO:tasks.workunit.client.1.smithi040.stdout:3/148: write d1/d9/d18/f2c [629806,60581] 0 2024-04-24T23:13:46.006 INFO:tasks.workunit.client.1.smithi040.stdout:7/196: sync 2024-04-24T23:13:46.008 INFO:tasks.workunit.client.0.smithi031.stdout:0/544: creat da/d18/fb5 x:0 0 0 2024-04-24T23:13:46.008 INFO:tasks.workunit.client.0.smithi031.stdout:0/545: readlink l2 0 2024-04-24T23:13:46.009 INFO:tasks.workunit.client.1.smithi040.stdout:8/118: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:46.011 INFO:tasks.workunit.client.1.smithi040.stdout:8/119: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:46.012 INFO:tasks.workunit.client.1.smithi040.stdout:8/120: stat d0 0 2024-04-24T23:13:46.018 INFO:tasks.workunit.client.0.smithi031.stdout:0/546: dwrite da/d18/d1a/d37/fa5 [0,4194304] 0 2024-04-24T23:13:46.022 INFO:tasks.workunit.client.1.smithi040.stdout:0/206: dwrite df/d1e/d2b/f29 [0,4194304] 0 2024-04-24T23:13:46.024 INFO:tasks.workunit.client.1.smithi040.stdout:4/128: creat da/d1d/f32 x:0 0 0 2024-04-24T23:13:46.025 INFO:tasks.workunit.client.1.smithi040.stdout:4/129: dread - da/d1d/f25 zero size 2024-04-24T23:13:46.030 INFO:tasks.workunit.client.0.smithi031.stdout:0/547: unlink da/d11/d6a/f8a 0 2024-04-24T23:13:46.031 INFO:tasks.workunit.client.1.smithi040.stdout:9/186: sync 2024-04-24T23:13:46.038 INFO:tasks.workunit.client.1.smithi040.stdout:3/149: fdatasync d1/fd 0 2024-04-24T23:13:46.040 INFO:tasks.workunit.client.1.smithi040.stdout:7/197: stat db/d2f/d37 0 2024-04-24T23:13:46.044 INFO:tasks.workunit.client.1.smithi040.stdout:1/132: mkdir da/d31 0 2024-04-24T23:13:46.046 INFO:tasks.workunit.client.1.smithi040.stdout:2/203: symlink da/d1c/d37/d39/d3f/l45 0 2024-04-24T23:13:46.048 INFO:tasks.workunit.client.1.smithi040.stdout:9/187: rename d7/da/d17/c2c to d7/da/d17/d40/c45 0 2024-04-24T23:13:46.049 INFO:tasks.workunit.client.1.smithi040.stdout:5/134: mkdir d2/d7/d8/db/d1e/d22/d27/d3c 0 2024-04-24T23:13:46.050 INFO:tasks.workunit.client.1.smithi040.stdout:5/135: write d2/d2f/d14/f17 [801643,58340] 0 2024-04-24T23:13:46.052 INFO:tasks.workunit.client.1.smithi040.stdout:6/141: truncate d7/de/ff 2852226 0 2024-04-24T23:13:46.054 INFO:tasks.workunit.client.1.smithi040.stdout:3/150: creat d1/d2d/de/f2f x:0 0 0 2024-04-24T23:13:46.056 INFO:tasks.workunit.client.1.smithi040.stdout:8/121: mkdir d0/d8/d22 0 2024-04-24T23:13:46.058 INFO:tasks.workunit.client.1.smithi040.stdout:8/122: chown d0/l2 1975143 1 2024-04-24T23:13:46.059 INFO:tasks.workunit.client.1.smithi040.stdout:5/136: dwrite d2/d7/d8/db/d1e/d22/d27/f32 [0,4194304] 0 2024-04-24T23:13:46.059 INFO:tasks.workunit.client.1.smithi040.stdout:2/204: symlink da/d1c/d37/d39/d3f/l46 0 2024-04-24T23:13:46.061 INFO:tasks.workunit.client.1.smithi040.stdout:9/188: creat d7/d1b/d2f/f46 x:0 0 0 2024-04-24T23:13:46.061 INFO:tasks.workunit.client.1.smithi040.stdout:6/142: dwrite d7/fb [0,4194304] 0 2024-04-24T23:13:46.063 INFO:tasks.workunit.client.1.smithi040.stdout:9/189: write f2 [5048367,67921] 0 2024-04-24T23:13:46.063 INFO:tasks.workunit.client.1.smithi040.stdout:5/137: chown d2/d7/d8/db/d1e/d22/c26 18531887 1 2024-04-24T23:13:46.063 INFO:tasks.workunit.client.0.smithi031.stdout:9/602: write d2/d43/f5d [1974739,49805] 0 2024-04-24T23:13:46.064 INFO:tasks.workunit.client.1.smithi040.stdout:7/198: creat db/d12/d3f/f49 x:0 0 0 2024-04-24T23:13:46.066 INFO:tasks.workunit.client.0.smithi031.stdout:6/628: write d13/d1b/d32/f8a [194505,90710] 0 2024-04-24T23:13:46.067 INFO:tasks.workunit.client.0.smithi031.stdout:9/603: readlink d2/d16/l38 0 2024-04-24T23:13:46.068 INFO:tasks.workunit.client.1.smithi040.stdout:9/190: write d7/f37 [214562,56952] 0 2024-04-24T23:13:46.068 INFO:tasks.workunit.client.1.smithi040.stdout:1/133: dread f2 [0,4194304] 0 2024-04-24T23:13:46.072 INFO:tasks.workunit.client.1.smithi040.stdout:0/207: creat df/f39 x:0 0 0 2024-04-24T23:13:46.072 INFO:tasks.workunit.client.1.smithi040.stdout:0/208: readlink df/l10 0 2024-04-24T23:13:46.074 INFO:tasks.workunit.client.0.smithi031.stdout:6/629: dread d13/d15/d1d/d23/d24/d8b/f9b [0,4194304] 0 2024-04-24T23:13:46.077 INFO:tasks.workunit.client.1.smithi040.stdout:2/205: mknod da/d10/d15/c47 0 2024-04-24T23:13:46.078 INFO:tasks.workunit.client.0.smithi031.stdout:9/604: dwrite d2/f42 [0,4194304] 0 2024-04-24T23:13:46.081 INFO:tasks.workunit.client.1.smithi040.stdout:6/143: creat d7/d1a/d20/f28 x:0 0 0 2024-04-24T23:13:46.081 INFO:tasks.workunit.client.0.smithi031.stdout:3/440: dwrite d6/d72/d1a/f22 [4194304,4194304] 0 2024-04-24T23:13:46.082 INFO:tasks.workunit.client.1.smithi040.stdout:2/206: dwrite da/f11 [0,4194304] 0 2024-04-24T23:13:46.083 INFO:tasks.workunit.client.1.smithi040.stdout:2/207: chown da/d1c/d37/d38/f3e 3651192 1 2024-04-24T23:13:46.084 INFO:tasks.workunit.client.0.smithi031.stdout:6/630: mknod d13/d15/d1d/d23/d24/d8b/cda 0 2024-04-24T23:13:46.091 INFO:tasks.workunit.client.0.smithi031.stdout:9/605: chown d2/dae/d14/d78/d84/d7e/l82 564 1 2024-04-24T23:13:46.091 INFO:tasks.workunit.client.0.smithi031.stdout:0/548: write da/d18/fb3 [702829,116632] 0 2024-04-24T23:13:46.095 INFO:tasks.workunit.client.1.smithi040.stdout:5/138: mkdir d2/d7/d8/db/d1e/d22/d27/d3d 0 2024-04-24T23:13:46.098 INFO:tasks.workunit.client.0.smithi031.stdout:6/631: unlink d13/d1b/f54 0 2024-04-24T23:13:46.099 INFO:tasks.workunit.client.0.smithi031.stdout:0/549: dwrite da/d18/d1a/d37/f4a [4194304,4194304] 0 2024-04-24T23:13:46.100 INFO:tasks.workunit.client.1.smithi040.stdout:4/130: dwrite da/f1a [0,4194304] 0 2024-04-24T23:13:46.101 INFO:tasks.workunit.client.1.smithi040.stdout:4/131: write da/f13 [3625377,32438] 0 2024-04-24T23:13:46.103 INFO:tasks.workunit.client.0.smithi031.stdout:4/604: sync 2024-04-24T23:13:46.103 INFO:tasks.workunit.client.0.smithi031.stdout:1/439: sync 2024-04-24T23:13:46.103 INFO:tasks.workunit.client.0.smithi031.stdout:5/436: sync 2024-04-24T23:13:46.103 INFO:tasks.workunit.client.0.smithi031.stdout:7/484: sync 2024-04-24T23:13:46.103 INFO:tasks.workunit.client.0.smithi031.stdout:8/591: sync 2024-04-24T23:13:46.103 INFO:tasks.workunit.client.0.smithi031.stdout:2/365: sync 2024-04-24T23:13:46.104 INFO:tasks.workunit.client.0.smithi031.stdout:0/550: write da/d11/f90 [3749885,101519] 0 2024-04-24T23:13:46.104 INFO:tasks.workunit.client.0.smithi031.stdout:9/606: unlink d2/f42 0 2024-04-24T23:13:46.106 INFO:tasks.workunit.client.0.smithi031.stdout:7/485: dread - d1/d45/d84/f96 zero size 2024-04-24T23:13:46.108 INFO:tasks.workunit.client.1.smithi040.stdout:1/134: readlink da/d12/l2c 0 2024-04-24T23:13:46.110 INFO:tasks.workunit.client.0.smithi031.stdout:8/592: chown d8/d1a/d1d/d35/d58/d47/f49 30466812 1 2024-04-24T23:13:46.112 INFO:tasks.workunit.client.1.smithi040.stdout:7/199: rename db/d2f/c32 to db/d2f/c4a 0 2024-04-24T23:13:46.113 INFO:tasks.workunit.client.0.smithi031.stdout:8/593: truncate d8/d1a/d1d/d50/dc0/d9d/fc6 823640 0 2024-04-24T23:13:46.116 INFO:tasks.workunit.client.0.smithi031.stdout:8/594: write d8/d1a/d1d/d35/d58/f4c [1072292,82966] 0 2024-04-24T23:13:46.117 INFO:tasks.workunit.client.0.smithi031.stdout:1/440: dread d1/d33/d36/f44 [0,4194304] 0 2024-04-24T23:13:46.117 INFO:tasks.workunit.client.0.smithi031.stdout:9/607: dwrite d2/dae/d14/d1d/f47 [0,4194304] 0 2024-04-24T23:13:46.119 INFO:tasks.workunit.client.1.smithi040.stdout:0/209: creat df/d12/d1a/f3a x:0 0 0 2024-04-24T23:13:46.119 INFO:tasks.workunit.client.0.smithi031.stdout:1/441: fdatasync d1/d33/d36/f44 0 2024-04-24T23:13:46.120 INFO:tasks.workunit.client.1.smithi040.stdout:0/210: stat c3 0 2024-04-24T23:13:46.120 INFO:tasks.workunit.client.1.smithi040.stdout:7/200: dwrite f2 [4194304,4194304] 0 2024-04-24T23:13:46.122 INFO:tasks.workunit.client.0.smithi031.stdout:6/632: mkdir d13/d1b/d32/d9c/ddb 0 2024-04-24T23:13:46.126 INFO:tasks.workunit.client.0.smithi031.stdout:6/633: read d13/d15/d1d/d23/d24/d43/f71 [136122,88310] 0 2024-04-24T23:13:46.126 INFO:tasks.workunit.client.1.smithi040.stdout:8/123: unlink d0/d8/l11 0 2024-04-24T23:13:46.132 INFO:tasks.workunit.client.1.smithi040.stdout:2/208: unlink da/d1c/l21 0 2024-04-24T23:13:46.133 INFO:tasks.workunit.client.0.smithi031.stdout:2/366: rmdir d5/d8/d45/d5b/d49 39 2024-04-24T23:13:46.135 INFO:tasks.workunit.client.1.smithi040.stdout:3/151: rmdir d1/d9/d15/d1c 0 2024-04-24T23:13:46.136 INFO:tasks.workunit.client.1.smithi040.stdout:3/152: write d1/d9/d18/f2c [1233689,71288] 0 2024-04-24T23:13:46.137 INFO:tasks.workunit.client.1.smithi040.stdout:3/153: write d1/d9/f20 [1838649,22926] 0 2024-04-24T23:13:46.140 INFO:tasks.workunit.client.1.smithi040.stdout:4/132: mknod da/d1d/d1e/c33 0 2024-04-24T23:13:46.142 INFO:tasks.workunit.client.1.smithi040.stdout:1/135: dread da/d12/d18/f1d [0,4194304] 0 2024-04-24T23:13:46.143 INFO:tasks.workunit.client.1.smithi040.stdout:1/136: write da/d12/d18/f22 [479013,106154] 0 2024-04-24T23:13:46.146 INFO:tasks.workunit.client.1.smithi040.stdout:9/191: creat d7/d1b/d2f/d1f/d34/f47 x:0 0 0 2024-04-24T23:13:46.152 INFO:tasks.workunit.client.1.smithi040.stdout:7/201: mkdir db/d1b/d4b 0 2024-04-24T23:13:46.152 INFO:tasks.workunit.client.0.smithi031.stdout:9/608: dread d2/f32 [0,4194304] 0 2024-04-24T23:13:46.154 INFO:tasks.workunit.client.0.smithi031.stdout:1/442: creat d1/d33/d36/f8d x:0 0 0 2024-04-24T23:13:46.155 INFO:tasks.workunit.client.1.smithi040.stdout:8/124: chown d0/d8/l9 10160 1 2024-04-24T23:13:46.159 INFO:tasks.workunit.client.0.smithi031.stdout:1/443: dwrite d1/f8 [4194304,4194304] 0 2024-04-24T23:13:46.159 INFO:tasks.workunit.client.1.smithi040.stdout:2/209: rename da/d10/f18 to da/d29/f48 0 2024-04-24T23:13:46.162 INFO:tasks.workunit.client.0.smithi031.stdout:3/441: truncate d6/df/d16/d57/f65 498368 0 2024-04-24T23:13:46.164 INFO:tasks.workunit.client.0.smithi031.stdout:6/634: truncate d13/d15/d1d/d23/d44/f80 325705 0 2024-04-24T23:13:46.165 INFO:tasks.workunit.client.1.smithi040.stdout:2/210: dwrite da/f40 [0,4194304] 0 2024-04-24T23:13:46.165 INFO:tasks.workunit.client.1.smithi040.stdout:3/154: rmdir d1 39 2024-04-24T23:13:46.166 INFO:tasks.workunit.client.1.smithi040.stdout:2/211: chown da/d10/d15/f2e 14778 1 2024-04-24T23:13:46.168 INFO:tasks.workunit.client.0.smithi031.stdout:1/444: dwrite d1/dc/f2b [0,4194304] 0 2024-04-24T23:13:46.176 INFO:tasks.workunit.client.0.smithi031.stdout:0/551: link da/d11/f33 da/d22/d62/d9c/fb6 0 2024-04-24T23:13:46.179 INFO:tasks.workunit.client.1.smithi040.stdout:6/144: sync 2024-04-24T23:13:46.181 INFO:tasks.workunit.client.1.smithi040.stdout:2/212: stat da/d10/l1f 0 2024-04-24T23:13:46.181 INFO:tasks.workunit.client.1.smithi040.stdout:1/137: mkdir da/d12/d18/d2a/d32 0 2024-04-24T23:13:46.185 INFO:tasks.workunit.client.0.smithi031.stdout:9/609: mknod d2/dae/d14/d18/cb6 0 2024-04-24T23:13:46.186 INFO:tasks.workunit.client.1.smithi040.stdout:5/139: write d2/d7/fe [3839257,67497] 0 2024-04-24T23:13:46.187 INFO:tasks.workunit.client.1.smithi040.stdout:5/140: dread - d2/d7/d8/d1f/f38 zero size 2024-04-24T23:13:46.187 INFO:tasks.workunit.client.0.smithi031.stdout:9/610: chown d2/dae/d14/d78/d84/d7e/l8b 2069234 1 2024-04-24T23:13:46.191 INFO:tasks.workunit.client.0.smithi031.stdout:5/437: dwrite d0/d6/d3b/d4b/dc/f23 [0,4194304] 0 2024-04-24T23:13:46.192 INFO:tasks.workunit.client.0.smithi031.stdout:3/442: unlink d6/d55/f73 0 2024-04-24T23:13:46.193 INFO:tasks.workunit.client.0.smithi031.stdout:5/438: write d0/d6/d12/f71 [405249,124036] 0 2024-04-24T23:13:46.193 INFO:tasks.workunit.client.0.smithi031.stdout:3/443: chown d6/d14 13670721 1 2024-04-24T23:13:46.197 INFO:tasks.workunit.client.0.smithi031.stdout:1/445: dread d1/da/f2c [0,4194304] 0 2024-04-24T23:13:46.198 INFO:tasks.workunit.client.0.smithi031.stdout:1/446: readlink d1/l50 0 2024-04-24T23:13:46.201 INFO:tasks.workunit.client.0.smithi031.stdout:6/635: rename d13/d15/l19 to d13/d15/d1d/d23/d4d/d7b/ldc 0 2024-04-24T23:13:46.202 INFO:tasks.workunit.client.1.smithi040.stdout:0/211: write df/d12/f25 [5495,118305] 0 2024-04-24T23:13:46.206 INFO:tasks.workunit.client.1.smithi040.stdout:6/145: symlink d7/d1a/d20/l29 0 2024-04-24T23:13:46.208 INFO:tasks.workunit.client.0.smithi031.stdout:0/552: mkdir da/d18/d1a/db7 0 2024-04-24T23:13:46.209 INFO:tasks.workunit.client.1.smithi040.stdout:1/138: mknod da/d12/d18/c33 0 2024-04-24T23:13:46.210 INFO:tasks.workunit.client.1.smithi040.stdout:1/139: stat da/d12/d18/f1d 0 2024-04-24T23:13:46.211 INFO:tasks.workunit.client.1.smithi040.stdout:8/125: symlink d0/d1f/l23 0 2024-04-24T23:13:46.212 INFO:tasks.workunit.client.0.smithi031.stdout:0/553: dread f6 [0,4194304] 0 2024-04-24T23:13:46.213 INFO:tasks.workunit.client.1.smithi040.stdout:6/146: dwrite d7/d1a/d20/f28 [0,4194304] 0 2024-04-24T23:13:46.214 INFO:tasks.workunit.client.1.smithi040.stdout:3/155: mkdir d1/d2d/de/d30 0 2024-04-24T23:13:46.215 INFO:tasks.workunit.client.0.smithi031.stdout:9/611: mkdir d2/dae/d14/d78/d84/d4a/dab/db7 0 2024-04-24T23:13:46.216 INFO:tasks.workunit.client.0.smithi031.stdout:9/612: fsync d2/f92 0 2024-04-24T23:13:46.221 INFO:tasks.workunit.client.1.smithi040.stdout:2/213: unlink da/l36 0 2024-04-24T23:13:46.221 INFO:tasks.workunit.client.1.smithi040.stdout:2/214: readlink da/d1c/d37/l43 0 2024-04-24T23:13:46.223 INFO:tasks.workunit.client.0.smithi031.stdout:3/444: mknod d6/d72/d60/c8f 0 2024-04-24T23:13:46.226 INFO:tasks.workunit.client.0.smithi031.stdout:5/439: mknod d0/d6/d3b/d4b/dc/c94 0 2024-04-24T23:13:46.226 INFO:tasks.workunit.client.0.smithi031.stdout:5/440: chown d0/f88 4 1 2024-04-24T23:13:46.229 INFO:tasks.workunit.client.0.smithi031.stdout:1/447: creat d1/d1d/f8e x:0 0 0 2024-04-24T23:13:46.239 INFO:tasks.workunit.client.1.smithi040.stdout:1/140: rmdir da/d12/d18 39 2024-04-24T23:13:46.239 INFO:tasks.workunit.client.1.smithi040.stdout:6/147: mkdir d7/de/d15/d2a 0 2024-04-24T23:13:46.240 INFO:tasks.workunit.client.1.smithi040.stdout:6/148: readlink d7/de/l14 0 2024-04-24T23:13:46.241 INFO:tasks.workunit.client.0.smithi031.stdout:7/486: getdents d1 0 2024-04-24T23:13:46.243 INFO:tasks.workunit.client.1.smithi040.stdout:2/215: symlink da/d1c/d37/d39/d3f/l49 0 2024-04-24T23:13:46.247 INFO:tasks.workunit.client.1.smithi040.stdout:7/202: link c9 db/c4c 0 2024-04-24T23:13:46.252 INFO:tasks.workunit.client.1.smithi040.stdout:3/156: mknod d1/d9/c31 0 2024-04-24T23:13:46.253 INFO:tasks.workunit.client.0.smithi031.stdout:3/445: truncate d6/f48 990667 0 2024-04-24T23:13:46.253 INFO:tasks.workunit.client.1.smithi040.stdout:1/141: fsync da/f1f 0 2024-04-24T23:13:46.257 INFO:tasks.workunit.client.1.smithi040.stdout:6/149: mknod d7/d1a/c2b 0 2024-04-24T23:13:46.258 INFO:tasks.workunit.client.1.smithi040.stdout:0/212: mknod df/d1e/d20/d21/c3b 0 2024-04-24T23:13:46.258 INFO:tasks.workunit.client.0.smithi031.stdout:2/367: link d5/d8/d6e/d70/c7a d5/d8/d45/d5b/d49/d6f/c7b 0 2024-04-24T23:13:46.259 INFO:tasks.workunit.client.1.smithi040.stdout:6/150: chown d7/de/d15/f17 2894872 1 2024-04-24T23:13:46.260 INFO:tasks.workunit.client.1.smithi040.stdout:6/151: readlink d7/de/l19 0 2024-04-24T23:13:46.261 INFO:tasks.workunit.client.0.smithi031.stdout:0/554: write da/d11/f59 [556691,84954] 0 2024-04-24T23:13:46.261 INFO:tasks.workunit.client.1.smithi040.stdout:1/142: dwrite da/d12/d14/f2d [0,4194304] 0 2024-04-24T23:13:46.262 INFO:tasks.workunit.client.1.smithi040.stdout:7/203: symlink db/d1b/d44/l4d 0 2024-04-24T23:13:46.262 INFO:tasks.workunit.client.1.smithi040.stdout:4/133: sync 2024-04-24T23:13:46.264 INFO:tasks.workunit.client.1.smithi040.stdout:7/204: readlink db/d12/l1f 0 2024-04-24T23:13:46.264 INFO:tasks.workunit.client.0.smithi031.stdout:7/487: symlink d1/d2/dd/lab 0 2024-04-24T23:13:46.264 INFO:tasks.workunit.client.1.smithi040.stdout:6/152: write d7/d1a/d20/f23 [997044,43818] 0 2024-04-24T23:13:46.265 INFO:tasks.workunit.client.0.smithi031.stdout:7/488: stat d1/d2/d83 0 2024-04-24T23:13:46.265 INFO:tasks.workunit.client.1.smithi040.stdout:8/126: dwrite d0/d8/dc/f17 [8388608,4194304] 0 2024-04-24T23:13:46.265 INFO:tasks.workunit.client.0.smithi031.stdout:7/489: chown d1/d2/d60/la7 4986648 1 2024-04-24T23:13:46.266 INFO:tasks.workunit.client.1.smithi040.stdout:8/127: chown d0/d16/d19 2993 1 2024-04-24T23:13:46.267 INFO:tasks.workunit.client.1.smithi040.stdout:7/205: write db/d12/d20/f22 [596865,121892] 0 2024-04-24T23:13:46.271 INFO:tasks.workunit.client.0.smithi031.stdout:3/446: creat d6/d14/d1d/d59/f90 x:0 0 0 2024-04-24T23:13:46.271 INFO:tasks.workunit.client.1.smithi040.stdout:6/153: truncate d7/de/d15/d1e/f27 768427 0 2024-04-24T23:13:46.272 INFO:tasks.workunit.client.0.smithi031.stdout:3/447: write d6/d72/d60/f64 [1501243,122421] 0 2024-04-24T23:13:46.275 INFO:tasks.workunit.client.1.smithi040.stdout:4/134: rmdir da/d1d 39 2024-04-24T23:13:46.277 INFO:tasks.workunit.client.0.smithi031.stdout:1/448: unlink d1/d33/d41/c8c 0 2024-04-24T23:13:46.278 INFO:tasks.workunit.client.0.smithi031.stdout:1/449: dread - d1/d1d/f8e zero size 2024-04-24T23:13:46.279 INFO:tasks.workunit.client.0.smithi031.stdout:2/368: creat d5/f7c x:0 0 0 2024-04-24T23:13:46.280 INFO:tasks.workunit.client.1.smithi040.stdout:0/213: symlink df/l3c 0 2024-04-24T23:13:46.281 INFO:tasks.workunit.client.0.smithi031.stdout:0/555: symlink da/d20/lb8 0 2024-04-24T23:13:46.286 INFO:tasks.workunit.client.1.smithi040.stdout:7/206: rmdir db/d12/d3f 39 2024-04-24T23:13:46.287 INFO:tasks.workunit.client.1.smithi040.stdout:7/207: chown db/d12/c2d 1077378 1 2024-04-24T23:13:46.288 INFO:tasks.workunit.client.0.smithi031.stdout:0/556: dwrite da/fd [4194304,4194304] 0 2024-04-24T23:13:46.288 INFO:tasks.workunit.client.0.smithi031.stdout:9/613: truncate d2/dae/d14/d1d/f24 3237840 0 2024-04-24T23:13:46.289 INFO:tasks.workunit.client.0.smithi031.stdout:3/448: rmdir d6/df/d16/d53 39 2024-04-24T23:13:46.291 INFO:tasks.workunit.client.1.smithi040.stdout:3/157: dread d1/f8 [0,4194304] 0 2024-04-24T23:13:46.295 INFO:tasks.workunit.client.1.smithi040.stdout:4/135: rmdir da 39 2024-04-24T23:13:46.297 INFO:tasks.workunit.client.0.smithi031.stdout:1/450: creat d1/d33/d36/f8f x:0 0 0 2024-04-24T23:13:46.298 INFO:tasks.workunit.client.1.smithi040.stdout:0/214: write df/d14/d1b/f36 [895425,56839] 0 2024-04-24T23:13:46.298 INFO:tasks.workunit.client.0.smithi031.stdout:5/441: dwrite d0/d6/d32/f39 [0,4194304] 0 2024-04-24T23:13:46.299 INFO:tasks.workunit.client.1.smithi040.stdout:9/192: sync 2024-04-24T23:13:46.300 INFO:tasks.workunit.client.1.smithi040.stdout:2/216: getdents da/d1c/d37 0 2024-04-24T23:13:46.300 INFO:tasks.workunit.client.1.smithi040.stdout:9/193: readlink d7/da/d17/l1d 0 2024-04-24T23:13:46.301 INFO:tasks.workunit.client.1.smithi040.stdout:9/194: stat d7/da/d17/l1d 0 2024-04-24T23:13:46.302 INFO:tasks.workunit.client.0.smithi031.stdout:7/490: link d1/d2/f12 d1/d2/dd/d33/d5e/d64/d6d/fac 0 2024-04-24T23:13:46.308 INFO:tasks.workunit.client.1.smithi040.stdout:7/208: creat db/d43/f4e x:0 0 0 2024-04-24T23:13:46.309 INFO:tasks.workunit.client.0.smithi031.stdout:9/614: mkdir d2/dae/d14/d78/d84/d4a/dab/db8 0 2024-04-24T23:13:46.310 INFO:tasks.workunit.client.0.smithi031.stdout:9/615: chown d2/dae/f41 7857351 1 2024-04-24T23:13:46.311 INFO:tasks.workunit.client.1.smithi040.stdout:6/154: dwrite f2 [0,4194304] 0 2024-04-24T23:13:46.315 INFO:tasks.workunit.client.0.smithi031.stdout:3/449: dread d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:46.317 INFO:tasks.workunit.client.0.smithi031.stdout:1/451: mkdir d1/da/d61/d90 0 2024-04-24T23:13:46.320 INFO:tasks.workunit.client.0.smithi031.stdout:1/452: read d1/f8 [1853195,121207] 0 2024-04-24T23:13:46.321 INFO:tasks.workunit.client.0.smithi031.stdout:0/557: dwrite da/d18/d1a/d37/f75 [0,4194304] 0 2024-04-24T23:13:46.322 INFO:tasks.workunit.client.1.smithi040.stdout:2/217: creat da/d29/f4a x:0 0 0 2024-04-24T23:13:46.323 INFO:tasks.workunit.client.0.smithi031.stdout:0/558: chown da/d18/d1a/f63 3518800 1 2024-04-24T23:13:46.328 INFO:tasks.workunit.client.0.smithi031.stdout:1/453: dread d1/dc/f2b [0,4194304] 0 2024-04-24T23:13:46.329 INFO:tasks.workunit.client.0.smithi031.stdout:3/450: stat d6/df/d16/d57/c5c 0 2024-04-24T23:13:46.332 INFO:tasks.workunit.client.0.smithi031.stdout:9/616: write d2/d16/f33 [1554132,84485] 0 2024-04-24T23:13:46.332 INFO:tasks.workunit.client.0.smithi031.stdout:3/451: write d6/d14/d1d/d59/f6c [226881,45908] 0 2024-04-24T23:13:46.333 INFO:tasks.workunit.client.0.smithi031.stdout:0/559: dwrite da/d11/f59 [0,4194304] 0 2024-04-24T23:13:46.334 INFO:tasks.workunit.client.1.smithi040.stdout:1/143: rename da/c28 to da/c34 0 2024-04-24T23:13:46.335 INFO:tasks.workunit.client.1.smithi040.stdout:1/144: fsync da/d12/d14/f24 0 2024-04-24T23:13:46.339 INFO:tasks.workunit.client.1.smithi040.stdout:2/218: symlink da/d1c/d37/l4b 0 2024-04-24T23:13:46.340 INFO:tasks.workunit.client.1.smithi040.stdout:9/195: truncate d7/da/d17/d40/f42 1466856 0 2024-04-24T23:13:46.341 INFO:tasks.workunit.client.1.smithi040.stdout:8/128: link d0/d16/l1e d0/d8/d22/l24 0 2024-04-24T23:13:46.343 INFO:tasks.workunit.client.1.smithi040.stdout:7/209: creat db/d1b/d4b/f4f x:0 0 0 2024-04-24T23:13:46.346 INFO:tasks.workunit.client.1.smithi040.stdout:3/158: creat d1/f32 x:0 0 0 2024-04-24T23:13:46.348 INFO:tasks.workunit.client.1.smithi040.stdout:8/129: dwrite d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:46.348 INFO:tasks.workunit.client.1.smithi040.stdout:6/155: mkdir d7/d2c 0 2024-04-24T23:13:46.349 INFO:tasks.workunit.client.0.smithi031.stdout:9/617: mknod d2/dae/d14/d78/cb9 0 2024-04-24T23:13:46.349 INFO:tasks.workunit.client.1.smithi040.stdout:8/130: chown d0/d8/l9 20727368 1 2024-04-24T23:13:46.351 INFO:tasks.workunit.client.1.smithi040.stdout:1/145: chown da/d12/d18/c33 6 1 2024-04-24T23:13:46.352 INFO:tasks.workunit.client.1.smithi040.stdout:1/146: dread - da/d12/d23/f30 zero size 2024-04-24T23:13:46.353 INFO:tasks.workunit.client.1.smithi040.stdout:0/215: rename df/d14/d1b/l31 to df/l3d 0 2024-04-24T23:13:46.353 INFO:tasks.workunit.client.1.smithi040.stdout:6/156: read d7/de/d15/d1e/f27 [501813,4494] 0 2024-04-24T23:13:46.355 INFO:tasks.workunit.client.1.smithi040.stdout:5/141: sync 2024-04-24T23:13:46.356 INFO:tasks.workunit.client.0.smithi031.stdout:7/491: creat d1/d45/d75/d9f/fad x:0 0 0 2024-04-24T23:13:46.357 INFO:tasks.workunit.client.1.smithi040.stdout:8/131: dwrite d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:46.359 INFO:tasks.workunit.client.1.smithi040.stdout:7/210: mknod db/d12/d20/c50 0 2024-04-24T23:13:46.359 INFO:tasks.workunit.client.0.smithi031.stdout:3/452: getdents d6/d72/d89 0 2024-04-24T23:13:46.360 INFO:tasks.workunit.client.0.smithi031.stdout:3/453: dread - d6/d72/d1a/d25/f74 zero size 2024-04-24T23:13:46.361 INFO:tasks.workunit.client.1.smithi040.stdout:3/159: fsync d1/d2d/f24 0 2024-04-24T23:13:46.367 INFO:tasks.workunit.client.1.smithi040.stdout:8/132: dwrite d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:46.370 INFO:tasks.workunit.client.0.smithi031.stdout:0/560: creat da/d20/d8e/fb9 x:0 0 0 2024-04-24T23:13:46.371 INFO:tasks.workunit.client.1.smithi040.stdout:7/211: dwrite db/d12/f29 [0,4194304] 0 2024-04-24T23:13:46.371 INFO:tasks.workunit.client.0.smithi031.stdout:9/618: dread d2/dae/f10 [0,4194304] 0 2024-04-24T23:13:46.372 INFO:tasks.workunit.client.0.smithi031.stdout:7/492: mkdir d1/d2/dd/d33/d5e/d64/d6d/dae 0 2024-04-24T23:13:46.373 INFO:tasks.workunit.client.0.smithi031.stdout:9/619: chown d2/dae/d14/d78/d84/da6/caa 849195 1 2024-04-24T23:13:46.374 INFO:tasks.workunit.client.0.smithi031.stdout:7/493: read d1/d45/d75/d9f/f80 [93681,92672] 0 2024-04-24T23:13:46.375 INFO:tasks.workunit.client.1.smithi040.stdout:9/196: dread d7/da/f39 [0,4194304] 0 2024-04-24T23:13:46.381 INFO:tasks.workunit.client.1.smithi040.stdout:0/216: write f6 [2043879,28531] 0 2024-04-24T23:13:46.386 INFO:tasks.workunit.client.0.smithi031.stdout:7/494: dwrite d1/d2/dd/f7d [0,4194304] 0 2024-04-24T23:13:46.386 INFO:tasks.workunit.client.1.smithi040.stdout:4/136: rename da/d1d/f25 to da/d1d/d1e/f34 0 2024-04-24T23:13:46.386 INFO:tasks.workunit.client.1.smithi040.stdout:2/219: rename da to da/d29/d4c 22 2024-04-24T23:13:46.388 INFO:tasks.workunit.client.0.smithi031.stdout:0/561: mkdir da/d11/d6a/dba 0 2024-04-24T23:13:46.391 INFO:tasks.workunit.client.0.smithi031.stdout:9/620: rename d2/c44 to d2/dae/d14/d78/d84/d4a/dab/dad/db5/cba 0 2024-04-24T23:13:46.393 INFO:tasks.workunit.client.0.smithi031.stdout:5/442: truncate d0/d6/d3b/d4b/dc/f47 98992 0 2024-04-24T23:13:46.394 INFO:tasks.workunit.client.0.smithi031.stdout:1/454: dwrite d1/dc/d11/f38 [0,4194304] 0 2024-04-24T23:13:46.396 INFO:tasks.workunit.client.0.smithi031.stdout:7/495: truncate d1/f28 674470 0 2024-04-24T23:13:46.397 INFO:tasks.workunit.client.0.smithi031.stdout:7/496: dread d1/d45/d75/d9f/f80 [0,4194304] 0 2024-04-24T23:13:46.398 INFO:tasks.workunit.client.1.smithi040.stdout:2/220: dwrite da/d10/f2b [0,4194304] 0 2024-04-24T23:13:46.400 INFO:tasks.workunit.client.0.smithi031.stdout:7/497: truncate d1/d45/d75/d9f/d65/f98 113848 0 2024-04-24T23:13:46.401 INFO:tasks.workunit.client.0.smithi031.stdout:7/498: chown d1/d2/d60 0 1 2024-04-24T23:13:46.408 INFO:tasks.workunit.client.1.smithi040.stdout:8/133: symlink d0/d16/l25 0 2024-04-24T23:13:46.409 INFO:tasks.workunit.client.1.smithi040.stdout:8/134: fdatasync d0/d8/dc/f17 0 2024-04-24T23:13:46.410 INFO:tasks.workunit.client.1.smithi040.stdout:7/212: symlink db/d2f/l51 0 2024-04-24T23:13:46.411 INFO:tasks.workunit.client.1.smithi040.stdout:9/197: creat d7/d16/f48 x:0 0 0 2024-04-24T23:13:46.411 INFO:tasks.workunit.client.1.smithi040.stdout:1/147: mkdir da/d35 0 2024-04-24T23:13:46.413 INFO:tasks.workunit.client.1.smithi040.stdout:4/137: creat da/d1d/d2b/f35 x:0 0 0 2024-04-24T23:13:46.417 INFO:tasks.workunit.client.1.smithi040.stdout:2/221: mkdir da/d10/d4d 0 2024-04-24T23:13:46.417 INFO:tasks.workunit.client.1.smithi040.stdout:0/217: dwrite df/d12/d1a/f2e [0,4194304] 0 2024-04-24T23:13:46.418 INFO:tasks.workunit.client.1.smithi040.stdout:3/160: mknod d1/d9/c33 0 2024-04-24T23:13:46.419 INFO:tasks.workunit.client.1.smithi040.stdout:2/222: write da/d10/d15/f2c [294503,14184] 0 2024-04-24T23:13:46.419 INFO:tasks.workunit.client.1.smithi040.stdout:3/161: chown d1/d9/f17 81699 1 2024-04-24T23:13:46.419 INFO:tasks.workunit.client.1.smithi040.stdout:8/135: mknod d0/d16/d19/c26 0 2024-04-24T23:13:46.421 INFO:tasks.workunit.client.1.smithi040.stdout:9/198: creat d7/d1b/d2f/d1f/d34/f49 x:0 0 0 2024-04-24T23:13:46.422 INFO:tasks.workunit.client.1.smithi040.stdout:6/157: link d7/de/c13 d7/d2c/c2d 0 2024-04-24T23:13:46.422 INFO:tasks.workunit.client.1.smithi040.stdout:8/136: chown d0/l2 3495 1 2024-04-24T23:13:46.424 INFO:tasks.workunit.client.1.smithi040.stdout:3/162: creat d1/d2d/de/f34 x:0 0 0 2024-04-24T23:13:46.424 INFO:tasks.workunit.client.1.smithi040.stdout:6/158: chown d7/fb 128086 1 2024-04-24T23:13:46.424 INFO:tasks.workunit.client.1.smithi040.stdout:4/138: mknod da/c36 0 2024-04-24T23:13:46.426 INFO:tasks.workunit.client.1.smithi040.stdout:9/199: mknod d7/da/c4a 0 2024-04-24T23:13:46.426 INFO:tasks.workunit.client.1.smithi040.stdout:6/159: write f2 [73266,8646] 0 2024-04-24T23:13:46.427 INFO:tasks.workunit.client.1.smithi040.stdout:4/139: chown da/d1f/f2c 176 1 2024-04-24T23:13:46.428 INFO:tasks.workunit.client.1.smithi040.stdout:1/148: dread da/d12/d18/f1d [0,4194304] 0 2024-04-24T23:13:46.429 INFO:tasks.workunit.client.0.smithi031.stdout:4/605: sync 2024-04-24T23:13:46.430 INFO:tasks.workunit.client.0.smithi031.stdout:7/499: stat d1/d2/dd/f15 0 2024-04-24T23:13:46.430 INFO:tasks.workunit.client.1.smithi040.stdout:4/140: truncate da/d1d/f32 4363 0 2024-04-24T23:13:46.430 INFO:tasks.workunit.client.0.smithi031.stdout:5/443: mkdir d0/d7/d95 0 2024-04-24T23:13:46.430 INFO:tasks.workunit.client.1.smithi040.stdout:8/137: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:46.432 INFO:tasks.workunit.client.1.smithi040.stdout:3/163: dread d1/d2d/de/f26 [0,4194304] 0 2024-04-24T23:13:46.432 INFO:tasks.workunit.client.0.smithi031.stdout:1/455: write d1/dc/d11/f20 [1518013,65950] 0 2024-04-24T23:13:46.434 INFO:tasks.workunit.client.1.smithi040.stdout:4/141: dread da/d1d/d2b/f30 [0,4194304] 0 2024-04-24T23:13:46.435 INFO:tasks.workunit.client.1.smithi040.stdout:8/138: dread d0/d8/dc/f17 [4194304,4194304] 0 2024-04-24T23:13:46.437 INFO:tasks.workunit.client.1.smithi040.stdout:0/218: creat df/d1e/f3e x:0 0 0 2024-04-24T23:13:46.437 INFO:tasks.workunit.client.1.smithi040.stdout:9/200: symlink d7/d1b/d2f/d1f/l4b 0 2024-04-24T23:13:46.438 INFO:tasks.workunit.client.1.smithi040.stdout:6/160: unlink d7/de/d15/d1e/f27 0 2024-04-24T23:13:46.440 INFO:tasks.workunit.client.1.smithi040.stdout:9/201: rename d7/d1b/d2f to d7/d1b/d2f/d4c 22 2024-04-24T23:13:46.440 INFO:tasks.workunit.client.1.smithi040.stdout:4/142: dread da/d1d/f21 [0,4194304] 0 2024-04-24T23:13:46.441 INFO:tasks.workunit.client.1.smithi040.stdout:9/202: readlink d7/d1b/d2f/d1f/l4b 0 2024-04-24T23:13:46.442 INFO:tasks.workunit.client.1.smithi040.stdout:4/143: write da/d1d/d2b/f30 [1563535,40012] 0 2024-04-24T23:13:46.445 INFO:tasks.workunit.client.0.smithi031.stdout:1/456: creat d1/d33/d36/f91 x:0 0 0 2024-04-24T23:13:46.448 INFO:tasks.workunit.client.0.smithi031.stdout:7/500: creat d1/d2/d83/faf x:0 0 0 2024-04-24T23:13:46.452 INFO:tasks.workunit.client.1.smithi040.stdout:0/219: creat df/d1e/d20/f3f x:0 0 0 2024-04-24T23:13:46.453 INFO:tasks.workunit.client.1.smithi040.stdout:1/149: symlink da/d35/l36 0 2024-04-24T23:13:46.454 INFO:tasks.workunit.client.1.smithi040.stdout:1/150: chown da/d12/f1c 5631029 1 2024-04-24T23:13:46.454 INFO:tasks.workunit.client.1.smithi040.stdout:1/151: chown da/d12/d18/c2b 7 1 2024-04-24T23:13:46.458 INFO:tasks.workunit.client.1.smithi040.stdout:6/161: creat d7/de/d15/f2e x:0 0 0 2024-04-24T23:13:46.460 INFO:tasks.workunit.client.1.smithi040.stdout:1/152: dwrite da/f1f [0,4194304] 0 2024-04-24T23:13:46.461 INFO:tasks.workunit.client.0.smithi031.stdout:5/444: rename d0/d6/d32/f46 to d0/d6/d3b/d4b/f96 0 2024-04-24T23:13:46.462 INFO:tasks.workunit.client.0.smithi031.stdout:5/445: chown d0/d6/d3b/d4b/d1d/c2f 14634 1 2024-04-24T23:13:46.466 INFO:tasks.workunit.client.0.smithi031.stdout:1/457: dread d1/d33/f47 [0,4194304] 0 2024-04-24T23:13:46.467 INFO:tasks.workunit.client.0.smithi031.stdout:0/562: write da/d22/f54 [1000610,35709] 0 2024-04-24T23:13:46.468 INFO:tasks.workunit.client.0.smithi031.stdout:1/458: dread - d1/da/d12/d7e/d7b/d4e/f72 zero size 2024-04-24T23:13:46.471 INFO:tasks.workunit.client.0.smithi031.stdout:5/446: symlink d0/d6/d3b/d4b/d1d/d2e/d28/l97 0 2024-04-24T23:13:46.473 INFO:tasks.workunit.client.0.smithi031.stdout:1/459: mknod d1/da/d61/c92 0 2024-04-24T23:13:46.476 INFO:tasks.workunit.client.0.smithi031.stdout:5/447: creat d0/d6/d3b/d4f/f98 x:0 0 0 2024-04-24T23:13:46.477 INFO:tasks.workunit.client.0.smithi031.stdout:0/563: dwrite da/d18/d5a/fb2 [0,4194304] 0 2024-04-24T23:13:46.478 INFO:tasks.workunit.client.1.smithi040.stdout:9/203: rename d7/d16 to d7/d1b/d2f/d1f/d31/d4d 0 2024-04-24T23:13:46.486 INFO:tasks.workunit.client.1.smithi040.stdout:5/142: truncate d2/d7/fe 3108926 0 2024-04-24T23:13:46.487 INFO:tasks.workunit.client.1.smithi040.stdout:5/143: read - d2/d7/d8/f39 zero size 2024-04-24T23:13:46.487 INFO:tasks.workunit.client.0.smithi031.stdout:0/564: symlink da/d11/d6a/lbb 0 2024-04-24T23:13:46.488 INFO:tasks.workunit.client.1.smithi040.stdout:5/144: write d2/d2f/d14/f17 [3194407,53875] 0 2024-04-24T23:13:46.490 INFO:tasks.workunit.client.0.smithi031.stdout:5/448: symlink d0/d6/d3b/d4b/d1d/d2e/d28/l99 0 2024-04-24T23:13:46.491 INFO:tasks.workunit.client.0.smithi031.stdout:5/449: chown d0/d6/d3b/d4b/d1d/d2e/c76 208 1 2024-04-24T23:13:46.492 INFO:tasks.workunit.client.0.smithi031.stdout:5/450: fdatasync d0/d6/d3b/d4b/d1d/d2e/f52 0 2024-04-24T23:13:46.494 INFO:tasks.workunit.client.1.smithi040.stdout:8/139: truncate d0/d8/dc/f17 8688900 0 2024-04-24T23:13:46.495 INFO:tasks.workunit.client.1.smithi040.stdout:1/153: rename da/d12/l1b to da/d12/d14/d25/l37 0 2024-04-24T23:13:46.496 INFO:tasks.workunit.client.1.smithi040.stdout:1/154: chown da/d12/d18/d2a 227022 1 2024-04-24T23:13:46.500 INFO:tasks.workunit.client.0.smithi031.stdout:1/460: dread d1/da/d10/f51 [0,4194304] 0 2024-04-24T23:13:46.501 INFO:tasks.workunit.client.1.smithi040.stdout:2/223: write da/d10/d15/f2e [495765,119300] 0 2024-04-24T23:13:46.502 INFO:tasks.workunit.client.1.smithi040.stdout:7/213: dwrite f6 [0,4194304] 0 2024-04-24T23:13:46.502 INFO:tasks.workunit.client.1.smithi040.stdout:6/162: dread d7/de/ff [0,4194304] 0 2024-04-24T23:13:46.503 INFO:tasks.workunit.client.0.smithi031.stdout:5/451: creat d0/d6/d3b/d4b/dc/f9a x:0 0 0 2024-04-24T23:13:46.505 INFO:tasks.workunit.client.1.smithi040.stdout:8/140: symlink d0/d8/dc/l27 0 2024-04-24T23:13:46.508 INFO:tasks.workunit.client.0.smithi031.stdout:1/461: rename d1/d33/f48 to d1/da/d12/d7e/d7b/f93 0 2024-04-24T23:13:46.509 INFO:tasks.workunit.client.1.smithi040.stdout:0/220: creat df/d1e/f40 x:0 0 0 2024-04-24T23:13:46.511 INFO:tasks.workunit.client.1.smithi040.stdout:1/155: mkdir da/d35/d38 0 2024-04-24T23:13:46.513 INFO:tasks.workunit.client.0.smithi031.stdout:4/606: write d3/d23/d3b/f56 [1066044,114354] 0 2024-04-24T23:13:46.517 INFO:tasks.workunit.client.0.smithi031.stdout:4/607: dread d3/d23/d37/d86/f9a [0,4194304] 0 2024-04-24T23:13:46.522 INFO:tasks.workunit.client.0.smithi031.stdout:4/608: dwrite d3/fbc [0,4194304] 0 2024-04-24T23:13:46.556 INFO:tasks.workunit.client.0.smithi031.stdout:8/595: sync 2024-04-24T23:13:46.580 INFO:tasks.workunit.client.1.smithi040.stdout:5/145: mkdir d2/d7/d8/db/d1e/d22/d3e 0 2024-04-24T23:13:46.580 INFO:tasks.workunit.client.1.smithi040.stdout:3/164: write d1/f5 [2161054,99467] 0 2024-04-24T23:13:46.580 INFO:tasks.workunit.client.0.smithi031.stdout:5/452: fdatasync d0/d6/d3b/d4b/dc/f44 0 2024-04-24T23:13:46.581 INFO:tasks.workunit.client.1.smithi040.stdout:5/146: dread - d2/d7/f3b zero size 2024-04-24T23:13:46.581 INFO:tasks.workunit.client.1.smithi040.stdout:3/165: write d1/d2d/de/f34 [493737,130842] 0 2024-04-24T23:13:46.583 INFO:tasks.workunit.client.1.smithi040.stdout:2/224: creat da/d1c/d37/d39/f4e x:0 0 0 2024-04-24T23:13:46.584 INFO:tasks.workunit.client.1.smithi040.stdout:2/225: chown da/d10/f34 401849516 1 2024-04-24T23:13:46.585 INFO:tasks.workunit.client.1.smithi040.stdout:6/163: mkdir d7/de/d15/d2f 0 2024-04-24T23:13:46.585 INFO:tasks.workunit.client.1.smithi040.stdout:2/226: stat da/d1c/d37/d39/d3f/l46 0 2024-04-24T23:13:46.585 INFO:tasks.workunit.client.1.smithi040.stdout:3/166: dread d1/fd [0,4194304] 0 2024-04-24T23:13:46.588 INFO:tasks.workunit.client.1.smithi040.stdout:7/214: rename db/d1b to db/d12/d20/d52 0 2024-04-24T23:13:46.590 INFO:tasks.workunit.client.0.smithi031.stdout:4/609: symlink d3/d20/d25/d60/lc1 0 2024-04-24T23:13:46.594 INFO:tasks.workunit.client.0.smithi031.stdout:8/596: mknod d8/d15/d37/d43/d7b/d89/cd2 0 2024-04-24T23:13:46.594 INFO:tasks.workunit.client.0.smithi031.stdout:7/501: dwrite d1/d45/d75/d9f/fa5 [0,4194304] 0 2024-04-24T23:13:46.595 INFO:tasks.workunit.client.0.smithi031.stdout:8/597: fdatasync d8/d15/d9a/fb1 0 2024-04-24T23:13:46.596 INFO:tasks.workunit.client.0.smithi031.stdout:7/502: write d1/d2/dd/d33/d5e/d64/f91 [807211,111112] 0 2024-04-24T23:13:46.596 INFO:tasks.workunit.client.1.smithi040.stdout:0/221: rmdir df/d14/d1b 39 2024-04-24T23:13:46.596 INFO:tasks.workunit.client.1.smithi040.stdout:0/222: fdatasync df/f39 0 2024-04-24T23:13:46.597 INFO:tasks.workunit.client.1.smithi040.stdout:0/223: readlink df/d14/l27 0 2024-04-24T23:13:46.598 INFO:tasks.workunit.client.1.smithi040.stdout:4/144: write da/f28 [499721,44853] 0 2024-04-24T23:13:46.599 INFO:tasks.workunit.client.1.smithi040.stdout:4/145: truncate da/f1c 5106497 0 2024-04-24T23:13:46.600 INFO:tasks.workunit.client.0.smithi031.stdout:5/453: write d0/d6/d3b/d4b/dc/f23 [1709370,9879] 0 2024-04-24T23:13:46.603 INFO:tasks.workunit.client.1.smithi040.stdout:5/147: unlink d2/d7/l9 0 2024-04-24T23:13:46.606 INFO:tasks.workunit.client.1.smithi040.stdout:6/164: symlink d7/de/d15/l30 0 2024-04-24T23:13:46.608 INFO:tasks.workunit.client.1.smithi040.stdout:2/227: mkdir da/d1c/d4f 0 2024-04-24T23:13:46.609 INFO:tasks.workunit.client.1.smithi040.stdout:5/148: dwrite d2/d7/d8/f34 [0,4194304] 0 2024-04-24T23:13:46.613 INFO:tasks.workunit.client.0.smithi031.stdout:4/610: mkdir d3/d5e/dc2 0 2024-04-24T23:13:46.617 INFO:tasks.workunit.client.0.smithi031.stdout:8/598: fsync d8/d15/d37/f5d 0 2024-04-24T23:13:46.619 INFO:tasks.workunit.client.0.smithi031.stdout:8/599: stat d8/d15/d37/d43/d7b/d87 0 2024-04-24T23:13:46.619 INFO:tasks.workunit.client.0.smithi031.stdout:5/454: dread d0/d6/d3b/d4b/f83 [0,4194304] 0 2024-04-24T23:13:46.621 INFO:tasks.workunit.client.0.smithi031.stdout:5/455: chown d0/d6/d3b 596 1 2024-04-24T23:13:46.621 INFO:tasks.workunit.client.0.smithi031.stdout:7/503: symlink d1/d45/da0/d4f/lb0 0 2024-04-24T23:13:46.623 INFO:tasks.workunit.client.1.smithi040.stdout:1/156: creat da/d12/d18/d2a/d32/f39 x:0 0 0 2024-04-24T23:13:46.625 INFO:tasks.workunit.client.1.smithi040.stdout:8/141: dread d0/d8/dc/f17 [4194304,4194304] 0 2024-04-24T23:13:46.626 INFO:tasks.workunit.client.1.smithi040.stdout:8/142: chown d0/d16/l15 299249 1 2024-04-24T23:13:46.629 INFO:tasks.workunit.client.1.smithi040.stdout:1/157: dwrite da/d12/f21 [0,4194304] 0 2024-04-24T23:13:46.629 INFO:tasks.workunit.client.0.smithi031.stdout:8/600: mkdir d8/d15/d37/d43/d7b/dd3 0 2024-04-24T23:13:46.632 INFO:tasks.workunit.client.0.smithi031.stdout:1/462: write d1/da/d12/d7e/d7b/f6a [3274652,82437] 0 2024-04-24T23:13:46.633 INFO:tasks.workunit.client.1.smithi040.stdout:2/228: mknod da/d1c/c50 0 2024-04-24T23:13:46.635 INFO:tasks.workunit.client.0.smithi031.stdout:8/601: dread d8/d1a/d73/f80 [0,4194304] 0 2024-04-24T23:13:46.638 INFO:tasks.workunit.client.1.smithi040.stdout:5/149: rename d2/d7/d8/c12 to d2/d7/d8/db/d1e/c3f 0 2024-04-24T23:13:46.639 INFO:tasks.workunit.client.1.smithi040.stdout:2/229: dread f8 [0,4194304] 0 2024-04-24T23:13:46.642 INFO:tasks.workunit.client.1.smithi040.stdout:7/215: rename db/d12/d20/d52/d44/f48 to db/d12/d20/d52/d44/f53 0 2024-04-24T23:13:46.643 INFO:tasks.workunit.client.1.smithi040.stdout:7/216: chown db/f3c 1363 1 2024-04-24T23:13:46.647 INFO:tasks.workunit.client.0.smithi031.stdout:1/463: creat d1/da/d12/d7e/d7b/d4e/f94 x:0 0 0 2024-04-24T23:13:46.649 INFO:tasks.workunit.client.0.smithi031.stdout:7/504: mkdir d1/d2/dd/d33/d5e/d64/d6d/dae/db1 0 2024-04-24T23:13:46.652 INFO:tasks.workunit.client.0.smithi031.stdout:1/464: dread d1/da/d12/d7e/d7b/f93 [0,4194304] 0 2024-04-24T23:13:46.653 INFO:tasks.workunit.client.0.smithi031.stdout:1/465: chown d1/da/d12/d7e/d7b 297536135 1 2024-04-24T23:13:46.653 INFO:tasks.workunit.client.0.smithi031.stdout:1/466: chown d1/da/d12/f83 2 1 2024-04-24T23:13:46.654 INFO:tasks.workunit.client.0.smithi031.stdout:8/602: truncate d8/f14 4804928 0 2024-04-24T23:13:46.654 INFO:tasks.workunit.client.0.smithi031.stdout:1/467: write d1/d3e/d46/f59 [855598,1600] 0 2024-04-24T23:13:46.655 INFO:tasks.workunit.client.1.smithi040.stdout:6/165: creat d7/d2c/f31 x:0 0 0 2024-04-24T23:13:46.657 INFO:tasks.workunit.client.0.smithi031.stdout:4/611: creat d3/d20/d25/fc3 x:0 0 0 2024-04-24T23:13:46.657 INFO:tasks.workunit.client.0.smithi031.stdout:4/612: chown d3/f26 299 1 2024-04-24T23:13:46.658 INFO:tasks.workunit.client.1.smithi040.stdout:8/143: mkdir d0/d28 0 2024-04-24T23:13:46.658 INFO:tasks.workunit.client.0.smithi031.stdout:5/456: mkdir d0/d6/d3b/d4b/dc/d8b/d9b 0 2024-04-24T23:13:46.662 INFO:tasks.workunit.client.1.smithi040.stdout:9/204: mkdir d7/d1b/d2f/d1f/d4e 0 2024-04-24T23:13:46.662 INFO:tasks.workunit.client.1.smithi040.stdout:3/167: dwrite d1/d9/d15/f1b [4194304,4194304] 0 2024-04-24T23:13:46.663 INFO:tasks.workunit.client.0.smithi031.stdout:7/505: rmdir d1/d45 39 2024-04-24T23:13:46.665 INFO:tasks.workunit.client.0.smithi031.stdout:8/603: read d8/f22 [237861,111085] 0 2024-04-24T23:13:46.669 INFO:tasks.workunit.client.1.smithi040.stdout:2/230: symlink da/d1c/d37/l51 0 2024-04-24T23:13:46.677 INFO:tasks.workunit.client.1.smithi040.stdout:6/166: symlink d7/d1a/l32 0 2024-04-24T23:13:46.679 INFO:tasks.workunit.client.1.smithi040.stdout:6/167: write d7/fd [149712,101259] 0 2024-04-24T23:13:46.681 INFO:tasks.workunit.client.1.smithi040.stdout:8/144: fdatasync d0/d8/dc/f17 0 2024-04-24T23:13:46.682 INFO:tasks.workunit.client.1.smithi040.stdout:0/224: dwrite df/d1e/f2f [4194304,4194304] 0 2024-04-24T23:13:46.683 INFO:tasks.workunit.client.1.smithi040.stdout:4/146: write f5 [1497000,13078] 0 2024-04-24T23:13:46.687 INFO:tasks.workunit.client.1.smithi040.stdout:6/168: dwrite d7/fb [0,4194304] 0 2024-04-24T23:13:46.691 INFO:tasks.workunit.client.1.smithi040.stdout:1/158: write f2 [5398568,104283] 0 2024-04-24T23:13:46.692 INFO:tasks.workunit.client.1.smithi040.stdout:4/147: dwrite da/d1d/d2b/f31 [0,4194304] 0 2024-04-24T23:13:46.694 INFO:tasks.workunit.client.1.smithi040.stdout:5/150: mknod d2/d7/d8/d1f/d31/c40 0 2024-04-24T23:13:46.695 INFO:tasks.workunit.client.0.smithi031.stdout:7/506: creat d1/d2/fb2 x:0 0 0 2024-04-24T23:13:46.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/613: write d3/d20/d2d/f50 [1726032,55403] 0 2024-04-24T23:13:46.701 INFO:tasks.workunit.client.1.smithi040.stdout:6/169: dwrite d7/d2c/f31 [0,4194304] 0 2024-04-24T23:13:46.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/507: rmdir d1/d2/d83 39 2024-04-24T23:13:46.704 INFO:tasks.workunit.client.0.smithi031.stdout:8/604: dwrite d8/f38 [12582912,4194304] 0 2024-04-24T23:13:46.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/457: write d0/d6/d3b/d4b/dc/d25/f60 [220151,69680] 0 2024-04-24T23:13:46.707 INFO:tasks.workunit.client.1.smithi040.stdout:2/231: dwrite da/d29/f48 [0,4194304] 0 2024-04-24T23:13:46.713 INFO:tasks.workunit.client.1.smithi040.stdout:0/225: mkdir df/d1e/d20/d21/d41 0 2024-04-24T23:13:46.716 INFO:tasks.workunit.client.0.smithi031.stdout:8/605: rename l5 to d8/d1a/d1d/ld4 0 2024-04-24T23:13:46.719 INFO:tasks.workunit.client.1.smithi040.stdout:9/205: unlink d7/f15 0 2024-04-24T23:13:46.722 INFO:tasks.workunit.client.1.smithi040.stdout:3/168: link d1/d2d/de/f2f d1/d9/d25/f35 0 2024-04-24T23:13:46.722 INFO:tasks.workunit.client.1.smithi040.stdout:3/169: chown d1/d2d 59001756 1 2024-04-24T23:13:46.723 INFO:tasks.workunit.client.1.smithi040.stdout:3/170: write d1/d9/f20 [1792555,28425] 0 2024-04-24T23:13:46.724 INFO:tasks.workunit.client.1.smithi040.stdout:1/159: chown da/cb 99310712 1 2024-04-24T23:13:46.725 INFO:tasks.workunit.client.1.smithi040.stdout:4/148: unlink f6 0 2024-04-24T23:13:46.730 INFO:tasks.workunit.client.1.smithi040.stdout:1/160: dwrite da/d12/d14/f24 [0,4194304] 0 2024-04-24T23:13:46.731 INFO:tasks.workunit.client.0.smithi031.stdout:7/508: dread d1/d2/dd/d33/d5e/d64/faa [0,4194304] 0 2024-04-24T23:13:46.731 INFO:tasks.workunit.client.1.smithi040.stdout:0/226: truncate df/d14/d1b/f36 1790794 0 2024-04-24T23:13:46.732 INFO:tasks.workunit.client.1.smithi040.stdout:9/206: read d7/ff [4858211,47538] 0 2024-04-24T23:13:46.737 INFO:tasks.workunit.client.1.smithi040.stdout:4/149: unlink l0 0 2024-04-24T23:13:46.737 INFO:tasks.workunit.client.1.smithi040.stdout:4/150: chown da/l17 10291010 1 2024-04-24T23:13:46.739 INFO:tasks.workunit.client.1.smithi040.stdout:7/217: truncate db/f24 1392873 0 2024-04-24T23:13:46.739 INFO:tasks.workunit.client.1.smithi040.stdout:5/151: getdents d2/d2f/d23 0 2024-04-24T23:13:46.739 INFO:tasks.workunit.client.0.smithi031.stdout:6/636: sync 2024-04-24T23:13:46.740 INFO:tasks.workunit.client.1.smithi040.stdout:6/170: mknod d7/de/d15/d2f/c33 0 2024-04-24T23:13:46.740 INFO:tasks.workunit.client.1.smithi040.stdout:2/232: read f7 [322278,128653] 0 2024-04-24T23:13:46.740 INFO:tasks.workunit.client.0.smithi031.stdout:4/614: write d3/d20/d2d/f39 [1300101,98747] 0 2024-04-24T23:13:46.740 INFO:tasks.workunit.client.1.smithi040.stdout:6/171: fdatasync d7/de/d15/f2e 0 2024-04-24T23:13:46.742 INFO:tasks.workunit.client.1.smithi040.stdout:8/145: creat d0/d8/f29 x:0 0 0 2024-04-24T23:13:46.743 INFO:tasks.workunit.client.1.smithi040.stdout:1/161: readlink da/l13 0 2024-04-24T23:13:46.744 INFO:tasks.workunit.client.1.smithi040.stdout:1/162: truncate da/d12/f1c 5088266 0 2024-04-24T23:13:46.748 INFO:tasks.workunit.client.0.smithi031.stdout:4/615: mkdir d3/d5e/dc4 0 2024-04-24T23:13:46.750 INFO:tasks.workunit.client.1.smithi040.stdout:0/227: mknod df/d1e/d20/d21/c42 0 2024-04-24T23:13:46.751 INFO:tasks.workunit.client.0.smithi031.stdout:4/616: read d3/d23/d3b/fb4 [1180498,53117] 0 2024-04-24T23:13:46.751 INFO:tasks.workunit.client.0.smithi031.stdout:6/637: mkdir d13/d15/d1d/ddd 0 2024-04-24T23:13:46.752 INFO:tasks.workunit.client.0.smithi031.stdout:4/617: chown d3/d20/d2d/d44/l6f 4790154 1 2024-04-24T23:13:46.753 INFO:tasks.workunit.client.1.smithi040.stdout:9/207: chown d7/d1b/d2f/d1f/d31/d4d/f43 1380 1 2024-04-24T23:13:46.754 INFO:tasks.workunit.client.0.smithi031.stdout:5/458: write d0/d6/d3b/d4b/dc/f84 [2159006,114664] 0 2024-04-24T23:13:46.755 INFO:tasks.workunit.client.1.smithi040.stdout:0/228: dwrite df/d12/f22 [0,4194304] 0 2024-04-24T23:13:46.764 INFO:tasks.workunit.client.0.smithi031.stdout:4/618: truncate d3/f1b 565081 0 2024-04-24T23:13:46.765 INFO:tasks.workunit.client.0.smithi031.stdout:4/619: stat d3/d23/d37/fa9 0 2024-04-24T23:13:46.770 INFO:tasks.workunit.client.0.smithi031.stdout:5/459: unlink d0/d6/d3b/d4b/f83 0 2024-04-24T23:13:46.771 INFO:tasks.workunit.client.0.smithi031.stdout:5/460: fsync d0/d6/d3b/d4b/d1d/d2e/f79 0 2024-04-24T23:13:46.775 INFO:tasks.workunit.client.0.smithi031.stdout:8/606: mknod d8/d1a/d1d/d35/cd5 0 2024-04-24T23:13:46.779 INFO:tasks.workunit.client.0.smithi031.stdout:4/620: fsync d3/d23/d69/d3f/f6d 0 2024-04-24T23:13:46.779 INFO:tasks.workunit.client.0.smithi031.stdout:6/638: write d13/d1b/d32/db4/d79/f84 [1519791,30930] 0 2024-04-24T23:13:46.784 INFO:tasks.workunit.client.0.smithi031.stdout:4/621: chown d3/fb 95 1 2024-04-24T23:13:46.788 INFO:tasks.workunit.client.0.smithi031.stdout:4/622: creat d3/d23/fc5 x:0 0 0 2024-04-24T23:13:46.789 INFO:tasks.workunit.client.0.smithi031.stdout:4/623: write d3/da3/faf [814957,122282] 0 2024-04-24T23:13:46.789 INFO:tasks.workunit.client.1.smithi040.stdout:4/151: rmdir da 39 2024-04-24T23:13:46.789 INFO:tasks.workunit.client.0.smithi031.stdout:5/461: write d0/f93 [1585541,11864] 0 2024-04-24T23:13:46.790 INFO:tasks.workunit.client.1.smithi040.stdout:3/171: write d1/f16 [935612,120293] 0 2024-04-24T23:13:46.791 INFO:tasks.workunit.client.1.smithi040.stdout:0/229: truncate df/f39 822434 0 2024-04-24T23:13:46.791 INFO:tasks.workunit.client.0.smithi031.stdout:8/607: write d8/d15/d37/d62/f90 [567451,60557] 0 2024-04-24T23:13:46.792 INFO:tasks.workunit.client.1.smithi040.stdout:2/233: mkdir da/d1c/d37/d39/d3f/d52 0 2024-04-24T23:13:46.792 INFO:tasks.workunit.client.1.smithi040.stdout:6/172: mknod d7/de/d15/c34 0 2024-04-24T23:13:46.793 INFO:tasks.workunit.client.1.smithi040.stdout:5/152: dread d2/d2f/d14/f17 [0,4194304] 0 2024-04-24T23:13:46.794 INFO:tasks.workunit.client.1.smithi040.stdout:0/230: write df/d12/f25 [403091,61888] 0 2024-04-24T23:13:46.794 INFO:tasks.workunit.client.1.smithi040.stdout:3/172: read - d1/d2d/de/f2f zero size 2024-04-24T23:13:46.795 INFO:tasks.workunit.client.0.smithi031.stdout:6/639: write d13/d15/d1d/d23/d24/d43/d68/db8/fcc [858439,46238] 0 2024-04-24T23:13:46.800 INFO:tasks.workunit.client.0.smithi031.stdout:6/640: unlink f12 0 2024-04-24T23:13:46.803 INFO:tasks.workunit.client.0.smithi031.stdout:5/462: symlink d0/d6/d3b/d4b/dc/d86/d91/l9c 0 2024-04-24T23:13:46.804 INFO:tasks.workunit.client.0.smithi031.stdout:5/463: stat d0/d6/d3b/d4b/d1d/d2e/d28/c3f 0 2024-04-24T23:13:46.805 INFO:tasks.workunit.client.0.smithi031.stdout:5/464: write d0/d6/d3b/d4b/dc/f9a [230015,27164] 0 2024-04-24T23:13:46.806 INFO:tasks.workunit.client.0.smithi031.stdout:4/624: link l1 d3/d20/d2d/d44/d5b/lc6 0 2024-04-24T23:13:46.807 INFO:tasks.workunit.client.0.smithi031.stdout:8/608: dread d8/d15/d37/f5e [0,4194304] 0 2024-04-24T23:13:46.809 INFO:tasks.workunit.client.0.smithi031.stdout:4/625: symlink d3/d23/d69/lc7 0 2024-04-24T23:13:46.812 INFO:tasks.workunit.client.0.smithi031.stdout:4/626: fsync d3/d20/d25/d60/fc0 0 2024-04-24T23:13:46.812 INFO:tasks.workunit.client.1.smithi040.stdout:5/153: chown d2/c2d 6 1 2024-04-24T23:13:46.812 INFO:tasks.workunit.client.1.smithi040.stdout:9/208: dwrite f4 [0,4194304] 0 2024-04-24T23:13:46.813 INFO:tasks.workunit.client.1.smithi040.stdout:8/146: mknod d0/d16/d1b/c2a 0 2024-04-24T23:13:46.813 INFO:tasks.workunit.client.1.smithi040.stdout:3/173: chown d1/f32 500 1 2024-04-24T23:13:46.814 INFO:tasks.workunit.client.0.smithi031.stdout:5/465: dwrite d0/d6/d3b/f68 [0,4194304] 0 2024-04-24T23:13:46.816 INFO:tasks.workunit.client.1.smithi040.stdout:6/173: creat d7/de/d15/d2f/f35 x:0 0 0 2024-04-24T23:13:46.816 INFO:tasks.workunit.client.1.smithi040.stdout:7/218: link db/lc db/d2f/l54 0 2024-04-24T23:13:46.817 INFO:tasks.workunit.client.1.smithi040.stdout:9/209: write d7/d1b/d2f/d1f/d34/f47 [120278,43601] 0 2024-04-24T23:13:46.818 INFO:tasks.workunit.client.1.smithi040.stdout:7/219: fsync db/f31 0 2024-04-24T23:13:46.825 INFO:tasks.workunit.client.0.smithi031.stdout:6/641: dread d13/d15/d1d/f8c [0,4194304] 0 2024-04-24T23:13:46.826 INFO:tasks.workunit.client.1.smithi040.stdout:0/231: creat df/d1e/d20/f43 x:0 0 0 2024-04-24T23:13:46.826 INFO:tasks.workunit.client.0.smithi031.stdout:6/642: dread - d13/d15/d1d/d23/d24/d43/fcb zero size 2024-04-24T23:13:46.828 INFO:tasks.workunit.client.1.smithi040.stdout:8/147: symlink d0/d1f/l2b 0 2024-04-24T23:13:46.828 INFO:tasks.workunit.client.0.smithi031.stdout:8/609: truncate d8/d1a/d1d/d35/f8f 3514126 0 2024-04-24T23:13:46.829 INFO:tasks.workunit.client.0.smithi031.stdout:8/610: fdatasync d8/d15/d37/d43/d7b/f8c 0 2024-04-24T23:13:46.829 INFO:tasks.workunit.client.0.smithi031.stdout:8/611: readlink d8/lf 0 2024-04-24T23:13:46.830 INFO:tasks.workunit.client.1.smithi040.stdout:3/174: mkdir d1/d9/d18/d36 0 2024-04-24T23:13:46.831 INFO:tasks.workunit.client.1.smithi040.stdout:0/232: dwrite df/d12/f1f [8388608,4194304] 0 2024-04-24T23:13:46.831 INFO:tasks.workunit.client.1.smithi040.stdout:0/233: readlink df/d14/l27 0 2024-04-24T23:13:46.832 INFO:tasks.workunit.client.0.smithi031.stdout:6/643: mkdir d13/d1b/d32/db4/d79/dd7/dde 0 2024-04-24T23:13:46.833 INFO:tasks.workunit.client.1.smithi040.stdout:0/234: write f2 [4718762,73169] 0 2024-04-24T23:13:46.833 INFO:tasks.workunit.client.1.smithi040.stdout:3/175: truncate d1/d9/d18/f1f 143123 0 2024-04-24T23:13:46.833 INFO:tasks.workunit.client.0.smithi031.stdout:6/644: chown d13/d15/d1d/d23/d2d/dbf 9485588 1 2024-04-24T23:13:46.835 INFO:tasks.workunit.client.0.smithi031.stdout:5/466: link d0/d6/d3b/d4b/f72 d0/d6/d3b/d4b/d1d/f9d 0 2024-04-24T23:13:46.840 INFO:tasks.workunit.client.0.smithi031.stdout:8/612: creat d8/d15/fd6 x:0 0 0 2024-04-24T23:13:46.842 INFO:tasks.workunit.client.0.smithi031.stdout:6/645: rename d13/d15/l3a to d13/d15/d1d/d23/d2d/dbf/ldf 0 2024-04-24T23:13:46.843 INFO:tasks.workunit.client.0.smithi031.stdout:4/627: dread d3/d20/d2d/d44/f84 [0,4194304] 0 2024-04-24T23:13:46.843 INFO:tasks.workunit.client.0.smithi031.stdout:5/467: dwrite d0/d6/d3b/d4b/d1d/d2e/f52 [0,4194304] 0 2024-04-24T23:13:46.843 INFO:tasks.workunit.client.0.smithi031.stdout:8/613: write d8/d15/d37/f95 [1414631,114722] 0 2024-04-24T23:13:46.844 INFO:tasks.workunit.client.1.smithi040.stdout:7/220: rmdir db/d12 39 2024-04-24T23:13:46.848 INFO:tasks.workunit.client.0.smithi031.stdout:8/614: dread d8/d15/d9a/fb2 [0,4194304] 0 2024-04-24T23:13:46.849 INFO:tasks.workunit.client.1.smithi040.stdout:8/148: rmdir d0/d8 39 2024-04-24T23:13:46.856 INFO:tasks.workunit.client.0.smithi031.stdout:8/615: mknod d8/d15/d37/d43/dbd/cd7 0 2024-04-24T23:13:46.860 INFO:tasks.workunit.client.0.smithi031.stdout:8/616: dread d8/f38 [12582912,4194304] 0 2024-04-24T23:13:46.862 INFO:tasks.workunit.client.0.smithi031.stdout:5/468: unlink d0/d7/d56/d7c/d7e/f5a 0 2024-04-24T23:13:46.863 INFO:tasks.workunit.client.1.smithi040.stdout:6/174: mknod d7/c36 0 2024-04-24T23:13:46.865 INFO:tasks.workunit.client.1.smithi040.stdout:9/210: symlink d7/da/d17/d40/l4f 0 2024-04-24T23:13:46.866 INFO:tasks.workunit.client.0.smithi031.stdout:8/617: chown d8/f18 0 1 2024-04-24T23:13:46.867 INFO:tasks.workunit.client.1.smithi040.stdout:4/152: creat da/f37 x:0 0 0 2024-04-24T23:13:46.868 INFO:tasks.workunit.client.0.smithi031.stdout:8/618: read d8/d15/f2a [385444,123077] 0 2024-04-24T23:13:46.868 INFO:tasks.workunit.client.0.smithi031.stdout:2/369: sync 2024-04-24T23:13:46.868 INFO:tasks.workunit.client.0.smithi031.stdout:3/454: sync 2024-04-24T23:13:46.870 INFO:tasks.workunit.client.1.smithi040.stdout:5/154: truncate d2/d2f/d14/f17 3390832 0 2024-04-24T23:13:46.871 INFO:tasks.workunit.client.1.smithi040.stdout:5/155: chown d2/l5 31568 1 2024-04-24T23:13:46.872 INFO:tasks.workunit.client.1.smithi040.stdout:5/156: write d2/f6 [2309869,94779] 0 2024-04-24T23:13:46.873 INFO:tasks.workunit.client.0.smithi031.stdout:4/628: write d3/d20/fa4 [2280156,15004] 0 2024-04-24T23:13:46.874 INFO:tasks.workunit.client.0.smithi031.stdout:8/619: dread d8/d15/d37/f95 [0,4194304] 0 2024-04-24T23:13:46.875 INFO:tasks.workunit.client.1.smithi040.stdout:3/176: mkdir d1/d2d/de/d30/d37 0 2024-04-24T23:13:46.875 INFO:tasks.workunit.client.0.smithi031.stdout:5/469: dread d0/d6/d3b/f4d [0,4194304] 0 2024-04-24T23:13:46.876 INFO:tasks.workunit.client.1.smithi040.stdout:8/149: dwrite d0/d8/f29 [0,4194304] 0 2024-04-24T23:13:46.877 INFO:tasks.workunit.client.1.smithi040.stdout:6/175: rename d7/c21 to d7/de/d15/d2f/c37 0 2024-04-24T23:13:46.877 INFO:tasks.workunit.client.1.smithi040.stdout:6/176: dread - d7/de/d15/f17 zero size 2024-04-24T23:13:46.878 INFO:tasks.workunit.client.0.smithi031.stdout:2/370: unlink d5/d28/l59 0 2024-04-24T23:13:46.879 INFO:tasks.workunit.client.1.smithi040.stdout:9/211: creat d7/d1b/d2f/d1f/d31/d3b/f50 x:0 0 0 2024-04-24T23:13:46.881 INFO:tasks.workunit.client.1.smithi040.stdout:7/221: rmdir db/d12/d20/d52/d44 39 2024-04-24T23:13:46.884 INFO:tasks.workunit.client.1.smithi040.stdout:4/153: mknod da/d1d/d2b/c38 0 2024-04-24T23:13:46.886 INFO:tasks.workunit.client.0.smithi031.stdout:4/629: mkdir d3/d20/d25/d60/dc8 0 2024-04-24T23:13:46.887 INFO:tasks.workunit.client.0.smithi031.stdout:4/630: chown d3/d20/d2d/d44/d5b/d5d/f96 479780 1 2024-04-24T23:13:46.889 INFO:tasks.workunit.client.0.smithi031.stdout:5/470: truncate d0/d7/d56/f61 889260 0 2024-04-24T23:13:46.890 INFO:tasks.workunit.client.1.smithi040.stdout:4/154: dwrite da/f19 [0,4194304] 0 2024-04-24T23:13:46.893 INFO:tasks.workunit.client.1.smithi040.stdout:8/150: symlink d0/d8/dc/l2c 0 2024-04-24T23:13:46.894 INFO:tasks.workunit.client.1.smithi040.stdout:6/177: symlink d7/de/d15/d2f/l38 0 2024-04-24T23:13:46.899 INFO:tasks.workunit.client.0.smithi031.stdout:2/371: symlink d5/d8/d15/l7d 0 2024-04-24T23:13:46.905 INFO:tasks.workunit.client.1.smithi040.stdout:5/157: fsync d2/d2f/d14/f17 0 2024-04-24T23:13:46.906 INFO:tasks.workunit.client.1.smithi040.stdout:4/155: rmdir da/d1d/d1e 39 2024-04-24T23:13:46.907 INFO:tasks.workunit.client.1.smithi040.stdout:4/156: dread - da/f37 zero size 2024-04-24T23:13:46.908 INFO:tasks.workunit.client.0.smithi031.stdout:2/372: creat d5/d8/d45/d5b/d49/f7e x:0 0 0 2024-04-24T23:13:46.909 INFO:tasks.workunit.client.1.smithi040.stdout:1/163: sync 2024-04-24T23:13:46.910 INFO:tasks.workunit.client.0.smithi031.stdout:3/455: getdents d6/df/d16 0 2024-04-24T23:13:46.911 INFO:tasks.workunit.client.1.smithi040.stdout:8/151: creat d0/d1f/f2d x:0 0 0 2024-04-24T23:13:46.911 INFO:tasks.workunit.client.1.smithi040.stdout:4/157: dwrite da/f1a [0,4194304] 0 2024-04-24T23:13:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:46 smithi040 ceph-mon[33128]: pgmap v26: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 21 MiB/s rd, 136 MiB/s wr, 316 op/s 2024-04-24T23:13:46.912 INFO:tasks.workunit.client.0.smithi031.stdout:5/471: creat d0/d6/d3b/d4b/d1d/f9e x:0 0 0 2024-04-24T23:13:46.914 INFO:tasks.workunit.client.1.smithi040.stdout:6/178: mknod d7/de/d15/c39 0 2024-04-24T23:13:46.916 INFO:tasks.workunit.client.0.smithi031.stdout:2/373: unlink d5/f7c 0 2024-04-24T23:13:46.919 INFO:tasks.workunit.client.0.smithi031.stdout:3/456: stat d6/d55/l56 0 2024-04-24T23:13:46.920 INFO:tasks.workunit.client.0.smithi031.stdout:3/457: write d6/f7b [920043,81210] 0 2024-04-24T23:13:46.923 INFO:tasks.workunit.client.1.smithi040.stdout:0/235: link df/d1e/d20/c24 df/c44 0 2024-04-24T23:13:46.925 INFO:tasks.workunit.client.0.smithi031.stdout:8/620: write d8/d1a/d1d/d50/dc0/d9d/fc6 [698275,51871] 0 2024-04-24T23:13:46.926 INFO:tasks.workunit.client.1.smithi040.stdout:8/152: dwrite d0/d8/f29 [4194304,4194304] 0 2024-04-24T23:13:46.926 INFO:tasks.workunit.client.0.smithi031.stdout:2/374: mkdir d5/d28/d4e/d7f 0 2024-04-24T23:13:46.931 INFO:tasks.workunit.client.1.smithi040.stdout:4/158: readlink da/l16 0 2024-04-24T23:13:46.935 INFO:tasks.workunit.client.0.smithi031.stdout:8/621: rename d8/d15/f3d to d8/d15/d37/d43/dbd/dc4/fd8 0 2024-04-24T23:13:46.936 INFO:tasks.workunit.client.0.smithi031.stdout:8/622: write d8/d1a/f9b [338934,24754] 0 2024-04-24T23:13:46.937 INFO:tasks.workunit.client.1.smithi040.stdout:9/212: truncate d7/d1b/d2f/d28/f29 2381900 0 2024-04-24T23:13:46.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/623: dread - d8/d15/d37/d43/d7b/d87/fa0 zero size 2024-04-24T23:13:46.940 INFO:tasks.workunit.client.1.smithi040.stdout:7/222: creat db/f55 x:0 0 0 2024-04-24T23:13:46.943 INFO:tasks.workunit.client.1.smithi040.stdout:3/177: getdents d1 0 2024-04-24T23:13:46.944 INFO:tasks.workunit.client.1.smithi040.stdout:1/164: getdents da/d35/d38 0 2024-04-24T23:13:46.944 INFO:tasks.workunit.client.1.smithi040.stdout:9/213: dwrite d7/d1b/d2f/d1f/d31/d3b/f50 [0,4194304] 0 2024-04-24T23:13:46.945 INFO:tasks.workunit.client.1.smithi040.stdout:4/159: rename da/f28 to da/d1d/d2b/f39 0 2024-04-24T23:13:46.946 INFO:tasks.workunit.client.0.smithi031.stdout:5/472: write d0/d6/d3b/d4b/dc/f47 [831782,75441] 0 2024-04-24T23:13:46.951 INFO:tasks.workunit.client.1.smithi040.stdout:0/236: symlink df/d14/l45 0 2024-04-24T23:13:46.953 INFO:tasks.workunit.client.1.smithi040.stdout:9/214: dwrite d7/d1b/d2f/f46 [0,4194304] 0 2024-04-24T23:13:46.953 INFO:tasks.workunit.client.0.smithi031.stdout:2/375: truncate d5/d6/f4d 3504786 0 2024-04-24T23:13:46.955 INFO:tasks.workunit.client.1.smithi040.stdout:9/215: write d7/d1b/d2f/d1f/d31/f35 [1001096,49600] 0 2024-04-24T23:13:46.956 INFO:tasks.workunit.client.1.smithi040.stdout:9/216: read - d7/d1b/d2f/d1f/d31/d4d/f43 zero size 2024-04-24T23:13:46.957 INFO:tasks.workunit.client.1.smithi040.stdout:9/217: write d7/d1b/d2f/d1f/d31/d4d/f43 [549033,127353] 0 2024-04-24T23:13:46.958 INFO:tasks.workunit.client.1.smithi040.stdout:8/153: truncate d0/d8/dc/f17 2801613 0 2024-04-24T23:13:46.962 INFO:tasks.workunit.client.0.smithi031.stdout:8/624: dread d8/f14 [0,4194304] 0 2024-04-24T23:13:46.962 INFO:tasks.workunit.client.0.smithi031.stdout:2/376: dwrite d5/d8/d15/f22 [0,4194304] 0 2024-04-24T23:13:46.963 INFO:tasks.workunit.client.1.smithi040.stdout:6/179: write d7/de/d15/d1e/f22 [190125,73211] 0 2024-04-24T23:13:46.963 INFO:tasks.workunit.client.1.smithi040.stdout:9/218: fsync d7/d1b/d2f/d1f/d31/f35 0 2024-04-24T23:13:46.964 INFO:tasks.workunit.client.1.smithi040.stdout:7/223: readlink db/d12/d20/d52/d44/l4d 0 2024-04-24T23:13:46.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/625: dread - d8/d1a/d1d/d50/dc0/d9d/fbe zero size 2024-04-24T23:13:46.969 INFO:tasks.workunit.client.1.smithi040.stdout:6/180: dwrite d7/de/d15/f2e [0,4194304] 0 2024-04-24T23:13:46.970 INFO:tasks.workunit.client.1.smithi040.stdout:3/178: creat d1/d9/d18/f38 x:0 0 0 2024-04-24T23:13:46.973 INFO:tasks.workunit.client.1.smithi040.stdout:1/165: creat da/d35/f3a x:0 0 0 2024-04-24T23:13:46.974 INFO:tasks.workunit.client.1.smithi040.stdout:1/166: write da/d12/f21 [4028990,44775] 0 2024-04-24T23:13:46.975 INFO:tasks.workunit.client.1.smithi040.stdout:1/167: readlink da/d35/l36 0 2024-04-24T23:13:46.977 INFO:tasks.workunit.client.1.smithi040.stdout:1/168: write da/d12/d18/d2a/d32/f39 [312921,46243] 0 2024-04-24T23:13:46.977 INFO:tasks.workunit.client.1.smithi040.stdout:1/169: chown da/d12/d18/f22 3 1 2024-04-24T23:13:46.978 INFO:tasks.workunit.client.1.smithi040.stdout:4/160: mknod da/d1f/c3a 0 2024-04-24T23:13:46.983 INFO:tasks.workunit.client.1.smithi040.stdout:0/237: mknod df/d14/d1b/c46 0 2024-04-24T23:13:46.988 INFO:tasks.workunit.client.1.smithi040.stdout:7/224: mkdir db/d12/d20/d56 0 2024-04-24T23:13:46.993 INFO:tasks.workunit.client.1.smithi040.stdout:3/179: rmdir d1/d9/d25 39 2024-04-24T23:13:46.995 INFO:tasks.workunit.client.1.smithi040.stdout:6/181: creat d7/d1a/d20/f3a x:0 0 0 2024-04-24T23:13:46.998 INFO:tasks.workunit.client.0.smithi031.stdout:5/473: mknod d0/d6/d12/c9f 0 2024-04-24T23:13:46.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:46 smithi031 ceph-mon[26411]: pgmap v26: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 21 MiB/s rd, 136 MiB/s wr, 316 op/s 2024-04-24T23:13:46.999 INFO:tasks.workunit.client.1.smithi040.stdout:1/170: creat da/d12/d23/f3b x:0 0 0 2024-04-24T23:13:47.000 INFO:tasks.workunit.client.1.smithi040.stdout:1/171: truncate da/d12/d18/d2a/d32/f39 989176 0 2024-04-24T23:13:47.001 INFO:tasks.workunit.client.1.smithi040.stdout:4/161: symlink da/d1d/d2b/l3b 0 2024-04-24T23:13:47.001 INFO:tasks.workunit.client.1.smithi040.stdout:1/172: stat da/d12/d18/f1d 0 2024-04-24T23:13:47.002 INFO:tasks.workunit.client.0.smithi031.stdout:8/626: rmdir d8/d1a/d74 39 2024-04-24T23:13:47.005 INFO:tasks.workunit.client.0.smithi031.stdout:3/458: getdents d6/d72/d1a/d25 0 2024-04-24T23:13:47.009 INFO:tasks.workunit.client.0.smithi031.stdout:5/474: creat d0/d6/d32/d6b/fa0 x:0 0 0 2024-04-24T23:13:47.011 INFO:tasks.workunit.client.0.smithi031.stdout:2/377: mknod d5/d6/c80 0 2024-04-24T23:13:47.015 INFO:tasks.workunit.client.0.smithi031.stdout:3/459: mkdir d6/d14/d91 0 2024-04-24T23:13:47.015 INFO:tasks.workunit.client.0.smithi031.stdout:5/475: dwrite d0/d6/d32/d6b/f70 [0,4194304] 0 2024-04-24T23:13:47.016 INFO:tasks.workunit.client.1.smithi040.stdout:2/234: sync 2024-04-24T23:13:47.016 INFO:tasks.workunit.client.1.smithi040.stdout:6/182: symlink d7/de/l3b 0 2024-04-24T23:13:47.016 INFO:tasks.workunit.client.0.smithi031.stdout:5/476: chown d0/d6/d3b/d4b/f96 42299365 1 2024-04-24T23:13:47.017 INFO:tasks.workunit.client.0.smithi031.stdout:5/477: readlink d0/d6/d3b/d4f/l8e 0 2024-04-24T23:13:47.018 INFO:tasks.workunit.client.0.smithi031.stdout:8/627: rename d8/d1a/d1d/c71 to d8/d15/d37/d43/dbd/cd9 0 2024-04-24T23:13:47.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/478: dread - d0/d6/d3b/d4b/d1d/f9e zero size 2024-04-24T23:13:47.020 INFO:tasks.workunit.client.0.smithi031.stdout:5/479: write d0/d7/d74/f82 [360273,109195] 0 2024-04-24T23:13:47.021 INFO:tasks.workunit.client.0.smithi031.stdout:3/460: dread - d6/df/d16/d2a/f68 zero size 2024-04-24T23:13:47.021 INFO:tasks.workunit.client.0.smithi031.stdout:9/621: sync 2024-04-24T23:13:47.022 INFO:tasks.workunit.client.0.smithi031.stdout:5/480: write d0/d6/d3b/f8d [204832,74453] 0 2024-04-24T23:13:47.023 INFO:tasks.workunit.client.0.smithi031.stdout:9/622: stat d2/dae/d14/d1d/f28 0 2024-04-24T23:13:47.023 INFO:tasks.workunit.client.1.smithi040.stdout:1/173: readlink da/d12/d14/d25/l37 0 2024-04-24T23:13:47.024 INFO:tasks.workunit.client.1.smithi040.stdout:6/183: dwrite d7/d1a/d20/f28 [0,4194304] 0 2024-04-24T23:13:47.024 INFO:tasks.workunit.client.0.smithi031.stdout:9/623: fdatasync d2/dae/d14/d18/fb2 0 2024-04-24T23:13:47.026 INFO:tasks.workunit.client.1.smithi040.stdout:6/184: fsync d7/de/d15/d1e/f22 0 2024-04-24T23:13:47.026 INFO:tasks.workunit.client.1.smithi040.stdout:0/238: symlink df/d1e/l47 0 2024-04-24T23:13:47.029 INFO:tasks.workunit.client.0.smithi031.stdout:2/378: rename d5/d28/c52 to d5/d28/d43/c81 0 2024-04-24T23:13:47.029 INFO:tasks.workunit.client.0.smithi031.stdout:2/379: fdatasync d5/d28/d43/f53 0 2024-04-24T23:13:47.030 INFO:tasks.workunit.client.1.smithi040.stdout:8/154: link d0/d16/c20 d0/d16/c2e 0 2024-04-24T23:13:47.030 INFO:tasks.workunit.client.1.smithi040.stdout:8/155: read - d0/d1f/f2d zero size 2024-04-24T23:13:47.035 INFO:tasks.workunit.client.1.smithi040.stdout:7/225: fdatasync db/d2f/f35 0 2024-04-24T23:13:47.037 INFO:tasks.workunit.client.0.smithi031.stdout:3/461: dread d6/d14/f37 [0,4194304] 0 2024-04-24T23:13:47.038 INFO:tasks.workunit.client.1.smithi040.stdout:3/180: mkdir d1/d9/d39 0 2024-04-24T23:13:47.039 INFO:tasks.workunit.client.0.smithi031.stdout:9/624: fdatasync d2/dae/d14/f46 0 2024-04-24T23:13:47.041 INFO:tasks.workunit.client.1.smithi040.stdout:7/226: dwrite db/d12/d20/d52/d44/f47 [0,4194304] 0 2024-04-24T23:13:47.041 INFO:tasks.workunit.client.1.smithi040.stdout:7/227: fdatasync db/f3c 0 2024-04-24T23:13:47.042 INFO:tasks.workunit.client.1.smithi040.stdout:2/235: symlink da/d29/l53 0 2024-04-24T23:13:47.043 INFO:tasks.workunit.client.1.smithi040.stdout:2/236: dread - da/d10/d15/f32 zero size 2024-04-24T23:13:47.044 INFO:tasks.workunit.client.0.smithi031.stdout:3/462: dwrite d6/df/d16/d2a/f31 [0,4194304] 0 2024-04-24T23:13:47.048 INFO:tasks.workunit.client.0.smithi031.stdout:2/380: creat d5/d28/d4e/f82 x:0 0 0 2024-04-24T23:13:47.049 INFO:tasks.workunit.client.1.smithi040.stdout:5/158: sync 2024-04-24T23:13:47.049 INFO:tasks.workunit.client.0.smithi031.stdout:2/381: dread - d5/d6/df/f79 zero size 2024-04-24T23:13:47.050 INFO:tasks.workunit.client.1.smithi040.stdout:4/162: mkdir da/d1d/d3c 0 2024-04-24T23:13:47.050 INFO:tasks.workunit.client.0.smithi031.stdout:2/382: chown d5/d28/d4e/d5e/l71 842278 1 2024-04-24T23:13:47.052 INFO:tasks.workunit.client.0.smithi031.stdout:7/509: sync 2024-04-24T23:13:47.052 INFO:tasks.workunit.client.0.smithi031.stdout:6/646: sync 2024-04-24T23:13:47.052 INFO:tasks.workunit.client.0.smithi031.stdout:0/565: sync 2024-04-24T23:13:47.052 INFO:tasks.workunit.client.0.smithi031.stdout:4/631: sync 2024-04-24T23:13:47.052 INFO:tasks.workunit.client.0.smithi031.stdout:1/468: sync 2024-04-24T23:13:47.052 INFO:tasks.workunit.client.1.smithi040.stdout:1/174: creat da/d12/d23/f3c x:0 0 0 2024-04-24T23:13:47.054 INFO:tasks.workunit.client.0.smithi031.stdout:6/647: readlink d13/d15/d1d/l7a 0 2024-04-24T23:13:47.056 INFO:tasks.workunit.client.0.smithi031.stdout:3/463: dwrite d6/d14/d1d/f24 [0,4194304] 0 2024-04-24T23:13:47.056 INFO:tasks.workunit.client.0.smithi031.stdout:1/469: read d1/d33/f47 [2151180,110151] 0 2024-04-24T23:13:47.058 INFO:tasks.workunit.client.1.smithi040.stdout:0/239: stat df/d1e/d2b/f29 0 2024-04-24T23:13:47.059 INFO:tasks.workunit.client.0.smithi031.stdout:1/470: chown d1/da/d42 2 1 2024-04-24T23:13:47.059 INFO:tasks.workunit.client.1.smithi040.stdout:0/240: chown df/d14/d1b/f28 478 1 2024-04-24T23:13:47.061 INFO:tasks.workunit.client.1.smithi040.stdout:0/241: read df/d14/d1b/f36 [1350736,89368] 0 2024-04-24T23:13:47.062 INFO:tasks.workunit.client.0.smithi031.stdout:2/383: dwrite d5/f46 [0,4194304] 0 2024-04-24T23:13:47.063 INFO:tasks.workunit.client.1.smithi040.stdout:0/242: chown df/l10 73939526 1 2024-04-24T23:13:47.063 INFO:tasks.workunit.client.0.smithi031.stdout:9/625: rename d2/dae/d14/d1d/c63 to d2/dae/d14/d78/d84/cbb 0 2024-04-24T23:13:47.064 INFO:tasks.workunit.client.1.smithi040.stdout:9/219: getdents d7/d1b/d2f 0 2024-04-24T23:13:47.065 INFO:tasks.workunit.client.1.smithi040.stdout:9/220: chown d7/d1b/d2f/d1f/d31/d3b 35 1 2024-04-24T23:13:47.067 INFO:tasks.workunit.client.0.smithi031.stdout:5/481: creat d0/d6/d3b/d4b/d1d/d2e/fa1 x:0 0 0 2024-04-24T23:13:47.071 INFO:tasks.workunit.client.1.smithi040.stdout:3/181: creat d1/d9/d25/f3a x:0 0 0 2024-04-24T23:13:47.072 INFO:tasks.workunit.client.0.smithi031.stdout:4/632: creat d3/d20/d25/d7e/fc9 x:0 0 0 2024-04-24T23:13:47.072 INFO:tasks.workunit.client.1.smithi040.stdout:7/228: stat db/d2f/c4a 0 2024-04-24T23:13:47.074 INFO:tasks.workunit.client.1.smithi040.stdout:2/237: mknod da/d1c/d37/d38/c54 0 2024-04-24T23:13:47.074 INFO:tasks.workunit.client.1.smithi040.stdout:9/221: dwrite d7/d1b/d2f/d1f/d31/d4d/f43 [0,4194304] 0 2024-04-24T23:13:47.075 INFO:tasks.workunit.client.1.smithi040.stdout:2/238: readlink da/d1c/d37/l43 0 2024-04-24T23:13:47.076 INFO:tasks.workunit.client.1.smithi040.stdout:9/222: readlink d7/da/d17/d40/l4f 0 2024-04-24T23:13:47.077 INFO:tasks.workunit.client.1.smithi040.stdout:5/159: dread d2/d7/d8/f13 [0,4194304] 0 2024-04-24T23:13:47.078 INFO:tasks.workunit.client.0.smithi031.stdout:3/464: unlink d6/d14/d1d/d69/f6d 0 2024-04-24T23:13:47.079 INFO:tasks.workunit.client.0.smithi031.stdout:3/465: write d6/d14/f7d [1763933,6032] 0 2024-04-24T23:13:47.080 INFO:tasks.workunit.client.1.smithi040.stdout:6/185: unlink d7/de/c13 0 2024-04-24T23:13:47.080 INFO:tasks.workunit.client.1.smithi040.stdout:6/186: chown d7/de/d15/c16 556969143 1 2024-04-24T23:13:47.082 INFO:tasks.workunit.client.1.smithi040.stdout:8/156: symlink d0/d8/d22/l2f 0 2024-04-24T23:13:47.083 INFO:tasks.workunit.client.0.smithi031.stdout:1/471: creat d1/da/d42/d69/f95 x:0 0 0 2024-04-24T23:13:47.085 INFO:tasks.workunit.client.0.smithi031.stdout:1/472: read d1/da/d10/f51 [263184,49874] 0 2024-04-24T23:13:47.085 INFO:tasks.workunit.client.0.smithi031.stdout:0/566: rename da/d11/d6a/f79 to da/d11/d6a/dba/fbc 0 2024-04-24T23:13:47.088 INFO:tasks.workunit.client.1.smithi040.stdout:0/243: rmdir df/d1e 39 2024-04-24T23:13:47.093 INFO:tasks.workunit.client.1.smithi040.stdout:3/182: mknod d1/d9/d15/c3b 0 2024-04-24T23:13:47.097 INFO:tasks.workunit.client.1.smithi040.stdout:2/239: mkdir da/d29/d55 0 2024-04-24T23:13:47.098 INFO:tasks.workunit.client.0.smithi031.stdout:6/648: dwrite d13/d1b/f49 [0,4194304] 0 2024-04-24T23:13:47.099 INFO:tasks.workunit.client.1.smithi040.stdout:4/163: rmdir da/d1d/d1e 39 2024-04-24T23:13:47.099 INFO:tasks.workunit.client.1.smithi040.stdout:4/164: chown da 492274 1 2024-04-24T23:13:47.100 INFO:tasks.workunit.client.0.smithi031.stdout:6/649: write d13/d1b/d32/f8a [801612,47820] 0 2024-04-24T23:13:47.100 INFO:tasks.workunit.client.1.smithi040.stdout:4/165: dread da/d1d/f32 [0,4194304] 0 2024-04-24T23:13:47.101 INFO:tasks.workunit.client.0.smithi031.stdout:8/628: write d8/d1a/d1d/d35/d58/d47/f72 [1097751,101516] 0 2024-04-24T23:13:47.101 INFO:tasks.workunit.client.1.smithi040.stdout:9/223: mknod d7/d1b/d2f/c51 0 2024-04-24T23:13:47.102 INFO:tasks.workunit.client.1.smithi040.stdout:9/224: fdatasync d7/d1b/d2f/d1f/d31/d4d/f48 0 2024-04-24T23:13:47.107 INFO:tasks.workunit.client.0.smithi031.stdout:3/466: symlink d6/d72/d29/l92 0 2024-04-24T23:13:47.109 INFO:tasks.workunit.client.1.smithi040.stdout:6/187: rmdir d7/de/d15 39 2024-04-24T23:13:47.111 INFO:tasks.workunit.client.0.smithi031.stdout:4/633: write d3/d23/d69/f13 [822746,91509] 0 2024-04-24T23:13:47.112 INFO:tasks.workunit.client.0.smithi031.stdout:1/473: rename d1/d31/l5d to d1/d33/d41/l96 0 2024-04-24T23:13:47.113 INFO:tasks.workunit.client.0.smithi031.stdout:1/474: fdatasync d1/da/d12/d7e/d7b/d4e/f72 0 2024-04-24T23:13:47.114 INFO:tasks.workunit.client.1.smithi040.stdout:0/244: unlink fb 0 2024-04-24T23:13:47.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/567: fsync da/d18/d1a/d37/d41/f66 0 2024-04-24T23:13:47.117 INFO:tasks.workunit.client.0.smithi031.stdout:2/384: symlink d5/d8/l83 0 2024-04-24T23:13:47.117 INFO:tasks.workunit.client.1.smithi040.stdout:2/240: mknod da/d1c/d37/d39/c56 0 2024-04-24T23:13:47.119 INFO:tasks.workunit.client.0.smithi031.stdout:9/626: mknod d2/dae/d14/d78/d84/d4a/cbc 0 2024-04-24T23:13:47.123 INFO:tasks.workunit.client.1.smithi040.stdout:4/166: unlink da/d1f/f2c 0 2024-04-24T23:13:47.124 INFO:tasks.workunit.client.0.smithi031.stdout:7/510: mknod d1/d45/d75/d9f/d93/cb3 0 2024-04-24T23:13:47.129 INFO:tasks.workunit.client.0.smithi031.stdout:1/475: dread d1/d31/f39 [0,4194304] 0 2024-04-24T23:13:47.130 INFO:tasks.workunit.client.0.smithi031.stdout:3/467: dread d6/d72/d1a/d25/f3d [4194304,4194304] 0 2024-04-24T23:13:47.131 INFO:tasks.workunit.client.1.smithi040.stdout:9/225: creat d7/d1b/d2f/d1f/d34/f52 x:0 0 0 2024-04-24T23:13:47.131 INFO:tasks.workunit.client.0.smithi031.stdout:3/468: write d6/f6b [971699,55399] 0 2024-04-24T23:13:47.133 INFO:tasks.workunit.client.1.smithi040.stdout:1/175: sync 2024-04-24T23:13:47.133 INFO:tasks.workunit.client.1.smithi040.stdout:1/176: stat da/d12/d14 0 2024-04-24T23:13:47.134 INFO:tasks.workunit.client.0.smithi031.stdout:3/469: dread d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:47.136 INFO:tasks.workunit.client.1.smithi040.stdout:3/183: dwrite d1/d9/f27 [0,4194304] 0 2024-04-24T23:13:47.136 INFO:tasks.workunit.client.0.smithi031.stdout:6/650: symlink d13/d15/d1d/d23/d4d/d7b/le0 0 2024-04-24T23:13:47.138 INFO:tasks.workunit.client.1.smithi040.stdout:6/188: symlink d7/d1a/d20/l3c 0 2024-04-24T23:13:47.139 INFO:tasks.workunit.client.1.smithi040.stdout:6/189: chown d7/c11 0 1 2024-04-24T23:13:47.139 INFO:tasks.workunit.client.1.smithi040.stdout:8/157: mknod d0/d28/c30 0 2024-04-24T23:13:47.145 INFO:tasks.workunit.client.1.smithi040.stdout:2/241: creat da/d1c/f57 x:0 0 0 2024-04-24T23:13:47.145 INFO:tasks.workunit.client.1.smithi040.stdout:2/242: stat da/f20 0 2024-04-24T23:13:47.146 INFO:tasks.workunit.client.0.smithi031.stdout:0/568: mkdir da/d18/d5a/d7d/dbd 0 2024-04-24T23:13:47.147 INFO:tasks.workunit.client.0.smithi031.stdout:0/569: readlink da/d20/d4e/l6f 0 2024-04-24T23:13:47.149 INFO:tasks.workunit.client.0.smithi031.stdout:2/385: rename d5/d8/d6e/d70 to d5/d8/d15/d56/d84 0 2024-04-24T23:13:47.150 INFO:tasks.workunit.client.0.smithi031.stdout:2/386: chown d5/d8/d45/d5b/d49/f7e 0 1 2024-04-24T23:13:47.150 INFO:tasks.workunit.client.0.smithi031.stdout:5/482: creat d0/d6/d3b/d4b/dc/fa2 x:0 0 0 2024-04-24T23:13:47.151 INFO:tasks.workunit.client.0.smithi031.stdout:5/483: stat d0/d6/d12/f63 0 2024-04-24T23:13:47.152 INFO:tasks.workunit.client.0.smithi031.stdout:9/627: dread - d2/dae/d14/d78/d84/d4a/f76 zero size 2024-04-24T23:13:47.155 INFO:tasks.workunit.client.1.smithi040.stdout:4/167: dwrite da/d1d/d2b/f39 [0,4194304] 0 2024-04-24T23:13:47.156 INFO:tasks.workunit.client.1.smithi040.stdout:1/177: creat da/d12/d18/d2a/f3d x:0 0 0 2024-04-24T23:13:47.160 INFO:tasks.workunit.client.1.smithi040.stdout:5/160: rename d2/c2d to d2/d7/c41 0 2024-04-24T23:13:47.161 INFO:tasks.workunit.client.0.smithi031.stdout:4/634: dwrite d3/d20/d25/d7e/fa1 [0,4194304] 0 2024-04-24T23:13:47.163 INFO:tasks.workunit.client.0.smithi031.stdout:6/651: write f11 [1489538,126694] 0 2024-04-24T23:13:47.165 INFO:tasks.workunit.client.0.smithi031.stdout:6/652: stat d13/d1b/d32/c65 0 2024-04-24T23:13:47.166 INFO:tasks.workunit.client.1.smithi040.stdout:0/245: unlink df/d1e/l47 0 2024-04-24T23:13:47.167 INFO:tasks.workunit.client.0.smithi031.stdout:6/653: truncate d13/d15/d1d/d23/d2d/f3d 4691771 0 2024-04-24T23:13:47.167 INFO:tasks.workunit.client.1.smithi040.stdout:0/246: write df/f39 [1724381,24990] 0 2024-04-24T23:13:47.168 INFO:tasks.workunit.client.1.smithi040.stdout:0/247: fdatasync df/d12/f22 0 2024-04-24T23:13:47.171 INFO:tasks.workunit.client.0.smithi031.stdout:1/476: write d1/f18 [2477181,65341] 0 2024-04-24T23:13:47.176 INFO:tasks.workunit.client.1.smithi040.stdout:7/229: rename db/d12/d3f/f49 to db/d12/d20/d52/d4b/f57 0 2024-04-24T23:13:47.177 INFO:tasks.workunit.client.0.smithi031.stdout:0/570: creat da/d18/d1a/d76/fbe x:0 0 0 2024-04-24T23:13:47.181 INFO:tasks.workunit.client.0.smithi031.stdout:5/484: creat d0/d7/fa3 x:0 0 0 2024-04-24T23:13:47.184 INFO:tasks.workunit.client.1.smithi040.stdout:9/226: truncate f4 5075986 0 2024-04-24T23:13:47.184 INFO:tasks.workunit.client.1.smithi040.stdout:2/243: creat da/d29/d55/f58 x:0 0 0 2024-04-24T23:13:47.186 INFO:tasks.workunit.client.1.smithi040.stdout:0/248: symlink df/d14/d1b/l48 0 2024-04-24T23:13:47.188 INFO:tasks.workunit.client.1.smithi040.stdout:1/178: creat da/d35/d38/f3e x:0 0 0 2024-04-24T23:13:47.191 INFO:tasks.workunit.client.0.smithi031.stdout:3/470: mknod d6/c93 0 2024-04-24T23:13:47.192 INFO:tasks.workunit.client.0.smithi031.stdout:3/471: fsync d6/f6b 0 2024-04-24T23:13:47.193 INFO:tasks.workunit.client.0.smithi031.stdout:2/387: dwrite d5/d8/d45/f4b [0,4194304] 0 2024-04-24T23:13:47.193 INFO:tasks.workunit.client.1.smithi040.stdout:4/168: rename da/f20 to da/d1f/f3d 0 2024-04-24T23:13:47.194 INFO:tasks.workunit.client.0.smithi031.stdout:2/388: stat d5/d28/d43/f55 0 2024-04-24T23:13:47.197 INFO:tasks.workunit.client.0.smithi031.stdout:7/511: write d1/f28 [1020731,18048] 0 2024-04-24T23:13:47.198 INFO:tasks.workunit.client.0.smithi031.stdout:6/654: mknod d13/d15/d1d/d23/d2d/dbf/ce1 0 2024-04-24T23:13:47.198 INFO:tasks.workunit.client.0.smithi031.stdout:7/512: stat d1/d45/d75/d9f/d66/l7f 0 2024-04-24T23:13:47.200 INFO:tasks.workunit.client.1.smithi040.stdout:5/161: symlink d2/d7/d8/db/d1e/d22/d3e/l42 0 2024-04-24T23:13:47.200 INFO:tasks.workunit.client.1.smithi040.stdout:5/162: stat d2/d2f/l29 0 2024-04-24T23:13:47.201 INFO:tasks.workunit.client.1.smithi040.stdout:9/227: rmdir d7/d1b/d2f 39 2024-04-24T23:13:47.203 INFO:tasks.workunit.client.0.smithi031.stdout:1/477: rename d1/d33/d36/c79 to d1/da/d61/c97 0 2024-04-24T23:13:47.204 INFO:tasks.workunit.client.0.smithi031.stdout:1/478: chown d1/da/d42/l57 1 1 2024-04-24T23:13:47.209 INFO:tasks.workunit.client.0.smithi031.stdout:9/628: mkdir d2/d43/d9d/dbd 0 2024-04-24T23:13:47.210 INFO:tasks.workunit.client.0.smithi031.stdout:9/629: readlink d2/dae/d14/d1d/d45/d65/l80 0 2024-04-24T23:13:47.211 INFO:tasks.workunit.client.1.smithi040.stdout:9/228: truncate d7/f37 1251560 0 2024-04-24T23:13:47.212 INFO:tasks.workunit.client.1.smithi040.stdout:0/249: mkdir df/d12/d49 0 2024-04-24T23:13:47.213 INFO:tasks.workunit.client.0.smithi031.stdout:4/635: mknod d3/d20/d25/d95/cca 0 2024-04-24T23:13:47.213 INFO:tasks.workunit.client.1.smithi040.stdout:0/250: write df/d1e/d20/f43 [824497,105533] 0 2024-04-24T23:13:47.213 INFO:tasks.workunit.client.1.smithi040.stdout:0/251: stat df/d12/f22 0 2024-04-24T23:13:47.214 INFO:tasks.workunit.client.0.smithi031.stdout:2/389: rmdir d5/d28/d4e 39 2024-04-24T23:13:47.217 INFO:tasks.workunit.client.1.smithi040.stdout:2/244: rename da/d10/d15/c24 to da/d1c/c59 0 2024-04-24T23:13:47.218 INFO:tasks.workunit.client.1.smithi040.stdout:2/245: write da/d10/d15/f32 [544122,123107] 0 2024-04-24T23:13:47.218 INFO:tasks.workunit.client.0.smithi031.stdout:7/513: mknod d1/d2/dd/cb4 0 2024-04-24T23:13:47.220 INFO:tasks.workunit.client.1.smithi040.stdout:4/169: creat da/d1d/d3c/f3e x:0 0 0 2024-04-24T23:13:47.221 INFO:tasks.workunit.client.0.smithi031.stdout:8/629: getdents d8/d15/d37/d43/da5 0 2024-04-24T23:13:47.224 INFO:tasks.workunit.client.1.smithi040.stdout:8/158: sync 2024-04-24T23:13:47.224 INFO:tasks.workunit.client.1.smithi040.stdout:6/190: sync 2024-04-24T23:13:47.224 INFO:tasks.workunit.client.1.smithi040.stdout:3/184: sync 2024-04-24T23:13:47.225 INFO:tasks.workunit.client.1.smithi040.stdout:3/185: stat d1/d2d 0 2024-04-24T23:13:47.226 INFO:tasks.workunit.client.1.smithi040.stdout:8/159: read - d0/d1f/f2d zero size 2024-04-24T23:13:47.226 INFO:tasks.workunit.client.1.smithi040.stdout:4/170: dwrite da/f1c [4194304,4194304] 0 2024-04-24T23:13:47.226 INFO:tasks.workunit.client.0.smithi031.stdout:1/479: mkdir d1/da/d12/d7e/d7b/d4e/d98 0 2024-04-24T23:13:47.227 INFO:tasks.workunit.client.1.smithi040.stdout:9/229: chown d7/d1b/d2f/d1f/d31/d4d/f43 608182 1 2024-04-24T23:13:47.228 INFO:tasks.workunit.client.1.smithi040.stdout:4/171: write da/d1d/d2b/f35 [912429,82139] 0 2024-04-24T23:13:47.228 INFO:tasks.workunit.client.0.smithi031.stdout:5/485: fsync d0/d7/f58 0 2024-04-24T23:13:47.232 INFO:tasks.workunit.client.1.smithi040.stdout:5/163: rename d2/d2f/d23 to d2/d7/d8/db/d1e/d22/d27/d3c/d43 0 2024-04-24T23:13:47.234 INFO:tasks.workunit.client.0.smithi031.stdout:3/472: unlink d6/d72/d1a/l3f 0 2024-04-24T23:13:47.237 INFO:tasks.workunit.client.0.smithi031.stdout:4/636: creat d3/d23/d37/fcb x:0 0 0 2024-04-24T23:13:47.240 INFO:tasks.workunit.client.1.smithi040.stdout:7/230: getdents db/d43 0 2024-04-24T23:13:47.242 INFO:tasks.workunit.client.0.smithi031.stdout:3/473: dwrite d6/d72/d60/f84 [0,4194304] 0 2024-04-24T23:13:47.242 INFO:tasks.workunit.client.1.smithi040.stdout:6/191: write d7/de/d15/d1e/f22 [4274765,128065] 0 2024-04-24T23:13:47.243 INFO:tasks.workunit.client.0.smithi031.stdout:3/474: readlink d6/d14/d1d/l49 0 2024-04-24T23:13:47.244 INFO:tasks.workunit.client.0.smithi031.stdout:7/514: unlink d1/d2/fb2 0 2024-04-24T23:13:47.245 INFO:tasks.workunit.client.0.smithi031.stdout:3/475: chown d6/df/d16/d53/f67 837 1 2024-04-24T23:13:47.246 INFO:tasks.workunit.client.0.smithi031.stdout:8/630: creat d8/d15/d37/d43/d7b/d87/dad/fda x:0 0 0 2024-04-24T23:13:47.247 INFO:tasks.workunit.client.1.smithi040.stdout:0/252: mknod df/d14/c4a 0 2024-04-24T23:13:47.248 INFO:tasks.workunit.client.1.smithi040.stdout:4/172: rmdir da/d1d 39 2024-04-24T23:13:47.248 INFO:tasks.workunit.client.1.smithi040.stdout:0/253: truncate df/d1e/d20/d21/f32 930761 0 2024-04-24T23:13:47.249 INFO:tasks.workunit.client.1.smithi040.stdout:0/254: write df/d1e/f2f [2103724,64346] 0 2024-04-24T23:13:47.250 INFO:tasks.workunit.client.1.smithi040.stdout:1/179: getdents da/d35 0 2024-04-24T23:13:47.250 INFO:tasks.workunit.client.1.smithi040.stdout:0/255: chown df/d1e 220839672 1 2024-04-24T23:13:47.252 INFO:tasks.workunit.client.0.smithi031.stdout:4/637: creat d3/d23/d3b/fcc x:0 0 0 2024-04-24T23:13:47.257 INFO:tasks.workunit.client.0.smithi031.stdout:7/515: creat d1/d45/da0/d4f/fb5 x:0 0 0 2024-04-24T23:13:47.265 INFO:tasks.workunit.client.0.smithi031.stdout:0/571: getdents da/d11/d6a/dba 0 2024-04-24T23:13:47.267 INFO:tasks.workunit.client.0.smithi031.stdout:4/638: dread d3/d23/f3a [0,4194304] 0 2024-04-24T23:13:47.273 INFO:tasks.workunit.client.0.smithi031.stdout:4/639: dwrite d3/da3/faf [0,4194304] 0 2024-04-24T23:13:47.275 INFO:tasks.workunit.client.0.smithi031.stdout:9/630: rename d2/dae/d14/d18 to d2/d16/dbe 0 2024-04-24T23:13:47.276 INFO:tasks.workunit.client.0.smithi031.stdout:3/476: write d6/df/d16/d2a/f68 [81350,81423] 0 2024-04-24T23:13:47.277 INFO:tasks.workunit.client.0.smithi031.stdout:2/390: mknod d5/d28/d4e/d7f/c85 0 2024-04-24T23:13:47.278 INFO:tasks.workunit.client.0.smithi031.stdout:6/655: getdents d13/d1b/dd0 0 2024-04-24T23:13:47.282 INFO:tasks.workunit.client.0.smithi031.stdout:0/572: fdatasync da/d18/d1a/d37/d41/f66 0 2024-04-24T23:13:47.284 INFO:tasks.workunit.client.0.smithi031.stdout:0/573: read da/d18/d1a/d37/f5e [259902,89774] 0 2024-04-24T23:13:47.285 INFO:tasks.workunit.client.0.smithi031.stdout:5/486: creat d0/d6/d3b/d4b/d1d/fa4 x:0 0 0 2024-04-24T23:13:47.287 INFO:tasks.workunit.client.0.smithi031.stdout:4/640: unlink d3/d5e/f98 0 2024-04-24T23:13:47.287 INFO:tasks.workunit.client.1.smithi040.stdout:6/192: truncate d7/de/ff 2772700 0 2024-04-24T23:13:47.288 INFO:tasks.workunit.client.0.smithi031.stdout:0/574: dread da/d22/f30 [0,4194304] 0 2024-04-24T23:13:47.288 INFO:tasks.workunit.client.1.smithi040.stdout:1/180: creat da/d12/d14/f3f x:0 0 0 2024-04-24T23:13:47.288 INFO:tasks.workunit.client.0.smithi031.stdout:9/631: mknod d2/dae/d14/d78/d84/d4a/d4b/d96/cbf 0 2024-04-24T23:13:47.289 INFO:tasks.workunit.client.1.smithi040.stdout:1/181: write da/d35/f3a [633111,98204] 0 2024-04-24T23:13:47.290 INFO:tasks.workunit.client.0.smithi031.stdout:4/641: write d3/d23/d69/f30 [4494749,86702] 0 2024-04-24T23:13:47.290 INFO:tasks.workunit.client.0.smithi031.stdout:3/477: creat d6/d14/d1d/d69/f94 x:0 0 0 2024-04-24T23:13:47.291 INFO:tasks.workunit.client.1.smithi040.stdout:1/182: write da/d12/d18/f22 [1572154,23989] 0 2024-04-24T23:13:47.292 INFO:tasks.workunit.client.1.smithi040.stdout:1/183: chown da/d12/d18/f27 105 1 2024-04-24T23:13:47.292 INFO:tasks.workunit.client.1.smithi040.stdout:1/184: chown da/d12/f1c 844188841 1 2024-04-24T23:13:47.295 INFO:tasks.workunit.client.1.smithi040.stdout:4/173: mkdir da/d3f 0 2024-04-24T23:13:47.296 INFO:tasks.workunit.client.0.smithi031.stdout:5/487: dwrite d0/d6/d3b/d4b/d1d/fa4 [0,4194304] 0 2024-04-24T23:13:47.301 INFO:tasks.workunit.client.1.smithi040.stdout:9/230: rename d7/da/d17/l2d to d7/da/l53 0 2024-04-24T23:13:47.301 INFO:tasks.workunit.client.1.smithi040.stdout:4/174: dread da/d1f/f3d [0,4194304] 0 2024-04-24T23:13:47.302 INFO:tasks.workunit.client.1.smithi040.stdout:9/231: write d7/d1b/d2f/d1f/d34/f49 [835653,95044] 0 2024-04-24T23:13:47.303 INFO:tasks.workunit.client.0.smithi031.stdout:0/575: dwrite da/f9e [0,4194304] 0 2024-04-24T23:13:47.305 INFO:tasks.workunit.client.1.smithi040.stdout:5/164: creat d2/d7/d8/db/d1e/d22/f44 x:0 0 0 2024-04-24T23:13:47.308 INFO:tasks.workunit.client.1.smithi040.stdout:2/246: link da/d1c/l2a da/d10/l5a 0 2024-04-24T23:13:47.309 INFO:tasks.workunit.client.0.smithi031.stdout:7/516: dwrite d1/d2/dd/d33/d5e/d64/f8e [0,4194304] 0 2024-04-24T23:13:47.310 INFO:tasks.workunit.client.0.smithi031.stdout:7/517: fdatasync d1/d2/dd/f97 0 2024-04-24T23:13:47.311 INFO:tasks.workunit.client.1.smithi040.stdout:6/193: creat d7/de/d15/d1e/f3d x:0 0 0 2024-04-24T23:13:47.311 INFO:tasks.workunit.client.0.smithi031.stdout:1/480: link d1/d31/c3c d1/da/d10/c99 0 2024-04-24T23:13:47.313 INFO:tasks.workunit.client.0.smithi031.stdout:1/481: write d1/d1d/f6d [524651,64516] 0 2024-04-24T23:13:47.314 INFO:tasks.workunit.client.0.smithi031.stdout:1/482: truncate d1/f85 389941 0 2024-04-24T23:13:47.316 INFO:tasks.workunit.client.1.smithi040.stdout:8/160: sync 2024-04-24T23:13:47.316 INFO:tasks.workunit.client.1.smithi040.stdout:3/186: getdents d1/d9 0 2024-04-24T23:13:47.317 INFO:tasks.workunit.client.0.smithi031.stdout:9/632: chown d2/dae/c30 0 1 2024-04-24T23:13:47.318 INFO:tasks.workunit.client.1.smithi040.stdout:1/185: rmdir da/d35 39 2024-04-24T23:13:47.319 INFO:tasks.workunit.client.1.smithi040.stdout:6/194: dwrite d7/de/d15/d1e/f3d [0,4194304] 0 2024-04-24T23:13:47.320 INFO:tasks.workunit.client.0.smithi031.stdout:3/478: mkdir d6/df/d16/d2a/d95 0 2024-04-24T23:13:47.322 INFO:tasks.workunit.client.1.smithi040.stdout:6/195: write d7/de/d15/d1e/f3d [531866,19421] 0 2024-04-24T23:13:47.323 INFO:tasks.workunit.client.0.smithi031.stdout:1/483: dwrite d1/dc/d11/f13 [8388608,4194304] 0 2024-04-24T23:13:47.324 INFO:tasks.workunit.client.0.smithi031.stdout:3/479: write d6/d14/d1d/d59/f5e [898985,34530] 0 2024-04-24T23:13:47.324 INFO:tasks.workunit.client.1.smithi040.stdout:5/165: chown d2/d2f/d14/c1c 1102 1 2024-04-24T23:13:47.325 INFO:tasks.workunit.client.1.smithi040.stdout:8/161: dwrite d0/d1f/f2d [0,4194304] 0 2024-04-24T23:13:47.326 INFO:tasks.workunit.client.1.smithi040.stdout:6/196: write d7/de/d15/d1e/f3d [1700081,71278] 0 2024-04-24T23:13:47.328 INFO:tasks.workunit.client.1.smithi040.stdout:2/247: symlink da/d1c/d37/l5b 0 2024-04-24T23:13:47.331 INFO:tasks.workunit.client.0.smithi031.stdout:5/488: mknod d0/d6/d3b/d4b/dc/d8b/ca5 0 2024-04-24T23:13:47.335 INFO:tasks.workunit.client.1.smithi040.stdout:3/187: readlink d1/d9/l11 0 2024-04-24T23:13:47.335 INFO:tasks.workunit.client.0.smithi031.stdout:8/631: getdents d8/d1a/d73 0 2024-04-24T23:13:47.336 INFO:tasks.workunit.client.1.smithi040.stdout:0/256: getdents df/d14/d1b 0 2024-04-24T23:13:47.338 INFO:tasks.workunit.client.0.smithi031.stdout:0/576: mkdir da/d86/dbf 0 2024-04-24T23:13:47.338 INFO:tasks.workunit.client.1.smithi040.stdout:0/257: write df/d12/d1a/f3a [576627,44274] 0 2024-04-24T23:13:47.345 INFO:tasks.workunit.client.1.smithi040.stdout:4/175: mkdir da/d3f/d40 0 2024-04-24T23:13:47.345 INFO:tasks.workunit.client.1.smithi040.stdout:5/166: dread d2/d7/d8/db/d1e/f2e [0,4194304] 0 2024-04-24T23:13:47.345 INFO:tasks.workunit.client.1.smithi040.stdout:3/188: dwrite d1/f5 [0,4194304] 0 2024-04-24T23:13:47.347 INFO:tasks.workunit.client.1.smithi040.stdout:3/189: chown d1/d2d/cc 2 1 2024-04-24T23:13:47.347 INFO:tasks.workunit.client.0.smithi031.stdout:0/577: dwrite da/d18/d1a/d37/d41/d92/fa8 [0,4194304] 0 2024-04-24T23:13:47.351 INFO:tasks.workunit.client.1.smithi040.stdout:3/190: dread d1/f8 [0,4194304] 0 2024-04-24T23:13:47.351 INFO:tasks.workunit.client.0.smithi031.stdout:9/633: mkdir d2/d43/d9d/dc0 0 2024-04-24T23:13:47.354 INFO:tasks.workunit.client.0.smithi031.stdout:2/391: link d5/d8/d15/f5f d5/d28/f86 0 2024-04-24T23:13:47.354 INFO:tasks.workunit.client.0.smithi031.stdout:5/489: dread d0/d6/d3b/d4b/d1d/f9d [0,4194304] 0 2024-04-24T23:13:47.359 INFO:tasks.workunit.client.0.smithi031.stdout:4/642: creat d3/d20/d25/d60/d63/db9/fcd x:0 0 0 2024-04-24T23:13:47.359 INFO:tasks.workunit.client.0.smithi031.stdout:3/480: readlink d6/df/d16/d2a/l39 0 2024-04-24T23:13:47.361 INFO:tasks.workunit.client.1.smithi040.stdout:8/162: rename d0/d16/d1b/c2a to d0/d16/d19/c31 0 2024-04-24T23:13:47.361 INFO:tasks.workunit.client.0.smithi031.stdout:9/634: dwrite d2/dae/f8f [0,4194304] 0 2024-04-24T23:13:47.362 INFO:tasks.workunit.client.1.smithi040.stdout:8/163: rename d0 to d0/d1f/d32 22 2024-04-24T23:13:47.362 INFO:tasks.workunit.client.0.smithi031.stdout:1/484: truncate d1/dc/fe 3316571 0 2024-04-24T23:13:47.363 INFO:tasks.workunit.client.0.smithi031.stdout:1/485: chown d1/da/d42 11543 1 2024-04-24T23:13:47.366 INFO:tasks.workunit.client.1.smithi040.stdout:0/258: rmdir df/d1e/d20/d21 39 2024-04-24T23:13:47.368 INFO:tasks.workunit.client.1.smithi040.stdout:0/259: write df/d1e/d20/f43 [1240309,80055] 0 2024-04-24T23:13:47.368 INFO:tasks.workunit.client.1.smithi040.stdout:1/186: symlink da/d35/d38/l40 0 2024-04-24T23:13:47.369 INFO:tasks.workunit.client.0.smithi031.stdout:8/632: write d8/d15/d37/f5e [2400827,116666] 0 2024-04-24T23:13:47.372 INFO:tasks.workunit.client.1.smithi040.stdout:5/167: symlink d2/d7/d8/db/l45 0 2024-04-24T23:13:47.378 INFO:tasks.workunit.client.1.smithi040.stdout:3/191: mkdir d1/d9/d18/d3c 0 2024-04-24T23:13:47.379 INFO:tasks.workunit.client.1.smithi040.stdout:3/192: chown d1/d9/d15/f1d 54280695 1 2024-04-24T23:13:47.379 INFO:tasks.workunit.client.0.smithi031.stdout:2/392: creat d5/f87 x:0 0 0 2024-04-24T23:13:47.380 INFO:tasks.workunit.client.1.smithi040.stdout:3/193: write d1/d2d/de/f34 [1075996,15787] 0 2024-04-24T23:13:47.382 INFO:tasks.workunit.client.1.smithi040.stdout:7/231: sync 2024-04-24T23:13:47.384 INFO:tasks.workunit.client.1.smithi040.stdout:7/232: write db/d12/d3f/f41 [1811884,76255] 0 2024-04-24T23:13:47.384 INFO:tasks.workunit.client.0.smithi031.stdout:4/643: mkdir d3/d23/d69/dce 0 2024-04-24T23:13:47.385 INFO:tasks.workunit.client.0.smithi031.stdout:7/518: dread d1/d45/da0/f5f [0,4194304] 0 2024-04-24T23:13:47.387 INFO:tasks.workunit.client.1.smithi040.stdout:4/176: mkdir da/d3f/d40/d41 0 2024-04-24T23:13:47.388 INFO:tasks.workunit.client.1.smithi040.stdout:8/164: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:47.389 INFO:tasks.workunit.client.1.smithi040.stdout:6/197: dwrite d7/d2c/f31 [0,4194304] 0 2024-04-24T23:13:47.390 INFO:tasks.workunit.client.1.smithi040.stdout:6/198: chown d7/de/d15/d2f/l38 3166 1 2024-04-24T23:13:47.392 INFO:tasks.workunit.client.1.smithi040.stdout:0/260: unlink df/d1e/d2b/l38 0 2024-04-24T23:13:47.392 INFO:tasks.workunit.client.0.smithi031.stdout:3/481: dwrite d6/d72/f7c [0,4194304] 0 2024-04-24T23:13:47.394 INFO:tasks.workunit.client.0.smithi031.stdout:6/656: link d13/d1b/d32/db4/lbb d13/d15/d1d/d23/d44/d47/d69/le2 0 2024-04-24T23:13:47.398 INFO:tasks.workunit.client.1.smithi040.stdout:1/187: unlink da/d12/d14/d25/l37 0 2024-04-24T23:13:47.400 INFO:tasks.workunit.client.0.smithi031.stdout:2/393: creat d5/d8/d45/d5b/d49/f88 x:0 0 0 2024-04-24T23:13:47.401 INFO:tasks.workunit.client.1.smithi040.stdout:2/248: sync 2024-04-24T23:13:47.401 INFO:tasks.workunit.client.0.smithi031.stdout:0/578: dread da/d22/f27 [0,4194304] 0 2024-04-24T23:13:47.402 INFO:tasks.workunit.client.1.smithi040.stdout:2/249: stat da/d1c/d37/d39 0 2024-04-24T23:13:47.403 INFO:tasks.workunit.client.1.smithi040.stdout:2/250: write da/f19 [3076978,28430] 0 2024-04-24T23:13:47.404 INFO:tasks.workunit.client.1.smithi040.stdout:2/251: chown da/d29/f48 57 1 2024-04-24T23:13:47.404 INFO:tasks.workunit.client.1.smithi040.stdout:2/252: stat da/d10/f2b 0 2024-04-24T23:13:47.405 INFO:tasks.workunit.client.0.smithi031.stdout:4/644: symlink d3/da3/lcf 0 2024-04-24T23:13:47.407 INFO:tasks.workunit.client.1.smithi040.stdout:3/194: rename d1/d9/d25/l28 to d1/d2d/de/d30/l3d 0 2024-04-24T23:13:47.418 INFO:tasks.workunit.client.1.smithi040.stdout:9/232: truncate d7/fe 817096 0 2024-04-24T23:13:47.419 INFO:tasks.workunit.client.1.smithi040.stdout:7/233: truncate db/f2b 714650 0 2024-04-24T23:13:47.422 INFO:tasks.workunit.client.0.smithi031.stdout:3/482: rename d6/df/c45 to d6/df/d16/d2a/d95/c96 0 2024-04-24T23:13:47.423 INFO:tasks.workunit.client.1.smithi040.stdout:8/165: chown d0/d16/c20 761052353 1 2024-04-24T23:13:47.425 INFO:tasks.workunit.client.0.smithi031.stdout:6/657: mkdir d13/d1b/de3 0 2024-04-24T23:13:47.426 INFO:tasks.workunit.client.0.smithi031.stdout:6/658: dread - d13/d15/d1d/d23/d24/d63/f67 zero size 2024-04-24T23:13:47.427 INFO:tasks.workunit.client.1.smithi040.stdout:6/199: unlink d7/cc 0 2024-04-24T23:13:47.429 INFO:tasks.workunit.client.0.smithi031.stdout:2/394: readlink d5/d8/d45/d5b/d49/l47 0 2024-04-24T23:13:47.429 INFO:tasks.workunit.client.1.smithi040.stdout:0/261: dread df/d1e/d2b/f29 [0,4194304] 0 2024-04-24T23:13:47.430 INFO:tasks.workunit.client.0.smithi031.stdout:8/633: dread d8/d15/f69 [0,4194304] 0 2024-04-24T23:13:47.430 INFO:tasks.workunit.client.0.smithi031.stdout:0/579: fdatasync da/d11/d6a/f6e 0 2024-04-24T23:13:47.432 INFO:tasks.workunit.client.0.smithi031.stdout:4/645: mknod d3/d23/d37/d86/cd0 0 2024-04-24T23:13:47.433 INFO:tasks.workunit.client.1.smithi040.stdout:5/168: getdents d2/d2f/d14 0 2024-04-24T23:13:47.433 INFO:tasks.workunit.client.1.smithi040.stdout:6/200: dwrite d7/de/d15/f17 [0,4194304] 0 2024-04-24T23:13:47.434 INFO:tasks.workunit.client.0.smithi031.stdout:9/635: creat d2/dae/d14/d1d/d45/fc1 x:0 0 0 2024-04-24T23:13:47.435 INFO:tasks.workunit.client.1.smithi040.stdout:1/188: truncate da/d12/d18/f1d 3048027 0 2024-04-24T23:13:47.436 INFO:tasks.workunit.client.0.smithi031.stdout:7/519: mkdir d1/d2/dd/d33/d5e/d64/d6d/da1/db6 0 2024-04-24T23:13:47.442 INFO:tasks.workunit.client.1.smithi040.stdout:2/253: rename da/d10/f26 to da/d1c/f5c 0 2024-04-24T23:13:47.442 INFO:tasks.workunit.client.1.smithi040.stdout:2/254: dread - da/d29/d55/f58 zero size 2024-04-24T23:13:47.443 INFO:tasks.workunit.client.0.smithi031.stdout:3/483: unlink d6/d72/d1a/d25/f5f 0 2024-04-24T23:13:47.445 INFO:tasks.workunit.client.0.smithi031.stdout:7/520: dwrite d1/d2/dd/d33/d5e/d64/f91 [0,4194304] 0 2024-04-24T23:13:47.446 INFO:tasks.workunit.client.0.smithi031.stdout:1/486: write d1/f66 [700531,54016] 0 2024-04-24T23:13:47.446 INFO:tasks.workunit.client.0.smithi031.stdout:3/484: write d6/df/d16/d2a/f4f [4620804,4255] 0 2024-04-24T23:13:47.447 INFO:tasks.workunit.client.1.smithi040.stdout:9/233: symlink d7/d1b/d2f/d1f/d31/l54 0 2024-04-24T23:13:47.448 INFO:tasks.workunit.client.1.smithi040.stdout:2/255: dwrite da/d10/f2b [0,4194304] 0 2024-04-24T23:13:47.448 INFO:tasks.workunit.client.0.smithi031.stdout:1/487: write d1/d1d/f8e [117879,58941] 0 2024-04-24T23:13:47.448 INFO:tasks.workunit.client.0.smithi031.stdout:2/395: symlink d5/d6/d29/l89 0 2024-04-24T23:13:47.449 INFO:tasks.workunit.client.1.smithi040.stdout:2/256: dread - da/d10/f34 zero size 2024-04-24T23:13:47.449 INFO:tasks.workunit.client.1.smithi040.stdout:2/257: dread - da/d10/f31 zero size 2024-04-24T23:13:47.449 INFO:tasks.workunit.client.0.smithi031.stdout:1/488: stat d1/d33/d41 0 2024-04-24T23:13:47.450 INFO:tasks.workunit.client.0.smithi031.stdout:7/521: readlink d1/d2/d60/l8c 0 2024-04-24T23:13:47.450 INFO:tasks.workunit.client.1.smithi040.stdout:7/234: rmdir db/d12/d20/d52/d4b 39 2024-04-24T23:13:47.457 INFO:tasks.workunit.client.1.smithi040.stdout:8/166: mknod d0/d8/dc/c33 0 2024-04-24T23:13:47.458 INFO:tasks.workunit.client.0.smithi031.stdout:0/580: fdatasync da/d22/f27 0 2024-04-24T23:13:47.463 INFO:tasks.workunit.client.0.smithi031.stdout:4/646: rmdir d3/d23/d69/d3f 39 2024-04-24T23:13:47.470 INFO:tasks.workunit.client.0.smithi031.stdout:9/636: dwrite d2/f32 [4194304,4194304] 0 2024-04-24T23:13:47.472 INFO:tasks.workunit.client.1.smithi040.stdout:1/189: unlink da/d35/f3a 0 2024-04-24T23:13:47.478 INFO:tasks.workunit.client.1.smithi040.stdout:1/190: dwrite da/d12/d14/f2d [0,4194304] 0 2024-04-24T23:13:47.480 INFO:tasks.workunit.client.1.smithi040.stdout:1/191: write da/d12/d14/f3f [45639,36117] 0 2024-04-24T23:13:47.483 INFO:tasks.workunit.client.1.smithi040.stdout:9/234: readlink d7/da/l14 0 2024-04-24T23:13:47.491 INFO:tasks.workunit.client.1.smithi040.stdout:4/177: rename da/d1f/f3d to da/d1d/d1e/f42 0 2024-04-24T23:13:47.491 INFO:tasks.workunit.client.1.smithi040.stdout:4/178: chown da/l17 15120599 1 2024-04-24T23:13:47.492 INFO:tasks.workunit.client.0.smithi031.stdout:8/634: fdatasync d8/d1a/d1d/f7d 0 2024-04-24T23:13:47.494 INFO:tasks.workunit.client.1.smithi040.stdout:8/167: mkdir d0/d8/dc/d34 0 2024-04-24T23:13:47.495 INFO:tasks.workunit.client.1.smithi040.stdout:8/168: chown d0/d16 251583 1 2024-04-24T23:13:47.497 INFO:tasks.workunit.client.1.smithi040.stdout:0/262: mkdir df/d12/d4b 0 2024-04-24T23:13:47.498 INFO:tasks.workunit.client.1.smithi040.stdout:0/263: chown l5 426 1 2024-04-24T23:13:47.498 INFO:tasks.workunit.client.1.smithi040.stdout:0/264: write df/f34 [365456,101999] 0 2024-04-24T23:13:47.500 INFO:tasks.workunit.client.0.smithi031.stdout:0/581: read da/d18/d1a/d37/f26 [1765787,65319] 0 2024-04-24T23:13:47.502 INFO:tasks.workunit.client.0.smithi031.stdout:4/647: symlink d3/d23/d37/ld1 0 2024-04-24T23:13:47.507 INFO:tasks.workunit.client.1.smithi040.stdout:3/195: link d1/d9/d15/c3b d1/d9/d18/c3e 0 2024-04-24T23:13:47.510 INFO:tasks.workunit.client.1.smithi040.stdout:1/192: mkdir da/d12/d18/d2a/d41 0 2024-04-24T23:13:47.510 INFO:tasks.workunit.client.1.smithi040.stdout:1/193: chown da/d12/d18/f27 10 1 2024-04-24T23:13:47.514 INFO:tasks.workunit.client.1.smithi040.stdout:9/235: rename d7/d1b to d7/da/d17/d40/d55 0 2024-04-24T23:13:47.542 INFO:tasks.workunit.client.1.smithi040.stdout:6/201: sync 2024-04-24T23:13:47.593 INFO:tasks.workunit.client.1.smithi040.stdout:7/235: sync 2024-04-24T23:13:47.626 INFO:tasks.workunit.client.1.smithi040.stdout:9/236: sync 2024-04-24T23:13:47.628 INFO:tasks.workunit.client.0.smithi031.stdout:7/522: creat d1/d45/d75/d9f/d65/da4/fb7 x:0 0 0 2024-04-24T23:13:47.630 INFO:tasks.workunit.client.1.smithi040.stdout:9/237: dread d7/da/d17/d40/d55/d2f/d1f/d31/d4d/f43 [0,4194304] 0 2024-04-24T23:13:47.631 INFO:tasks.workunit.client.0.smithi031.stdout:1/489: write d1/d33/f47 [505451,65295] 0 2024-04-24T23:13:47.632 INFO:tasks.workunit.client.0.smithi031.stdout:8/635: creat d8/d15/d37/d62/fdb x:0 0 0 2024-04-24T23:13:47.634 INFO:tasks.workunit.client.1.smithi040.stdout:3/196: mknod d1/d2d/de/c3f 0 2024-04-24T23:13:47.635 INFO:tasks.workunit.client.0.smithi031.stdout:0/582: mkdir da/d11/d6a/dba/dc0 0 2024-04-24T23:13:47.637 INFO:tasks.workunit.client.1.smithi040.stdout:2/258: link da/d1c/d37/l4b da/d1c/d37/d39/l5d 0 2024-04-24T23:13:47.639 INFO:tasks.workunit.client.1.smithi040.stdout:4/179: creat da/d3f/d40/d41/f43 x:0 0 0 2024-04-24T23:13:47.639 INFO:tasks.workunit.client.0.smithi031.stdout:6/659: getdents d13/d15/d1d/d23 0 2024-04-24T23:13:47.641 INFO:tasks.workunit.client.0.smithi031.stdout:6/660: fdatasync d13/d15/d1d/d23/d24/d43/d68/fb5 0 2024-04-24T23:13:47.641 INFO:tasks.workunit.client.1.smithi040.stdout:6/202: mkdir d7/de/d15/d1e/d3e 0 2024-04-24T23:13:47.642 INFO:tasks.workunit.client.0.smithi031.stdout:3/485: creat d6/d72/d1a/f97 x:0 0 0 2024-04-24T23:13:47.643 INFO:tasks.workunit.client.1.smithi040.stdout:5/169: dwrite d2/d7/d8/db/d1e/d22/f25 [0,4194304] 0 2024-04-24T23:13:47.643 INFO:tasks.workunit.client.1.smithi040.stdout:6/203: write d7/fb [288198,2223] 0 2024-04-24T23:13:47.643 INFO:tasks.workunit.client.1.smithi040.stdout:7/236: symlink db/d2f/l58 0 2024-04-24T23:13:47.645 INFO:tasks.workunit.client.0.smithi031.stdout:2/396: rmdir d5/d8/d6e 0 2024-04-24T23:13:47.647 INFO:tasks.workunit.client.0.smithi031.stdout:7/523: mkdir d1/d45/d75/d81/db8 0 2024-04-24T23:13:47.651 INFO:tasks.workunit.client.0.smithi031.stdout:1/490: symlink d1/da/d10/l9a 0 2024-04-24T23:13:47.651 INFO:tasks.workunit.client.0.smithi031.stdout:9/637: dwrite d2/d16/dbe/f31 [0,4194304] 0 2024-04-24T23:13:47.654 INFO:tasks.workunit.client.0.smithi031.stdout:8/636: write d8/d1a/d1d/d50/dc0/d9d/fc6 [632400,121618] 0 2024-04-24T23:13:47.657 INFO:tasks.workunit.client.0.smithi031.stdout:0/583: mknod da/d18/d1a/cc1 0 2024-04-24T23:13:47.657 INFO:tasks.workunit.client.0.smithi031.stdout:8/637: chown d8/d15/d37/d43/d7b/d87/dad/cc9 0 1 2024-04-24T23:13:47.661 INFO:tasks.workunit.client.1.smithi040.stdout:8/169: dwrite d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:47.662 INFO:tasks.workunit.client.1.smithi040.stdout:8/170: write d0/d8/dc/f17 [539201,33464] 0 2024-04-24T23:13:47.663 INFO:tasks.workunit.client.1.smithi040.stdout:3/197: mknod d1/d2d/de/c40 0 2024-04-24T23:13:47.665 INFO:tasks.workunit.client.0.smithi031.stdout:3/486: fdatasync d6/df/d16/f35 0 2024-04-24T23:13:47.668 INFO:tasks.workunit.client.1.smithi040.stdout:8/171: dwrite d0/d1f/f2d [0,4194304] 0 2024-04-24T23:13:47.676 INFO:tasks.workunit.client.1.smithi040.stdout:2/259: creat da/d10/d15/f5e x:0 0 0 2024-04-24T23:13:47.677 INFO:tasks.workunit.client.0.smithi031.stdout:1/491: rmdir d1/da/d12/d7e 39 2024-04-24T23:13:47.678 INFO:tasks.workunit.client.1.smithi040.stdout:2/260: truncate da/d10/f31 71247 0 2024-04-24T23:13:47.678 INFO:tasks.workunit.client.1.smithi040.stdout:4/180: symlink da/d1d/d2b/l44 0 2024-04-24T23:13:47.679 INFO:tasks.workunit.client.1.smithi040.stdout:4/181: chown da/cc 426022567 1 2024-04-24T23:13:47.681 INFO:tasks.workunit.client.0.smithi031.stdout:4/648: write d3/d23/d69/d3f/f6d [1226978,54433] 0 2024-04-24T23:13:47.683 INFO:tasks.workunit.client.0.smithi031.stdout:8/638: unlink d8/f14 0 2024-04-24T23:13:47.683 INFO:tasks.workunit.client.0.smithi031.stdout:4/649: truncate d3/d20/d25/d60/d63/db9/fcd 844296 0 2024-04-24T23:13:47.684 INFO:tasks.workunit.client.1.smithi040.stdout:5/170: creat d2/d7/d8/db/d1e/f46 x:0 0 0 2024-04-24T23:13:47.684 INFO:tasks.workunit.client.1.smithi040.stdout:7/237: rmdir db/d12/d20/d52/d44 39 2024-04-24T23:13:47.685 INFO:tasks.workunit.client.0.smithi031.stdout:7/524: write d1/d45/da0/f36 [1876458,51512] 0 2024-04-24T23:13:47.686 INFO:tasks.workunit.client.1.smithi040.stdout:7/238: write db/d2f/f30 [1677807,1486] 0 2024-04-24T23:13:47.686 INFO:tasks.workunit.client.0.smithi031.stdout:6/661: mkdir d13/d15/d1d/d23/d44/d47/d69/dd5/de4 0 2024-04-24T23:13:47.687 INFO:tasks.workunit.client.1.smithi040.stdout:7/239: truncate db/d2f/f30 2209999 0 2024-04-24T23:13:47.688 INFO:tasks.workunit.client.1.smithi040.stdout:0/265: creat df/d12/f4c x:0 0 0 2024-04-24T23:13:47.689 INFO:tasks.workunit.client.1.smithi040.stdout:0/266: write df/d1e/f3e [1043251,56175] 0 2024-04-24T23:13:47.689 INFO:tasks.workunit.client.0.smithi031.stdout:3/487: symlink d6/df/d16/d53/l98 0 2024-04-24T23:13:47.690 INFO:tasks.workunit.client.1.smithi040.stdout:9/238: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/c56 0 2024-04-24T23:13:47.691 INFO:tasks.workunit.client.0.smithi031.stdout:2/397: dwrite d5/d3d/f62 [0,4194304] 0 2024-04-24T23:13:47.693 INFO:tasks.workunit.client.0.smithi031.stdout:7/525: dread d1/d45/da0/f5f [0,4194304] 0 2024-04-24T23:13:47.694 INFO:tasks.workunit.client.0.smithi031.stdout:1/492: mkdir d1/d33/d41/d9b 0 2024-04-24T23:13:47.695 INFO:tasks.workunit.client.1.smithi040.stdout:3/198: creat d1/d2d/de/d30/f41 x:0 0 0 2024-04-24T23:13:47.695 INFO:tasks.workunit.client.1.smithi040.stdout:0/267: dwrite df/d12/d1a/f2e [0,4194304] 0 2024-04-24T23:13:47.697 INFO:tasks.workunit.client.1.smithi040.stdout:1/194: link da/d12/d18/f27 da/d12/f42 0 2024-04-24T23:13:47.698 INFO:tasks.workunit.client.1.smithi040.stdout:1/195: chown da/d12/d14/f24 0 1 2024-04-24T23:13:47.699 INFO:tasks.workunit.client.0.smithi031.stdout:4/650: unlink d3/d23/f3d 0 2024-04-24T23:13:47.700 INFO:tasks.workunit.client.1.smithi040.stdout:1/196: write da/d12/d14/f2d [3669568,117462] 0 2024-04-24T23:13:47.701 INFO:tasks.workunit.client.1.smithi040.stdout:8/172: rmdir d0/d1f 39 2024-04-24T23:13:47.705 INFO:tasks.workunit.client.1.smithi040.stdout:2/261: mknod da/d1c/d37/d39/d3f/c5f 0 2024-04-24T23:13:47.709 INFO:tasks.workunit.client.0.smithi031.stdout:7/526: dread d1/d2/dd/d33/d5e/d64/d6d/fac [0,4194304] 0 2024-04-24T23:13:47.710 INFO:tasks.workunit.client.0.smithi031.stdout:3/488: chown d6/c13 0 1 2024-04-24T23:13:47.711 INFO:tasks.workunit.client.0.smithi031.stdout:7/527: dread - d1/d45/da0/d4f/fb5 zero size 2024-04-24T23:13:47.712 INFO:tasks.workunit.client.1.smithi040.stdout:5/171: rmdir d2/d7/d8/d1f/d31 39 2024-04-24T23:13:47.716 INFO:tasks.workunit.client.0.smithi031.stdout:9/638: rename d2/d43/fa9 to d2/dae/d14/d78/d84/d4a/d4b/fc2 0 2024-04-24T23:13:47.717 INFO:tasks.workunit.client.1.smithi040.stdout:7/240: dread - db/d12/d20/d52/d4b/f4f zero size 2024-04-24T23:13:47.718 INFO:tasks.workunit.client.0.smithi031.stdout:9/639: chown d2/dae/d14/d78/d84/d7e/l8b 55002333 1 2024-04-24T23:13:47.720 INFO:tasks.workunit.client.0.smithi031.stdout:1/493: truncate d1/dc/f70 4693552 0 2024-04-24T23:13:47.724 INFO:tasks.workunit.client.1.smithi040.stdout:0/268: symlink df/d12/d1a/l4d 0 2024-04-24T23:13:47.728 INFO:tasks.workunit.client.1.smithi040.stdout:3/199: rename d1/d2d/de/f26 to d1/d2d/d2b/f42 0 2024-04-24T23:13:47.728 INFO:tasks.workunit.client.1.smithi040.stdout:1/197: rename da/d12 to da/d12/d18/d43 22 2024-04-24T23:13:47.729 INFO:tasks.workunit.client.1.smithi040.stdout:1/198: chown da/d12/d14/d25 3 1 2024-04-24T23:13:47.729 INFO:tasks.workunit.client.1.smithi040.stdout:3/200: write d1/d9/d18/f2c [989410,40663] 0 2024-04-24T23:13:47.733 INFO:tasks.workunit.client.0.smithi031.stdout:3/489: dread d6/f48 [0,4194304] 0 2024-04-24T23:13:47.734 INFO:tasks.workunit.client.0.smithi031.stdout:3/490: chown d6/d72/d1a/d25/f54 7563991 1 2024-04-24T23:13:47.735 INFO:tasks.workunit.client.1.smithi040.stdout:8/173: dread d0/d1f/f2d [0,4194304] 0 2024-04-24T23:13:47.736 INFO:tasks.workunit.client.1.smithi040.stdout:2/262: creat da/d1c/d37/d39/f60 x:0 0 0 2024-04-24T23:13:47.736 INFO:tasks.workunit.client.0.smithi031.stdout:7/528: dread d1/d2/f11 [0,4194304] 0 2024-04-24T23:13:47.737 INFO:tasks.workunit.client.0.smithi031.stdout:2/398: unlink d5/d8/d15/f42 0 2024-04-24T23:13:47.738 INFO:tasks.workunit.client.1.smithi040.stdout:4/182: mkdir da/d45 0 2024-04-24T23:13:47.739 INFO:tasks.workunit.client.1.smithi040.stdout:4/183: fsync da/f18 0 2024-04-24T23:13:47.739 INFO:tasks.workunit.client.1.smithi040.stdout:4/184: fsync da/f1a 0 2024-04-24T23:13:47.742 INFO:tasks.workunit.client.1.smithi040.stdout:6/204: creat d7/f3f x:0 0 0 2024-04-24T23:13:47.742 INFO:tasks.workunit.client.0.smithi031.stdout:6/662: dwrite d13/d1b/d32/d9c/fc4 [0,4194304] 0 2024-04-24T23:13:47.744 INFO:tasks.workunit.client.0.smithi031.stdout:8/639: rename d8/d1a/d1d/d35/d58/d47/c6a to d8/d15/d37/d43/d7b/dc8/cdc 0 2024-04-24T23:13:47.745 INFO:tasks.workunit.client.1.smithi040.stdout:5/172: creat d2/d7/d8/db/d1e/d22/d27/d3c/f47 x:0 0 0 2024-04-24T23:13:47.746 INFO:tasks.workunit.client.1.smithi040.stdout:6/205: dwrite d7/fd [0,4194304] 0 2024-04-24T23:13:47.747 INFO:tasks.workunit.client.0.smithi031.stdout:9/640: rmdir d2/dae/d14/d78/d84 39 2024-04-24T23:13:47.747 INFO:tasks.workunit.client.1.smithi040.stdout:6/206: dread - d7/de/d15/d2f/f35 zero size 2024-04-24T23:13:47.749 INFO:tasks.workunit.client.1.smithi040.stdout:7/241: write db/d12/d20/d52/d44/f47 [2593936,45590] 0 2024-04-24T23:13:47.751 INFO:tasks.workunit.client.1.smithi040.stdout:9/239: fdatasync f4 0 2024-04-24T23:13:47.752 INFO:tasks.workunit.client.1.smithi040.stdout:9/240: fsync d7/f37 0 2024-04-24T23:13:47.754 INFO:tasks.workunit.client.1.smithi040.stdout:9/241: truncate d7/da/d17/d40/d55/d2f/f46 4798344 0 2024-04-24T23:13:47.757 INFO:tasks.workunit.client.1.smithi040.stdout:0/269: rename df/d12/f25 to df/d12/d1a/f4e 0 2024-04-24T23:13:47.760 INFO:tasks.workunit.client.0.smithi031.stdout:2/399: mknod d5/d6/d29/d4f/c8a 0 2024-04-24T23:13:47.761 INFO:tasks.workunit.client.0.smithi031.stdout:2/400: fsync d5/d8/d15/f50 0 2024-04-24T23:13:47.764 INFO:tasks.workunit.client.1.smithi040.stdout:3/201: mkdir d1/d2d/de/d30/d43 0 2024-04-24T23:13:47.767 INFO:tasks.workunit.client.0.smithi031.stdout:7/529: write d1/d2/dd/d33/d5e/d64/faa [377026,126916] 0 2024-04-24T23:13:47.767 INFO:tasks.workunit.client.1.smithi040.stdout:3/202: dread d1/f16 [0,4194304] 0 2024-04-24T23:13:47.768 INFO:tasks.workunit.client.1.smithi040.stdout:3/203: fsync d1/f3 0 2024-04-24T23:13:47.769 INFO:tasks.workunit.client.1.smithi040.stdout:8/174: creat d0/d16/d19/f35 x:0 0 0 2024-04-24T23:13:47.770 INFO:tasks.workunit.client.1.smithi040.stdout:8/175: fsync d0/d8/f29 0 2024-04-24T23:13:47.775 INFO:tasks.workunit.client.0.smithi031.stdout:3/491: creat d6/d14/d91/f99 x:0 0 0 2024-04-24T23:13:47.777 INFO:tasks.workunit.client.1.smithi040.stdout:4/185: rmdir da/d1d/d3c 39 2024-04-24T23:13:47.777 INFO:tasks.workunit.client.1.smithi040.stdout:4/186: readlink l1 0 2024-04-24T23:13:47.777 INFO:tasks.workunit.client.1.smithi040.stdout:1/199: dread da/ff [0,4194304] 0 2024-04-24T23:13:47.781 INFO:tasks.workunit.client.1.smithi040.stdout:5/173: mkdir d2/d7/d8/db/d1e/d22/d3e/d48 0 2024-04-24T23:13:47.782 INFO:tasks.workunit.client.1.smithi040.stdout:5/174: fsync d2/d7/fe 0 2024-04-24T23:13:47.783 INFO:tasks.workunit.client.0.smithi031.stdout:2/401: symlink d5/d28/d4e/d5e/l8b 0 2024-04-24T23:13:47.789 INFO:tasks.workunit.client.0.smithi031.stdout:7/530: chown d1/d2/dd/c59 1632 1 2024-04-24T23:13:47.790 INFO:tasks.workunit.client.0.smithi031.stdout:8/640: mknod d8/d15/dcf/cdd 0 2024-04-24T23:13:47.790 INFO:tasks.workunit.client.0.smithi031.stdout:2/402: dwrite d5/f36 [0,4194304] 0 2024-04-24T23:13:47.792 INFO:tasks.workunit.client.0.smithi031.stdout:8/641: write d8/d15/d37/d43/d7b/f8c [4999336,23285] 0 2024-04-24T23:13:47.798 INFO:tasks.workunit.client.0.smithi031.stdout:2/403: mknod d5/d28/d4e/c8c 0 2024-04-24T23:13:47.799 INFO:tasks.workunit.client.0.smithi031.stdout:6/663: dwrite d13/d15/d1d/d23/d44/f80 [0,4194304] 0 2024-04-24T23:13:47.802 INFO:tasks.workunit.client.0.smithi031.stdout:9/641: link d2/dae/d14/d1d/d45/f72 d2/dae/d14/d78/d84/d4a/dab/dad/fc3 0 2024-04-24T23:13:47.803 INFO:tasks.workunit.client.0.smithi031.stdout:9/642: write d2/d43/f70 [729548,47416] 0 2024-04-24T23:13:47.803 INFO:tasks.workunit.client.0.smithi031.stdout:8/642: creat d8/d1a/d1d/d35/d58/d47/fde x:0 0 0 2024-04-24T23:13:47.806 INFO:tasks.workunit.client.1.smithi040.stdout:3/204: mkdir d1/d2d/d44 0 2024-04-24T23:13:47.810 INFO:tasks.workunit.client.0.smithi031.stdout:2/404: creat d5/d8/d45/f8d x:0 0 0 2024-04-24T23:13:47.811 INFO:tasks.workunit.client.0.smithi031.stdout:2/405: chown d5/d8/d45/d5b/d49/f6d 300 1 2024-04-24T23:13:47.811 INFO:tasks.workunit.client.1.smithi040.stdout:8/176: rename d0/d8/dc/c33 to d0/d8/dc/d34/c36 0 2024-04-24T23:13:47.812 INFO:tasks.workunit.client.1.smithi040.stdout:1/200: chown da/c34 411238 1 2024-04-24T23:13:47.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/643: creat d8/d15/d37/d43/d7b/d89/fdf x:0 0 0 2024-04-24T23:13:47.818 INFO:tasks.workunit.client.0.smithi031.stdout:7/531: rmdir d1/d2/dd/d33/d5e/d64/d6d/da1/db6 0 2024-04-24T23:13:47.819 INFO:tasks.workunit.client.0.smithi031.stdout:9/643: dwrite d2/dae/f13 [0,4194304] 0 2024-04-24T23:13:47.820 INFO:tasks.workunit.client.0.smithi031.stdout:9/644: fdatasync d2/dae/d14/d78/d84/da6/fa7 0 2024-04-24T23:13:47.826 INFO:tasks.workunit.client.0.smithi031.stdout:7/532: symlink d1/d45/d75/d81/lb9 0 2024-04-24T23:13:47.832 INFO:tasks.workunit.client.0.smithi031.stdout:9/645: unlink d2/dae/d14/d78/d84/l89 0 2024-04-24T23:13:47.833 INFO:tasks.workunit.client.0.smithi031.stdout:3/492: truncate d6/d72/d60/f64 323707 0 2024-04-24T23:13:47.835 INFO:tasks.workunit.client.0.smithi031.stdout:8/644: read - d8/d15/d37/d62/fbb zero size 2024-04-24T23:13:47.837 INFO:tasks.workunit.client.0.smithi031.stdout:7/533: stat d1/d2/dd/d33/f77 0 2024-04-24T23:13:47.838 INFO:tasks.workunit.client.1.smithi040.stdout:2/263: truncate da/d10/d15/f32 287491 0 2024-04-24T23:13:47.838 INFO:tasks.workunit.client.1.smithi040.stdout:3/205: mkdir d1/d9/d15/d45 0 2024-04-24T23:13:47.841 INFO:tasks.workunit.client.0.smithi031.stdout:3/493: creat d6/d14/d1d/d59/f9a x:0 0 0 2024-04-24T23:13:47.842 INFO:tasks.workunit.client.1.smithi040.stdout:8/177: creat d0/d16/d19/f37 x:0 0 0 2024-04-24T23:13:47.842 INFO:tasks.workunit.client.0.smithi031.stdout:3/494: truncate d6/df/d16/d2a/f5a 1235411 0 2024-04-24T23:13:47.843 INFO:tasks.workunit.client.1.smithi040.stdout:2/264: dwrite da/d10/f31 [0,4194304] 0 2024-04-24T23:13:47.843 INFO:tasks.workunit.client.0.smithi031.stdout:8/645: fdatasync d8/d15/f57 0 2024-04-24T23:13:47.843 INFO:tasks.workunit.client.1.smithi040.stdout:8/178: write d0/d8/f29 [1213166,84884] 0 2024-04-24T23:13:47.847 INFO:tasks.workunit.client.1.smithi040.stdout:1/201: rmdir da/d12/d23 39 2024-04-24T23:13:47.849 INFO:tasks.workunit.client.1.smithi040.stdout:6/207: write d7/de/ff [517988,127528] 0 2024-04-24T23:13:47.849 INFO:tasks.workunit.client.1.smithi040.stdout:1/202: chown da/d12/d18/c33 1 1 2024-04-24T23:13:47.850 INFO:tasks.workunit.client.0.smithi031.stdout:7/534: symlink d1/d2/dd/d33/lba 0 2024-04-24T23:13:47.850 INFO:tasks.workunit.client.0.smithi031.stdout:7/535: stat d1/d2/c1a 0 2024-04-24T23:13:47.850 INFO:tasks.workunit.client.1.smithi040.stdout:4/187: link da/d1d/d1e/f24 da/d1d/d2b/f46 0 2024-04-24T23:13:47.851 INFO:tasks.workunit.client.0.smithi031.stdout:7/536: chown d1/d45/d75/d9f/d65/c71 172351 1 2024-04-24T23:13:47.851 INFO:tasks.workunit.client.1.smithi040.stdout:4/188: write da/f1a [8719381,91035] 0 2024-04-24T23:13:47.853 INFO:tasks.workunit.client.0.smithi031.stdout:9/646: rename d2/dae/d14/d1d/f24 to d2/dae/d85/fc4 0 2024-04-24T23:13:47.853 INFO:tasks.workunit.client.0.smithi031.stdout:2/406: truncate d5/d8/d15/f34 1192833 0 2024-04-24T23:13:47.854 INFO:tasks.workunit.client.0.smithi031.stdout:2/407: readlink d5/d6/d29/d26/l74 0 2024-04-24T23:13:47.855 INFO:tasks.workunit.client.0.smithi031.stdout:2/408: read d5/f13 [78684,113648] 0 2024-04-24T23:13:47.855 INFO:tasks.workunit.client.1.smithi040.stdout:0/270: truncate f2 7621413 0 2024-04-24T23:13:47.856 INFO:tasks.workunit.client.1.smithi040.stdout:0/271: write df/d12/d1a/f3a [432399,61714] 0 2024-04-24T23:13:47.857 INFO:tasks.workunit.client.1.smithi040.stdout:0/272: write df/d12/f22 [4223790,26859] 0 2024-04-24T23:13:47.862 INFO:tasks.workunit.client.0.smithi031.stdout:3/495: dread d6/df/d16/d2a/f2e [0,4194304] 0 2024-04-24T23:13:47.862 INFO:tasks.workunit.client.1.smithi040.stdout:7/242: creat db/d12/d20/d52/f59 x:0 0 0 2024-04-24T23:13:47.862 INFO:tasks.workunit.client.1.smithi040.stdout:7/243: readlink db/d2f/d37/l38 0 2024-04-24T23:13:47.863 INFO:tasks.workunit.client.0.smithi031.stdout:8/646: creat d8/d1a/d1d/d35/d58/d5b/fe0 x:0 0 0 2024-04-24T23:13:47.866 INFO:tasks.workunit.client.1.smithi040.stdout:9/242: creat d7/f57 x:0 0 0 2024-04-24T23:13:47.867 INFO:tasks.workunit.client.0.smithi031.stdout:7/537: fdatasync d1/d2/dd/d33/f4e 0 2024-04-24T23:13:47.871 INFO:tasks.workunit.client.1.smithi040.stdout:3/206: creat d1/d2d/f46 x:0 0 0 2024-04-24T23:13:47.874 INFO:tasks.workunit.client.0.smithi031.stdout:7/538: dwrite d1/d2/f7 [0,4194304] 0 2024-04-24T23:13:47.876 INFO:tasks.workunit.client.0.smithi031.stdout:7/539: fdatasync d1/d2/dd/f17 0 2024-04-24T23:13:47.876 INFO:tasks.workunit.client.1.smithi040.stdout:3/207: dwrite d1/d9/d18/f1f [0,4194304] 0 2024-04-24T23:13:47.877 INFO:tasks.workunit.client.0.smithi031.stdout:3/496: write d6/df/d16/d2a/f2e [217810,97063] 0 2024-04-24T23:13:47.878 INFO:tasks.workunit.client.0.smithi031.stdout:7/540: dread - d1/d45/da0/d4f/fb5 zero size 2024-04-24T23:13:47.879 INFO:tasks.workunit.client.1.smithi040.stdout:8/179: rmdir d0/d28 39 2024-04-24T23:13:47.882 INFO:tasks.workunit.client.1.smithi040.stdout:8/180: dread - d0/d16/d19/f35 zero size 2024-04-24T23:13:47.889 INFO:tasks.workunit.client.1.smithi040.stdout:1/203: unlink da/d12/d18/d2a/d32/f39 0 2024-04-24T23:13:47.889 INFO:tasks.workunit.client.1.smithi040.stdout:1/204: readlink da/d35/d38/l40 0 2024-04-24T23:13:47.890 INFO:tasks.workunit.client.1.smithi040.stdout:4/189: rmdir da/d3f 39 2024-04-24T23:13:47.893 INFO:tasks.workunit.client.0.smithi031.stdout:2/409: creat d5/d8/d15/d65/f8e x:0 0 0 2024-04-24T23:13:47.894 INFO:tasks.workunit.client.0.smithi031.stdout:2/410: chown d5/d6/d29/d4f/c8a 5535989 1 2024-04-24T23:13:47.896 INFO:tasks.workunit.client.1.smithi040.stdout:1/205: dwrite da/d12/d18/d2a/f3d [0,4194304] 0 2024-04-24T23:13:47.899 INFO:tasks.workunit.client.1.smithi040.stdout:7/244: readlink db/d12/l13 0 2024-04-24T23:13:47.900 INFO:tasks.workunit.client.1.smithi040.stdout:7/245: rename db to db/d12/d20/d5a 22 2024-04-24T23:13:47.901 INFO:tasks.workunit.client.1.smithi040.stdout:9/243: mknod d7/da/d17/d40/c58 0 2024-04-24T23:13:47.908 INFO:tasks.workunit.client.1.smithi040.stdout:2/265: creat da/d1c/d4f/f61 x:0 0 0 2024-04-24T23:13:47.909 INFO:tasks.workunit.client.1.smithi040.stdout:2/266: write da/d29/f48 [3516568,16131] 0 2024-04-24T23:13:47.912 INFO:tasks.workunit.client.0.smithi031.stdout:7/541: rename d1/f2b to d1/d2/dd/d33/d5e/d64/d6d/fbb 0 2024-04-24T23:13:47.912 INFO:tasks.workunit.client.0.smithi031.stdout:5/490: sync 2024-04-24T23:13:47.912 INFO:tasks.workunit.client.0.smithi031.stdout:7/542: readlink d1/d2/d60/l8c 0 2024-04-24T23:13:47.914 INFO:tasks.workunit.client.0.smithi031.stdout:8/647: dread d8/d15/d37/d62/fb3 [0,4194304] 0 2024-04-24T23:13:47.914 INFO:tasks.workunit.client.0.smithi031.stdout:3/497: write d6/d72/d1a/f80 [818644,25064] 0 2024-04-24T23:13:47.917 INFO:tasks.workunit.client.0.smithi031.stdout:3/498: dread d6/df/d16/f1c [0,4194304] 0 2024-04-24T23:13:47.922 INFO:tasks.workunit.client.1.smithi040.stdout:5/175: rename d2/d7/d8/db/d1e/d22/l2b to d2/d7/d8/db/d1e/d22/l49 0 2024-04-24T23:13:47.924 INFO:tasks.workunit.client.1.smithi040.stdout:5/176: write d2/d7/d8/db/d1e/d22/d27/f32 [4109023,125400] 0 2024-04-24T23:13:47.927 INFO:tasks.workunit.client.0.smithi031.stdout:8/648: rename d8/d1a/d1d/d35/d58/d47 to d8/d15/d37/d43/d5c/de1 0 2024-04-24T23:13:47.929 INFO:tasks.workunit.client.0.smithi031.stdout:2/411: link d5/d28/d4e/d5e/f60 d5/d8/d15/d56/f8f 0 2024-04-24T23:13:47.930 INFO:tasks.workunit.client.1.smithi040.stdout:1/206: mknod da/d35/c44 0 2024-04-24T23:13:47.931 INFO:tasks.workunit.client.0.smithi031.stdout:2/412: write d5/d6/d29/f72 [1145894,46446] 0 2024-04-24T23:13:47.931 INFO:tasks.workunit.client.0.smithi031.stdout:5/491: write d0/d7/f58 [178784,108471] 0 2024-04-24T23:13:47.932 INFO:tasks.workunit.client.0.smithi031.stdout:2/413: write d5/f46 [2196147,8695] 0 2024-04-24T23:13:47.933 INFO:tasks.workunit.client.1.smithi040.stdout:7/246: rmdir db/d12/d20/d52/d4b 39 2024-04-24T23:13:47.933 INFO:tasks.workunit.client.1.smithi040.stdout:7/247: readlink db/d12/l26 0 2024-04-24T23:13:47.934 INFO:tasks.workunit.client.0.smithi031.stdout:7/543: symlink d1/d2/lbc 0 2024-04-24T23:13:47.939 INFO:tasks.workunit.client.0.smithi031.stdout:3/499: rename d6/d72 to d6/df/d16/d2a/d95/d9b 0 2024-04-24T23:13:47.940 INFO:tasks.workunit.client.0.smithi031.stdout:2/414: dwrite d5/d6/d29/d4f/f77 [0,4194304] 0 2024-04-24T23:13:47.940 INFO:tasks.workunit.client.0.smithi031.stdout:3/500: stat d6/d14/d1d/d69 0 2024-04-24T23:13:47.943 INFO:tasks.workunit.client.1.smithi040.stdout:3/208: creat d1/d2a/f47 x:0 0 0 2024-04-24T23:13:47.946 INFO:tasks.workunit.client.1.smithi040.stdout:2/267: creat da/d29/d55/f62 x:0 0 0 2024-04-24T23:13:47.947 INFO:tasks.workunit.client.0.smithi031.stdout:8/649: truncate d8/d15/d37/d43/d5c/de1/f53 2315255 0 2024-04-24T23:13:47.947 INFO:tasks.workunit.client.1.smithi040.stdout:7/248: dread db/d12/d20/f21 [4194304,4194304] 0 2024-04-24T23:13:47.947 INFO:tasks.workunit.client.0.smithi031.stdout:7/544: mkdir d1/d2/dd/d33/d5e/d64/d6d/dbd 0 2024-04-24T23:13:47.949 INFO:tasks.workunit.client.1.smithi040.stdout:8/181: mkdir d0/d38 0 2024-04-24T23:13:47.951 INFO:tasks.workunit.client.1.smithi040.stdout:8/182: fdatasync d0/d16/d19/f35 0 2024-04-24T23:13:47.953 INFO:tasks.workunit.client.1.smithi040.stdout:2/268: dwrite da/f40 [0,4194304] 0 2024-04-24T23:13:47.953 INFO:tasks.workunit.client.1.smithi040.stdout:2/269: stat da/l17 0 2024-04-24T23:13:47.954 INFO:tasks.workunit.client.1.smithi040.stdout:6/208: link d7/de/l3b d7/de/d15/d1e/l40 0 2024-04-24T23:13:47.958 INFO:tasks.workunit.client.0.smithi031.stdout:5/492: rename d0/d7/d56/f8c to d0/d6/d3b/d4b/dc/d8b/fa6 0 2024-04-24T23:13:47.959 INFO:tasks.workunit.client.1.smithi040.stdout:2/270: dwrite da/d10/f34 [0,4194304] 0 2024-04-24T23:13:47.965 INFO:tasks.workunit.client.1.smithi040.stdout:5/177: mkdir d2/d7/d8/db/d1e/d22/d27/d3c/d4a 0 2024-04-24T23:13:47.971 INFO:tasks.workunit.client.1.smithi040.stdout:5/178: dwrite d2/d7/d8/f39 [0,4194304] 0 2024-04-24T23:13:47.973 INFO:tasks.workunit.client.1.smithi040.stdout:5/179: chown d2/d7/d8/db/d1e/d22/d27/d3c/f47 6886 1 2024-04-24T23:13:47.973 INFO:tasks.workunit.client.1.smithi040.stdout:5/180: stat d2/d2f/l29 0 2024-04-24T23:13:47.974 INFO:tasks.workunit.client.1.smithi040.stdout:5/181: dread - d2/d2f/f33 zero size 2024-04-24T23:13:47.978 INFO:tasks.workunit.client.1.smithi040.stdout:7/249: creat db/d2f/f5b x:0 0 0 2024-04-24T23:13:47.979 INFO:tasks.workunit.client.1.smithi040.stdout:7/250: write db/f3c [486882,59088] 0 2024-04-24T23:13:47.987 INFO:tasks.workunit.client.1.smithi040.stdout:2/271: rmdir da/d1c 39 2024-04-24T23:13:47.992 INFO:tasks.workunit.client.1.smithi040.stdout:1/207: truncate da/d12/f42 592750 0 2024-04-24T23:13:47.993 INFO:tasks.workunit.client.1.smithi040.stdout:2/272: dwrite da/d29/f4a [0,4194304] 0 2024-04-24T23:13:47.994 INFO:tasks.workunit.client.1.smithi040.stdout:9/244: creat d7/f59 x:0 0 0 2024-04-24T23:13:47.996 INFO:tasks.workunit.client.1.smithi040.stdout:3/209: creat d1/d2d/d44/f48 x:0 0 0 2024-04-24T23:13:47.996 INFO:tasks.workunit.client.1.smithi040.stdout:3/210: stat d1/d2d/de 0 2024-04-24T23:13:48.006 INFO:tasks.workunit.client.1.smithi040.stdout:4/190: getdents da/d1d/d2b 0 2024-04-24T23:13:48.006 INFO:tasks.workunit.client.1.smithi040.stdout:0/273: link df/d1e/d20/c24 df/d1e/d20/d21/c4f 0 2024-04-24T23:13:48.007 INFO:tasks.workunit.client.1.smithi040.stdout:7/251: dread db/d2f/f35 [0,4194304] 0 2024-04-24T23:13:48.010 INFO:tasks.workunit.client.0.smithi031.stdout:7/545: unlink d1/d45/d75/d9f/fad 0 2024-04-24T23:13:48.012 INFO:tasks.workunit.client.1.smithi040.stdout:6/209: truncate d7/f9 293661 0 2024-04-24T23:13:48.013 INFO:tasks.workunit.client.1.smithi040.stdout:6/210: fdatasync d7/fb 0 2024-04-24T23:13:48.014 INFO:tasks.workunit.client.1.smithi040.stdout:2/273: creat da/d10/d15/f63 x:0 0 0 2024-04-24T23:13:48.016 INFO:tasks.workunit.client.1.smithi040.stdout:9/245: unlink d7/da/d17/d40/d55/d2f/c51 0 2024-04-24T23:13:48.017 INFO:tasks.workunit.client.0.smithi031.stdout:3/501: truncate d6/df/d16/d2a/d95/d9b/d1a/d25/f54 702674 0 2024-04-24T23:13:48.018 INFO:tasks.workunit.client.1.smithi040.stdout:9/246: dread d7/ff [4194304,4194304] 0 2024-04-24T23:13:48.019 INFO:tasks.workunit.client.0.smithi031.stdout:2/415: rename d5/d8/d15/d56/d84/d75/f76 to d5/d3d/f90 0 2024-04-24T23:13:48.021 INFO:tasks.workunit.client.0.smithi031.stdout:5/493: rmdir d0/d7/d56 39 2024-04-24T23:13:48.024 INFO:tasks.workunit.client.1.smithi040.stdout:7/252: dread db/d12/f16 [0,4194304] 0 2024-04-24T23:13:48.025 INFO:tasks.workunit.client.1.smithi040.stdout:0/274: mknod df/d1e/d2b/c50 0 2024-04-24T23:13:48.027 INFO:tasks.workunit.client.0.smithi031.stdout:8/650: symlink d8/d1a/d74/le2 0 2024-04-24T23:13:48.030 INFO:tasks.workunit.client.1.smithi040.stdout:1/208: creat da/d12/d23/f45 x:0 0 0 2024-04-24T23:13:48.036 INFO:tasks.workunit.client.0.smithi031.stdout:7/546: dread d1/d2/dd/f15 [4194304,4194304] 0 2024-04-24T23:13:48.036 INFO:tasks.workunit.client.1.smithi040.stdout:9/247: write d7/da/f39 [866268,33799] 0 2024-04-24T23:13:48.036 INFO:tasks.workunit.client.1.smithi040.stdout:9/248: readlink d7/da/l3e 0 2024-04-24T23:13:48.037 INFO:tasks.workunit.client.0.smithi031.stdout:2/416: rename d5/f24 to d5/d8/d15/d56/d84/d75/f91 0 2024-04-24T23:13:48.040 INFO:tasks.workunit.client.1.smithi040.stdout:3/211: link d1/d9/d15/f1b d1/d9/d15/d45/f49 0 2024-04-24T23:13:48.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/651: write d8/d15/d37/d43/d5c/de1/f49 [3428213,79776] 0 2024-04-24T23:13:48.041 INFO:tasks.workunit.client.1.smithi040.stdout:7/253: write f2 [7441410,8212] 0 2024-04-24T23:13:48.042 INFO:tasks.workunit.client.0.smithi031.stdout:8/652: chown d8/d15/d37/d43/c75 1 1 2024-04-24T23:13:48.043 INFO:tasks.workunit.client.1.smithi040.stdout:4/191: dread f9 [0,4194304] 0 2024-04-24T23:13:48.043 INFO:tasks.workunit.client.1.smithi040.stdout:0/275: mkdir df/d1e/d20/d21/d51 0 2024-04-24T23:13:48.044 INFO:tasks.workunit.client.1.smithi040.stdout:4/192: fdatasync da/d1d/d2b/f35 0 2024-04-24T23:13:48.044 INFO:tasks.workunit.client.0.smithi031.stdout:3/502: link d6/df/d16/d2a/f68 d6/d14/d1d/d69/f9c 0 2024-04-24T23:13:48.047 INFO:tasks.workunit.client.1.smithi040.stdout:2/274: creat da/d10/d4d/f64 x:0 0 0 2024-04-24T23:13:48.048 INFO:tasks.workunit.client.1.smithi040.stdout:2/275: chown da/d10/d15/c47 3904767 1 2024-04-24T23:13:48.049 INFO:tasks.workunit.client.0.smithi031.stdout:2/417: chown d5/d8/d15/c1b 13423418 1 2024-04-24T23:13:48.050 INFO:tasks.workunit.client.0.smithi031.stdout:5/494: truncate d0/d6/d12/f5d 385155 0 2024-04-24T23:13:48.053 INFO:tasks.workunit.client.1.smithi040.stdout:9/249: creat d7/da/d17/d40/d55/d2f/d1f/f5a x:0 0 0 2024-04-24T23:13:48.054 INFO:tasks.workunit.client.0.smithi031.stdout:7/547: dwrite d1/d2/d83/faf [0,4194304] 0 2024-04-24T23:13:48.055 INFO:tasks.workunit.client.0.smithi031.stdout:7/548: dread - d1/d2/dd/f95 zero size 2024-04-24T23:13:48.055 INFO:tasks.workunit.client.0.smithi031.stdout:8/653: rmdir d8/d15/d37 39 2024-04-24T23:13:48.058 INFO:tasks.workunit.client.1.smithi040.stdout:3/212: unlink d1/f5 0 2024-04-24T23:13:48.059 INFO:tasks.workunit.client.1.smithi040.stdout:3/213: fsync d1/d9/d18/f38 0 2024-04-24T23:13:48.059 INFO:tasks.workunit.client.0.smithi031.stdout:5/495: dwrite d0/d6/f1a [0,4194304] 0 2024-04-24T23:13:48.063 INFO:tasks.workunit.client.1.smithi040.stdout:7/254: unlink db/d12/d20/d52/f59 0 2024-04-24T23:13:48.066 INFO:tasks.workunit.client.1.smithi040.stdout:0/276: unlink l5 0 2024-04-24T23:13:48.067 INFO:tasks.workunit.client.0.smithi031.stdout:2/418: creat d5/d6/d29/d26/f92 x:0 0 0 2024-04-24T23:13:48.072 INFO:tasks.workunit.client.1.smithi040.stdout:0/277: dwrite df/f39 [0,4194304] 0 2024-04-24T23:13:48.099 INFO:tasks.workunit.client.1.smithi040.stdout:8/183: sync 2024-04-24T23:13:48.115 INFO:tasks.workunit.client.0.smithi031.stdout:8/654: write d8/d15/d37/d43/d7b/d89/fdf [533614,76965] 0 2024-04-24T23:13:48.119 INFO:tasks.workunit.client.1.smithi040.stdout:9/250: creat d7/da/d17/d40/d55/d2f/d1f/d34/f5b x:0 0 0 2024-04-24T23:13:48.120 INFO:tasks.workunit.client.1.smithi040.stdout:1/209: dwrite da/d12/d23/f30 [0,4194304] 0 2024-04-24T23:13:48.121 INFO:tasks.workunit.client.0.smithi031.stdout:3/503: write d6/d14/f37 [1159013,61106] 0 2024-04-24T23:13:48.125 INFO:tasks.workunit.client.0.smithi031.stdout:8/655: dwrite d8/d15/d37/d43/d5c/de1/f49 [4194304,4194304] 0 2024-04-24T23:13:48.125 INFO:tasks.workunit.client.1.smithi040.stdout:3/214: mknod d1/d2d/de/c4a 0 2024-04-24T23:13:48.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/656: chown d8/d1a/d1d/d35/d58 121 1 2024-04-24T23:13:48.133 INFO:tasks.workunit.client.0.smithi031.stdout:3/504: unlink d6/df/d16/d2a/d95/d9b/d29/l92 0 2024-04-24T23:13:48.136 INFO:tasks.workunit.client.0.smithi031.stdout:7/549: rename d1/d45/d75/d9f/d65/da4 to d1/dbe 0 2024-04-24T23:13:48.138 INFO:tasks.workunit.client.1.smithi040.stdout:4/193: mkdir da/d3f/d40/d47 0 2024-04-24T23:13:48.140 INFO:tasks.workunit.client.0.smithi031.stdout:2/419: creat d5/d8/d15/f93 x:0 0 0 2024-04-24T23:13:48.141 INFO:tasks.workunit.client.1.smithi040.stdout:2/276: symlink da/d1c/d37/d39/d3f/l65 0 2024-04-24T23:13:48.142 INFO:tasks.workunit.client.1.smithi040.stdout:2/277: fsync da/d29/f48 0 2024-04-24T23:13:48.142 INFO:tasks.workunit.client.0.smithi031.stdout:3/505: creat d6/d14/d1d/f9d x:0 0 0 2024-04-24T23:13:48.143 INFO:tasks.workunit.client.0.smithi031.stdout:3/506: chown d6/l75 1021 1 2024-04-24T23:13:48.144 INFO:tasks.workunit.client.1.smithi040.stdout:1/210: mknod da/d12/d18/d2a/c46 0 2024-04-24T23:13:48.144 INFO:tasks.workunit.client.0.smithi031.stdout:3/507: readlink d6/df/d16/d2a/l44 0 2024-04-24T23:13:48.145 INFO:tasks.workunit.client.0.smithi031.stdout:3/508: stat d6/df/d16/d2a/c8e 0 2024-04-24T23:13:48.147 INFO:tasks.workunit.client.1.smithi040.stdout:9/251: rename d7/ff to d7/da/d17/d40/d55/d2f/d1f/d34/f5c 0 2024-04-24T23:13:48.148 INFO:tasks.workunit.client.1.smithi040.stdout:9/252: dread - d7/da/d17/d40/d55/d2f/d1f/f5a zero size 2024-04-24T23:13:48.148 INFO:tasks.workunit.client.0.smithi031.stdout:7/550: chown d1/d45/da0/c88 400828 1 2024-04-24T23:13:48.149 INFO:tasks.workunit.client.1.smithi040.stdout:7/255: dread f2 [0,4194304] 0 2024-04-24T23:13:48.151 INFO:tasks.workunit.client.1.smithi040.stdout:0/278: mkdir df/d1e/d20/d21/d41/d52 0 2024-04-24T23:13:48.153 INFO:tasks.workunit.client.1.smithi040.stdout:5/182: sync 2024-04-24T23:13:48.153 INFO:tasks.workunit.client.1.smithi040.stdout:8/184: creat d0/d28/f39 x:0 0 0 2024-04-24T23:13:48.154 INFO:tasks.workunit.client.1.smithi040.stdout:5/183: chown d2/d7/d8/f34 139 1 2024-04-24T23:13:48.155 INFO:tasks.workunit.client.1.smithi040.stdout:4/194: dread - da/d1d/d3c/f3e zero size 2024-04-24T23:13:48.155 INFO:tasks.workunit.client.0.smithi031.stdout:3/509: symlink d6/df/d16/d2a/l9e 0 2024-04-24T23:13:48.158 INFO:tasks.workunit.client.0.smithi031.stdout:7/551: truncate d1/f53 3849977 0 2024-04-24T23:13:48.158 INFO:tasks.workunit.client.1.smithi040.stdout:8/185: dwrite d0/d8/f29 [0,4194304] 0 2024-04-24T23:13:48.159 INFO:tasks.workunit.client.0.smithi031.stdout:8/657: write d8/d15/d37/d43/f4b [1804820,100054] 0 2024-04-24T23:13:48.160 INFO:tasks.workunit.client.0.smithi031.stdout:7/552: truncate d1/d45/da0/d4f/f9b 399702 0 2024-04-24T23:13:48.162 INFO:tasks.workunit.client.1.smithi040.stdout:1/211: mkdir da/d35/d38/d47 0 2024-04-24T23:13:48.163 INFO:tasks.workunit.client.0.smithi031.stdout:7/553: read d1/d45/d75/d9f/f7c [600519,91283] 0 2024-04-24T23:13:48.165 INFO:tasks.workunit.client.1.smithi040.stdout:3/215: rename d1/d2d/f46 to d1/d9/d15/d45/f4b 0 2024-04-24T23:13:48.168 INFO:tasks.workunit.client.0.smithi031.stdout:3/510: creat d6/df/d16/d2a/d95/d9b/d1a/d25/f9f x:0 0 0 2024-04-24T23:13:48.169 INFO:tasks.workunit.client.0.smithi031.stdout:8/658: dwrite d8/d15/d37/d43/dbd/dc4/fc7 [0,4194304] 0 2024-04-24T23:13:48.176 INFO:tasks.workunit.client.1.smithi040.stdout:7/256: mknod db/d12/d20/c5c 0 2024-04-24T23:13:48.178 INFO:tasks.workunit.client.0.smithi031.stdout:5/496: rename d0/d6/d3b/d4b/dc/f24 to d0/d7/d56/fa7 0 2024-04-24T23:13:48.179 INFO:tasks.workunit.client.1.smithi040.stdout:0/279: fsync df/d1e/d20/f43 0 2024-04-24T23:13:48.180 INFO:tasks.workunit.client.0.smithi031.stdout:5/497: write d0/d6/d3b/d4b/dc/d25/f62 [4454026,68626] 0 2024-04-24T23:13:48.183 INFO:tasks.workunit.client.0.smithi031.stdout:3/511: creat d6/df/d16/d2a/d95/d9b/d29/fa0 x:0 0 0 2024-04-24T23:13:48.184 INFO:tasks.workunit.client.0.smithi031.stdout:2/420: getdents d5/d8/d15/d56/d84 0 2024-04-24T23:13:48.185 INFO:tasks.workunit.client.0.smithi031.stdout:3/512: chown d6/df/d16/c3c 12050039 1 2024-04-24T23:13:48.187 INFO:tasks.workunit.client.0.smithi031.stdout:8/659: rmdir d8/d1a/d1d/d50/dc0/d9d 39 2024-04-24T23:13:48.191 INFO:tasks.workunit.client.1.smithi040.stdout:1/212: creat da/d12/d18/d2a/d32/f48 x:0 0 0 2024-04-24T23:13:48.194 INFO:tasks.workunit.client.0.smithi031.stdout:7/554: mknod d1/cbf 0 2024-04-24T23:13:48.198 INFO:tasks.workunit.client.1.smithi040.stdout:3/216: creat d1/d9/d15/d45/f4c x:0 0 0 2024-04-24T23:13:48.202 INFO:tasks.workunit.client.1.smithi040.stdout:6/211: sync 2024-04-24T23:13:48.202 INFO:tasks.workunit.client.1.smithi040.stdout:9/253: symlink d7/da/d17/d40/d55/l5d 0 2024-04-24T23:13:48.202 INFO:tasks.workunit.client.1.smithi040.stdout:6/212: stat d7/de/l14 0 2024-04-24T23:13:48.202 INFO:tasks.workunit.client.1.smithi040.stdout:6/213: fdatasync d7/de/d15/f2e 0 2024-04-24T23:13:48.205 INFO:tasks.workunit.client.1.smithi040.stdout:7/257: write f2 [5671069,61274] 0 2024-04-24T23:13:48.205 INFO:tasks.workunit.client.0.smithi031.stdout:2/421: dread d5/d6/d29/d26/f63 [0,4194304] 0 2024-04-24T23:13:48.205 INFO:tasks.workunit.client.0.smithi031.stdout:5/498: dwrite d0/d6/d3b/d4b/dc/d8b/fa6 [0,4194304] 0 2024-04-24T23:13:48.206 INFO:tasks.workunit.client.0.smithi031.stdout:8/660: mknod d8/d15/d37/d43/ce3 0 2024-04-24T23:13:48.209 INFO:tasks.workunit.client.0.smithi031.stdout:7/555: creat d1/d45/da0/fc0 x:0 0 0 2024-04-24T23:13:48.210 INFO:tasks.workunit.client.0.smithi031.stdout:7/556: chown d1/d45/d75/d9f/d65 6202 1 2024-04-24T23:13:48.212 INFO:tasks.workunit.client.0.smithi031.stdout:0/584: sync 2024-04-24T23:13:48.213 INFO:tasks.workunit.client.1.smithi040.stdout:4/195: mkdir da/d45/d48 0 2024-04-24T23:13:48.214 INFO:tasks.workunit.client.1.smithi040.stdout:2/278: link da/cc da/d10/c66 0 2024-04-24T23:13:48.217 INFO:tasks.workunit.client.1.smithi040.stdout:8/186: link d0/d8/f29 d0/d8/dc/f3a 0 2024-04-24T23:13:48.218 INFO:tasks.workunit.client.0.smithi031.stdout:7/557: dwrite d1/d45/da0/fc0 [0,4194304] 0 2024-04-24T23:13:48.220 INFO:tasks.workunit.client.1.smithi040.stdout:2/279: dwrite da/d10/d15/f2c [0,4194304] 0 2024-04-24T23:13:48.221 INFO:tasks.workunit.client.1.smithi040.stdout:1/213: creat da/d12/d18/f49 x:0 0 0 2024-04-24T23:13:48.224 INFO:tasks.workunit.client.1.smithi040.stdout:3/217: chown d1/d9/l11 9 1 2024-04-24T23:13:48.227 INFO:tasks.workunit.client.0.smithi031.stdout:8/661: rename d8/d15/f69 to d8/d1a/d1d/d50/dc0/d9d/fe4 0 2024-04-24T23:13:48.228 INFO:tasks.workunit.client.0.smithi031.stdout:7/558: mkdir d1/d2/d83/dc1 0 2024-04-24T23:13:48.230 INFO:tasks.workunit.client.1.smithi040.stdout:7/258: stat db/c4c 0 2024-04-24T23:13:48.232 INFO:tasks.workunit.client.0.smithi031.stdout:5/499: dread d0/d6/d3b/d4b/dc/f84 [0,4194304] 0 2024-04-24T23:13:48.232 INFO:tasks.workunit.client.1.smithi040.stdout:1/214: dwrite da/d35/d38/f3e [0,4194304] 0 2024-04-24T23:13:48.234 INFO:tasks.workunit.client.0.smithi031.stdout:7/559: dread d1/d45/da0/f5f [0,4194304] 0 2024-04-24T23:13:48.235 INFO:tasks.workunit.client.1.smithi040.stdout:5/184: creat d2/d7/d8/d1f/f4b x:0 0 0 2024-04-24T23:13:48.236 INFO:tasks.workunit.client.0.smithi031.stdout:2/422: link d5/d6/d29/d26/f3f d5/d8/f94 0 2024-04-24T23:13:48.237 INFO:tasks.workunit.client.1.smithi040.stdout:3/218: dwrite d1/f3 [0,4194304] 0 2024-04-24T23:13:48.238 INFO:tasks.workunit.client.0.smithi031.stdout:7/560: mkdir d1/d45/d84/dc2 0 2024-04-24T23:13:48.238 INFO:tasks.workunit.client.1.smithi040.stdout:4/196: mknod da/d3f/c49 0 2024-04-24T23:13:48.238 INFO:tasks.workunit.client.1.smithi040.stdout:1/215: truncate da/d12/d14/f3f 316196 0 2024-04-24T23:13:48.240 INFO:tasks.workunit.client.1.smithi040.stdout:1/216: chown da/c34 34393 1 2024-04-24T23:13:48.241 INFO:tasks.workunit.client.1.smithi040.stdout:5/185: dread - d2/d7/d8/db/d1e/d22/f44 zero size 2024-04-24T23:13:48.241 INFO:tasks.workunit.client.1.smithi040.stdout:7/259: dread db/f10 [0,4194304] 0 2024-04-24T23:13:48.242 INFO:tasks.workunit.client.1.smithi040.stdout:2/280: creat da/d1c/d37/d39/d3f/f67 x:0 0 0 2024-04-24T23:13:48.242 INFO:tasks.workunit.client.0.smithi031.stdout:2/423: mknod d5/d28/c95 0 2024-04-24T23:13:48.242 INFO:tasks.workunit.client.1.smithi040.stdout:3/219: write d1/d2a/f47 [451843,12927] 0 2024-04-24T23:13:48.243 INFO:tasks.workunit.client.1.smithi040.stdout:8/187: creat d0/d16/f3b x:0 0 0 2024-04-24T23:13:48.244 INFO:tasks.workunit.client.1.smithi040.stdout:8/188: chown d0/d8 13 1 2024-04-24T23:13:48.245 INFO:tasks.workunit.client.0.smithi031.stdout:2/424: symlink d5/d6/d29/l96 0 2024-04-24T23:13:48.247 INFO:tasks.workunit.client.1.smithi040.stdout:0/280: link df/c1d df/d12/d4b/c53 0 2024-04-24T23:13:48.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/585: dread da/d18/d1a/d37/fa5 [0,4194304] 0 2024-04-24T23:13:48.250 INFO:tasks.workunit.client.1.smithi040.stdout:0/281: fdatasync df/d1e/f40 0 2024-04-24T23:13:48.252 INFO:tasks.workunit.client.0.smithi031.stdout:7/561: getdents d1/d2/dd/d33/d5e 0 2024-04-24T23:13:48.257 INFO:tasks.workunit.client.1.smithi040.stdout:2/281: creat da/d1c/d37/d38/f68 x:0 0 0 2024-04-24T23:13:48.257 INFO:tasks.workunit.client.1.smithi040.stdout:4/197: dwrite da/d1d/d2b/f39 [0,4194304] 0 2024-04-24T23:13:48.258 INFO:tasks.workunit.client.1.smithi040.stdout:7/260: symlink db/d12/d20/l5d 0 2024-04-24T23:13:48.258 INFO:tasks.workunit.client.0.smithi031.stdout:0/586: link da/d18/d1a/d37/d41/c71 da/d18/cc2 0 2024-04-24T23:13:48.260 INFO:tasks.workunit.client.0.smithi031.stdout:7/562: rmdir d1/d45/d84/dc2 0 2024-04-24T23:13:48.261 INFO:tasks.workunit.client.1.smithi040.stdout:8/189: dwrite d0/d16/d19/f37 [0,4194304] 0 2024-04-24T23:13:48.261 INFO:tasks.workunit.client.1.smithi040.stdout:6/214: getdents d7/d1a 0 2024-04-24T23:13:48.262 INFO:tasks.workunit.client.1.smithi040.stdout:5/186: dwrite d2/d7/d8/db/d1e/d22/f44 [0,4194304] 0 2024-04-24T23:13:48.265 INFO:tasks.workunit.client.0.smithi031.stdout:7/563: creat d1/d45/fc3 x:0 0 0 2024-04-24T23:13:48.266 INFO:tasks.workunit.client.1.smithi040.stdout:1/217: creat da/d12/f4a x:0 0 0 2024-04-24T23:13:48.266 INFO:tasks.workunit.client.0.smithi031.stdout:0/587: dwrite da/d18/d5a/fb2 [4194304,4194304] 0 2024-04-24T23:13:48.268 INFO:tasks.workunit.client.0.smithi031.stdout:7/564: fsync d1/d45/da0/d4f/fb5 0 2024-04-24T23:13:48.269 INFO:tasks.workunit.client.1.smithi040.stdout:4/198: rename da/d1f/c26 to da/d1f/c4a 0 2024-04-24T23:13:48.272 INFO:tasks.workunit.client.1.smithi040.stdout:0/282: link df/d1e/d20/f33 df/d12/d49/f54 0 2024-04-24T23:13:48.278 INFO:tasks.workunit.client.1.smithi040.stdout:7/261: dwrite db/f10 [0,4194304] 0 2024-04-24T23:13:48.279 INFO:tasks.workunit.client.1.smithi040.stdout:1/218: dwrite da/d12/d14/f3f [0,4194304] 0 2024-04-24T23:13:48.281 INFO:tasks.workunit.client.1.smithi040.stdout:2/282: rename da/d1c/d37/d38/f3e to da/d29/f69 0 2024-04-24T23:13:48.283 INFO:tasks.workunit.client.1.smithi040.stdout:2/283: chown da/d1c/d37/d39/f4e 0 1 2024-04-24T23:13:48.284 INFO:tasks.workunit.client.1.smithi040.stdout:2/284: stat da/f19 0 2024-04-24T23:13:48.284 INFO:tasks.workunit.client.1.smithi040.stdout:2/285: fsync da/f11 0 2024-04-24T23:13:48.285 INFO:tasks.workunit.client.1.smithi040.stdout:6/215: symlink d7/de/d15/d1e/l41 0 2024-04-24T23:13:48.287 INFO:tasks.workunit.client.0.smithi031.stdout:8/662: dwrite d8/f18 [4194304,4194304] 0 2024-04-24T23:13:48.291 INFO:tasks.workunit.client.1.smithi040.stdout:4/199: write da/d1d/d1e/f42 [1009355,65310] 0 2024-04-24T23:13:48.293 INFO:tasks.workunit.client.1.smithi040.stdout:0/283: creat df/d12/d1a/f55 x:0 0 0 2024-04-24T23:13:48.295 INFO:tasks.workunit.client.0.smithi031.stdout:2/425: rmdir d5/d28 39 2024-04-24T23:13:48.297 INFO:tasks.workunit.client.1.smithi040.stdout:3/220: getdents d1/d9/d15 0 2024-04-24T23:13:48.297 INFO:tasks.workunit.client.0.smithi031.stdout:2/426: write d5/d6/df/f79 [892798,23106] 0 2024-04-24T23:13:48.299 INFO:tasks.workunit.client.1.smithi040.stdout:1/219: creat da/d12/d18/d2a/d32/f4b x:0 0 0 2024-04-24T23:13:48.300 INFO:tasks.workunit.client.1.smithi040.stdout:8/190: truncate d0/d8/dc/f3a 6298850 0 2024-04-24T23:13:48.301 INFO:tasks.workunit.client.0.smithi031.stdout:8/663: dwrite d8/d15/d37/d62/f90 [0,4194304] 0 2024-04-24T23:13:48.305 INFO:tasks.workunit.client.1.smithi040.stdout:7/262: rename db/d12/f33 to db/d43/f5e 0 2024-04-24T23:13:48.306 INFO:tasks.workunit.client.0.smithi031.stdout:0/588: rmdir da/d18 39 2024-04-24T23:13:48.307 INFO:tasks.workunit.client.0.smithi031.stdout:0/589: write da/d22/f4b [1470994,61371] 0 2024-04-24T23:13:48.310 INFO:tasks.workunit.client.1.smithi040.stdout:2/286: mknod da/d1c/d37/d39/c6a 0 2024-04-24T23:13:48.311 INFO:tasks.workunit.client.0.smithi031.stdout:8/664: rmdir d8/d1a/d1d/d35/d58 39 2024-04-24T23:13:48.313 INFO:tasks.workunit.client.1.smithi040.stdout:6/216: creat d7/d1a/f42 x:0 0 0 2024-04-24T23:13:48.314 INFO:tasks.workunit.client.1.smithi040.stdout:4/200: creat da/d3f/f4b x:0 0 0 2024-04-24T23:13:48.315 INFO:tasks.workunit.client.1.smithi040.stdout:6/217: write d7/d2c/f31 [1937970,35503] 0 2024-04-24T23:13:48.317 INFO:tasks.workunit.client.0.smithi031.stdout:8/665: rmdir d8/d1a/d1d/d50 39 2024-04-24T23:13:48.317 INFO:tasks.workunit.client.1.smithi040.stdout:2/287: dwrite da/d10/d4d/f64 [0,4194304] 0 2024-04-24T23:13:48.321 INFO:tasks.workunit.client.1.smithi040.stdout:8/191: creat d0/d8/dc/d34/f3c x:0 0 0 2024-04-24T23:13:48.321 INFO:tasks.workunit.client.1.smithi040.stdout:6/218: dread d7/d1a/d20/f28 [0,4194304] 0 2024-04-24T23:13:48.323 INFO:tasks.workunit.client.1.smithi040.stdout:8/192: write d0/d16/d19/f37 [2596815,13983] 0 2024-04-24T23:13:48.327 INFO:tasks.workunit.client.0.smithi031.stdout:7/565: dwrite d1/f9 [4194304,4194304] 0 2024-04-24T23:13:48.331 INFO:tasks.workunit.client.0.smithi031.stdout:7/566: dread d1/d2/f12 [0,4194304] 0 2024-04-24T23:13:48.332 INFO:tasks.workunit.client.1.smithi040.stdout:0/284: symlink df/d12/d49/l56 0 2024-04-24T23:13:48.333 INFO:tasks.workunit.client.0.smithi031.stdout:2/427: write d5/d8/f10 [763763,118386] 0 2024-04-24T23:13:48.334 INFO:tasks.workunit.client.0.smithi031.stdout:7/567: write d1/d2/d83/faf [2170389,80714] 0 2024-04-24T23:13:48.337 INFO:tasks.workunit.client.1.smithi040.stdout:4/201: rename da/d1d/d2b/c38 to da/d1f/c4c 0 2024-04-24T23:13:48.338 INFO:tasks.workunit.client.0.smithi031.stdout:0/590: dread da/fe [0,4194304] 0 2024-04-24T23:13:48.338 INFO:tasks.workunit.client.1.smithi040.stdout:4/202: chown da/d1d 50731 1 2024-04-24T23:13:48.340 INFO:tasks.workunit.client.1.smithi040.stdout:0/285: dread df/d12/f2a [0,4194304] 0 2024-04-24T23:13:48.341 INFO:tasks.workunit.client.1.smithi040.stdout:4/203: dread f9 [0,4194304] 0 2024-04-24T23:13:48.342 INFO:tasks.workunit.client.1.smithi040.stdout:4/204: stat da/l16 0 2024-04-24T23:13:48.343 INFO:tasks.workunit.client.1.smithi040.stdout:4/205: write da/d1d/d1e/f23 [335481,109998] 0 2024-04-24T23:13:48.347 INFO:tasks.workunit.client.1.smithi040.stdout:9/254: sync 2024-04-24T23:13:48.347 INFO:tasks.workunit.client.1.smithi040.stdout:6/219: unlink d7/de/d15/f2e 0 2024-04-24T23:13:48.347 INFO:tasks.workunit.client.1.smithi040.stdout:6/220: write f2 [1615764,7089] 0 2024-04-24T23:13:48.348 INFO:tasks.workunit.client.1.smithi040.stdout:6/221: write d7/de/d15/d2f/f35 [905356,3553] 0 2024-04-24T23:13:48.350 INFO:tasks.workunit.client.0.smithi031.stdout:8/666: dwrite d8/d1a/f7e [0,4194304] 0 2024-04-24T23:13:48.356 INFO:tasks.workunit.client.0.smithi031.stdout:2/428: rename d5/d8/d15/f34 to d5/d28/d43/f97 0 2024-04-24T23:13:48.359 INFO:tasks.workunit.client.1.smithi040.stdout:2/288: rename da/fe to da/d29/f6b 0 2024-04-24T23:13:48.362 INFO:tasks.workunit.client.1.smithi040.stdout:3/221: rmdir d1/d9/d18/d3c 0 2024-04-24T23:13:48.363 INFO:tasks.workunit.client.0.smithi031.stdout:8/667: dread d8/fa8 [0,4194304] 0 2024-04-24T23:13:48.367 INFO:tasks.workunit.client.1.smithi040.stdout:4/206: creat da/d1d/d1e/f4d x:0 0 0 2024-04-24T23:13:48.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/568: dwrite d1/d2/f46 [0,4194304] 0 2024-04-24T23:13:48.371 INFO:tasks.workunit.client.0.smithi031.stdout:0/591: dwrite da/d18/d5a/d7d/f9b [0,4194304] 0 2024-04-24T23:13:48.372 INFO:tasks.workunit.client.1.smithi040.stdout:4/207: dread da/d1d/d2b/f39 [0,4194304] 0 2024-04-24T23:13:48.372 INFO:tasks.workunit.client.1.smithi040.stdout:8/193: unlink d0/d16/l1e 0 2024-04-24T23:13:48.372 INFO:tasks.workunit.client.1.smithi040.stdout:4/208: dread - da/f2f zero size 2024-04-24T23:13:48.382 INFO:tasks.workunit.client.0.smithi031.stdout:2/429: write d5/d28/f86 [881759,6602] 0 2024-04-24T23:13:48.382 INFO:tasks.workunit.client.0.smithi031.stdout:1/494: sync 2024-04-24T23:13:48.383 INFO:tasks.workunit.client.0.smithi031.stdout:4/651: sync 2024-04-24T23:13:48.384 INFO:tasks.workunit.client.0.smithi031.stdout:0/592: mkdir da/d18/d5a/d7d/dc3 0 2024-04-24T23:13:48.385 INFO:tasks.workunit.client.0.smithi031.stdout:0/593: fsync da/d18/f85 0 2024-04-24T23:13:48.389 INFO:tasks.workunit.client.0.smithi031.stdout:8/668: creat d8/d1a/d1d/d35/d58/d5b/fe5 x:0 0 0 2024-04-24T23:13:48.389 INFO:tasks.workunit.client.1.smithi040.stdout:9/255: dread f4 [0,4194304] 0 2024-04-24T23:13:48.390 INFO:tasks.workunit.client.1.smithi040.stdout:3/222: creat d1/d2d/d2b/f4d x:0 0 0 2024-04-24T23:13:48.393 INFO:tasks.workunit.client.0.smithi031.stdout:1/495: symlink d1/d31/l9c 0 2024-04-24T23:13:48.394 INFO:tasks.workunit.client.0.smithi031.stdout:1/496: dread - d1/d33/d36/f71 zero size 2024-04-24T23:13:48.394 INFO:tasks.workunit.client.1.smithi040.stdout:8/194: mknod d0/d8/d22/c3d 0 2024-04-24T23:13:48.396 INFO:tasks.workunit.client.1.smithi040.stdout:6/222: dwrite d7/de/d15/f1f [4194304,4194304] 0 2024-04-24T23:13:48.396 INFO:tasks.workunit.client.1.smithi040.stdout:4/209: creat da/d3f/d40/d41/f4e x:0 0 0 2024-04-24T23:13:48.398 INFO:tasks.workunit.client.0.smithi031.stdout:7/569: link d1/d45/d75/d9f/c39 d1/d2/d60/cc4 0 2024-04-24T23:13:48.400 INFO:tasks.workunit.client.0.smithi031.stdout:7/570: write d1/d45/d75/d9f/d66/f94 [842698,105622] 0 2024-04-24T23:13:48.401 INFO:tasks.workunit.client.1.smithi040.stdout:0/286: write f2 [5423834,72790] 0 2024-04-24T23:13:48.401 INFO:tasks.workunit.client.1.smithi040.stdout:7/263: getdents db/d12/d20/d52/d44 0 2024-04-24T23:13:48.403 INFO:tasks.workunit.client.1.smithi040.stdout:7/264: chown db/d12/l1f 101625817 1 2024-04-24T23:13:48.409 INFO:tasks.workunit.client.1.smithi040.stdout:3/223: mkdir d1/d9/d15/d45/d4e 0 2024-04-24T23:13:48.410 INFO:tasks.workunit.client.1.smithi040.stdout:0/287: dwrite df/d12/d1a/f3a [0,4194304] 0 2024-04-24T23:13:48.411 INFO:tasks.workunit.client.0.smithi031.stdout:8/669: creat d8/d15/d37/d43/d5c/db5/fe6 x:0 0 0 2024-04-24T23:13:48.413 INFO:tasks.workunit.client.1.smithi040.stdout:8/195: truncate d0/d1f/f2d 77704 0 2024-04-24T23:13:48.415 INFO:tasks.workunit.client.0.smithi031.stdout:2/430: rename d5/f36 to d5/d8/f98 0 2024-04-24T23:13:48.419 INFO:tasks.workunit.client.1.smithi040.stdout:5/187: sync 2024-04-24T23:13:48.425 INFO:tasks.workunit.client.1.smithi040.stdout:9/256: mkdir d7/d5e 0 2024-04-24T23:13:48.428 INFO:tasks.workunit.client.1.smithi040.stdout:2/289: dwrite da/d10/d15/f32 [0,4194304] 0 2024-04-24T23:13:48.429 INFO:tasks.workunit.client.0.smithi031.stdout:2/431: truncate d5/d8/d15/f2b 3993820 0 2024-04-24T23:13:48.430 INFO:tasks.workunit.client.0.smithi031.stdout:4/652: dwrite d3/f1b [0,4194304] 0 2024-04-24T23:13:48.431 INFO:tasks.workunit.client.0.smithi031.stdout:0/594: getdents da/d86 0 2024-04-24T23:13:48.438 INFO:tasks.workunit.client.0.smithi031.stdout:1/497: getdents d1/da/d10 0 2024-04-24T23:13:48.439 INFO:tasks.workunit.client.0.smithi031.stdout:7/571: truncate d1/d2/dd/d33/d5e/d64/f91 1097075 0 2024-04-24T23:13:48.440 INFO:tasks.workunit.client.0.smithi031.stdout:1/498: truncate d1/d1d/f7a 9026874 0 2024-04-24T23:13:48.440 INFO:tasks.workunit.client.1.smithi040.stdout:4/210: symlink da/d45/d48/l4f 0 2024-04-24T23:13:48.443 INFO:tasks.workunit.client.0.smithi031.stdout:0/595: fsync da/ff 0 2024-04-24T23:13:48.444 INFO:tasks.workunit.client.0.smithi031.stdout:9/647: sync 2024-04-24T23:13:48.444 INFO:tasks.workunit.client.0.smithi031.stdout:6/664: sync 2024-04-24T23:13:48.445 INFO:tasks.workunit.client.0.smithi031.stdout:4/653: chown d3/d20/d2d/d44/f4c 7670 1 2024-04-24T23:13:48.445 INFO:tasks.workunit.client.1.smithi040.stdout:2/290: fsync da/d10/d15/f32 0 2024-04-24T23:13:48.446 INFO:tasks.workunit.client.0.smithi031.stdout:9/648: fdatasync d2/dae/d14/f39 0 2024-04-24T23:13:48.448 INFO:tasks.workunit.client.0.smithi031.stdout:8/670: creat d8/d15/fe7 x:0 0 0 2024-04-24T23:13:48.448 INFO:tasks.workunit.client.1.smithi040.stdout:9/257: read d7/da/d17/d40/d55/d2f/d28/f29 [2268946,26233] 0 2024-04-24T23:13:48.448 INFO:tasks.workunit.client.1.smithi040.stdout:3/224: dwrite d1/f16 [4194304,4194304] 0 2024-04-24T23:13:48.449 INFO:tasks.workunit.client.1.smithi040.stdout:9/258: fsync d7/f59 0 2024-04-24T23:13:48.453 INFO:tasks.workunit.client.1.smithi040.stdout:1/220: sync 2024-04-24T23:13:48.454 INFO:tasks.workunit.client.1.smithi040.stdout:0/288: creat df/d1e/d20/d21/d51/f57 x:0 0 0 2024-04-24T23:13:48.458 INFO:tasks.workunit.client.1.smithi040.stdout:8/196: mknod d0/d8/c3e 0 2024-04-24T23:13:48.460 INFO:tasks.workunit.client.1.smithi040.stdout:9/259: dread d7/da/d17/d40/d55/d2f/d1f/d31/f35 [0,4194304] 0 2024-04-24T23:13:48.463 INFO:tasks.workunit.client.0.smithi031.stdout:4/654: creat d3/d20/d2d/fd2 x:0 0 0 2024-04-24T23:13:48.464 INFO:tasks.workunit.client.1.smithi040.stdout:5/188: getdents d2/d7/d8/db/d1e/d22/d3e/d48 0 2024-04-24T23:13:48.465 INFO:tasks.workunit.client.0.smithi031.stdout:6/665: mknod d13/d15/d1d/d23/d44/d47/d69/ce5 0 2024-04-24T23:13:48.469 INFO:tasks.workunit.client.1.smithi040.stdout:2/291: mkdir da/d10/d4d/d6c 0 2024-04-24T23:13:48.470 INFO:tasks.workunit.client.0.smithi031.stdout:8/671: symlink d8/d15/d37/d43/dbd/dc4/le8 0 2024-04-24T23:13:48.472 INFO:tasks.workunit.client.0.smithi031.stdout:2/432: dwrite d5/d28/d43/f97 [0,4194304] 0 2024-04-24T23:13:48.474 INFO:tasks.workunit.client.0.smithi031.stdout:1/499: symlink d1/da/d12/l9d 0 2024-04-24T23:13:48.474 INFO:tasks.workunit.client.0.smithi031.stdout:0/596: write da/d22/f30 [829218,34211] 0 2024-04-24T23:13:48.475 INFO:tasks.workunit.client.1.smithi040.stdout:3/225: symlink d1/d9/l4f 0 2024-04-24T23:13:48.476 INFO:tasks.workunit.client.0.smithi031.stdout:7/572: mkdir d1/d2/dd/d33/d5e/d64/d6d/dae/db1/dc5 0 2024-04-24T23:13:48.480 INFO:tasks.workunit.client.1.smithi040.stdout:1/221: rename da/d35/c44 to da/d35/d38/d47/c4c 0 2024-04-24T23:13:48.481 INFO:tasks.workunit.client.1.smithi040.stdout:8/197: mkdir d0/d16/d1b/d3f 0 2024-04-24T23:13:48.482 INFO:tasks.workunit.client.1.smithi040.stdout:6/223: link d7/d1a/d20/l29 d7/d2c/l43 0 2024-04-24T23:13:48.483 INFO:tasks.workunit.client.1.smithi040.stdout:3/226: dwrite d1/f16 [4194304,4194304] 0 2024-04-24T23:13:48.483 INFO:tasks.workunit.client.0.smithi031.stdout:9/649: dread d2/dae/d85/fc4 [0,4194304] 0 2024-04-24T23:13:48.484 INFO:tasks.workunit.client.1.smithi040.stdout:6/224: stat d7/d1a/d20/l3c 0 2024-04-24T23:13:48.484 INFO:tasks.workunit.client.1.smithi040.stdout:8/198: write d0/d16/d19/f35 [980790,101722] 0 2024-04-24T23:13:48.484 INFO:tasks.workunit.client.0.smithi031.stdout:9/650: write d2/f75 [1403723,75766] 0 2024-04-24T23:13:48.485 INFO:tasks.workunit.client.1.smithi040.stdout:9/260: mknod d7/da/d17/c5f 0 2024-04-24T23:13:48.485 INFO:tasks.workunit.client.1.smithi040.stdout:8/199: chown d0/d38 1326688 1 2024-04-24T23:13:48.488 INFO:tasks.workunit.client.0.smithi031.stdout:4/655: creat d3/d20/d2d/d44/fd3 x:0 0 0 2024-04-24T23:13:48.489 INFO:tasks.workunit.client.1.smithi040.stdout:4/211: truncate da/f18 704158 0 2024-04-24T23:13:48.490 INFO:tasks.workunit.client.0.smithi031.stdout:2/433: dread d5/d28/d43/f53 [0,4194304] 0 2024-04-24T23:13:48.490 INFO:tasks.workunit.client.0.smithi031.stdout:4/656: dread - d3/d20/d25/d60/d63/fbf zero size 2024-04-24T23:13:48.491 INFO:tasks.workunit.client.0.smithi031.stdout:6/666: write d13/d15/f27 [5124955,72283] 0 2024-04-24T23:13:48.491 INFO:tasks.workunit.client.1.smithi040.stdout:5/189: rmdir d2 39 2024-04-24T23:13:48.492 INFO:tasks.workunit.client.1.smithi040.stdout:9/261: dwrite f2 [4194304,4194304] 0 2024-04-24T23:13:48.493 INFO:tasks.workunit.client.1.smithi040.stdout:9/262: stat d7/da/d17/d40/d55/l5d 0 2024-04-24T23:13:48.494 INFO:tasks.workunit.client.1.smithi040.stdout:7/265: getdents db/d12/d20/d52 0 2024-04-24T23:13:48.494 INFO:tasks.workunit.client.0.smithi031.stdout:8/672: creat d8/d15/d37/d43/d7b/fe9 x:0 0 0 2024-04-24T23:13:48.495 INFO:tasks.workunit.client.1.smithi040.stdout:9/263: stat d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f50 0 2024-04-24T23:13:48.496 INFO:tasks.workunit.client.0.smithi031.stdout:1/500: rmdir d1/da/d12 39 2024-04-24T23:13:48.501 INFO:tasks.workunit.client.0.smithi031.stdout:7/573: creat d1/d2/dd/d33/d5e/d64/d6d/dae/db1/fc6 x:0 0 0 2024-04-24T23:13:48.506 INFO:tasks.workunit.client.1.smithi040.stdout:1/222: mkdir da/d12/d18/d2a/d4d 0 2024-04-24T23:13:48.507 INFO:tasks.workunit.client.1.smithi040.stdout:6/225: mkdir d7/de/d15/d44 0 2024-04-24T23:13:48.511 INFO:tasks.workunit.client.0.smithi031.stdout:6/667: write d13/d15/d1d/d23/d44/d47/d69/faa [1047493,117966] 0 2024-04-24T23:13:48.513 INFO:tasks.workunit.client.0.smithi031.stdout:6/668: write d13/d15/d1d/d23/d2d/fac [666579,44342] 0 2024-04-24T23:13:48.513 INFO:tasks.workunit.client.1.smithi040.stdout:3/227: rename d1/d9/d18/l12 to d1/d2d/de/d30/l50 0 2024-04-24T23:13:48.515 INFO:tasks.workunit.client.0.smithi031.stdout:0/597: mknod da/d11/d6a/dba/dc0/cc4 0 2024-04-24T23:13:48.516 INFO:tasks.workunit.client.1.smithi040.stdout:0/289: truncate df/f39 2347842 0 2024-04-24T23:13:48.518 INFO:tasks.workunit.client.0.smithi031.stdout:7/574: symlink d1/d45/d75/d81/lc7 0 2024-04-24T23:13:48.518 INFO:tasks.workunit.client.1.smithi040.stdout:2/292: dwrite f8 [0,4194304] 0 2024-04-24T23:13:48.519 INFO:tasks.workunit.client.0.smithi031.stdout:7/575: stat d1/d45/d75/d81 0 2024-04-24T23:13:48.519 INFO:tasks.workunit.client.1.smithi040.stdout:0/290: stat df/d1e/d20/d21/d41/d52 0 2024-04-24T23:13:48.519 INFO:tasks.workunit.client.1.smithi040.stdout:3/228: dread d1/d2d/f24 [0,4194304] 0 2024-04-24T23:13:48.520 INFO:tasks.workunit.client.0.smithi031.stdout:4/657: write d3/d23/d37/d86/f9a [1637614,33030] 0 2024-04-24T23:13:48.521 INFO:tasks.workunit.client.1.smithi040.stdout:3/229: dread - d1/d9/d18/f38 zero size 2024-04-24T23:13:48.522 INFO:tasks.workunit.client.0.smithi031.stdout:2/434: mkdir d5/d99 0 2024-04-24T23:13:48.524 INFO:tasks.workunit.client.0.smithi031.stdout:2/435: chown d5/d8/d45/d5b/d49/d6f/c7b 242931 1 2024-04-24T23:13:48.524 INFO:tasks.workunit.client.1.smithi040.stdout:7/266: symlink db/d43/l5f 0 2024-04-24T23:13:48.524 INFO:tasks.workunit.client.1.smithi040.stdout:3/230: dread d1/fd [4194304,4194304] 0 2024-04-24T23:13:48.531 INFO:tasks.workunit.client.1.smithi040.stdout:7/267: dwrite db/d12/d20/f21 [4194304,4194304] 0 2024-04-24T23:13:48.532 INFO:tasks.workunit.client.1.smithi040.stdout:7/268: chown db/d12/d20/c2a 16479 1 2024-04-24T23:13:48.533 INFO:tasks.workunit.client.1.smithi040.stdout:9/264: truncate d7/da/d17/d40/d55/d2f/d28/f29 1745522 0 2024-04-24T23:13:48.534 INFO:tasks.workunit.client.1.smithi040.stdout:1/223: creat da/d12/d14/f4e x:0 0 0 2024-04-24T23:13:48.535 INFO:tasks.workunit.client.0.smithi031.stdout:6/669: truncate d13/d15/d1d/d23/d24/d43/d68/fa3 660674 0 2024-04-24T23:13:48.537 INFO:tasks.workunit.client.0.smithi031.stdout:0/598: fsync da/d20/d4e/f57 0 2024-04-24T23:13:48.539 INFO:tasks.workunit.client.1.smithi040.stdout:4/212: dwrite f9 [0,4194304] 0 2024-04-24T23:13:48.542 INFO:tasks.workunit.client.0.smithi031.stdout:9/651: rename d2/dae/d14/d78/d84/d4a/d4b/c95 to d2/dae/d14/d78/d84/d4a/d4b/cc5 0 2024-04-24T23:13:48.548 INFO:tasks.workunit.client.1.smithi040.stdout:8/200: fsync d0/d1f/f2d 0 2024-04-24T23:13:48.551 INFO:tasks.workunit.client.0.smithi031.stdout:1/501: creat d1/da/d12/d7e/f9e x:0 0 0 2024-04-24T23:13:48.557 INFO:tasks.workunit.client.0.smithi031.stdout:0/599: creat da/d11/d6a/fc5 x:0 0 0 2024-04-24T23:13:48.557 INFO:tasks.workunit.client.0.smithi031.stdout:1/502: dread d1/f66 [0,4194304] 0 2024-04-24T23:13:48.558 INFO:tasks.workunit.client.0.smithi031.stdout:1/503: truncate d1/f66 757539 0 2024-04-24T23:13:48.559 INFO:tasks.workunit.client.0.smithi031.stdout:1/504: fsync d1/da/d42/f77 0 2024-04-24T23:13:48.561 INFO:tasks.workunit.client.0.smithi031.stdout:7/576: rename d1/l2e to d1/d2/d60/lc8 0 2024-04-24T23:13:48.561 INFO:tasks.workunit.client.1.smithi040.stdout:5/190: dread d2/d7/d8/db/d1e/f2e [0,4194304] 0 2024-04-24T23:13:48.561 INFO:tasks.workunit.client.1.smithi040.stdout:5/191: fsync d2/d7/d8/db/d1e/d22/f25 0 2024-04-24T23:13:48.561 INFO:tasks.workunit.client.0.smithi031.stdout:9/652: fdatasync d2/dae/d14/d1d/d45/f73 0 2024-04-24T23:13:48.562 INFO:tasks.workunit.client.1.smithi040.stdout:5/192: readlink d2/d7/d8/db/d1e/d22/d3e/l42 0 2024-04-24T23:13:48.563 INFO:tasks.workunit.client.1.smithi040.stdout:3/231: mknod d1/d2d/d44/c51 0 2024-04-24T23:13:48.563 INFO:tasks.workunit.client.0.smithi031.stdout:4/658: creat d3/d20/d25/d60/dc8/fd4 x:0 0 0 2024-04-24T23:13:48.568 INFO:tasks.workunit.client.0.smithi031.stdout:6/670: dread d13/d15/d1d/d23/d24/d43/f5a [0,4194304] 0 2024-04-24T23:13:48.573 INFO:tasks.workunit.client.1.smithi040.stdout:9/265: creat d7/da/d17/d40/d55/d2f/d1f/d31/d4d/f60 x:0 0 0 2024-04-24T23:13:48.574 INFO:tasks.workunit.client.1.smithi040.stdout:9/266: write d7/f57 [900715,44664] 0 2024-04-24T23:13:48.575 INFO:tasks.workunit.client.0.smithi031.stdout:0/600: fdatasync da/fb 0 2024-04-24T23:13:48.580 INFO:tasks.workunit.client.0.smithi031.stdout:7/577: rename d1/d2/d60/c56 to d1/d2/dd/cc9 0 2024-04-24T23:13:48.583 INFO:tasks.workunit.client.1.smithi040.stdout:8/201: creat d0/d16/d1b/f40 x:0 0 0 2024-04-24T23:13:48.585 INFO:tasks.workunit.client.1.smithi040.stdout:0/291: mkdir df/d12/d58 0 2024-04-24T23:13:48.586 INFO:tasks.workunit.client.0.smithi031.stdout:7/578: dwrite d1/d45/d75/d9f/f35 [4194304,4194304] 0 2024-04-24T23:13:48.587 INFO:tasks.workunit.client.0.smithi031.stdout:2/436: link d5/d28/d4e/d5e/l8b d5/d6/df/l9a 0 2024-04-24T23:13:48.591 INFO:tasks.workunit.client.1.smithi040.stdout:5/193: chown d2/d7/d8/d1f/c3a 1427133496 1 2024-04-24T23:13:48.592 INFO:tasks.workunit.client.0.smithi031.stdout:4/659: dwrite d3/d23/d69/d3f/f57 [0,4194304] 0 2024-04-24T23:13:48.594 INFO:tasks.workunit.client.1.smithi040.stdout:3/232: unlink d1/d9/d18/f38 0 2024-04-24T23:13:48.598 INFO:tasks.workunit.client.1.smithi040.stdout:5/194: dwrite d2/d7/d8/db/d1e/d22/f25 [0,4194304] 0 2024-04-24T23:13:48.599 INFO:tasks.workunit.client.1.smithi040.stdout:5/195: read - d2/d7/f36 zero size 2024-04-24T23:13:48.600 INFO:tasks.workunit.client.1.smithi040.stdout:1/224: mkdir da/d12/d18/d2a/d41/d4f 0 2024-04-24T23:13:48.601 INFO:tasks.workunit.client.1.smithi040.stdout:1/225: chown da/d12/d23 3907347 1 2024-04-24T23:13:48.601 INFO:tasks.workunit.client.1.smithi040.stdout:6/226: link d7/de/d15/c39 d7/d1a/c45 0 2024-04-24T23:13:48.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/671: write d13/d15/d1d/d23/d24/d8b/f9b [934365,28368] 0 2024-04-24T23:13:48.605 INFO:tasks.workunit.client.1.smithi040.stdout:4/213: mknod da/c50 0 2024-04-24T23:13:48.607 INFO:tasks.workunit.client.0.smithi031.stdout:0/601: write da/d11/f25 [1085825,127435] 0 2024-04-24T23:13:48.609 INFO:tasks.workunit.client.0.smithi031.stdout:7/579: creat d1/d2/dd/d33/d5e/d64/fca x:0 0 0 2024-04-24T23:13:48.611 INFO:tasks.workunit.client.1.smithi040.stdout:2/293: link da/d10/d15/l3b da/d1c/d37/l6d 0 2024-04-24T23:13:48.612 INFO:tasks.workunit.client.1.smithi040.stdout:6/227: dwrite d7/de/d15/f26 [0,4194304] 0 2024-04-24T23:13:48.614 INFO:tasks.workunit.client.0.smithi031.stdout:1/505: dwrite d1/dc/d11/d21/f28 [0,4194304] 0 2024-04-24T23:13:48.614 INFO:tasks.workunit.client.0.smithi031.stdout:4/660: symlink d3/d20/d2d/d44/d5b/d5d/ld5 0 2024-04-24T23:13:48.615 INFO:tasks.workunit.client.1.smithi040.stdout:3/233: creat d1/d2d/d44/f52 x:0 0 0 2024-04-24T23:13:48.616 INFO:tasks.workunit.client.1.smithi040.stdout:7/269: creat db/f60 x:0 0 0 2024-04-24T23:13:48.621 INFO:tasks.workunit.client.1.smithi040.stdout:2/294: dread da/f20 [0,4194304] 0 2024-04-24T23:13:48.621 INFO:tasks.workunit.client.1.smithi040.stdout:1/226: creat da/d12/d18/d2a/d41/f50 x:0 0 0 2024-04-24T23:13:48.622 INFO:tasks.workunit.client.1.smithi040.stdout:4/214: mknod da/d45/c51 0 2024-04-24T23:13:48.623 INFO:tasks.workunit.client.0.smithi031.stdout:0/602: dread - da/d18/d5a/f5b zero size 2024-04-24T23:13:48.624 INFO:tasks.workunit.client.1.smithi040.stdout:9/267: truncate d7/da/d17/d40/f42 1138995 0 2024-04-24T23:13:48.627 INFO:tasks.workunit.client.0.smithi031.stdout:4/661: creat d3/d23/d3b/fd6 x:0 0 0 2024-04-24T23:13:48.629 INFO:tasks.workunit.client.1.smithi040.stdout:6/228: mkdir d7/de/d15/d1e/d46 0 2024-04-24T23:13:48.629 INFO:tasks.workunit.client.0.smithi031.stdout:9/653: getdents d2/dae/d14/d78/d84/da6 0 2024-04-24T23:13:48.630 INFO:tasks.workunit.client.1.smithi040.stdout:6/229: chown d7/de/d15/d2f 203704 1 2024-04-24T23:13:48.631 INFO:tasks.workunit.client.0.smithi031.stdout:9/654: truncate d2/f12 5929908 0 2024-04-24T23:13:48.631 INFO:tasks.workunit.client.1.smithi040.stdout:3/234: chown d1/d9/d15/c3b 122 1 2024-04-24T23:13:48.633 INFO:tasks.workunit.client.1.smithi040.stdout:9/268: dwrite d7/da/d17/d40/d55/d2f/d1f/d34/f52 [0,4194304] 0 2024-04-24T23:13:48.633 INFO:tasks.workunit.client.1.smithi040.stdout:9/269: stat f2 0 2024-04-24T23:13:48.634 INFO:tasks.workunit.client.0.smithi031.stdout:0/603: mknod da/d22/d62/d9c/cc6 0 2024-04-24T23:13:48.634 INFO:tasks.workunit.client.0.smithi031.stdout:1/506: dwrite d1/da/d10/f51 [4194304,4194304] 0 2024-04-24T23:13:48.635 INFO:tasks.workunit.client.1.smithi040.stdout:9/270: write d7/da/d17/d40/d55/d2f/f46 [4525546,66538] 0 2024-04-24T23:13:48.639 INFO:tasks.workunit.client.1.smithi040.stdout:4/215: rename da/d1d/d2b/f30 to da/d1d/d2b/f52 0 2024-04-24T23:13:48.640 INFO:tasks.workunit.client.1.smithi040.stdout:4/216: write da/f13 [4251446,78616] 0 2024-04-24T23:13:48.641 INFO:tasks.workunit.client.0.smithi031.stdout:2/437: getdents d5/d28/d43 0 2024-04-24T23:13:48.642 INFO:tasks.workunit.client.0.smithi031.stdout:9/655: mknod d2/dae/d14/d78/d84/d7e/cc6 0 2024-04-24T23:13:48.643 INFO:tasks.workunit.client.0.smithi031.stdout:0/604: dwrite da/d22/f54 [0,4194304] 0 2024-04-24T23:13:48.648 INFO:tasks.workunit.client.1.smithi040.stdout:1/227: dread da/d12/d18/f27 [0,4194304] 0 2024-04-24T23:13:48.649 INFO:tasks.workunit.client.0.smithi031.stdout:9/656: chown d2/dae/d14/d78/d84/d4a/f71 354 1 2024-04-24T23:13:48.649 INFO:tasks.workunit.client.0.smithi031.stdout:0/605: write da/d18/d1a/d37/d41/d92/fa8 [2327166,17324] 0 2024-04-24T23:13:48.650 INFO:tasks.workunit.client.0.smithi031.stdout:5/500: sync 2024-04-24T23:13:48.651 INFO:tasks.workunit.client.0.smithi031.stdout:1/507: symlink d1/da/d12/d7e/d7b/l9f 0 2024-04-24T23:13:48.652 INFO:tasks.workunit.client.0.smithi031.stdout:3/513: sync 2024-04-24T23:13:48.653 INFO:tasks.workunit.client.0.smithi031.stdout:5/501: truncate d0/d6/d3b/d4b/dc/f9a 396380 0 2024-04-24T23:13:48.655 INFO:tasks.workunit.client.1.smithi040.stdout:3/235: creat d1/d9/d15/f53 x:0 0 0 2024-04-24T23:13:48.656 INFO:tasks.workunit.client.1.smithi040.stdout:6/230: dread d7/d1a/d20/f28 [0,4194304] 0 2024-04-24T23:13:48.656 INFO:tasks.workunit.client.1.smithi040.stdout:3/236: chown d1/d9/d2e 7086737 1 2024-04-24T23:13:48.657 INFO:tasks.workunit.client.0.smithi031.stdout:2/438: mknod d5/d6/d29/d26/c9b 0 2024-04-24T23:13:48.663 INFO:tasks.workunit.client.0.smithi031.stdout:6/672: write d13/f14 [1317757,117726] 0 2024-04-24T23:13:48.667 INFO:tasks.workunit.client.0.smithi031.stdout:2/439: dwrite d5/d8/d45/d5b/d49/f7e [0,4194304] 0 2024-04-24T23:13:48.667 INFO:tasks.workunit.client.0.smithi031.stdout:7/580: dread d1/f13 [0,4194304] 0 2024-04-24T23:13:48.669 INFO:tasks.workunit.client.0.smithi031.stdout:0/606: rename da/d18/d1a/d37/f4a to da/d18/d1a/d76/d82/fc7 0 2024-04-24T23:13:48.676 INFO:tasks.workunit.client.0.smithi031.stdout:5/502: unlink d0/d6/d3b/d4b/d1d/d2e/l85 0 2024-04-24T23:13:48.677 INFO:tasks.workunit.client.1.smithi040.stdout:4/217: dread da/d1d/d2b/f52 [0,4194304] 0 2024-04-24T23:13:48.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/440: dwrite d5/d28/f40 [0,4194304] 0 2024-04-24T23:13:48.678 INFO:tasks.workunit.client.1.smithi040.stdout:8/202: getdents d0/d16/d1b 0 2024-04-24T23:13:48.679 INFO:tasks.workunit.client.1.smithi040.stdout:8/203: stat d0/d16/c21 0 2024-04-24T23:13:48.680 INFO:tasks.workunit.client.0.smithi031.stdout:1/508: dread d1/dc/d11/d21/f4b [0,4194304] 0 2024-04-24T23:13:48.682 INFO:tasks.workunit.client.1.smithi040.stdout:1/228: creat da/d35/d38/d47/f51 x:0 0 0 2024-04-24T23:13:48.682 INFO:tasks.workunit.client.0.smithi031.stdout:1/509: write d1/da/d10/f4d [2990738,81465] 0 2024-04-24T23:13:48.683 INFO:tasks.workunit.client.1.smithi040.stdout:0/292: truncate f6 3149695 0 2024-04-24T23:13:48.683 INFO:tasks.workunit.client.1.smithi040.stdout:1/229: write da/d12/d18/f49 [419191,105581] 0 2024-04-24T23:13:48.684 INFO:tasks.workunit.client.0.smithi031.stdout:4/662: truncate d3/d23/d69/d3f/f61 1630024 0 2024-04-24T23:13:48.685 INFO:tasks.workunit.client.1.smithi040.stdout:7/270: creat db/f61 x:0 0 0 2024-04-24T23:13:48.686 INFO:tasks.workunit.client.0.smithi031.stdout:6/673: truncate d13/f87 142093 0 2024-04-24T23:13:48.687 INFO:tasks.workunit.client.1.smithi040.stdout:9/271: dread d7/da/d17/d40/d55/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:13:48.688 INFO:tasks.workunit.client.1.smithi040.stdout:6/231: creat d7/d1a/d20/f47 x:0 0 0 2024-04-24T23:13:48.690 INFO:tasks.workunit.client.1.smithi040.stdout:7/271: dread db/d12/d20/d52/f23 [0,4194304] 0 2024-04-24T23:13:48.690 INFO:tasks.workunit.client.1.smithi040.stdout:6/232: truncate d7/f3f 749576 0 2024-04-24T23:13:48.690 INFO:tasks.workunit.client.1.smithi040.stdout:2/295: link da/d10/c1a da/d10/d4d/d6c/c6e 0 2024-04-24T23:13:48.690 INFO:tasks.workunit.client.1.smithi040.stdout:7/272: stat db/f31 0 2024-04-24T23:13:48.693 INFO:tasks.workunit.client.1.smithi040.stdout:4/218: mknod da/d45/c53 0 2024-04-24T23:13:48.693 INFO:tasks.workunit.client.0.smithi031.stdout:0/607: fsync da/d18/d1a/d37/fa5 0 2024-04-24T23:13:48.694 INFO:tasks.workunit.client.0.smithi031.stdout:0/608: dread - da/d11/f7f zero size 2024-04-24T23:13:48.695 INFO:tasks.workunit.client.1.smithi040.stdout:6/233: dread d7/de/d15/d2f/f35 [0,4194304] 0 2024-04-24T23:13:48.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/609: readlink da/d18/d1a/l7a 0 2024-04-24T23:13:48.696 INFO:tasks.workunit.client.1.smithi040.stdout:8/204: mkdir d0/d16/d41 0 2024-04-24T23:13:48.696 INFO:tasks.workunit.client.1.smithi040.stdout:8/205: chown d0 3453 1 2024-04-24T23:13:48.696 INFO:tasks.workunit.client.1.smithi040.stdout:6/234: write d7/de/d15/d2f/f35 [943333,54294] 0 2024-04-24T23:13:48.699 INFO:tasks.workunit.client.1.smithi040.stdout:3/237: creat d1/d9/d15/d45/d4e/f54 x:0 0 0 2024-04-24T23:13:48.702 INFO:tasks.workunit.client.1.smithi040.stdout:0/293: dread df/d14/d1b/f28 [0,4194304] 0 2024-04-24T23:13:48.708 INFO:tasks.workunit.client.1.smithi040.stdout:2/296: mknod da/d1c/d37/d38/c6f 0 2024-04-24T23:13:48.708 INFO:tasks.workunit.client.1.smithi040.stdout:2/297: chown da/cc 843523 1 2024-04-24T23:13:48.713 INFO:tasks.workunit.client.1.smithi040.stdout:6/235: mknod d7/d1a/d20/c48 0 2024-04-24T23:13:48.715 INFO:tasks.workunit.client.1.smithi040.stdout:4/219: dwrite da/d1d/d2b/f52 [4194304,4194304] 0 2024-04-24T23:13:48.715 INFO:tasks.workunit.client.0.smithi031.stdout:8/673: sync 2024-04-24T23:13:48.716 INFO:tasks.workunit.client.1.smithi040.stdout:1/230: creat da/d31/f52 x:0 0 0 2024-04-24T23:13:48.717 INFO:tasks.workunit.client.1.smithi040.stdout:1/231: dread - da/d12/d23/f45 zero size 2024-04-24T23:13:48.717 INFO:tasks.workunit.client.1.smithi040.stdout:4/220: write da/d1d/d2b/f35 [1817364,96138] 0 2024-04-24T23:13:48.718 INFO:tasks.workunit.client.1.smithi040.stdout:4/221: fdatasync da/d3f/f4b 0 2024-04-24T23:13:48.720 INFO:tasks.workunit.client.1.smithi040.stdout:1/232: dread - da/d12/d18/d2a/d32/f4b zero size 2024-04-24T23:13:48.720 INFO:tasks.workunit.client.1.smithi040.stdout:9/272: mkdir d7/da/d17/d40/d55/d61 0 2024-04-24T23:13:48.721 INFO:tasks.workunit.client.1.smithi040.stdout:3/238: write d1/d9/d25/f35 [525471,105306] 0 2024-04-24T23:13:48.726 INFO:tasks.workunit.client.1.smithi040.stdout:7/273: dwrite db/f1c [0,4194304] 0 2024-04-24T23:13:48.727 INFO:tasks.workunit.client.1.smithi040.stdout:7/274: write db/f60 [237212,104570] 0 2024-04-24T23:13:48.728 INFO:tasks.workunit.client.1.smithi040.stdout:4/222: unlink f4 0 2024-04-24T23:13:48.729 INFO:tasks.workunit.client.1.smithi040.stdout:0/294: creat df/d12/d58/f59 x:0 0 0 2024-04-24T23:13:48.730 INFO:tasks.workunit.client.1.smithi040.stdout:7/275: dread db/d2f/f35 [0,4194304] 0 2024-04-24T23:13:48.732 INFO:tasks.workunit.client.1.smithi040.stdout:8/206: creat d0/d8/f42 x:0 0 0 2024-04-24T23:13:48.734 INFO:tasks.workunit.client.1.smithi040.stdout:9/273: symlink d7/da/d17/d40/d55/d2f/d28/l62 0 2024-04-24T23:13:48.735 INFO:tasks.workunit.client.1.smithi040.stdout:9/274: rename d7/da/d17/d40/d55/d2f/d1f/d4e to d7/da/d17/d40/d55/d2f/d1f/d4e/d63 22 2024-04-24T23:13:48.739 INFO:tasks.workunit.client.1.smithi040.stdout:3/239: mkdir d1/d9/d18/d36/d55 0 2024-04-24T23:13:48.739 INFO:tasks.workunit.client.1.smithi040.stdout:2/298: dread da/d1c/d37/f41 [0,4194304] 0 2024-04-24T23:13:48.744 INFO:tasks.workunit.client.1.smithi040.stdout:9/275: dwrite d7/f59 [0,4194304] 0 2024-04-24T23:13:48.744 INFO:tasks.workunit.client.1.smithi040.stdout:3/240: unlink d1/f3 0 2024-04-24T23:13:48.746 INFO:tasks.workunit.client.1.smithi040.stdout:3/241: write d1/d9/d25/f3a [664317,130468] 0 2024-04-24T23:13:48.748 INFO:tasks.workunit.client.1.smithi040.stdout:8/207: rmdir d0/d16/d41 0 2024-04-24T23:13:48.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:48 smithi031 ceph-mon[26411]: pgmap v27: 65 pgs: 65 active+clean; 1.5 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 185 MiB/s wr, 400 op/s 2024-04-24T23:13:48.749 INFO:tasks.workunit.client.1.smithi040.stdout:2/299: symlink da/d1c/d37/d39/d3f/d52/l70 0 2024-04-24T23:13:48.749 INFO:tasks.workunit.client.1.smithi040.stdout:8/208: read d0/d16/d19/f35 [909058,10367] 0 2024-04-24T23:13:48.751 INFO:tasks.workunit.client.1.smithi040.stdout:9/276: creat d7/da/d17/d40/d55/d2f/f64 x:0 0 0 2024-04-24T23:13:48.751 INFO:tasks.workunit.client.1.smithi040.stdout:1/233: link da/l19 da/l53 0 2024-04-24T23:13:48.753 INFO:tasks.workunit.client.1.smithi040.stdout:7/276: getdents db 0 2024-04-24T23:13:48.754 INFO:tasks.workunit.client.1.smithi040.stdout:1/234: readlink da/d12/d18/l29 0 2024-04-24T23:13:48.754 INFO:tasks.workunit.client.1.smithi040.stdout:2/300: creat da/d10/f71 x:0 0 0 2024-04-24T23:13:48.755 INFO:tasks.workunit.client.1.smithi040.stdout:2/301: fsync da/f11 0 2024-04-24T23:13:48.756 INFO:tasks.workunit.client.1.smithi040.stdout:7/277: write db/f55 [477929,48522] 0 2024-04-24T23:13:48.756 INFO:tasks.workunit.client.1.smithi040.stdout:1/235: write da/d12/f4a [382955,84482] 0 2024-04-24T23:13:48.756 INFO:tasks.workunit.client.1.smithi040.stdout:8/209: dread d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:13:48.758 INFO:tasks.workunit.client.1.smithi040.stdout:9/277: truncate d7/da/d17/d40/d55/d2f/d1f/f5a 662875 0 2024-04-24T23:13:48.760 INFO:tasks.workunit.client.1.smithi040.stdout:1/236: rmdir da/d12/d23 39 2024-04-24T23:13:48.761 INFO:tasks.workunit.client.1.smithi040.stdout:9/278: dread d7/da/d17/d40/d55/d2f/d1f/f5a [0,4194304] 0 2024-04-24T23:13:48.762 INFO:tasks.workunit.client.1.smithi040.stdout:1/237: dread f2 [0,4194304] 0 2024-04-24T23:13:48.763 INFO:tasks.workunit.client.1.smithi040.stdout:1/238: fsync da/d12/d14/f3f 0 2024-04-24T23:13:48.768 INFO:tasks.workunit.client.1.smithi040.stdout:2/302: link da/d10/d4d/f64 da/f72 0 2024-04-24T23:13:48.771 INFO:tasks.workunit.client.1.smithi040.stdout:2/303: creat da/d1c/d37/d39/f73 x:0 0 0 2024-04-24T23:13:48.775 INFO:tasks.workunit.client.1.smithi040.stdout:7/278: dread db/d12/f29 [0,4194304] 0 2024-04-24T23:13:48.775 INFO:tasks.workunit.client.1.smithi040.stdout:2/304: dread da/f72 [0,4194304] 0 2024-04-24T23:13:48.775 INFO:tasks.workunit.client.1.smithi040.stdout:7/279: stat db/f3c 0 2024-04-24T23:13:48.776 INFO:tasks.workunit.client.1.smithi040.stdout:7/280: chown db/d2f 1664746 1 2024-04-24T23:13:48.777 INFO:tasks.workunit.client.1.smithi040.stdout:2/305: mknod da/d1c/c74 0 2024-04-24T23:13:48.778 INFO:tasks.workunit.client.1.smithi040.stdout:1/239: dread da/d12/d14/f2d [0,4194304] 0 2024-04-24T23:13:48.780 INFO:tasks.workunit.client.1.smithi040.stdout:7/281: mkdir db/d12/d62 0 2024-04-24T23:13:48.782 INFO:tasks.workunit.client.1.smithi040.stdout:6/236: truncate d7/de/d15/f1f 4563315 0 2024-04-24T23:13:48.783 INFO:tasks.workunit.client.1.smithi040.stdout:1/240: creat da/d12/f54 x:0 0 0 2024-04-24T23:13:48.784 INFO:tasks.workunit.client.1.smithi040.stdout:7/282: symlink db/d12/d20/l63 0 2024-04-24T23:13:48.787 INFO:tasks.workunit.client.1.smithi040.stdout:1/241: write da/d12/d18/d2a/f3d [2512073,6083] 0 2024-04-24T23:13:48.788 INFO:tasks.workunit.client.1.smithi040.stdout:2/306: dwrite da/d10/d15/f32 [0,4194304] 0 2024-04-24T23:13:48.788 INFO:tasks.workunit.client.1.smithi040.stdout:7/283: rmdir db/d12/d20/d52/d44 39 2024-04-24T23:13:48.791 INFO:tasks.workunit.client.1.smithi040.stdout:6/237: link d7/d1a/d20/f3a d7/de/d15/d1e/f49 0 2024-04-24T23:13:48.791 INFO:tasks.workunit.client.0.smithi031.stdout:1/510: creat d1/da/d12/d65/fa0 x:0 0 0 2024-04-24T23:13:48.791 INFO:tasks.workunit.client.0.smithi031.stdout:4/663: fdatasync d3/d23/d37/f7c 0 2024-04-24T23:13:48.792 INFO:tasks.workunit.client.0.smithi031.stdout:8/674: chown d8/d15/d37/d43/c75 165 1 2024-04-24T23:13:48.793 INFO:tasks.workunit.client.1.smithi040.stdout:1/242: write da/d35/d38/d47/f51 [753106,20451] 0 2024-04-24T23:13:48.793 INFO:tasks.workunit.client.0.smithi031.stdout:3/514: creat d6/fa1 x:0 0 0 2024-04-24T23:13:48.794 INFO:tasks.workunit.client.0.smithi031.stdout:7/581: truncate d1/d2/dd/d33/d5e/d64/faa 3997112 0 2024-04-24T23:13:48.794 INFO:tasks.workunit.client.1.smithi040.stdout:0/295: dwrite df/d1e/d20/f33 [0,4194304] 0 2024-04-24T23:13:48.794 INFO:tasks.workunit.client.1.smithi040.stdout:8/210: write d0/d8/dc/f3a [4524293,52814] 0 2024-04-24T23:13:48.795 INFO:tasks.workunit.client.0.smithi031.stdout:1/511: read d1/d33/f47 [1383876,91336] 0 2024-04-24T23:13:48.796 INFO:tasks.workunit.client.1.smithi040.stdout:1/243: read - da/d12/f54 zero size 2024-04-24T23:13:48.797 INFO:tasks.workunit.client.0.smithi031.stdout:3/515: chown d6/f48 12335 1 2024-04-24T23:13:48.799 INFO:tasks.workunit.client.1.smithi040.stdout:0/296: write df/d1e/d20/f43 [1294227,34045] 0 2024-04-24T23:13:48.802 INFO:tasks.workunit.client.1.smithi040.stdout:6/238: dread d7/de/d15/f26 [0,4194304] 0 2024-04-24T23:13:48.802 INFO:tasks.workunit.client.0.smithi031.stdout:5/503: mkdir d0/d6/d3b/d4b/dc/d8b/d9b/da8 0 2024-04-24T23:13:48.803 INFO:tasks.workunit.client.0.smithi031.stdout:9/657: getdents d2/d43 0 2024-04-24T23:13:48.803 INFO:tasks.workunit.client.1.smithi040.stdout:9/279: dwrite f4 [4194304,4194304] 0 2024-04-24T23:13:48.804 INFO:tasks.workunit.client.0.smithi031.stdout:6/674: dread d13/d15/d1d/d23/d44/d47/fd6 [0,4194304] 0 2024-04-24T23:13:48.805 INFO:tasks.workunit.client.1.smithi040.stdout:9/280: chown d7/da/d17/d40/d55/d2f/d1f/d31/c38 2173 1 2024-04-24T23:13:48.805 INFO:tasks.workunit.client.0.smithi031.stdout:0/610: dwrite da/d11/d6a/dba/fbc [0,4194304] 0 2024-04-24T23:13:48.806 INFO:tasks.workunit.client.1.smithi040.stdout:2/307: mknod da/d1c/d37/c75 0 2024-04-24T23:13:48.808 INFO:tasks.workunit.client.0.smithi031.stdout:8/675: dread d8/d1a/f7e [0,4194304] 0 2024-04-24T23:13:48.810 INFO:tasks.workunit.client.1.smithi040.stdout:8/211: symlink d0/d28/l43 0 2024-04-24T23:13:48.816 INFO:tasks.workunit.client.1.smithi040.stdout:8/212: dwrite d0/d28/f39 [0,4194304] 0 2024-04-24T23:13:48.817 INFO:tasks.workunit.client.0.smithi031.stdout:4/664: dread d3/d23/d37/d86/f9a [0,4194304] 0 2024-04-24T23:13:48.819 INFO:tasks.workunit.client.1.smithi040.stdout:7/284: write db/d12/d20/d52/d44/f47 [888342,100289] 0 2024-04-24T23:13:48.820 INFO:tasks.workunit.client.1.smithi040.stdout:7/285: write db/d12/d20/f22 [4553355,31875] 0 2024-04-24T23:13:48.821 INFO:tasks.workunit.client.0.smithi031.stdout:7/582: fsync d1/f13 0 2024-04-24T23:13:48.821 INFO:tasks.workunit.client.0.smithi031.stdout:4/665: write d3/d23/d69/f30 [3873871,105940] 0 2024-04-24T23:13:48.821 INFO:tasks.workunit.client.0.smithi031.stdout:7/583: read d1/d2/f46 [1532086,119808] 0 2024-04-24T23:13:48.823 INFO:tasks.workunit.client.0.smithi031.stdout:4/666: truncate d3/d23/d3b/f56 2148057 0 2024-04-24T23:13:48.826 INFO:tasks.workunit.client.0.smithi031.stdout:1/512: creat d1/d33/d36/fa1 x:0 0 0 2024-04-24T23:13:48.830 INFO:tasks.workunit.client.0.smithi031.stdout:1/513: write d1/d33/d36/fa1 [412720,41252] 0 2024-04-24T23:13:48.832 INFO:tasks.workunit.client.0.smithi031.stdout:7/584: dwrite d1/d45/fc3 [0,4194304] 0 2024-04-24T23:13:48.832 INFO:tasks.workunit.client.1.smithi040.stdout:2/308: rmdir da/d1c/d37/d39/d3f 39 2024-04-24T23:13:48.832 INFO:tasks.workunit.client.1.smithi040.stdout:2/309: stat da/d10 0 2024-04-24T23:13:48.833 INFO:tasks.workunit.client.1.smithi040.stdout:2/310: read da/d10/d15/f2c [1927226,87622] 0 2024-04-24T23:13:48.834 INFO:tasks.workunit.client.1.smithi040.stdout:5/196: sync 2024-04-24T23:13:48.835 INFO:tasks.workunit.client.0.smithi031.stdout:7/585: write d1/d2/dd/d33/d5e/d64/d6d/dae/db1/fc6 [639594,15474] 0 2024-04-24T23:13:48.835 INFO:tasks.workunit.client.1.smithi040.stdout:1/244: mkdir da/d12/d55 0 2024-04-24T23:13:48.835 INFO:tasks.workunit.client.1.smithi040.stdout:5/197: truncate d2/f6 4893413 0 2024-04-24T23:13:48.838 INFO:tasks.workunit.client.0.smithi031.stdout:3/516: dread f0 [4194304,4194304] 0 2024-04-24T23:13:48.839 INFO:tasks.workunit.client.1.smithi040.stdout:8/213: rmdir d0/d8/d22 39 2024-04-24T23:13:48.845 INFO:tasks.workunit.client.1.smithi040.stdout:7/286: rename db/f2b to db/d12/d62/f64 0 2024-04-24T23:13:48.847 INFO:tasks.workunit.client.0.smithi031.stdout:3/517: dwrite d6/d14/d1d/d69/f94 [0,4194304] 0 2024-04-24T23:13:48.848 INFO:tasks.workunit.client.0.smithi031.stdout:0/611: dread da/d18/d1a/d37/f23 [0,4194304] 0 2024-04-24T23:13:48.848 INFO:tasks.workunit.client.1.smithi040.stdout:6/239: symlink d7/l4a 0 2024-04-24T23:13:48.848 INFO:tasks.workunit.client.1.smithi040.stdout:6/240: readlink d7/d1a/l32 0 2024-04-24T23:13:48.849 INFO:tasks.workunit.client.1.smithi040.stdout:6/241: chown d7/d1a 89440 1 2024-04-24T23:13:48.849 INFO:tasks.workunit.client.1.smithi040.stdout:6/242: fsync d7/de/d15/f17 0 2024-04-24T23:13:48.851 INFO:tasks.workunit.client.0.smithi031.stdout:0/612: write da/d22/f54 [897361,8568] 0 2024-04-24T23:13:48.851 INFO:tasks.workunit.client.0.smithi031.stdout:3/518: write d6/df/d16/d2a/d95/d9b/d60/f84 [503674,95490] 0 2024-04-24T23:13:48.862 INFO:tasks.workunit.client.0.smithi031.stdout:9/658: unlink d2/dae/d14/d78/d84/cbb 0 2024-04-24T23:13:48.862 INFO:tasks.workunit.client.1.smithi040.stdout:1/245: truncate f2 1051896 0 2024-04-24T23:13:48.869 INFO:tasks.workunit.client.1.smithi040.stdout:8/214: unlink d0/l6 0 2024-04-24T23:13:48.869 INFO:tasks.workunit.client.1.smithi040.stdout:8/215: dread - d0/d8/dc/d34/f3c zero size 2024-04-24T23:13:48.869 INFO:tasks.workunit.client.1.smithi040.stdout:8/216: fdatasync d0/d8/dc/f3a 0 2024-04-24T23:13:48.869 INFO:tasks.workunit.client.1.smithi040.stdout:8/217: dread - d0/d8/dc/d34/f3c zero size 2024-04-24T23:13:48.870 INFO:tasks.workunit.client.0.smithi031.stdout:8/676: rename d8/d15/d37/d62/f90 to d8/d1a/d1d/d50/dc0/d9d/fea 0 2024-04-24T23:13:48.870 INFO:tasks.workunit.client.0.smithi031.stdout:5/504: write d0/d6/d3b/d4b/d1d/d2e/f54 [940605,105251] 0 2024-04-24T23:13:48.871 INFO:tasks.workunit.client.0.smithi031.stdout:6/675: dwrite d13/d15/d1d/d23/d4d/f88 [0,4194304] 0 2024-04-24T23:13:48.878 INFO:tasks.workunit.client.1.smithi040.stdout:6/243: creat d7/de/d15/f4b x:0 0 0 2024-04-24T23:13:48.880 INFO:tasks.workunit.client.1.smithi040.stdout:9/281: link l5 d7/d5e/l65 0 2024-04-24T23:13:48.881 INFO:tasks.workunit.client.0.smithi031.stdout:0/613: creat da/d11/fc8 x:0 0 0 2024-04-24T23:13:48.883 INFO:tasks.workunit.client.0.smithi031.stdout:4/667: mknod d3/d23/d69/dce/cd7 0 2024-04-24T23:13:48.884 INFO:tasks.workunit.client.0.smithi031.stdout:4/668: stat d3/d23/d3b/fcc 0 2024-04-24T23:13:48.884 INFO:tasks.workunit.client.0.smithi031.stdout:4/669: stat d3/f1b 0 2024-04-24T23:13:48.885 INFO:tasks.workunit.client.0.smithi031.stdout:4/670: write d3/d20/d25/d7e/fc9 [17615,19882] 0 2024-04-24T23:13:48.887 INFO:tasks.workunit.client.0.smithi031.stdout:4/671: chown d3/d20/d2d/d44/c38 2803 1 2024-04-24T23:13:48.890 INFO:tasks.workunit.client.1.smithi040.stdout:1/246: readlink da/l19 0 2024-04-24T23:13:48.891 INFO:tasks.workunit.client.1.smithi040.stdout:1/247: readlink da/l13 0 2024-04-24T23:13:48.892 INFO:tasks.workunit.client.0.smithi031.stdout:9/659: mkdir d2/dae/d14/d78/d84/d7e/dc7 0 2024-04-24T23:13:48.894 INFO:tasks.workunit.client.0.smithi031.stdout:9/660: write d2/dae/d14/d78/d84/d4a/fb1 [228777,27115] 0 2024-04-24T23:13:48.896 INFO:tasks.workunit.client.1.smithi040.stdout:0/297: getdents df/d12 0 2024-04-24T23:13:48.898 INFO:tasks.workunit.client.0.smithi031.stdout:8/677: readlink d8/l20 0 2024-04-24T23:13:48.900 INFO:tasks.workunit.client.0.smithi031.stdout:5/505: readlink d0/d6/d3b/l78 0 2024-04-24T23:13:48.902 INFO:tasks.workunit.client.0.smithi031.stdout:6/676: mkdir d13/d15/d1d/d23/d44/da0/da6/de6 0 2024-04-24T23:13:48.903 INFO:tasks.workunit.client.0.smithi031.stdout:1/514: write d1/f2 [2888110,64171] 0 2024-04-24T23:13:48.904 INFO:tasks.workunit.client.0.smithi031.stdout:6/677: fdatasync d13/d15/d1d/d23/d2d/f3d 0 2024-04-24T23:13:48.905 INFO:tasks.workunit.client.0.smithi031.stdout:0/614: mkdir da/d20/d4e/dc9 0 2024-04-24T23:13:48.906 INFO:tasks.workunit.client.0.smithi031.stdout:3/519: fdatasync d6/d14/d1d/f24 0 2024-04-24T23:13:48.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:48 smithi040 ceph-mon[33128]: pgmap v27: 65 pgs: 65 active+clean; 1.5 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 185 MiB/s wr, 400 op/s 2024-04-24T23:13:48.978 INFO:tasks.workunit.client.0.smithi031.stdout:9/661: creat d2/dae/d14/d1d/d45/d99/d9c/fc8 x:0 0 0 2024-04-24T23:13:48.979 INFO:tasks.workunit.client.1.smithi040.stdout:0/298: mknod df/d14/d1b/c5a 0 2024-04-24T23:13:48.979 INFO:tasks.workunit.client.1.smithi040.stdout:0/299: fsync df/d1e/f3e 0 2024-04-24T23:13:48.980 INFO:tasks.workunit.client.1.smithi040.stdout:8/218: mkdir d0/d16/d1b/d3f/d44 0 2024-04-24T23:13:48.980 INFO:tasks.workunit.client.0.smithi031.stdout:5/506: symlink d0/d6/d32/la9 0 2024-04-24T23:13:48.982 INFO:tasks.workunit.client.1.smithi040.stdout:7/287: unlink db/f60 0 2024-04-24T23:13:48.983 INFO:tasks.workunit.client.1.smithi040.stdout:6/244: truncate d7/de/d15/f1f 3936468 0 2024-04-24T23:13:48.986 INFO:tasks.workunit.client.0.smithi031.stdout:0/615: creat da/d11/d6a/dba/fca x:0 0 0 2024-04-24T23:13:48.987 INFO:tasks.workunit.client.0.smithi031.stdout:0/616: chown da/d22/c43 5186 1 2024-04-24T23:13:48.988 INFO:tasks.workunit.client.0.smithi031.stdout:4/672: symlink d3/d23/d69/d3f/d9e/ld8 0 2024-04-24T23:13:48.989 INFO:tasks.workunit.client.1.smithi040.stdout:4/223: sync 2024-04-24T23:13:48.989 INFO:tasks.workunit.client.1.smithi040.stdout:2/311: link da/d10/d15/f2c da/d1c/d37/d39/f76 0 2024-04-24T23:13:48.991 INFO:tasks.workunit.client.0.smithi031.stdout:3/520: symlink d6/d14/la2 0 2024-04-24T23:13:48.991 INFO:tasks.workunit.client.1.smithi040.stdout:5/198: getdents d2/d7/d8/db/d1e/d22/d27 0 2024-04-24T23:13:48.995 INFO:tasks.workunit.client.1.smithi040.stdout:0/300: mkdir df/d12/d58/d5b 0 2024-04-24T23:13:48.995 INFO:tasks.workunit.client.1.smithi040.stdout:5/199: dread d2/d7/d8/db/d1e/d22/f25 [0,4194304] 0 2024-04-24T23:13:48.998 INFO:tasks.workunit.client.0.smithi031.stdout:6/678: write d13/d15/d1d/d23/d24/d43/f5a [2805078,74611] 0 2024-04-24T23:13:48.999 INFO:tasks.workunit.client.1.smithi040.stdout:9/282: truncate d7/da/d17/d40/d55/d2f/f46 2677241 0 2024-04-24T23:13:49.000 INFO:tasks.workunit.client.1.smithi040.stdout:8/219: symlink d0/d8/dc/l45 0 2024-04-24T23:13:49.002 INFO:tasks.workunit.client.1.smithi040.stdout:7/288: chown db/d43/l5f 238202667 1 2024-04-24T23:13:49.004 INFO:tasks.workunit.client.1.smithi040.stdout:6/245: rename d7/c1b to d7/de/d15/d44/c4c 0 2024-04-24T23:13:49.009 INFO:tasks.workunit.client.1.smithi040.stdout:5/200: readlink d2/d7/d8/db/d1e/d22/l49 0 2024-04-24T23:13:49.009 INFO:tasks.workunit.client.1.smithi040.stdout:6/246: dwrite d7/d1a/d20/f47 [0,4194304] 0 2024-04-24T23:13:49.010 INFO:tasks.workunit.client.1.smithi040.stdout:8/220: truncate d0/d1f/f2d 927306 0 2024-04-24T23:13:49.011 INFO:tasks.workunit.client.1.smithi040.stdout:5/201: write d2/d7/d8/d1f/f4b [418661,86795] 0 2024-04-24T23:13:49.011 INFO:tasks.workunit.client.1.smithi040.stdout:7/289: chown db/f2e 2568369 1 2024-04-24T23:13:49.011 INFO:tasks.workunit.client.1.smithi040.stdout:6/247: write d7/de/d15/d1e/f22 [1297827,39197] 0 2024-04-24T23:13:49.013 INFO:tasks.workunit.client.1.smithi040.stdout:7/290: fdatasync f6 0 2024-04-24T23:13:49.024 INFO:tasks.workunit.client.1.smithi040.stdout:7/291: readlink db/l1a 0 2024-04-24T23:13:49.029 INFO:tasks.workunit.client.1.smithi040.stdout:7/292: dwrite f2 [4194304,4194304] 0 2024-04-24T23:13:49.037 INFO:tasks.workunit.client.1.smithi040.stdout:2/312: rename da/d1c/d37/d38 to da/d1c/d37/d39/d77 0 2024-04-24T23:13:49.039 INFO:tasks.workunit.client.1.smithi040.stdout:7/293: dread db/d12/d20/d52/d44/f53 [0,4194304] 0 2024-04-24T23:13:49.040 INFO:tasks.workunit.client.1.smithi040.stdout:0/301: truncate df/d12/d1a/f3a 917043 0 2024-04-24T23:13:49.043 INFO:tasks.workunit.client.1.smithi040.stdout:9/283: rename d7/da/d17/d40/d55/d2f/d1f/d4e/c56 to d7/da/d17/c66 0 2024-04-24T23:13:49.046 INFO:tasks.workunit.client.1.smithi040.stdout:7/294: mknod db/d12/d20/c65 0 2024-04-24T23:13:49.049 INFO:tasks.workunit.client.1.smithi040.stdout:5/202: rename d2/d7/d8/db/d1e/l35 to d2/d7/d8/db/d1e/d22/d27/l4c 0 2024-04-24T23:13:49.051 INFO:tasks.workunit.client.1.smithi040.stdout:2/313: mkdir da/d1c/d37/d39/d3f/d52/d78 0 2024-04-24T23:13:49.052 INFO:tasks.workunit.client.1.smithi040.stdout:9/284: mkdir d7/da/d17/d40/d55/d2f/d28/d67 0 2024-04-24T23:13:49.057 INFO:tasks.workunit.client.0.smithi031.stdout:8/678: symlink d8/d15/d37/d43/daf/dca/leb 0 2024-04-24T23:13:49.057 INFO:tasks.workunit.client.0.smithi031.stdout:2/441: sync 2024-04-24T23:13:49.058 INFO:tasks.workunit.client.1.smithi040.stdout:2/314: write da/d29/f69 [669172,103015] 0 2024-04-24T23:13:49.060 INFO:tasks.workunit.client.1.smithi040.stdout:6/248: dwrite d7/de/d15/d1e/f49 [0,4194304] 0 2024-04-24T23:13:49.063 INFO:tasks.workunit.client.0.smithi031.stdout:4/673: rename d3/d20/d2d/f40 to d3/d23/d69/d3f/fd9 0 2024-04-24T23:13:49.064 INFO:tasks.workunit.client.0.smithi031.stdout:4/674: fdatasync d3/f2b 0 2024-04-24T23:13:49.064 INFO:tasks.workunit.client.1.smithi040.stdout:0/302: dread df/d1e/f2f [0,4194304] 0 2024-04-24T23:13:49.065 INFO:tasks.workunit.client.0.smithi031.stdout:3/521: creat d6/df/d16/d2a/d95/d9b/d29/fa3 x:0 0 0 2024-04-24T23:13:49.065 INFO:tasks.workunit.client.1.smithi040.stdout:8/221: dwrite d0/d1f/f2d [0,4194304] 0 2024-04-24T23:13:49.066 INFO:tasks.workunit.client.0.smithi031.stdout:4/675: fdatasync d3/d20/d2d/d44/d5b/d5d/f97 0 2024-04-24T23:13:49.067 INFO:tasks.workunit.client.1.smithi040.stdout:5/203: link d2/d7/d8/db/c37 d2/d7/d8/db/d1e/d22/d27/d3d/c4d 0 2024-04-24T23:13:49.067 INFO:tasks.workunit.client.1.smithi040.stdout:8/222: dread - d0/d8/f42 zero size 2024-04-24T23:13:49.068 INFO:tasks.workunit.client.0.smithi031.stdout:3/522: write d6/f7b [207465,22160] 0 2024-04-24T23:13:49.068 INFO:tasks.workunit.client.1.smithi040.stdout:0/303: read df/d14/d1b/f28 [1320580,73059] 0 2024-04-24T23:13:49.068 INFO:tasks.workunit.client.1.smithi040.stdout:3/242: sync 2024-04-24T23:13:49.068 INFO:tasks.workunit.client.0.smithi031.stdout:4/676: fdatasync d3/da3/fae 0 2024-04-24T23:13:49.069 INFO:tasks.workunit.client.1.smithi040.stdout:1/248: sync 2024-04-24T23:13:49.070 INFO:tasks.workunit.client.0.smithi031.stdout:6/679: symlink d13/d15/d1d/d23/d24/d43/d68/le7 0 2024-04-24T23:13:49.071 INFO:tasks.workunit.client.0.smithi031.stdout:6/680: fdatasync f11 0 2024-04-24T23:13:49.074 INFO:tasks.workunit.client.0.smithi031.stdout:9/662: getdents d2/dae/d14/d78/d84/da6 0 2024-04-24T23:13:49.076 INFO:tasks.workunit.client.1.smithi040.stdout:1/249: dwrite da/d12/d14/f24 [4194304,4194304] 0 2024-04-24T23:13:49.078 INFO:tasks.workunit.client.1.smithi040.stdout:1/250: write da/d35/d38/d47/f51 [695983,82142] 0 2024-04-24T23:13:49.083 INFO:tasks.workunit.client.1.smithi040.stdout:0/304: mknod df/d12/d49/c5c 0 2024-04-24T23:13:49.087 INFO:tasks.workunit.client.1.smithi040.stdout:1/251: dwrite da/d12/d18/d2a/f3d [4194304,4194304] 0 2024-04-24T23:13:49.088 INFO:tasks.workunit.client.1.smithi040.stdout:1/252: stat da/d12/d18/f22 0 2024-04-24T23:13:49.091 INFO:tasks.workunit.client.1.smithi040.stdout:2/315: mkdir da/d1c/d37/d39/d79 0 2024-04-24T23:13:49.095 INFO:tasks.workunit.client.1.smithi040.stdout:8/223: mknod d0/d38/c46 0 2024-04-24T23:13:49.095 INFO:tasks.workunit.client.1.smithi040.stdout:1/253: dwrite da/d12/d14/f3f [0,4194304] 0 2024-04-24T23:13:49.097 INFO:tasks.workunit.client.0.smithi031.stdout:3/523: symlink d6/df/d16/d2a/la4 0 2024-04-24T23:13:49.099 INFO:tasks.workunit.client.1.smithi040.stdout:6/249: rename d7/d2c/c2d to d7/de/d15/d44/c4d 0 2024-04-24T23:13:49.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/515: sync 2024-04-24T23:13:49.100 INFO:tasks.workunit.client.0.smithi031.stdout:7/586: sync 2024-04-24T23:13:49.101 INFO:tasks.workunit.client.1.smithi040.stdout:6/250: write d7/de/d15/f17 [5231292,12463] 0 2024-04-24T23:13:49.102 INFO:tasks.workunit.client.0.smithi031.stdout:7/587: truncate d1/d45/d75/d9f/d2c/f4c 904811 0 2024-04-24T23:13:49.102 INFO:tasks.workunit.client.0.smithi031.stdout:3/524: read d6/df/d16/d53/f67 [1470523,123656] 0 2024-04-24T23:13:49.103 INFO:tasks.workunit.client.0.smithi031.stdout:7/588: fsync d1/d2/dd/f95 0 2024-04-24T23:13:49.104 INFO:tasks.workunit.client.1.smithi040.stdout:6/251: write d7/d1a/d20/f47 [1731434,121420] 0 2024-04-24T23:13:49.106 INFO:tasks.workunit.client.0.smithi031.stdout:6/681: dread d13/d15/d1d/d23/d2d/f3d [0,4194304] 0 2024-04-24T23:13:49.106 INFO:tasks.workunit.client.1.smithi040.stdout:0/305: mknod df/d1e/d20/d21/d51/c5d 0 2024-04-24T23:13:49.107 INFO:tasks.workunit.client.1.smithi040.stdout:9/285: dwrite d7/da/d17/d40/f42 [0,4194304] 0 2024-04-24T23:13:49.108 INFO:tasks.workunit.client.0.smithi031.stdout:9/663: creat d2/d16/dbe/d7d/fc9 x:0 0 0 2024-04-24T23:13:49.109 INFO:tasks.workunit.client.0.smithi031.stdout:9/664: readlink d2/l37 0 2024-04-24T23:13:49.110 INFO:tasks.workunit.client.0.smithi031.stdout:9/665: write d2/dae/d14/f62 [5103774,36859] 0 2024-04-24T23:13:49.112 INFO:tasks.workunit.client.0.smithi031.stdout:0/617: write da/d18/d1a/d37/d41/f47 [2313429,83587] 0 2024-04-24T23:13:49.114 INFO:tasks.workunit.client.1.smithi040.stdout:2/316: mkdir da/d10/d7a 0 2024-04-24T23:13:49.114 INFO:tasks.workunit.client.1.smithi040.stdout:9/286: dwrite d7/da/d17/d40/d55/d2f/d1f/d34/f52 [0,4194304] 0 2024-04-24T23:13:49.116 INFO:tasks.workunit.client.1.smithi040.stdout:8/224: mkdir d0/d16/d1b/d47 0 2024-04-24T23:13:49.117 INFO:tasks.workunit.client.1.smithi040.stdout:2/317: read - da/d29/d55/f58 zero size 2024-04-24T23:13:49.118 INFO:tasks.workunit.client.1.smithi040.stdout:1/254: creat da/d12/d18/d2a/f56 x:0 0 0 2024-04-24T23:13:49.118 INFO:tasks.workunit.client.0.smithi031.stdout:5/507: write d0/d6/d3b/d4b/dc/f80 [671399,23578] 0 2024-04-24T23:13:49.120 INFO:tasks.workunit.client.1.smithi040.stdout:1/255: write da/d12/d18/f49 [1367217,91581] 0 2024-04-24T23:13:49.121 INFO:tasks.workunit.client.1.smithi040.stdout:9/287: dread d7/f57 [0,4194304] 0 2024-04-24T23:13:49.122 INFO:tasks.workunit.client.0.smithi031.stdout:8/679: dwrite d8/fa8 [0,4194304] 0 2024-04-24T23:13:49.123 INFO:tasks.workunit.client.0.smithi031.stdout:2/442: dwrite d5/d6/f1f [0,4194304] 0 2024-04-24T23:13:49.124 INFO:tasks.workunit.client.1.smithi040.stdout:3/243: rename d1/d9/d18/f1f to d1/d9/d18/d36/d55/f56 0 2024-04-24T23:13:49.124 INFO:tasks.workunit.client.0.smithi031.stdout:8/680: readlink d8/d1a/d74/l93 0 2024-04-24T23:13:49.125 INFO:tasks.workunit.client.0.smithi031.stdout:5/508: write d0/d6/d3b/d4f/f98 [761328,126991] 0 2024-04-24T23:13:49.128 INFO:tasks.workunit.client.0.smithi031.stdout:1/516: mkdir d1/d3e/da2 0 2024-04-24T23:13:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:1/517: truncate d1/f2 5405927 0 2024-04-24T23:13:49.130 INFO:tasks.workunit.client.0.smithi031.stdout:1/518: write d1/d1d/f8e [935604,124094] 0 2024-04-24T23:13:49.131 INFO:tasks.workunit.client.1.smithi040.stdout:4/224: sync 2024-04-24T23:13:49.137 INFO:tasks.workunit.client.1.smithi040.stdout:0/306: unlink df/d14/d1b/l48 0 2024-04-24T23:13:49.139 INFO:tasks.workunit.client.0.smithi031.stdout:6/682: creat d13/d1b/d32/db4/fe8 x:0 0 0 2024-04-24T23:13:49.144 INFO:tasks.workunit.client.1.smithi040.stdout:5/204: truncate d2/f6 4592248 0 2024-04-24T23:13:49.147 INFO:tasks.workunit.client.0.smithi031.stdout:6/683: dread d13/d1b/d32/f6d [4194304,4194304] 0 2024-04-24T23:13:49.148 INFO:tasks.workunit.client.0.smithi031.stdout:6/684: write d13/d15/f27 [4458763,119941] 0 2024-04-24T23:13:49.150 INFO:tasks.workunit.client.0.smithi031.stdout:6/685: truncate d13/d15/d1d/d23/d24/d43/d68/db8/fcc 1446400 0 2024-04-24T23:13:49.155 INFO:tasks.workunit.client.0.smithi031.stdout:7/589: dread d1/d2/dd/f23 [0,4194304] 0 2024-04-24T23:13:49.158 INFO:tasks.workunit.client.0.smithi031.stdout:2/443: symlink d5/d28/d4e/l9c 0 2024-04-24T23:13:49.161 INFO:tasks.workunit.client.1.smithi040.stdout:1/256: fsync da/d12/d23/f3b 0 2024-04-24T23:13:49.161 INFO:tasks.workunit.client.1.smithi040.stdout:1/257: read - da/d12/d18/d2a/d32/f4b zero size 2024-04-24T23:13:49.163 INFO:tasks.workunit.client.0.smithi031.stdout:3/525: rename d6/df/d16/d2a/f4f to d6/df/d16/d53/fa5 0 2024-04-24T23:13:49.166 INFO:tasks.workunit.client.1.smithi040.stdout:3/244: rmdir d1/d2d 39 2024-04-24T23:13:49.167 INFO:tasks.workunit.client.0.smithi031.stdout:1/519: fsync d1/da/f2c 0 2024-04-24T23:13:49.167 INFO:tasks.workunit.client.1.smithi040.stdout:1/258: dwrite da/d12/d18/f22 [0,4194304] 0 2024-04-24T23:13:49.169 INFO:tasks.workunit.client.0.smithi031.stdout:1/520: dread d1/f85 [0,4194304] 0 2024-04-24T23:13:49.169 INFO:tasks.workunit.client.1.smithi040.stdout:6/252: mkdir d7/de/d4e 0 2024-04-24T23:13:49.172 INFO:tasks.workunit.client.1.smithi040.stdout:4/225: rename da/l10 to da/d3f/l54 0 2024-04-24T23:13:49.173 INFO:tasks.workunit.client.1.smithi040.stdout:7/295: sync 2024-04-24T23:13:49.173 INFO:tasks.workunit.client.1.smithi040.stdout:4/226: fsync da/d3f/d40/d41/f43 0 2024-04-24T23:13:49.177 INFO:tasks.workunit.client.1.smithi040.stdout:0/307: dread df/d12/d1a/f3a [0,4194304] 0 2024-04-24T23:13:49.177 INFO:tasks.workunit.client.1.smithi040.stdout:1/259: dwrite da/d12/f4a [0,4194304] 0 2024-04-24T23:13:49.179 INFO:tasks.workunit.client.0.smithi031.stdout:4/677: getdents d3/d5e 0 2024-04-24T23:13:49.180 INFO:tasks.workunit.client.0.smithi031.stdout:4/678: chown d3/d23/d37/c49 186 1 2024-04-24T23:13:49.181 INFO:tasks.workunit.client.1.smithi040.stdout:8/225: link d0/d16/d1b/f40 d0/d16/d1b/d47/f48 0 2024-04-24T23:13:49.181 INFO:tasks.workunit.client.1.smithi040.stdout:9/288: dwrite d7/da/d17/d40/d55/d2f/d1f/f5a [0,4194304] 0 2024-04-24T23:13:49.181 INFO:tasks.workunit.client.0.smithi031.stdout:7/590: truncate d1/d2/dd/d33/d5e/d64/d6d/fac 1468650 0 2024-04-24T23:13:49.182 INFO:tasks.workunit.client.1.smithi040.stdout:8/226: chown d0/d8/dc/l27 4181 1 2024-04-24T23:13:49.183 INFO:tasks.workunit.client.1.smithi040.stdout:9/289: chown d7/da/d17/d40/d55/d61 207827 1 2024-04-24T23:13:49.184 INFO:tasks.workunit.client.1.smithi040.stdout:8/227: write d0/d8/dc/f3a [2708274,85670] 0 2024-04-24T23:13:49.188 INFO:tasks.workunit.client.0.smithi031.stdout:3/526: rename d6/df/d16/f7f to d6/df/d16/fa6 0 2024-04-24T23:13:49.193 INFO:tasks.workunit.client.0.smithi031.stdout:9/666: link d2/c81 d2/d43/d9d/dbd/cca 0 2024-04-24T23:13:49.193 INFO:tasks.workunit.client.1.smithi040.stdout:7/296: stat db/f18 0 2024-04-24T23:13:49.195 INFO:tasks.workunit.client.0.smithi031.stdout:6/686: rename d13/d1b/d32/db4/lbb to d13/d1b/d32/db4/le9 0 2024-04-24T23:13:49.197 INFO:tasks.workunit.client.1.smithi040.stdout:5/205: creat d2/d7/d8/d1f/d31/f4e x:0 0 0 2024-04-24T23:13:49.197 INFO:tasks.workunit.client.0.smithi031.stdout:1/521: dread d1/da/d10/f51 [0,4194304] 0 2024-04-24T23:13:49.198 INFO:tasks.workunit.client.0.smithi031.stdout:5/509: creat d0/faa x:0 0 0 2024-04-24T23:13:49.198 INFO:tasks.workunit.client.0.smithi031.stdout:4/679: readlink d3/l17 0 2024-04-24T23:13:49.199 INFO:tasks.workunit.client.0.smithi031.stdout:1/522: stat d1/da/d42/d69 0 2024-04-24T23:13:49.200 INFO:tasks.workunit.client.0.smithi031.stdout:7/591: creat d1/d2/dd/d33/fcb x:0 0 0 2024-04-24T23:13:49.202 INFO:tasks.workunit.client.0.smithi031.stdout:1/523: chown d1/dc/d11/d21/d5a/l89 332981 1 2024-04-24T23:13:49.203 INFO:tasks.workunit.client.0.smithi031.stdout:6/687: dwrite d13/d1b/d32/f8a [0,4194304] 0 2024-04-24T23:13:49.203 INFO:tasks.workunit.client.1.smithi040.stdout:5/206: dwrite d2/d7/f3b [0,4194304] 0 2024-04-24T23:13:49.206 INFO:tasks.workunit.client.1.smithi040.stdout:0/308: mknod df/d1e/d20/d21/d51/c5e 0 2024-04-24T23:13:49.208 INFO:tasks.workunit.client.1.smithi040.stdout:9/290: creat d7/da/d17/d40/d55/d2f/d1f/d4e/f68 x:0 0 0 2024-04-24T23:13:49.213 INFO:tasks.workunit.client.1.smithi040.stdout:3/245: mknod d1/d2d/de/d30/c57 0 2024-04-24T23:13:49.215 INFO:tasks.workunit.client.1.smithi040.stdout:5/207: dwrite d2/d7/d8/d1f/f38 [0,4194304] 0 2024-04-24T23:13:49.215 INFO:tasks.workunit.client.1.smithi040.stdout:6/253: symlink d7/de/d15/d1e/d3e/l4f 0 2024-04-24T23:13:49.217 INFO:tasks.workunit.client.1.smithi040.stdout:5/208: stat d2/d7/f3b 0 2024-04-24T23:13:49.224 INFO:tasks.workunit.client.1.smithi040.stdout:2/318: link da/d1c/d37/d39/f76 da/d1c/d37/d39/d3f/f7b 0 2024-04-24T23:13:49.224 INFO:tasks.workunit.client.1.smithi040.stdout:7/297: dwrite db/f31 [0,4194304] 0 2024-04-24T23:13:49.226 INFO:tasks.workunit.client.1.smithi040.stdout:7/298: write db/d12/d20/d52/d44/f47 [435699,109386] 0 2024-04-24T23:13:49.227 INFO:tasks.workunit.client.1.smithi040.stdout:0/309: rename df/d14/l45 to df/d1e/d20/d21/d51/l5f 0 2024-04-24T23:13:49.227 INFO:tasks.workunit.client.1.smithi040.stdout:7/299: truncate db/f31 4594990 0 2024-04-24T23:13:49.229 INFO:tasks.workunit.client.0.smithi031.stdout:7/592: fdatasync d1/d2/f46 0 2024-04-24T23:13:49.238 INFO:tasks.workunit.client.0.smithi031.stdout:8/681: rename d8/d15/d37/d43/c75 to d8/d1a/d1d/d35/d58/cec 0 2024-04-24T23:13:49.239 INFO:tasks.workunit.client.0.smithi031.stdout:2/444: truncate d5/d6/f1f 3227889 0 2024-04-24T23:13:49.242 INFO:tasks.workunit.client.0.smithi031.stdout:6/688: dread d13/d15/d1d/d23/d24/f9f [0,4194304] 0 2024-04-24T23:13:49.243 INFO:tasks.workunit.client.1.smithi040.stdout:6/254: symlink d7/de/d15/d1e/l50 0 2024-04-24T23:13:49.245 INFO:tasks.workunit.client.1.smithi040.stdout:5/209: chown d2/d7/c41 17 1 2024-04-24T23:13:49.247 INFO:tasks.workunit.client.0.smithi031.stdout:9/667: mkdir d2/dae/d14/d78/d84/d4a/d4b/dcb 0 2024-04-24T23:13:49.249 INFO:tasks.workunit.client.0.smithi031.stdout:3/527: dwrite d6/df/f61 [0,4194304] 0 2024-04-24T23:13:49.252 INFO:tasks.workunit.client.0.smithi031.stdout:7/593: dread d1/d45/d75/d9f/d66/f69 [0,4194304] 0 2024-04-24T23:13:49.254 INFO:tasks.workunit.client.0.smithi031.stdout:5/510: dread d0/d6/d32/f4 [0,4194304] 0 2024-04-24T23:13:49.256 INFO:tasks.workunit.client.0.smithi031.stdout:5/511: fdatasync d0/d6/d32/f39 0 2024-04-24T23:13:49.258 INFO:tasks.workunit.client.0.smithi031.stdout:3/528: dwrite d6/d14/d1d/d59/f9a [0,4194304] 0 2024-04-24T23:13:49.263 INFO:tasks.workunit.client.1.smithi040.stdout:7/300: mkdir db/d43/d66 0 2024-04-24T23:13:49.266 INFO:tasks.workunit.client.1.smithi040.stdout:3/246: write d1/d9/d15/f1b [1633376,31669] 0 2024-04-24T23:13:49.268 INFO:tasks.workunit.client.0.smithi031.stdout:1/524: dwrite d1/dc/d11/d21/d5a/f76 [0,4194304] 0 2024-04-24T23:13:49.270 INFO:tasks.workunit.client.0.smithi031.stdout:1/525: readlink d1/da/d42/l57 0 2024-04-24T23:13:49.271 INFO:tasks.workunit.client.1.smithi040.stdout:5/210: dread d2/d7/d8/f34 [0,4194304] 0 2024-04-24T23:13:49.273 INFO:tasks.workunit.client.0.smithi031.stdout:2/445: dread d5/d28/f37 [0,4194304] 0 2024-04-24T23:13:49.273 INFO:tasks.workunit.client.0.smithi031.stdout:6/689: dread - d13/d15/d1d/d23/d2d/d9e/fc1 zero size 2024-04-24T23:13:49.273 INFO:tasks.workunit.client.0.smithi031.stdout:2/446: readlink d5/d28/d4e/d5e/l71 0 2024-04-24T23:13:49.275 INFO:tasks.workunit.client.0.smithi031.stdout:9/668: dread - d2/dae/d85/f9b zero size 2024-04-24T23:13:49.279 INFO:tasks.workunit.client.1.smithi040.stdout:6/255: rename d7/de/d15 to d7/d1a/d20/d51 0 2024-04-24T23:13:49.279 INFO:tasks.workunit.client.1.smithi040.stdout:5/211: dwrite d2/d7/d8/f34 [4194304,4194304] 0 2024-04-24T23:13:49.284 INFO:tasks.workunit.client.1.smithi040.stdout:4/227: getdents da/d45/d48 0 2024-04-24T23:13:49.285 INFO:tasks.workunit.client.1.smithi040.stdout:9/291: truncate d7/da/d17/d40/f42 3114128 0 2024-04-24T23:13:49.285 INFO:tasks.workunit.client.1.smithi040.stdout:4/228: chown da/d1d/f32 1244084640 1 2024-04-24T23:13:49.286 INFO:tasks.workunit.client.1.smithi040.stdout:1/260: getdents da/d35/d38/d47 0 2024-04-24T23:13:49.288 INFO:tasks.workunit.client.0.smithi031.stdout:5/512: read - d0/d6/f40 zero size 2024-04-24T23:13:49.289 INFO:tasks.workunit.client.0.smithi031.stdout:5/513: stat d0/d6/d12/f71 0 2024-04-24T23:13:49.290 INFO:tasks.workunit.client.1.smithi040.stdout:8/228: link d0/l13 d0/d8/d22/l49 0 2024-04-24T23:13:49.293 INFO:tasks.workunit.client.1.smithi040.stdout:3/247: creat d1/d2d/de/d30/f58 x:0 0 0 2024-04-24T23:13:49.295 INFO:tasks.workunit.client.0.smithi031.stdout:1/526: rename d1/da/d42 to d1/d3e/da3 0 2024-04-24T23:13:49.302 INFO:tasks.workunit.client.0.smithi031.stdout:1/527: dwrite d1/da/d10/f4d [0,4194304] 0 2024-04-24T23:13:49.306 INFO:tasks.workunit.client.0.smithi031.stdout:4/680: getdents d3/d5e 0 2024-04-24T23:13:49.307 INFO:tasks.workunit.client.0.smithi031.stdout:4/681: chown d3/d20/d2d/d44/d5b/d5d/ld5 73393 1 2024-04-24T23:13:49.308 INFO:tasks.workunit.client.0.smithi031.stdout:4/682: fdatasync d3/d23/fc5 0 2024-04-24T23:13:49.314 INFO:tasks.workunit.client.1.smithi040.stdout:5/212: dwrite d2/d7/d8/f13 [0,4194304] 0 2024-04-24T23:13:49.315 INFO:tasks.workunit.client.1.smithi040.stdout:5/213: stat d2/d7/d8/d1f/d31 0 2024-04-24T23:13:49.316 INFO:tasks.workunit.client.1.smithi040.stdout:5/214: chown d2/d7/d8/db/d1e/d22/d27/f32 0 1 2024-04-24T23:13:49.316 INFO:tasks.workunit.client.1.smithi040.stdout:2/319: creat da/d1c/d37/d39/f7c x:0 0 0 2024-04-24T23:13:49.317 INFO:tasks.workunit.client.1.smithi040.stdout:2/320: stat da/d1c 0 2024-04-24T23:13:49.319 INFO:tasks.workunit.client.1.smithi040.stdout:8/229: mknod d0/d16/d19/c4a 0 2024-04-24T23:13:49.321 INFO:tasks.workunit.client.0.smithi031.stdout:1/528: creat d1/dc/d11/d21/fa4 x:0 0 0 2024-04-24T23:13:49.322 INFO:tasks.workunit.client.0.smithi031.stdout:2/447: truncate d5/d8/d15/f5f 5748 0 2024-04-24T23:13:49.323 INFO:tasks.workunit.client.1.smithi040.stdout:0/310: sync 2024-04-24T23:13:49.323 INFO:tasks.workunit.client.0.smithi031.stdout:2/448: dread - d5/d8/d45/f8d zero size 2024-04-24T23:13:49.324 INFO:tasks.workunit.client.1.smithi040.stdout:0/311: write df/d12/f22 [2229686,53676] 0 2024-04-24T23:13:49.325 INFO:tasks.workunit.client.0.smithi031.stdout:9/669: unlink d2/dae/d14/d78/d84/d4a/l8a 0 2024-04-24T23:13:49.326 INFO:tasks.workunit.client.0.smithi031.stdout:9/670: truncate d2/dae/d14/d78/d84/d4a/fa5 380351 0 2024-04-24T23:13:49.327 INFO:tasks.workunit.client.0.smithi031.stdout:4/683: creat d3/d5e/fda x:0 0 0 2024-04-24T23:13:49.329 INFO:tasks.workunit.client.0.smithi031.stdout:4/684: read d3/d23/d69/d3f/f6d [1335846,125717] 0 2024-04-24T23:13:49.330 INFO:tasks.workunit.client.0.smithi031.stdout:8/682: getdents d8/d1a/d1d/d35/d58/d5b 0 2024-04-24T23:13:49.331 INFO:tasks.workunit.client.0.smithi031.stdout:8/683: fsync d8/d15/d37/d43/f4b 0 2024-04-24T23:13:49.339 INFO:tasks.workunit.client.1.smithi040.stdout:9/292: truncate f2 222177 0 2024-04-24T23:13:49.348 INFO:tasks.workunit.client.0.smithi031.stdout:9/671: write d2/d16/f20 [5316517,59514] 0 2024-04-24T23:13:49.350 INFO:tasks.workunit.client.1.smithi040.stdout:5/215: symlink d2/d7/d8/db/d1e/d22/d27/l4f 0 2024-04-24T23:13:49.356 INFO:tasks.workunit.client.0.smithi031.stdout:8/684: creat d8/d15/d37/d43/daf/fed x:0 0 0 2024-04-24T23:13:49.357 INFO:tasks.workunit.client.1.smithi040.stdout:8/230: fdatasync d0/d16/d1b/f40 0 2024-04-24T23:13:49.358 INFO:tasks.workunit.client.0.smithi031.stdout:1/529: symlink d1/d33/d41/d9b/la5 0 2024-04-24T23:13:49.358 INFO:tasks.workunit.client.0.smithi031.stdout:1/530: chown d1/cd 123 1 2024-04-24T23:13:49.359 INFO:tasks.workunit.client.0.smithi031.stdout:1/531: chown d1/d31/f39 31 1 2024-04-24T23:13:49.360 INFO:tasks.workunit.client.0.smithi031.stdout:2/449: fdatasync d5/d8/f98 0 2024-04-24T23:13:49.361 INFO:tasks.workunit.client.0.smithi031.stdout:2/450: chown d5/d6/d29/d26/l74 10052317 1 2024-04-24T23:13:49.366 INFO:tasks.workunit.client.1.smithi040.stdout:6/256: rename d7/d1a/d20/d51/d44 to d7/d2c/d52 0 2024-04-24T23:13:49.368 INFO:tasks.workunit.client.1.smithi040.stdout:4/229: link da/d1d/f32 da/d1d/d2b/f55 0 2024-04-24T23:13:49.368 INFO:tasks.workunit.client.1.smithi040.stdout:4/230: chown da/d1d/d1e/f24 3 1 2024-04-24T23:13:49.369 INFO:tasks.workunit.client.1.smithi040.stdout:4/231: readlink l1 0 2024-04-24T23:13:49.372 INFO:tasks.workunit.client.1.smithi040.stdout:1/261: link da/c34 da/d12/d18/c57 0 2024-04-24T23:13:49.372 INFO:tasks.workunit.client.1.smithi040.stdout:0/312: write df/d1e/d2b/f29 [189834,33696] 0 2024-04-24T23:13:49.373 INFO:tasks.workunit.client.0.smithi031.stdout:0/618: sync 2024-04-24T23:13:49.374 INFO:tasks.workunit.client.0.smithi031.stdout:9/672: rmdir d2/dae/d14/d78/d84/d4a 39 2024-04-24T23:13:49.376 INFO:tasks.workunit.client.0.smithi031.stdout:5/514: link d0/d6/d3b/d4b/d1d/d2e/c76 d0/d6/d3b/d4b/dc/cab 0 2024-04-24T23:13:49.377 INFO:tasks.workunit.client.0.smithi031.stdout:5/515: truncate d0/f89 263518 0 2024-04-24T23:13:49.377 INFO:tasks.workunit.client.1.smithi040.stdout:5/216: creat d2/d7/d8/f50 x:0 0 0 2024-04-24T23:13:49.378 INFO:tasks.workunit.client.1.smithi040.stdout:5/217: chown d2/d7/d8/f50 785 1 2024-04-24T23:13:49.379 INFO:tasks.workunit.client.0.smithi031.stdout:1/532: creat d1/da/d12/d65/fa6 x:0 0 0 2024-04-24T23:13:49.381 INFO:tasks.workunit.client.0.smithi031.stdout:1/533: chown d1/da/f2c 38 1 2024-04-24T23:13:49.382 INFO:tasks.workunit.client.0.smithi031.stdout:4/685: dwrite d3/d20/fa4 [0,4194304] 0 2024-04-24T23:13:49.385 INFO:tasks.workunit.client.1.smithi040.stdout:3/248: creat d1/f59 x:0 0 0 2024-04-24T23:13:49.387 INFO:tasks.workunit.client.1.smithi040.stdout:8/231: fdatasync d0/d16/d1b/d47/f48 0 2024-04-24T23:13:49.389 INFO:tasks.workunit.client.0.smithi031.stdout:0/619: truncate da/d18/d1a/d37/f5f 1902851 0 2024-04-24T23:13:49.390 INFO:tasks.workunit.client.0.smithi031.stdout:4/686: dwrite d3/f85 [0,4194304] 0 2024-04-24T23:13:49.390 INFO:tasks.workunit.client.1.smithi040.stdout:6/257: fdatasync d7/d1a/d20/f28 0 2024-04-24T23:13:49.391 INFO:tasks.workunit.client.0.smithi031.stdout:4/687: fsync d3/d23/d3b/fd6 0 2024-04-24T23:13:49.391 INFO:tasks.workunit.client.1.smithi040.stdout:6/258: write d7/d1a/d20/f47 [3199047,22273] 0 2024-04-24T23:13:49.393 INFO:tasks.workunit.client.1.smithi040.stdout:9/293: symlink d7/da/l69 0 2024-04-24T23:13:49.396 INFO:tasks.workunit.client.1.smithi040.stdout:1/262: mkdir da/d12/d14/d58 0 2024-04-24T23:13:49.397 INFO:tasks.workunit.client.0.smithi031.stdout:1/534: creat d1/d33/d36/fa7 x:0 0 0 2024-04-24T23:13:49.398 INFO:tasks.workunit.client.1.smithi040.stdout:1/263: chown da/d31/f52 67230090 1 2024-04-24T23:13:49.405 INFO:tasks.workunit.client.1.smithi040.stdout:6/259: dwrite d7/d1a/d20/f47 [0,4194304] 0 2024-04-24T23:13:49.406 INFO:tasks.workunit.client.0.smithi031.stdout:9/673: creat d2/d43/d9d/dc0/fcc x:0 0 0 2024-04-24T23:13:49.406 INFO:tasks.workunit.client.1.smithi040.stdout:7/301: getdents db/d12/d20/d52/d4b 0 2024-04-24T23:13:49.407 INFO:tasks.workunit.client.0.smithi031.stdout:0/620: truncate da/f19 5036707 0 2024-04-24T23:13:49.407 INFO:tasks.workunit.client.1.smithi040.stdout:2/321: link da/l13 da/d29/d55/l7d 0 2024-04-24T23:13:49.412 INFO:tasks.workunit.client.0.smithi031.stdout:4/688: mkdir d3/d23/d69/d3f/d9e/ddb 0 2024-04-24T23:13:49.415 INFO:tasks.workunit.client.1.smithi040.stdout:2/322: dwrite da/d10/f2b [0,4194304] 0 2024-04-24T23:13:49.416 INFO:tasks.workunit.client.1.smithi040.stdout:3/249: dread d1/d9/f17 [0,4194304] 0 2024-04-24T23:13:49.416 INFO:tasks.workunit.client.0.smithi031.stdout:1/535: mkdir d1/d33/d41/d9b/da8 0 2024-04-24T23:13:49.417 INFO:tasks.workunit.client.1.smithi040.stdout:3/250: read d1/d9/d25/f3a [164345,47153] 0 2024-04-24T23:13:49.418 INFO:tasks.workunit.client.1.smithi040.stdout:3/251: readlink d1/d2d/de/d30/l3d 0 2024-04-24T23:13:49.425 INFO:tasks.workunit.client.0.smithi031.stdout:1/536: dwrite d1/d3e/da3/f77 [0,4194304] 0 2024-04-24T23:13:49.431 INFO:tasks.workunit.client.0.smithi031.stdout:5/516: creat d0/d6/d3b/d4b/fac x:0 0 0 2024-04-24T23:13:49.432 INFO:tasks.workunit.client.1.smithi040.stdout:4/232: write da/d1d/d1e/f24 [1034428,62253] 0 2024-04-24T23:13:49.434 INFO:tasks.workunit.client.0.smithi031.stdout:4/689: mkdir d3/d23/d37/ddc 0 2024-04-24T23:13:49.434 INFO:tasks.workunit.client.1.smithi040.stdout:4/233: read da/d1d/f32 [396,116782] 0 2024-04-24T23:13:49.436 INFO:tasks.workunit.client.0.smithi031.stdout:8/685: creat d8/d15/d37/d62/fee x:0 0 0 2024-04-24T23:13:49.437 INFO:tasks.workunit.client.0.smithi031.stdout:8/686: chown d8/d1a/d1d/d50/fb7 322380 1 2024-04-24T23:13:49.437 INFO:tasks.workunit.client.0.smithi031.stdout:2/451: link d5/d28/f86 d5/d8/d15/d56/d84/d75/f9d 0 2024-04-24T23:13:49.439 INFO:tasks.workunit.client.0.smithi031.stdout:8/687: stat d8/d1a/d1d/d35/d58/lbc 0 2024-04-24T23:13:49.440 INFO:tasks.workunit.client.1.smithi040.stdout:2/323: dread da/f19 [0,4194304] 0 2024-04-24T23:13:49.445 INFO:tasks.workunit.client.0.smithi031.stdout:0/621: creat da/d20/d4e/dc9/fcb x:0 0 0 2024-04-24T23:13:49.446 INFO:tasks.workunit.client.1.smithi040.stdout:5/218: write d2/d7/d8/db/d1e/d22/f25 [1858486,19507] 0 2024-04-24T23:13:49.446 INFO:tasks.workunit.client.0.smithi031.stdout:8/688: dwrite d8/d1a/d1d/d50/dc0/d9d/fbe [0,4194304] 0 2024-04-24T23:13:49.448 INFO:tasks.workunit.client.0.smithi031.stdout:8/689: write d8/fb [3394283,125377] 0 2024-04-24T23:13:49.448 INFO:tasks.workunit.client.0.smithi031.stdout:5/517: symlink d0/d6/d32/lad 0 2024-04-24T23:13:49.449 INFO:tasks.workunit.client.0.smithi031.stdout:4/690: mknod d3/d23/d37/d86/cdd 0 2024-04-24T23:13:49.455 INFO:tasks.workunit.client.0.smithi031.stdout:1/537: link d1/dc/d11/d21/f28 d1/d33/d41/d9b/da8/fa9 0 2024-04-24T23:13:49.463 INFO:tasks.workunit.client.0.smithi031.stdout:0/622: truncate da/d22/f27 2717914 0 2024-04-24T23:13:49.464 INFO:tasks.workunit.client.1.smithi040.stdout:3/252: mkdir d1/d9/d18/d36/d55/d5a 0 2024-04-24T23:13:49.464 INFO:tasks.workunit.client.1.smithi040.stdout:3/253: readlink d1/d9/l11 0 2024-04-24T23:13:49.466 INFO:tasks.workunit.client.1.smithi040.stdout:9/294: creat d7/da/d17/d40/d55/d2f/d28/d67/f6a x:0 0 0 2024-04-24T23:13:49.469 INFO:tasks.workunit.client.1.smithi040.stdout:1/264: creat da/d12/d18/d2a/d4d/f59 x:0 0 0 2024-04-24T23:13:49.471 INFO:tasks.workunit.client.0.smithi031.stdout:2/452: write d5/d8/f6c [411909,7449] 0 2024-04-24T23:13:49.472 INFO:tasks.workunit.client.0.smithi031.stdout:2/453: chown d5/d28/d43/f57 373544673 1 2024-04-24T23:13:49.473 INFO:tasks.workunit.client.1.smithi040.stdout:2/324: mkdir da/d1c/d7e 0 2024-04-24T23:13:49.474 INFO:tasks.workunit.client.1.smithi040.stdout:2/325: read - da/d1c/f57 zero size 2024-04-24T23:13:49.474 INFO:tasks.workunit.client.1.smithi040.stdout:1/265: dwrite da/d12/d14/f24 [0,4194304] 0 2024-04-24T23:13:49.476 INFO:tasks.workunit.client.1.smithi040.stdout:8/232: truncate d0/d28/f39 1463316 0 2024-04-24T23:13:49.477 INFO:tasks.workunit.client.0.smithi031.stdout:0/623: symlink da/d11/d6a/da2/lcc 0 2024-04-24T23:13:49.478 INFO:tasks.workunit.client.1.smithi040.stdout:7/302: symlink db/d12/d20/d56/l67 0 2024-04-24T23:13:49.486 INFO:tasks.workunit.client.0.smithi031.stdout:9/674: dwrite d2/dae/d14/d78/d84/d4a/f8d [0,4194304] 0 2024-04-24T23:13:49.487 INFO:tasks.workunit.client.0.smithi031.stdout:4/691: dread d3/f4a [0,4194304] 0 2024-04-24T23:13:49.488 INFO:tasks.workunit.client.1.smithi040.stdout:3/254: symlink d1/d2d/d44/l5b 0 2024-04-24T23:13:49.491 INFO:tasks.workunit.client.1.smithi040.stdout:9/295: unlink d7/da/d17/l27 0 2024-04-24T23:13:49.493 INFO:tasks.workunit.client.1.smithi040.stdout:0/313: getdents df/d12/d58 0 2024-04-24T23:13:49.495 INFO:tasks.workunit.client.1.smithi040.stdout:4/234: getdents da/d45/d48 0 2024-04-24T23:13:49.496 INFO:tasks.workunit.client.1.smithi040.stdout:0/314: dread df/d1e/d20/d21/f32 [0,4194304] 0 2024-04-24T23:13:49.496 INFO:tasks.workunit.client.0.smithi031.stdout:0/624: fdatasync da/d20/f52 0 2024-04-24T23:13:49.497 INFO:tasks.workunit.client.0.smithi031.stdout:1/538: dwrite d1/dc/d11/f24 [0,4194304] 0 2024-04-24T23:13:49.497 INFO:tasks.workunit.client.0.smithi031.stdout:0/625: chown da/d18/d1a/d37/d41 1600 1 2024-04-24T23:13:49.502 INFO:tasks.workunit.client.1.smithi040.stdout:2/326: write da/d1c/f5c [785700,60721] 0 2024-04-24T23:13:49.504 INFO:tasks.workunit.client.0.smithi031.stdout:1/539: dwrite d1/f8 [4194304,4194304] 0 2024-04-24T23:13:49.504 INFO:tasks.workunit.client.0.smithi031.stdout:5/518: rmdir d0/d7/d95 0 2024-04-24T23:13:49.509 INFO:tasks.workunit.client.1.smithi040.stdout:8/233: unlink d0/d8/f42 0 2024-04-24T23:13:49.511 INFO:tasks.workunit.client.0.smithi031.stdout:4/692: rename d3/d23/d37/fa9 to d3/d23/d37/d86/fde 0 2024-04-24T23:13:49.514 INFO:tasks.workunit.client.0.smithi031.stdout:8/690: truncate d8/d15/d37/f85 2652315 0 2024-04-24T23:13:49.519 INFO:tasks.workunit.client.0.smithi031.stdout:4/693: dwrite d3/d20/d25/d60/fc0 [0,4194304] 0 2024-04-24T23:13:49.520 INFO:tasks.workunit.client.0.smithi031.stdout:4/694: chown d3/d20/d2d/d44/d5b/d5d/f87 28 1 2024-04-24T23:13:49.522 INFO:tasks.workunit.client.0.smithi031.stdout:4/695: truncate d3/d5e/fb6 989917 0 2024-04-24T23:13:49.525 INFO:tasks.workunit.client.0.smithi031.stdout:1/540: truncate d1/f34 774273 0 2024-04-24T23:13:49.527 INFO:tasks.workunit.client.0.smithi031.stdout:1/541: write d1/da/d10/f4d [1853434,72768] 0 2024-04-24T23:13:49.528 INFO:tasks.workunit.client.0.smithi031.stdout:1/542: dread - d1/d33/d36/f71 zero size 2024-04-24T23:13:49.529 INFO:tasks.workunit.client.0.smithi031.stdout:7/594: sync 2024-04-24T23:13:49.529 INFO:tasks.workunit.client.0.smithi031.stdout:3/529: sync 2024-04-24T23:13:49.529 INFO:tasks.workunit.client.0.smithi031.stdout:6/690: sync 2024-04-24T23:13:49.530 INFO:tasks.workunit.client.0.smithi031.stdout:5/519: symlink d0/d7/d56/d7c/d7e/lae 0 2024-04-24T23:13:49.532 INFO:tasks.workunit.client.0.smithi031.stdout:5/520: write d0/d6/d3b/d4b/dc/d25/f69 [109291,22366] 0 2024-04-24T23:13:49.533 INFO:tasks.workunit.client.0.smithi031.stdout:9/675: mknod d2/dae/d14/d78/d84/d7e/dc7/ccd 0 2024-04-24T23:13:49.539 INFO:tasks.workunit.client.0.smithi031.stdout:7/595: dwrite d1/d2/dd/d33/d5e/d64/d6d/dae/db1/fc6 [0,4194304] 0 2024-04-24T23:13:49.541 INFO:tasks.workunit.client.0.smithi031.stdout:8/691: rename d8/d1a/c6e to d8/d1a/d74/cef 0 2024-04-24T23:13:49.544 INFO:tasks.workunit.client.0.smithi031.stdout:1/543: symlink d1/d3e/d46/laa 0 2024-04-24T23:13:49.546 INFO:tasks.workunit.client.1.smithi040.stdout:1/266: rename da/d12/d18/d2a/d32/f48 to da/d12/d18/d2a/f5a 0 2024-04-24T23:13:49.546 INFO:tasks.workunit.client.1.smithi040.stdout:3/255: mknod d1/d2d/d2b/c5c 0 2024-04-24T23:13:49.547 INFO:tasks.workunit.client.1.smithi040.stdout:1/267: stat da/d12/f54 0 2024-04-24T23:13:49.548 INFO:tasks.workunit.client.1.smithi040.stdout:3/256: chown d1/d2d/de 990230701 1 2024-04-24T23:13:49.548 INFO:tasks.workunit.client.0.smithi031.stdout:0/626: dwrite da/d18/d1a/d37/d41/f66 [0,4194304] 0 2024-04-24T23:13:49.548 INFO:tasks.workunit.client.0.smithi031.stdout:1/544: write d1/dc/d11/d21/d5a/f76 [1536085,56910] 0 2024-04-24T23:13:49.549 INFO:tasks.workunit.client.1.smithi040.stdout:0/315: unlink df/d1e/d20/f33 0 2024-04-24T23:13:49.549 INFO:tasks.workunit.client.1.smithi040.stdout:6/260: getdents d7/d2c 0 2024-04-24T23:13:49.550 INFO:tasks.workunit.client.1.smithi040.stdout:1/268: write da/d12/d14/f24 [2344928,100644] 0 2024-04-24T23:13:49.551 INFO:tasks.workunit.client.1.smithi040.stdout:5/219: creat d2/d7/d8/d1f/f51 x:0 0 0 2024-04-24T23:13:49.552 INFO:tasks.workunit.client.1.smithi040.stdout:1/269: readlink da/l19 0 2024-04-24T23:13:49.555 INFO:tasks.workunit.client.1.smithi040.stdout:1/270: write da/d12/d23/f30 [4590525,9040] 0 2024-04-24T23:13:49.556 INFO:tasks.workunit.client.1.smithi040.stdout:4/235: rename da/f13 to da/d3f/f56 0 2024-04-24T23:13:49.558 INFO:tasks.workunit.client.1.smithi040.stdout:1/271: write da/d12/d18/d2a/f3d [400181,91215] 0 2024-04-24T23:13:49.559 INFO:tasks.workunit.client.1.smithi040.stdout:4/236: write da/d1d/d1e/f42 [2025410,31846] 0 2024-04-24T23:13:49.559 INFO:tasks.workunit.client.0.smithi031.stdout:2/454: getdents d5/d8/d45/d5b/d49/d6f 0 2024-04-24T23:13:49.560 INFO:tasks.workunit.client.1.smithi040.stdout:1/272: chown da/d12/f1c 67772222 1 2024-04-24T23:13:49.562 INFO:tasks.workunit.client.1.smithi040.stdout:3/257: dread d1/d2d/de/f34 [0,4194304] 0 2024-04-24T23:13:49.563 INFO:tasks.workunit.client.0.smithi031.stdout:7/596: unlink d1/d2/dd/d33/l7a 0 2024-04-24T23:13:49.564 INFO:tasks.workunit.client.0.smithi031.stdout:7/597: chown d1/d2/d60/l8c 154374279 1 2024-04-24T23:13:49.564 INFO:tasks.workunit.client.0.smithi031.stdout:6/691: dread d13/d1b/f22 [0,4194304] 0 2024-04-24T23:13:49.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/696: rename d3/d20/d25/d60/lc1 to d3/d20/d2d/d44/ldf 0 2024-04-24T23:13:49.575 INFO:tasks.workunit.client.1.smithi040.stdout:0/316: dread df/d12/d1a/f2e [0,4194304] 0 2024-04-24T23:13:49.575 INFO:tasks.workunit.client.0.smithi031.stdout:0/627: mkdir da/d86/da3/dcd 0 2024-04-24T23:13:49.576 INFO:tasks.workunit.client.0.smithi031.stdout:0/628: chown da/c6d 3155928 1 2024-04-24T23:13:49.578 INFO:tasks.workunit.client.0.smithi031.stdout:1/545: fdatasync d1/d3e/d46/f5c 0 2024-04-24T23:13:49.578 INFO:tasks.workunit.client.0.smithi031.stdout:1/546: chown d1/l75 63 1 2024-04-24T23:13:49.579 INFO:tasks.workunit.client.0.smithi031.stdout:1/547: read - d1/da/d12/d65/fa0 zero size 2024-04-24T23:13:49.582 INFO:tasks.workunit.client.0.smithi031.stdout:9/676: mknod d2/cce 0 2024-04-24T23:13:49.585 INFO:tasks.workunit.client.0.smithi031.stdout:2/455: symlink d5/d8/d45/d5b/d49/l9e 0 2024-04-24T23:13:49.586 INFO:tasks.workunit.client.1.smithi040.stdout:2/327: truncate da/d10/f34 4129802 0 2024-04-24T23:13:49.588 INFO:tasks.workunit.client.0.smithi031.stdout:3/530: write d6/d14/d1d/f1e [2560735,8471] 0 2024-04-24T23:13:49.590 INFO:tasks.workunit.client.1.smithi040.stdout:5/220: mkdir d2/d7/d8/db/d1e/d22/d27/d3c/d52 0 2024-04-24T23:13:49.591 INFO:tasks.workunit.client.1.smithi040.stdout:5/221: fsync d2/d7/d8/db/d1e/d22/f44 0 2024-04-24T23:13:49.592 INFO:tasks.workunit.client.1.smithi040.stdout:8/234: rename d0/d16/c21 to d0/d16/d1b/c4b 0 2024-04-24T23:13:49.593 INFO:tasks.workunit.client.1.smithi040.stdout:1/273: rename da to da/d12/d18/d2a/d41/d5b 22 2024-04-24T23:13:49.594 INFO:tasks.workunit.client.0.smithi031.stdout:5/521: rename d0/d6/d3b/d4b/dc/d8b/ca5 to d0/d6/d32/d6b/caf 0 2024-04-24T23:13:49.597 INFO:tasks.workunit.client.1.smithi040.stdout:5/222: dwrite d2/d7/d8/db/d1e/d22/d27/d3c/f47 [0,4194304] 0 2024-04-24T23:13:49.599 INFO:tasks.workunit.client.0.smithi031.stdout:4/697: mknod d3/d23/d69/d3f/d9e/ce0 0 2024-04-24T23:13:49.601 INFO:tasks.workunit.client.1.smithi040.stdout:1/274: dwrite da/d12/f54 [0,4194304] 0 2024-04-24T23:13:49.603 INFO:tasks.workunit.client.0.smithi031.stdout:4/698: dread d3/f4a [0,4194304] 0 2024-04-24T23:13:49.604 INFO:tasks.workunit.client.0.smithi031.stdout:4/699: write d3/d20/d2d/d44/fd3 [508019,15842] 0 2024-04-24T23:13:49.605 INFO:tasks.workunit.client.0.smithi031.stdout:0/629: mknod da/d20/d8e/cce 0 2024-04-24T23:13:49.606 INFO:tasks.workunit.client.0.smithi031.stdout:4/700: dread - d3/d23/d37/fcb zero size 2024-04-24T23:13:49.607 INFO:tasks.workunit.client.0.smithi031.stdout:0/630: write da/d20/f44 [1626343,90868] 0 2024-04-24T23:13:49.608 INFO:tasks.workunit.client.1.smithi040.stdout:5/223: dread d2/d7/d8/f34 [4194304,4194304] 0 2024-04-24T23:13:49.609 INFO:tasks.workunit.client.1.smithi040.stdout:6/261: dwrite d7/d1a/d20/d51/d2f/f35 [0,4194304] 0 2024-04-24T23:13:49.610 INFO:tasks.workunit.client.1.smithi040.stdout:3/258: creat d1/d9/d18/d36/d55/f5d x:0 0 0 2024-04-24T23:13:49.611 INFO:tasks.workunit.client.0.smithi031.stdout:1/548: write d1/d31/f39 [803273,40287] 0 2024-04-24T23:13:49.612 INFO:tasks.workunit.client.1.smithi040.stdout:5/224: truncate d2/d7/f36 706976 0 2024-04-24T23:13:49.612 INFO:tasks.workunit.client.1.smithi040.stdout:6/262: read - d7/d1a/d20/d51/f4b zero size 2024-04-24T23:13:49.614 INFO:tasks.workunit.client.1.smithi040.stdout:0/317: write df/d1e/f2f [9267584,63497] 0 2024-04-24T23:13:49.614 INFO:tasks.workunit.client.1.smithi040.stdout:0/318: fsync df/d1e/f3e 0 2024-04-24T23:13:49.616 INFO:tasks.workunit.client.0.smithi031.stdout:2/456: creat d5/d8/d45/d5b/d49/f9f x:0 0 0 2024-04-24T23:13:49.618 INFO:tasks.workunit.client.1.smithi040.stdout:2/328: creat da/d29/f7f x:0 0 0 2024-04-24T23:13:49.619 INFO:tasks.workunit.client.1.smithi040.stdout:2/329: write da/d29/f4a [138654,90268] 0 2024-04-24T23:13:49.620 INFO:tasks.workunit.client.0.smithi031.stdout:7/598: mknod d1/d2/ccc 0 2024-04-24T23:13:49.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/531: read d6/df/d16/d53/f6e [3530122,46799] 0 2024-04-24T23:13:49.622 INFO:tasks.workunit.client.0.smithi031.stdout:6/692: mkdir d13/d15/dea 0 2024-04-24T23:13:49.622 INFO:tasks.workunit.client.1.smithi040.stdout:8/235: mkdir d0/d38/d4c 0 2024-04-24T23:13:49.623 INFO:tasks.workunit.client.0.smithi031.stdout:5/522: symlink d0/d6/d3b/d4b/dc/d8b/lb0 0 2024-04-24T23:13:49.624 INFO:tasks.workunit.client.1.smithi040.stdout:7/303: sync 2024-04-24T23:13:49.625 INFO:tasks.workunit.client.0.smithi031.stdout:6/693: write d13/d15/d1d/d23/d24/d43/d68/db8/fd1 [614846,11099] 0 2024-04-24T23:13:49.626 INFO:tasks.workunit.client.0.smithi031.stdout:8/692: creat d8/d1a/ff0 x:0 0 0 2024-04-24T23:13:49.628 INFO:tasks.workunit.client.1.smithi040.stdout:9/296: getdents d7/da/d17/d40/d55 0 2024-04-24T23:13:49.631 INFO:tasks.workunit.client.0.smithi031.stdout:0/631: creat da/d20/d4d/fcf x:0 0 0 2024-04-24T23:13:49.631 INFO:tasks.workunit.client.1.smithi040.stdout:4/237: symlink da/d1d/l57 0 2024-04-24T23:13:49.633 INFO:tasks.workunit.client.0.smithi031.stdout:6/694: dwrite d13/d15/d1d/d23/d44/f4e [0,4194304] 0 2024-04-24T23:13:49.634 INFO:tasks.workunit.client.1.smithi040.stdout:3/259: creat d1/d9/d18/f5e x:0 0 0 2024-04-24T23:13:49.634 INFO:tasks.workunit.client.1.smithi040.stdout:3/260: fsync d1/d9/d15/f1b 0 2024-04-24T23:13:49.638 INFO:tasks.workunit.client.1.smithi040.stdout:6/263: creat d7/d1a/f53 x:0 0 0 2024-04-24T23:13:49.640 INFO:tasks.workunit.client.0.smithi031.stdout:5/523: unlink d0/d6/d3b/f87 0 2024-04-24T23:13:49.642 INFO:tasks.workunit.client.1.smithi040.stdout:5/225: mknod d2/d7/d8/db/d1e/d22/d3e/c53 0 2024-04-24T23:13:49.644 INFO:tasks.workunit.client.1.smithi040.stdout:6/264: dread d7/d1a/d20/d51/d2f/f35 [0,4194304] 0 2024-04-24T23:13:49.644 INFO:tasks.workunit.client.1.smithi040.stdout:0/319: write df/d12/d1a/f4e [1768710,40048] 0 2024-04-24T23:13:49.645 INFO:tasks.workunit.client.0.smithi031.stdout:9/677: write d2/fc [1449572,72778] 0 2024-04-24T23:13:49.649 INFO:tasks.workunit.client.0.smithi031.stdout:4/701: link d3/d20/fa4 d3/d23/d69/d3f/fe1 0 2024-04-24T23:13:49.650 INFO:tasks.workunit.client.0.smithi031.stdout:3/532: dread f0 [0,4194304] 0 2024-04-24T23:13:49.650 INFO:tasks.workunit.client.0.smithi031.stdout:9/678: chown d2/dae/d14/d1d/d45/d65/l80 4986390 1 2024-04-24T23:13:49.652 INFO:tasks.workunit.client.0.smithi031.stdout:0/632: unlink da/d18/d5a/d7d/l87 0 2024-04-24T23:13:49.653 INFO:tasks.workunit.client.0.smithi031.stdout:9/679: stat d2/dae/d14/d78/d84/d7e/dc7 0 2024-04-24T23:13:49.653 INFO:tasks.workunit.client.0.smithi031.stdout:5/524: dwrite d0/d6/d3b/d4b/dc/fa2 [0,4194304] 0 2024-04-24T23:13:49.655 INFO:tasks.workunit.client.0.smithi031.stdout:2/457: dread d5/d28/d43/f67 [0,4194304] 0 2024-04-24T23:13:49.657 INFO:tasks.workunit.client.1.smithi040.stdout:8/236: rmdir d0/d16/d19 39 2024-04-24T23:13:49.659 INFO:tasks.workunit.client.0.smithi031.stdout:9/680: write d2/dae/d14/d78/d84/da6/fa7 [753496,65273] 0 2024-04-24T23:13:49.659 INFO:tasks.workunit.client.0.smithi031.stdout:2/458: stat d5/d8/d45/d5b 0 2024-04-24T23:13:49.660 INFO:tasks.workunit.client.0.smithi031.stdout:6/695: creat d13/d15/d1d/d23/d44/d47/d69/dd5/feb x:0 0 0 2024-04-24T23:13:49.666 INFO:tasks.workunit.client.0.smithi031.stdout:6/696: write d13/d15/d1d/d23/d24/d43/d68/fb5 [486517,84736] 0 2024-04-24T23:13:49.667 INFO:tasks.workunit.client.0.smithi031.stdout:0/633: dwrite da/d20/f44 [0,4194304] 0 2024-04-24T23:13:49.668 INFO:tasks.workunit.client.1.smithi040.stdout:4/238: mkdir da/d45/d58 0 2024-04-24T23:13:49.668 INFO:tasks.workunit.client.1.smithi040.stdout:4/239: stat f5 0 2024-04-24T23:13:49.669 INFO:tasks.workunit.client.0.smithi031.stdout:0/634: readlink da/l21 0 2024-04-24T23:13:49.669 INFO:tasks.workunit.client.1.smithi040.stdout:4/240: truncate da/d1d/d3c/f3e 424286 0 2024-04-24T23:13:49.671 INFO:tasks.workunit.client.0.smithi031.stdout:8/693: dread f3 [0,4194304] 0 2024-04-24T23:13:49.671 INFO:tasks.workunit.client.0.smithi031.stdout:9/681: dwrite d2/f32 [0,4194304] 0 2024-04-24T23:13:49.673 INFO:tasks.workunit.client.1.smithi040.stdout:1/275: rename da/d12/d23/c26 to da/d12/d18/d2a/d32/c5c 0 2024-04-24T23:13:49.673 INFO:tasks.workunit.client.0.smithi031.stdout:9/682: write d2/d43/f5d [611000,67271] 0 2024-04-24T23:13:49.678 INFO:tasks.workunit.client.0.smithi031.stdout:4/702: creat d3/d23/d3b/fe2 x:0 0 0 2024-04-24T23:13:49.679 INFO:tasks.workunit.client.0.smithi031.stdout:4/703: dread - d3/d20/d2d/fbe zero size 2024-04-24T23:13:49.684 INFO:tasks.workunit.client.1.smithi040.stdout:5/226: creat d2/d7/d8/db/d1e/d22/d27/f54 x:0 0 0 2024-04-24T23:13:49.686 INFO:tasks.workunit.client.1.smithi040.stdout:5/227: write d2/d7/d8/d1f/f51 [635779,67620] 0 2024-04-24T23:13:49.687 INFO:tasks.workunit.client.0.smithi031.stdout:2/459: symlink d5/d28/d4e/d5e/la0 0 2024-04-24T23:13:49.687 INFO:tasks.workunit.client.1.smithi040.stdout:5/228: chown d2/d7/d8/db/d1e/f2e 752329 1 2024-04-24T23:13:49.688 INFO:tasks.workunit.client.0.smithi031.stdout:2/460: fsync d5/d6/f35 0 2024-04-24T23:13:49.688 INFO:tasks.workunit.client.1.smithi040.stdout:0/320: stat c3 0 2024-04-24T23:13:49.689 INFO:tasks.workunit.client.1.smithi040.stdout:0/321: dread - df/d12/d1a/f55 zero size 2024-04-24T23:13:49.690 INFO:tasks.workunit.client.1.smithi040.stdout:2/330: getdents da/d1c/d37/d39/d79 0 2024-04-24T23:13:49.691 INFO:tasks.workunit.client.0.smithi031.stdout:0/635: creat da/d11/d6a/fd0 x:0 0 0 2024-04-24T23:13:49.694 INFO:tasks.workunit.client.1.smithi040.stdout:9/297: dread d7/f3a [0,4194304] 0 2024-04-24T23:13:49.695 INFO:tasks.workunit.client.0.smithi031.stdout:7/599: dwrite d1/d2/dd/d33/d5e/d64/d6d/fac [0,4194304] 0 2024-04-24T23:13:49.697 INFO:tasks.workunit.client.1.smithi040.stdout:0/322: dwrite df/d12/f2a [0,4194304] 0 2024-04-24T23:13:49.697 INFO:tasks.workunit.client.1.smithi040.stdout:1/276: creat da/d12/d18/d2a/f5d x:0 0 0 2024-04-24T23:13:49.702 INFO:tasks.workunit.client.0.smithi031.stdout:4/704: dread - d3/d20/d2d/f79 zero size 2024-04-24T23:13:49.702 INFO:tasks.workunit.client.1.smithi040.stdout:3/261: creat d1/d9/d2e/f5f x:0 0 0 2024-04-24T23:13:49.704 INFO:tasks.workunit.client.1.smithi040.stdout:3/262: truncate d1/d9/d15/d45/f4b 949778 0 2024-04-24T23:13:49.708 INFO:tasks.workunit.client.1.smithi040.stdout:5/229: creat d2/d7/d8/db/d1e/f55 x:0 0 0 2024-04-24T23:13:49.709 INFO:tasks.workunit.client.0.smithi031.stdout:2/461: creat d5/d28/d4e/fa1 x:0 0 0 2024-04-24T23:13:49.711 INFO:tasks.workunit.client.1.smithi040.stdout:3/263: dwrite d1/d9/d15/f53 [0,4194304] 0 2024-04-24T23:13:49.713 INFO:tasks.workunit.client.1.smithi040.stdout:2/331: creat da/d1c/d4f/f80 x:0 0 0 2024-04-24T23:13:49.713 INFO:tasks.workunit.client.0.smithi031.stdout:6/697: write d13/d15/d1d/d23/d24/d8b/fcd [313097,40321] 0 2024-04-24T23:13:49.717 INFO:tasks.workunit.client.1.smithi040.stdout:7/304: link db/d12/d20/d52/d4b/f4f db/d12/d20/d52/f68 0 2024-04-24T23:13:49.719 INFO:tasks.workunit.client.0.smithi031.stdout:8/694: mkdir d8/d15/d37/d43/d7b/dbf/dcc/df1 0 2024-04-24T23:13:49.721 INFO:tasks.workunit.client.0.smithi031.stdout:9/683: mknod d2/dae/ccf 0 2024-04-24T23:13:49.724 INFO:tasks.workunit.client.0.smithi031.stdout:3/533: dwrite d6/df/d16/d2a/d95/d9b/d1a/f62 [0,4194304] 0 2024-04-24T23:13:49.724 INFO:tasks.workunit.client.1.smithi040.stdout:9/298: dread f4 [0,4194304] 0 2024-04-24T23:13:49.725 INFO:tasks.workunit.client.0.smithi031.stdout:4/705: unlink d3/d23/d3b/f43 0 2024-04-24T23:13:49.726 INFO:tasks.workunit.client.0.smithi031.stdout:3/534: fsync d6/df/d16/d2a/d95/d9b/f7c 0 2024-04-24T23:13:49.727 INFO:tasks.workunit.client.0.smithi031.stdout:3/535: truncate d6/d14/d91/f99 469777 0 2024-04-24T23:13:49.728 INFO:tasks.workunit.client.0.smithi031.stdout:5/525: creat d0/d6/d3b/d4b/fb1 x:0 0 0 2024-04-24T23:13:49.729 INFO:tasks.workunit.client.1.smithi040.stdout:6/265: creat d7/f54 x:0 0 0 2024-04-24T23:13:49.729 INFO:tasks.workunit.client.0.smithi031.stdout:5/526: chown d0/d7/d56/d7c/d7e/l66 36668 1 2024-04-24T23:13:49.730 INFO:tasks.workunit.client.0.smithi031.stdout:5/527: readlink d0/d6/d3b/d4f/l8e 0 2024-04-24T23:13:49.733 INFO:tasks.workunit.client.1.smithi040.stdout:3/264: creat d1/d2d/f60 x:0 0 0 2024-04-24T23:13:49.733 INFO:tasks.workunit.client.1.smithi040.stdout:9/299: dwrite d7/da/d17/d40/d55/d2f/d1f/f5a [0,4194304] 0 2024-04-24T23:13:49.733 INFO:tasks.workunit.client.1.smithi040.stdout:6/266: truncate d7/d1a/d20/f23 1593900 0 2024-04-24T23:13:49.737 INFO:tasks.workunit.client.1.smithi040.stdout:8/237: creat d0/d8/f4d x:0 0 0 2024-04-24T23:13:49.739 INFO:tasks.workunit.client.1.smithi040.stdout:7/305: dread db/d12/d20/d52/f23 [0,4194304] 0 2024-04-24T23:13:49.742 INFO:tasks.workunit.client.1.smithi040.stdout:4/241: creat da/f59 x:0 0 0 2024-04-24T23:13:49.746 INFO:tasks.workunit.client.1.smithi040.stdout:4/242: read da/f1c [4623386,124732] 0 2024-04-24T23:13:49.746 INFO:tasks.workunit.client.1.smithi040.stdout:5/230: fdatasync d2/f6 0 2024-04-24T23:13:49.748 INFO:tasks.workunit.client.1.smithi040.stdout:4/243: write da/f2f [406162,48349] 0 2024-04-24T23:13:49.753 INFO:tasks.workunit.client.0.smithi031.stdout:0/636: creat da/d18/d5a/d7d/dbd/fd1 x:0 0 0 2024-04-24T23:13:49.753 INFO:tasks.workunit.client.0.smithi031.stdout:8/695: rename l4 to d8/d1a/d1d/d35/d58/d5b/lf2 0 2024-04-24T23:13:49.754 INFO:tasks.workunit.client.0.smithi031.stdout:4/706: creat d3/d20/d25/d60/d63/fe3 x:0 0 0 2024-04-24T23:13:49.754 INFO:tasks.workunit.client.0.smithi031.stdout:3/536: creat d6/d14/d1d/d69/fa7 x:0 0 0 2024-04-24T23:13:49.755 INFO:tasks.workunit.client.1.smithi040.stdout:3/265: rename d1/d9/d15/d45/d4e to d1/d9/d18/d36/d61 0 2024-04-24T23:13:49.755 INFO:tasks.workunit.client.0.smithi031.stdout:2/462: mkdir d5/d99/da2 0 2024-04-24T23:13:49.755 INFO:tasks.workunit.client.0.smithi031.stdout:0/637: dread - da/d11/d6a/dba/fca zero size 2024-04-24T23:13:49.756 INFO:tasks.workunit.client.0.smithi031.stdout:7/600: write d1/d2/f2d [4078246,13785] 0 2024-04-24T23:13:49.757 INFO:tasks.workunit.client.0.smithi031.stdout:4/707: stat d3/d20/d2d/d44/f4c 0 2024-04-24T23:13:49.758 INFO:tasks.workunit.client.0.smithi031.stdout:6/698: link f11 d13/d15/d1d/d23/d2d/dbf/fec 0 2024-04-24T23:13:49.758 INFO:tasks.workunit.client.0.smithi031.stdout:2/463: write d5/d6/f35 [2852680,111513] 0 2024-04-24T23:13:49.760 INFO:tasks.workunit.client.1.smithi040.stdout:8/238: unlink d0/d8/d22/l49 0 2024-04-24T23:13:49.761 INFO:tasks.workunit.client.0.smithi031.stdout:5/528: dread d0/d6/d3b/d4b/f72 [0,4194304] 0 2024-04-24T23:13:49.762 INFO:tasks.workunit.client.0.smithi031.stdout:2/464: read d5/d28/d43/f67 [3226731,104300] 0 2024-04-24T23:13:49.764 INFO:tasks.workunit.client.1.smithi040.stdout:8/239: dread d0/d1f/f2d [0,4194304] 0 2024-04-24T23:13:49.764 INFO:tasks.workunit.client.1.smithi040.stdout:2/332: write f7 [2239460,37792] 0 2024-04-24T23:13:49.764 INFO:tasks.workunit.client.1.smithi040.stdout:7/306: symlink db/d12/d20/d52/d4b/l69 0 2024-04-24T23:13:49.765 INFO:tasks.workunit.client.0.smithi031.stdout:8/696: dwrite d8/d1a/d1d/d35/fcd [0,4194304] 0 2024-04-24T23:13:49.767 INFO:tasks.workunit.client.0.smithi031.stdout:8/697: write d8/d15/d9a/fb1 [472068,59158] 0 2024-04-24T23:13:49.767 INFO:tasks.workunit.client.0.smithi031.stdout:3/537: rename d6/df/f82 to d6/df/d16/d53/fa8 0 2024-04-24T23:13:49.769 INFO:tasks.workunit.client.0.smithi031.stdout:3/538: readlink d6/l10 0 2024-04-24T23:13:49.770 INFO:tasks.workunit.client.0.smithi031.stdout:4/708: rmdir d3/d20/d2d/d44/d5b 39 2024-04-24T23:13:49.771 INFO:tasks.workunit.client.0.smithi031.stdout:4/709: chown d3/c1f 1224 1 2024-04-24T23:13:49.775 INFO:tasks.workunit.client.0.smithi031.stdout:8/698: dread d8/d15/d37/d43/f4b [0,4194304] 0 2024-04-24T23:13:49.775 INFO:tasks.workunit.client.1.smithi040.stdout:0/323: creat df/d1e/f60 x:0 0 0 2024-04-24T23:13:49.775 INFO:tasks.workunit.client.1.smithi040.stdout:1/277: write f2 [1713254,32204] 0 2024-04-24T23:13:49.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/465: mknod d5/d3d/ca3 0 2024-04-24T23:13:49.778 INFO:tasks.workunit.client.0.smithi031.stdout:2/466: fdatasync d5/d28/d4e/f82 0 2024-04-24T23:13:49.780 INFO:tasks.workunit.client.1.smithi040.stdout:5/231: unlink d2/d2f/d14/c1c 0 2024-04-24T23:13:49.781 INFO:tasks.workunit.client.1.smithi040.stdout:5/232: fsync d2/d7/d8/db/d1e/d22/f25 0 2024-04-24T23:13:49.782 INFO:tasks.workunit.client.1.smithi040.stdout:4/244: rename da/f19 to da/d3f/f5a 0 2024-04-24T23:13:49.782 INFO:tasks.workunit.client.1.smithi040.stdout:4/245: readlink l1 0 2024-04-24T23:13:49.783 INFO:tasks.workunit.client.1.smithi040.stdout:3/266: fdatasync d1/d9/d18/f19 0 2024-04-24T23:13:49.783 INFO:tasks.workunit.client.1.smithi040.stdout:2/333: dread da/d29/f4a [0,4194304] 0 2024-04-24T23:13:49.783 INFO:tasks.workunit.client.1.smithi040.stdout:5/233: chown d2/d7/d8/db/d1e/d22/d27/d3d 13 1 2024-04-24T23:13:49.784 INFO:tasks.workunit.client.0.smithi031.stdout:3/539: rename d6/df/d16/d2a/l9e to d6/df/d16/d2a/la9 0 2024-04-24T23:13:49.786 INFO:tasks.workunit.client.1.smithi040.stdout:5/234: write d2/d7/d8/db/d1e/d22/d27/f54 [247703,122474] 0 2024-04-24T23:13:49.786 INFO:tasks.workunit.client.0.smithi031.stdout:3/540: write d6/df/d16/d2a/d95/d9b/d1a/f97 [403112,83987] 0 2024-04-24T23:13:49.786 INFO:tasks.workunit.client.0.smithi031.stdout:4/710: chown d3/d20/d2d/d44/d5b/lc6 1796565 1 2024-04-24T23:13:49.787 INFO:tasks.workunit.client.0.smithi031.stdout:3/541: chown d6/d14/d1d/f24 8 1 2024-04-24T23:13:49.788 INFO:tasks.workunit.client.0.smithi031.stdout:0/638: creat da/fd2 x:0 0 0 2024-04-24T23:13:49.789 INFO:tasks.workunit.client.1.smithi040.stdout:8/240: creat d0/d1f/f4e x:0 0 0 2024-04-24T23:13:49.790 INFO:tasks.workunit.client.0.smithi031.stdout:0/639: write da/d18/d5a/d7d/dbd/fd1 [613718,88975] 0 2024-04-24T23:13:49.791 INFO:tasks.workunit.client.0.smithi031.stdout:0/640: write da/d18/f9f [1361211,123605] 0 2024-04-24T23:13:49.792 INFO:tasks.workunit.client.0.smithi031.stdout:2/467: rename d5/d8/l83 to d5/d6/d29/d4f/la4 0 2024-04-24T23:13:49.794 INFO:tasks.workunit.client.0.smithi031.stdout:2/468: chown d5/d8/d45/d5b/d49/c2c 28885 1 2024-04-24T23:13:49.795 INFO:tasks.workunit.client.0.smithi031.stdout:2/469: read d5/f46 [2331533,94206] 0 2024-04-24T23:13:49.796 INFO:tasks.workunit.client.0.smithi031.stdout:2/470: write d5/d3d/f62 [1285785,112962] 0 2024-04-24T23:13:49.797 INFO:tasks.workunit.client.0.smithi031.stdout:2/471: stat d5/d8/f98 0 2024-04-24T23:13:49.797 INFO:tasks.workunit.client.1.smithi040.stdout:7/307: mknod db/d12/d20/c6a 0 2024-04-24T23:13:49.803 INFO:tasks.workunit.client.0.smithi031.stdout:3/542: mknod d6/df/d16/d2a/caa 0 2024-04-24T23:13:49.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/278: mkdir da/d12/d14/d25/d5e 0 2024-04-24T23:13:49.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/279: fdatasync f2 0 2024-04-24T23:13:49.807 INFO:tasks.workunit.client.0.smithi031.stdout:0/641: creat da/d11/fd3 x:0 0 0 2024-04-24T23:13:49.810 INFO:tasks.workunit.client.1.smithi040.stdout:9/300: truncate d7/da/d17/d40/d55/d2f/d1f/d34/f49 479882 0 2024-04-24T23:13:49.811 INFO:tasks.workunit.client.0.smithi031.stdout:5/529: creat d0/d6/d3b/d4b/d1d/fb2 x:0 0 0 2024-04-24T23:13:49.820 INFO:tasks.workunit.client.0.smithi031.stdout:8/699: dwrite d8/d15/d37/f85 [0,4194304] 0 2024-04-24T23:13:49.822 INFO:tasks.workunit.client.1.smithi040.stdout:0/324: dwrite df/d12/f2a [4194304,4194304] 0 2024-04-24T23:13:49.822 INFO:tasks.workunit.client.0.smithi031.stdout:8/700: dread - d8/d15/d37/d43/daf/fed zero size 2024-04-24T23:13:49.822 INFO:tasks.workunit.client.1.smithi040.stdout:0/325: dread - df/d1e/d20/d21/d51/f57 zero size 2024-04-24T23:13:49.823 INFO:tasks.workunit.client.1.smithi040.stdout:5/235: creat d2/d7/d8/db/f56 x:0 0 0 2024-04-24T23:13:49.824 INFO:tasks.workunit.client.0.smithi031.stdout:0/642: creat da/d20/d8e/fd4 x:0 0 0 2024-04-24T23:13:49.835 INFO:tasks.workunit.client.0.smithi031.stdout:4/711: rename d3/d20/d2d/d44/f4c to d3/d20/d2d/d44/fe4 0 2024-04-24T23:13:49.836 INFO:tasks.workunit.client.1.smithi040.stdout:1/280: mknod da/d35/d38/c5f 0 2024-04-24T23:13:49.837 INFO:tasks.workunit.client.0.smithi031.stdout:2/472: dwrite d5/f58 [0,4194304] 0 2024-04-24T23:13:49.838 INFO:tasks.workunit.client.1.smithi040.stdout:7/308: dwrite db/d12/f29 [0,4194304] 0 2024-04-24T23:13:49.838 INFO:tasks.workunit.client.0.smithi031.stdout:2/473: read - d5/d28/d4e/f82 zero size 2024-04-24T23:13:49.839 INFO:tasks.workunit.client.0.smithi031.stdout:3/543: rename d6/d14/c4a to d6/df/d16/d2a/d95/cab 0 2024-04-24T23:13:49.839 INFO:tasks.workunit.client.0.smithi031.stdout:3/544: chown d6/fa1 109926593 1 2024-04-24T23:13:49.841 INFO:tasks.workunit.client.1.smithi040.stdout:6/267: getdents d7/d1a/d20/d51/d1e 0 2024-04-24T23:13:49.848 INFO:tasks.workunit.client.0.smithi031.stdout:5/530: mknod d0/d6/d3b/d4b/d1d/d2e/cb3 0 2024-04-24T23:13:49.850 INFO:tasks.workunit.client.1.smithi040.stdout:4/246: getdents da/d45/d58 0 2024-04-24T23:13:49.856 INFO:tasks.workunit.client.0.smithi031.stdout:2/474: creat d5/d8/d15/d56/d84/fa5 x:0 0 0 2024-04-24T23:13:49.858 INFO:tasks.workunit.client.1.smithi040.stdout:0/326: read df/d1e/d20/d21/f32 [7473,121922] 0 2024-04-24T23:13:49.859 INFO:tasks.workunit.client.0.smithi031.stdout:8/701: mknod d8/d1a/d1d/cf3 0 2024-04-24T23:13:49.859 INFO:tasks.workunit.client.1.smithi040.stdout:0/327: fdatasync df/d1e/d20/d21/d51/f57 0 2024-04-24T23:13:49.860 INFO:tasks.workunit.client.0.smithi031.stdout:0/643: mkdir da/d86/dbf/dd5 0 2024-04-24T23:13:49.862 INFO:tasks.workunit.client.0.smithi031.stdout:0/644: readlink da/d22/la6 0 2024-04-24T23:13:49.864 INFO:tasks.workunit.client.0.smithi031.stdout:0/645: chown da/d18/d1a/d37/f26 0 1 2024-04-24T23:13:49.864 INFO:tasks.workunit.client.1.smithi040.stdout:7/309: symlink db/d43/l6b 0 2024-04-24T23:13:49.865 INFO:tasks.workunit.client.0.smithi031.stdout:3/545: dwrite d6/df/d16/fa6 [0,4194304] 0 2024-04-24T23:13:49.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/475: rename d5/d28/d4e/d7f/c85 to d5/d8/d15/d56/ca6 0 2024-04-24T23:13:49.872 INFO:tasks.workunit.client.0.smithi031.stdout:8/702: mkdir d8/d15/d37/d43/d5c/df4 0 2024-04-24T23:13:49.874 INFO:tasks.workunit.client.1.smithi040.stdout:6/268: dread d7/d1a/d20/d51/d1e/f3d [0,4194304] 0 2024-04-24T23:13:49.874 INFO:tasks.workunit.client.0.smithi031.stdout:2/476: dwrite d5/d8/d45/d5b/d49/f7e [0,4194304] 0 2024-04-24T23:13:49.878 INFO:tasks.workunit.client.0.smithi031.stdout:5/531: write d0/d6/d32/f4 [1750488,79326] 0 2024-04-24T23:13:49.880 INFO:tasks.workunit.client.0.smithi031.stdout:5/532: write d0/d6/f1a [5169188,89489] 0 2024-04-24T23:13:49.880 INFO:tasks.workunit.client.0.smithi031.stdout:4/712: dwrite d3/f71 [0,4194304] 0 2024-04-24T23:13:49.881 INFO:tasks.workunit.client.1.smithi040.stdout:2/334: dwrite da/d1c/d37/d39/f76 [0,4194304] 0 2024-04-24T23:13:49.881 INFO:tasks.workunit.client.1.smithi040.stdout:0/328: readlink l1 0 2024-04-24T23:13:49.882 INFO:tasks.workunit.client.0.smithi031.stdout:5/533: fdatasync d0/d6/d3b/d4b/f7a 0 2024-04-24T23:13:49.884 INFO:tasks.workunit.client.1.smithi040.stdout:8/241: dwrite d0/d16/d19/f35 [0,4194304] 0 2024-04-24T23:13:49.884 INFO:tasks.workunit.client.0.smithi031.stdout:5/534: write d0/d6/d3b/d4f/f98 [1917347,97506] 0 2024-04-24T23:13:49.886 INFO:tasks.workunit.client.0.smithi031.stdout:5/535: stat d0/d6/d3b/d4b/dc/d86/d91/l9c 0 2024-04-24T23:13:49.888 INFO:tasks.workunit.client.1.smithi040.stdout:1/281: mknod da/c60 0 2024-04-24T23:13:49.889 INFO:tasks.workunit.client.0.smithi031.stdout:2/477: mkdir d5/d28/d4e/d7f/da7 0 2024-04-24T23:13:49.889 INFO:tasks.workunit.client.1.smithi040.stdout:1/282: write da/f1f [2201017,81268] 0 2024-04-24T23:13:49.893 INFO:tasks.workunit.client.1.smithi040.stdout:3/267: getdents d1/d9/d18/d36 0 2024-04-24T23:13:49.895 INFO:tasks.workunit.client.0.smithi031.stdout:0/646: truncate da/d18/d5a/d7d/dbd/fd1 287099 0 2024-04-24T23:13:49.896 INFO:tasks.workunit.client.0.smithi031.stdout:0/647: stat da/d18/d1a/c34 0 2024-04-24T23:13:49.898 INFO:tasks.workunit.client.0.smithi031.stdout:0/648: chown da/d11/d6a/da2 1024 1 2024-04-24T23:13:49.903 INFO:tasks.workunit.client.0.smithi031.stdout:5/536: mknod d0/d6/d3b/cb4 0 2024-04-24T23:13:49.904 INFO:tasks.workunit.client.0.smithi031.stdout:5/537: chown d0/d6/d3b/d4b/f7a 10 1 2024-04-24T23:13:49.904 INFO:tasks.workunit.client.0.smithi031.stdout:2/478: creat d5/d6/d29/d26/fa8 x:0 0 0 2024-04-24T23:13:49.905 INFO:tasks.workunit.client.0.smithi031.stdout:5/538: fsync d0/d6/d3b/d4b/d1d/d2e/f54 0 2024-04-24T23:13:49.906 INFO:tasks.workunit.client.1.smithi040.stdout:2/335: dread da/f40 [0,4194304] 0 2024-04-24T23:13:49.907 INFO:tasks.workunit.client.1.smithi040.stdout:2/336: stat da/d1c/d37/d39/f4e 0 2024-04-24T23:13:49.907 INFO:tasks.workunit.client.1.smithi040.stdout:1/283: mkdir da/d35/d61 0 2024-04-24T23:13:49.908 INFO:tasks.workunit.client.0.smithi031.stdout:4/713: symlink d3/d23/d69/d3f/d9e/ddb/le5 0 2024-04-24T23:13:49.908 INFO:tasks.workunit.client.1.smithi040.stdout:1/284: fsync da/d12/d18/d2a/d41/f50 0 2024-04-24T23:13:49.909 INFO:tasks.workunit.client.0.smithi031.stdout:0/649: rename da/d18/d1a/d76 to da/d18/d1a/dd6 0 2024-04-24T23:13:49.910 INFO:tasks.workunit.client.0.smithi031.stdout:0/650: chown da/d18/d1a/d37/d41/f47 39 1 2024-04-24T23:13:49.911 INFO:tasks.workunit.client.1.smithi040.stdout:3/268: mkdir d1/d9/d18/d36/d55/d62 0 2024-04-24T23:13:49.912 INFO:tasks.workunit.client.1.smithi040.stdout:5/236: getdents d2/d7 0 2024-04-24T23:13:49.912 INFO:tasks.workunit.client.0.smithi031.stdout:2/479: rmdir d5/d28/d4e/d5e 39 2024-04-24T23:13:49.912 INFO:tasks.workunit.client.1.smithi040.stdout:5/237: chown d2/d7/d8/db/d1e/d22/f25 3570 1 2024-04-24T23:13:49.913 INFO:tasks.workunit.client.0.smithi031.stdout:2/480: chown d5/d8/f10 12 1 2024-04-24T23:13:49.913 INFO:tasks.workunit.client.0.smithi031.stdout:5/539: creat d0/d7/d56/d7c/d7e/fb5 x:0 0 0 2024-04-24T23:13:49.914 INFO:tasks.workunit.client.1.smithi040.stdout:0/329: mkdir df/d1e/d20/d61 0 2024-04-24T23:13:49.914 INFO:tasks.workunit.client.0.smithi031.stdout:8/703: link d8/lf d8/d15/d37/d43/daf/dca/lf5 0 2024-04-24T23:13:49.918 INFO:tasks.workunit.client.0.smithi031.stdout:2/481: creat d5/d28/d4e/d7f/fa9 x:0 0 0 2024-04-24T23:13:49.919 INFO:tasks.workunit.client.1.smithi040.stdout:7/310: rename db/d43/d66 to db/d12/d20/d52/d6c 0 2024-04-24T23:13:49.920 INFO:tasks.workunit.client.1.smithi040.stdout:1/285: creat da/d12/d18/d2a/d41/f62 x:0 0 0 2024-04-24T23:13:49.922 INFO:tasks.workunit.client.1.smithi040.stdout:5/238: truncate d2/f6 1281676 0 2024-04-24T23:13:49.924 INFO:tasks.workunit.client.1.smithi040.stdout:5/239: write d2/d7/d8/db/d1e/f46 [47149,19971] 0 2024-04-24T23:13:49.925 INFO:tasks.workunit.client.1.smithi040.stdout:5/240: write d2/d7/d8/f13 [2195439,47421] 0 2024-04-24T23:13:49.926 INFO:tasks.workunit.client.1.smithi040.stdout:2/337: mkdir da/d81 0 2024-04-24T23:13:49.927 INFO:tasks.workunit.client.1.smithi040.stdout:2/338: read f8 [1030712,92904] 0 2024-04-24T23:13:49.928 INFO:tasks.workunit.client.1.smithi040.stdout:2/339: dread - da/d10/d15/f5e zero size 2024-04-24T23:13:49.928 INFO:tasks.workunit.client.1.smithi040.stdout:6/269: rename d7/d1a/d20/d51/d2f/l38 to d7/d1a/l55 0 2024-04-24T23:13:49.929 INFO:tasks.workunit.client.1.smithi040.stdout:3/269: creat d1/d2d/de/d30/d37/f63 x:0 0 0 2024-04-24T23:13:49.930 INFO:tasks.workunit.client.1.smithi040.stdout:2/340: readlink da/d1c/d37/d39/d3f/l49 0 2024-04-24T23:13:49.930 INFO:tasks.workunit.client.1.smithi040.stdout:0/330: mkdir df/d12/d4b/d62 0 2024-04-24T23:13:49.933 INFO:tasks.workunit.client.1.smithi040.stdout:7/311: creat db/d2f/f6d x:0 0 0 2024-04-24T23:13:49.933 INFO:tasks.workunit.client.1.smithi040.stdout:7/312: chown db/d43 6920442 1 2024-04-24T23:13:49.934 INFO:tasks.workunit.client.1.smithi040.stdout:7/313: chown db/d43/f4e 18935 1 2024-04-24T23:13:49.936 INFO:tasks.workunit.client.1.smithi040.stdout:6/270: unlink d7/fb 0 2024-04-24T23:13:49.936 INFO:tasks.workunit.client.1.smithi040.stdout:6/271: chown d7/d1a/d20 754043 1 2024-04-24T23:13:49.937 INFO:tasks.workunit.client.1.smithi040.stdout:6/272: chown d7/c11 1584 1 2024-04-24T23:13:49.937 INFO:tasks.workunit.client.1.smithi040.stdout:6/273: fdatasync f2 0 2024-04-24T23:13:49.937 INFO:tasks.workunit.client.1.smithi040.stdout:3/270: symlink d1/d9/l64 0 2024-04-24T23:13:49.938 INFO:tasks.workunit.client.1.smithi040.stdout:2/341: mknod da/d10/d15/c82 0 2024-04-24T23:13:49.939 INFO:tasks.workunit.client.1.smithi040.stdout:3/271: read d1/d9/f20 [4010991,43251] 0 2024-04-24T23:13:49.941 INFO:tasks.workunit.client.1.smithi040.stdout:8/242: truncate d0/d8/dc/f3a 1202764 0 2024-04-24T23:13:49.943 INFO:tasks.workunit.client.1.smithi040.stdout:7/314: write db/f55 [121672,15335] 0 2024-04-24T23:13:49.944 INFO:tasks.workunit.client.1.smithi040.stdout:3/272: dread d1/d9/d18/f2c [0,4194304] 0 2024-04-24T23:13:49.945 INFO:tasks.workunit.client.1.smithi040.stdout:3/273: chown d1/d9/l11 37784894 1 2024-04-24T23:13:49.945 INFO:tasks.workunit.client.1.smithi040.stdout:3/274: readlink d1/d9/l4f 0 2024-04-24T23:13:49.948 INFO:tasks.workunit.client.0.smithi031.stdout:1/549: sync 2024-04-24T23:13:49.949 INFO:tasks.workunit.client.0.smithi031.stdout:4/714: symlink d3/d23/d37/ddc/le6 0 2024-04-24T23:13:49.951 INFO:tasks.workunit.client.1.smithi040.stdout:6/274: readlink d7/d1a/l55 0 2024-04-24T23:13:49.955 INFO:tasks.workunit.client.1.smithi040.stdout:5/241: rename d2/d7/d8/db/d1e/d22 to d2/d7/d8/d1f/d57 0 2024-04-24T23:13:49.956 INFO:tasks.workunit.client.0.smithi031.stdout:1/550: readlink d1/dc/d11/d21/l40 0 2024-04-24T23:13:49.957 INFO:tasks.workunit.client.1.smithi040.stdout:7/315: mknod db/d12/d20/d52/d6c/c6e 0 2024-04-24T23:13:49.958 INFO:tasks.workunit.client.0.smithi031.stdout:8/704: dwrite d8/d15/d9a/fb1 [0,4194304] 0 2024-04-24T23:13:49.967 INFO:tasks.workunit.client.1.smithi040.stdout:0/331: truncate df/d12/d49/f54 457882 0 2024-04-24T23:13:49.967 INFO:tasks.workunit.client.1.smithi040.stdout:5/242: dread d2/d7/d8/db/d1e/f2e [0,4194304] 0 2024-04-24T23:13:49.969 INFO:tasks.workunit.client.0.smithi031.stdout:8/705: dwrite d8/d15/d37/d43/d7b/d89/fdf [0,4194304] 0 2024-04-24T23:13:49.971 INFO:tasks.workunit.client.1.smithi040.stdout:5/243: truncate d2/d7/d8/d1f/d57/d27/f32 4546232 0 2024-04-24T23:13:49.974 INFO:tasks.workunit.client.1.smithi040.stdout:6/275: write d7/d1a/d20/d51/d1e/f3d [654755,12183] 0 2024-04-24T23:13:49.974 INFO:tasks.workunit.client.1.smithi040.stdout:0/332: dwrite df/d1e/f2f [0,4194304] 0 2024-04-24T23:13:49.975 INFO:tasks.workunit.client.1.smithi040.stdout:6/276: write f2 [2495207,24555] 0 2024-04-24T23:13:49.979 INFO:tasks.workunit.client.0.smithi031.stdout:2/482: dread d5/d8/d15/f2b [0,4194304] 0 2024-04-24T23:13:49.980 INFO:tasks.workunit.client.1.smithi040.stdout:8/243: rename d0/d1f to d0/d16/d19/d4f 0 2024-04-24T23:13:49.984 INFO:tasks.workunit.client.1.smithi040.stdout:3/275: creat d1/d9/d18/d36/d55/d62/f65 x:0 0 0 2024-04-24T23:13:49.984 INFO:tasks.workunit.client.1.smithi040.stdout:3/276: dread - d1/f59 zero size 2024-04-24T23:13:49.988 INFO:tasks.workunit.client.1.smithi040.stdout:2/342: link da/d1c/d37/d39/d77/c6f da/d10/c83 0 2024-04-24T23:13:49.989 INFO:tasks.workunit.client.1.smithi040.stdout:2/343: dread - da/d1c/d4f/f61 zero size 2024-04-24T23:13:49.989 INFO:tasks.workunit.client.1.smithi040.stdout:2/344: fdatasync da/d1c/d4f/f61 0 2024-04-24T23:13:49.992 INFO:tasks.workunit.client.1.smithi040.stdout:5/244: creat d2/d7/d8/d1f/d57/d3e/f58 x:0 0 0 2024-04-24T23:13:49.995 INFO:tasks.workunit.client.1.smithi040.stdout:6/277: rmdir d7/d1a/d20/d51 39 2024-04-24T23:13:49.998 INFO:tasks.workunit.client.1.smithi040.stdout:5/245: dwrite d2/d7/d8/f39 [0,4194304] 0 2024-04-24T23:13:49.999 INFO:tasks.workunit.client.1.smithi040.stdout:8/244: rename d0/d16/d19/d4f/f2d to d0/d38/f50 0 2024-04-24T23:13:50.000 INFO:tasks.workunit.client.1.smithi040.stdout:5/246: truncate d2/d7/d8/f1b 1630672 0 2024-04-24T23:13:50.001 INFO:tasks.workunit.client.1.smithi040.stdout:3/277: creat d1/d9/d18/f66 x:0 0 0 2024-04-24T23:13:50.001 INFO:tasks.workunit.client.1.smithi040.stdout:5/247: dread - d2/d7/d8/f50 zero size 2024-04-24T23:13:50.005 INFO:tasks.workunit.client.1.smithi040.stdout:8/245: dread d0/d38/f50 [0,4194304] 0 2024-04-24T23:13:50.008 INFO:tasks.workunit.client.1.smithi040.stdout:7/316: dread db/d12/f16 [0,4194304] 0 2024-04-24T23:13:50.009 INFO:tasks.workunit.client.0.smithi031.stdout:5/540: truncate d0/d6/d12/f31 231917 0 2024-04-24T23:13:50.010 INFO:tasks.workunit.client.0.smithi031.stdout:5/541: fsync d0/d6/d3b/d4b/dc/fa2 0 2024-04-24T23:13:50.011 INFO:tasks.workunit.client.1.smithi040.stdout:5/248: dwrite d2/d7/d8/d1f/d31/f4e [0,4194304] 0 2024-04-24T23:13:50.012 INFO:tasks.workunit.client.0.smithi031.stdout:5/542: dread - d0/d6/d3b/d4b/d1d/d2e/f79 zero size 2024-04-24T23:13:50.012 INFO:tasks.workunit.client.1.smithi040.stdout:3/278: rename d1/d2d/d44/f48 to d1/d9/d18/d36/d55/f67 0 2024-04-24T23:13:50.016 INFO:tasks.workunit.client.1.smithi040.stdout:2/345: link da/f40 da/d10/f84 0 2024-04-24T23:13:50.016 INFO:tasks.workunit.client.0.smithi031.stdout:0/651: getdents da/d11/d6a/dba/dc0 0 2024-04-24T23:13:50.019 INFO:tasks.workunit.client.0.smithi031.stdout:1/551: chown d1/dc/d11/d21/l78 133 1 2024-04-24T23:13:50.020 INFO:tasks.workunit.client.1.smithi040.stdout:8/246: symlink d0/d16/d19/d4f/l51 0 2024-04-24T23:13:50.021 INFO:tasks.workunit.client.1.smithi040.stdout:2/346: dwrite da/d10/d15/f32 [0,4194304] 0 2024-04-24T23:13:50.021 INFO:tasks.workunit.client.0.smithi031.stdout:4/715: rename d3/d23/d37/d86/cdd to d3/d23/ce7 0 2024-04-24T23:13:50.024 INFO:tasks.workunit.client.1.smithi040.stdout:0/333: creat df/f63 x:0 0 0 2024-04-24T23:13:50.025 INFO:tasks.workunit.client.1.smithi040.stdout:0/334: write df/d1e/f40 [531285,56138] 0 2024-04-24T23:13:50.028 INFO:tasks.workunit.client.0.smithi031.stdout:2/483: rename d5/d8/d45/d5b/d49/d2f/f5d to d5/d8/d45/d5b/d49/d2f/faa 0 2024-04-24T23:13:50.030 INFO:tasks.workunit.client.1.smithi040.stdout:7/317: fdatasync db/d12/d20/d52/d4b/f57 0 2024-04-24T23:13:50.035 INFO:tasks.workunit.client.1.smithi040.stdout:3/279: mknod d1/d2d/de/d30/d37/c68 0 2024-04-24T23:13:50.036 INFO:tasks.workunit.client.0.smithi031.stdout:1/552: rename d1/dc/d11/c6e to d1/d33/d41/d9b/da8/cab 0 2024-04-24T23:13:50.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/706: symlink d8/d1a/d1d/d35/lf6 0 2024-04-24T23:13:50.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/707: readlink d8/l20 0 2024-04-24T23:13:50.046 INFO:tasks.workunit.client.0.smithi031.stdout:0/652: write da/d20/d4e/f51 [1702095,81597] 0 2024-04-24T23:13:50.046 INFO:tasks.workunit.client.0.smithi031.stdout:5/543: symlink d0/d6/d3b/d4b/dc/lb6 0 2024-04-24T23:13:50.047 INFO:tasks.workunit.client.0.smithi031.stdout:8/708: dwrite d8/f18 [4194304,4194304] 0 2024-04-24T23:13:50.049 INFO:tasks.workunit.client.0.smithi031.stdout:8/709: write d8/d15/d37/d43/daf/fed [1043416,38306] 0 2024-04-24T23:13:50.052 INFO:tasks.workunit.client.1.smithi040.stdout:5/249: mkdir d2/d7/d8/d1f/d57/d59 0 2024-04-24T23:13:50.053 INFO:tasks.workunit.client.1.smithi040.stdout:3/280: fdatasync d1/d9/d18/d36/d55/f56 0 2024-04-24T23:13:50.056 INFO:tasks.workunit.client.1.smithi040.stdout:8/247: mkdir d0/d52 0 2024-04-24T23:13:50.057 INFO:tasks.workunit.client.0.smithi031.stdout:4/716: symlink d3/d20/d25/le8 0 2024-04-24T23:13:50.057 INFO:tasks.workunit.client.1.smithi040.stdout:8/248: write d0/d16/d1b/d47/f48 [452266,35273] 0 2024-04-24T23:13:50.058 INFO:tasks.workunit.client.0.smithi031.stdout:2/484: dwrite d5/d8/d15/f2b [0,4194304] 0 2024-04-24T23:13:50.059 INFO:tasks.workunit.client.0.smithi031.stdout:2/485: stat d5/d28/d43/f67 0 2024-04-24T23:13:50.061 INFO:tasks.workunit.client.1.smithi040.stdout:6/278: getdents d7 0 2024-04-24T23:13:50.062 INFO:tasks.workunit.client.1.smithi040.stdout:0/335: mknod df/d14/c64 0 2024-04-24T23:13:50.063 INFO:tasks.workunit.client.0.smithi031.stdout:0/653: rmdir da/d20/d4e/dc9 39 2024-04-24T23:13:50.071 INFO:tasks.workunit.client.0.smithi031.stdout:1/553: mknod d1/da/cac 0 2024-04-24T23:13:50.072 INFO:tasks.workunit.client.1.smithi040.stdout:8/249: creat d0/d8/d22/f53 x:0 0 0 2024-04-24T23:13:50.073 INFO:tasks.workunit.client.1.smithi040.stdout:2/347: link da/d1c/d37/l51 da/d29/l85 0 2024-04-24T23:13:50.074 INFO:tasks.workunit.client.1.smithi040.stdout:8/250: write d0/d16/f3b [1030667,38169] 0 2024-04-24T23:13:50.075 INFO:tasks.workunit.client.0.smithi031.stdout:0/654: rmdir da/d86/da3 39 2024-04-24T23:13:50.076 INFO:tasks.workunit.client.0.smithi031.stdout:5/544: mknod d0/d6/d3b/d4b/dc/d8b/d9b/da8/cb7 0 2024-04-24T23:13:50.078 INFO:tasks.workunit.client.1.smithi040.stdout:5/250: link d2/d7/d8/f39 d2/d7/d8/d1f/d57/d59/f5a 0 2024-04-24T23:13:50.078 INFO:tasks.workunit.client.0.smithi031.stdout:8/710: write d8/d1a/d1d/d35/d58/f64 [948077,61547] 0 2024-04-24T23:13:50.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/717: dread d3/d20/d25/d7e/faa [0,4194304] 0 2024-04-24T23:13:50.083 INFO:tasks.workunit.client.1.smithi040.stdout:8/251: creat d0/d16/d1b/f54 x:0 0 0 2024-04-24T23:13:50.084 INFO:tasks.workunit.client.0.smithi031.stdout:8/711: dread d8/fa8 [0,4194304] 0 2024-04-24T23:13:50.085 INFO:tasks.workunit.client.0.smithi031.stdout:5/545: write d0/d6/d3b/d4b/f96 [3324154,26672] 0 2024-04-24T23:13:50.086 INFO:tasks.workunit.client.1.smithi040.stdout:7/318: rename db/ce to db/d12/c6f 0 2024-04-24T23:13:50.088 INFO:tasks.workunit.client.1.smithi040.stdout:2/348: write f8 [2369987,110037] 0 2024-04-24T23:13:50.089 INFO:tasks.workunit.client.1.smithi040.stdout:5/251: write d2/d7/d8/f34 [4352583,26474] 0 2024-04-24T23:13:50.094 INFO:tasks.workunit.client.0.smithi031.stdout:2/486: link d5/f5c d5/d6/df/fab 0 2024-04-24T23:13:50.095 INFO:tasks.workunit.client.0.smithi031.stdout:2/487: write d5/d6/d29/f61 [741481,7398] 0 2024-04-24T23:13:50.098 INFO:tasks.workunit.client.1.smithi040.stdout:6/279: rename d7/d1a/d20/d51/d1e/d3e to d7/d1a/d56 0 2024-04-24T23:13:50.098 INFO:tasks.workunit.client.0.smithi031.stdout:8/712: dread d8/d1a/d1d/d50/dc0/d9d/fea [0,4194304] 0 2024-04-24T23:13:50.099 INFO:tasks.workunit.client.0.smithi031.stdout:1/554: rename d1/da/d12/f82 to d1/da/fad 0 2024-04-24T23:13:50.099 INFO:tasks.workunit.client.0.smithi031.stdout:8/713: write d8/d15/d37/d43/d7b/d89/fdf [4109679,36872] 0 2024-04-24T23:13:50.100 INFO:tasks.workunit.client.1.smithi040.stdout:3/281: getdents d1/d9/d18 0 2024-04-24T23:13:50.100 INFO:tasks.workunit.client.1.smithi040.stdout:8/252: dwrite d0/d38/f50 [0,4194304] 0 2024-04-24T23:13:50.101 INFO:tasks.workunit.client.0.smithi031.stdout:4/718: mknod d3/d23/d69/ce9 0 2024-04-24T23:13:50.102 INFO:tasks.workunit.client.1.smithi040.stdout:3/282: readlink d1/d9/l1e 0 2024-04-24T23:13:50.102 INFO:tasks.workunit.client.1.smithi040.stdout:3/283: truncate d1/d2d/f60 1029864 0 2024-04-24T23:13:50.103 INFO:tasks.workunit.client.1.smithi040.stdout:6/280: dread d7/d1a/d20/f47 [0,4194304] 0 2024-04-24T23:13:50.103 INFO:tasks.workunit.client.1.smithi040.stdout:2/349: mknod da/d29/d55/c86 0 2024-04-24T23:13:50.104 INFO:tasks.workunit.client.1.smithi040.stdout:2/350: chown da/d1c/c1d 124334060 1 2024-04-24T23:13:50.106 INFO:tasks.workunit.client.1.smithi040.stdout:5/252: rename d2/d7/d8/d1f/d57/d3e/l42 to d2/d2f/l5b 0 2024-04-24T23:13:50.110 INFO:tasks.workunit.client.0.smithi031.stdout:0/655: rename da/d11/d6a/f6e to da/d22/db0/fd7 0 2024-04-24T23:13:50.116 INFO:tasks.workunit.client.0.smithi031.stdout:1/555: symlink d1/da/d12/d7e/d7b/d4e/lae 0 2024-04-24T23:13:50.117 INFO:tasks.workunit.client.1.smithi040.stdout:7/319: creat db/d12/d20/d52/f70 x:0 0 0 2024-04-24T23:13:50.118 INFO:tasks.workunit.client.0.smithi031.stdout:4/719: mkdir d3/d23/d69/d3f/dea 0 2024-04-24T23:13:50.119 INFO:tasks.workunit.client.0.smithi031.stdout:4/720: write d3/d23/d3b/fd6 [399110,22816] 0 2024-04-24T23:13:50.120 INFO:tasks.workunit.client.1.smithi040.stdout:6/281: symlink d7/d1a/d20/d51/d1e/l57 0 2024-04-24T23:13:50.120 INFO:tasks.workunit.client.0.smithi031.stdout:4/721: readlink d3/d23/d37/l91 0 2024-04-24T23:13:50.121 INFO:tasks.workunit.client.0.smithi031.stdout:4/722: write d3/d5e/f8d [629238,110597] 0 2024-04-24T23:13:50.123 INFO:tasks.workunit.client.1.smithi040.stdout:8/253: mkdir d0/d8/d55 0 2024-04-24T23:13:50.123 INFO:tasks.workunit.client.1.smithi040.stdout:7/320: dwrite db/d12/d3f/f41 [0,4194304] 0 2024-04-24T23:13:50.124 INFO:tasks.workunit.client.0.smithi031.stdout:5/546: rename d0/d6/d32/d6b/f70 to d0/d7/d56/d7c/d7e/fb8 0 2024-04-24T23:13:50.125 INFO:tasks.workunit.client.1.smithi040.stdout:8/254: readlink d0/d16/l18 0 2024-04-24T23:13:50.126 INFO:tasks.workunit.client.0.smithi031.stdout:5/547: dread - d0/d6/d3b/d4b/d1d/d2e/f79 zero size 2024-04-24T23:13:50.126 INFO:tasks.workunit.client.1.smithi040.stdout:8/255: fdatasync d0/d16/d1b/f54 0 2024-04-24T23:13:50.127 INFO:tasks.workunit.client.1.smithi040.stdout:9/301: sync 2024-04-24T23:13:50.128 INFO:tasks.workunit.client.1.smithi040.stdout:3/284: dwrite d1/d9/d18/f2c [0,4194304] 0 2024-04-24T23:13:50.128 INFO:tasks.workunit.client.0.smithi031.stdout:1/556: fdatasync d1/d33/d41/d9b/da8/fa9 0 2024-04-24T23:13:50.129 INFO:tasks.workunit.client.0.smithi031.stdout:5/548: read d0/d6/d3b/d4b/dc/f23 [939630,17595] 0 2024-04-24T23:13:50.131 INFO:tasks.workunit.client.0.smithi031.stdout:2/488: creat d5/d28/d4e/fac x:0 0 0 2024-04-24T23:13:50.131 INFO:tasks.workunit.client.0.smithi031.stdout:5/549: chown d0/d7/d56/d7c 5 1 2024-04-24T23:13:50.134 INFO:tasks.workunit.client.1.smithi040.stdout:8/256: mknod d0/d28/c56 0 2024-04-24T23:13:50.135 INFO:tasks.workunit.client.0.smithi031.stdout:4/723: mknod d3/d23/d37/d86/ceb 0 2024-04-24T23:13:50.139 INFO:tasks.workunit.client.1.smithi040.stdout:3/285: dwrite d1/d9/d2e/f5f [0,4194304] 0 2024-04-24T23:13:50.139 INFO:tasks.workunit.client.1.smithi040.stdout:9/302: dwrite d7/da/d17/d40/d55/d2f/f64 [0,4194304] 0 2024-04-24T23:13:50.142 INFO:tasks.workunit.client.1.smithi040.stdout:9/303: chown d7/da/d17/d40/d55/d2f/d1f/d31/d4d 1 1 2024-04-24T23:13:50.142 INFO:tasks.workunit.client.1.smithi040.stdout:6/282: write d7/d1a/d20/f28 [1618562,73413] 0 2024-04-24T23:13:50.146 INFO:tasks.workunit.client.0.smithi031.stdout:9/684: sync 2024-04-24T23:13:50.148 INFO:tasks.workunit.client.1.smithi040.stdout:7/321: creat db/f71 x:0 0 0 2024-04-24T23:13:50.148 INFO:tasks.workunit.client.0.smithi031.stdout:9/685: write d2/dae/d14/d1d/d45/d99/d9c/fc8 [31985,10800] 0 2024-04-24T23:13:50.148 INFO:tasks.workunit.client.1.smithi040.stdout:3/286: rmdir d1/d9/d18/d36/d61 39 2024-04-24T23:13:50.149 INFO:tasks.workunit.client.0.smithi031.stdout:2/489: fsync d5/d28/f54 0 2024-04-24T23:13:50.149 INFO:tasks.workunit.client.1.smithi040.stdout:7/322: chown f6 0 1 2024-04-24T23:13:50.151 INFO:tasks.workunit.client.0.smithi031.stdout:4/724: symlink d3/d23/d3b/lec 0 2024-04-24T23:13:50.151 INFO:tasks.workunit.client.0.smithi031.stdout:2/490: write d5/d8/d15/f2b [220435,98851] 0 2024-04-24T23:13:50.153 INFO:tasks.workunit.client.0.smithi031.stdout:0/656: link da/d22/f4b da/d18/d1a/d37/fd8 0 2024-04-24T23:13:50.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/491: fdatasync d5/d6/d29/d4f/f77 0 2024-04-24T23:13:50.154 INFO:tasks.workunit.client.1.smithi040.stdout:8/257: creat d0/d16/f57 x:0 0 0 2024-04-24T23:13:50.155 INFO:tasks.workunit.client.1.smithi040.stdout:8/258: fdatasync d0/d16/d19/f35 0 2024-04-24T23:13:50.155 INFO:tasks.workunit.client.0.smithi031.stdout:0/657: fdatasync da/d18/d1a/d37/d41/d92/fa8 0 2024-04-24T23:13:50.157 INFO:tasks.workunit.client.1.smithi040.stdout:7/323: mknod db/d12/d62/c72 0 2024-04-24T23:13:50.157 INFO:tasks.workunit.client.1.smithi040.stdout:7/324: chown db/d43/l6b 211 1 2024-04-24T23:13:50.158 INFO:tasks.workunit.client.1.smithi040.stdout:7/325: write db/d12/d20/d52/d44/f47 [1587509,88305] 0 2024-04-24T23:13:50.159 INFO:tasks.workunit.client.1.smithi040.stdout:9/304: fsync f2 0 2024-04-24T23:13:50.160 INFO:tasks.workunit.client.1.smithi040.stdout:6/283: symlink d7/d1a/d20/d51/d2a/l58 0 2024-04-24T23:13:50.161 INFO:tasks.workunit.client.0.smithi031.stdout:5/550: mknod d0/cb9 0 2024-04-24T23:13:50.168 INFO:tasks.workunit.client.0.smithi031.stdout:8/714: dwrite d8/d15/d37/d62/f7f [0,4194304] 0 2024-04-24T23:13:50.168 INFO:tasks.workunit.client.1.smithi040.stdout:8/259: unlink d0/d8/d22/f53 0 2024-04-24T23:13:50.173 INFO:tasks.workunit.client.0.smithi031.stdout:4/725: rename d3/d5e/fb6 to d3/d20/da7/fed 0 2024-04-24T23:13:50.173 INFO:tasks.workunit.client.0.smithi031.stdout:4/726: fdatasync d3/f11 0 2024-04-24T23:13:50.176 INFO:tasks.workunit.client.1.smithi040.stdout:9/305: read d7/da/d17/d40/f42 [1845864,56015] 0 2024-04-24T23:13:50.179 INFO:tasks.workunit.client.0.smithi031.stdout:1/557: write d1/da/fad [860833,1934] 0 2024-04-24T23:13:50.180 INFO:tasks.workunit.client.1.smithi040.stdout:6/284: symlink d7/d1a/d56/l59 0 2024-04-24T23:13:50.182 INFO:tasks.workunit.client.1.smithi040.stdout:3/287: link d1/d9/c31 d1/d9/d25/c69 0 2024-04-24T23:13:50.185 INFO:tasks.workunit.client.1.smithi040.stdout:8/260: mkdir d0/d16/d1b/d58 0 2024-04-24T23:13:50.187 INFO:tasks.workunit.client.1.smithi040.stdout:9/306: rmdir d7/da/d17/d40/d55/d2f/d1f/d31/d4d 39 2024-04-24T23:13:50.188 INFO:tasks.workunit.client.1.smithi040.stdout:9/307: read d7/f37 [593012,14504] 0 2024-04-24T23:13:50.190 INFO:tasks.workunit.client.0.smithi031.stdout:2/492: write d5/d28/f37 [1368048,64320] 0 2024-04-24T23:13:50.191 INFO:tasks.workunit.client.1.smithi040.stdout:7/326: write db/d12/d20/d52/d4b/f4f [913978,83998] 0 2024-04-24T23:13:50.192 INFO:tasks.workunit.client.1.smithi040.stdout:3/288: rename d1/d9/d25 to d1/d2a/d6a 0 2024-04-24T23:13:50.197 INFO:tasks.workunit.client.0.smithi031.stdout:1/558: creat d1/da/d12/d7e/faf x:0 0 0 2024-04-24T23:13:50.197 INFO:tasks.workunit.client.1.smithi040.stdout:7/327: fsync db/f61 0 2024-04-24T23:13:50.198 INFO:tasks.workunit.client.0.smithi031.stdout:1/559: write d1/d33/d36/f8f [90554,91121] 0 2024-04-24T23:13:50.198 INFO:tasks.workunit.client.0.smithi031.stdout:0/658: creat da/d18/d1a/db7/fd9 x:0 0 0 2024-04-24T23:13:50.201 INFO:tasks.workunit.client.1.smithi040.stdout:9/308: unlink d7/da/d17/d40/d55/d2f/d1f/d34/f5c 0 2024-04-24T23:13:50.202 INFO:tasks.workunit.client.1.smithi040.stdout:9/309: chown d7/da/d17/d40/d55/d2f/d1f/f5a 106 1 2024-04-24T23:13:50.203 INFO:tasks.workunit.client.0.smithi031.stdout:9/686: creat d2/dae/d14/d78/d84/fd0 x:0 0 0 2024-04-24T23:13:50.204 INFO:tasks.workunit.client.1.smithi040.stdout:9/310: write d7/da/d17/d40/d55/d2f/d1f/d34/f47 [903432,26944] 0 2024-04-24T23:13:50.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/560: rmdir d1/dc/d11 39 2024-04-24T23:13:50.207 INFO:tasks.workunit.client.1.smithi040.stdout:6/285: getdents d7/d2c/d52 0 2024-04-24T23:13:50.210 INFO:tasks.workunit.client.1.smithi040.stdout:3/289: dwrite d1/d2d/f24 [4194304,4194304] 0 2024-04-24T23:13:50.210 INFO:tasks.workunit.client.0.smithi031.stdout:4/727: truncate d3/d5e/f8d 2536717 0 2024-04-24T23:13:50.211 INFO:tasks.workunit.client.1.smithi040.stdout:3/290: write d1/d2d/de/d30/f41 [124175,89644] 0 2024-04-24T23:13:50.212 INFO:tasks.workunit.client.1.smithi040.stdout:3/291: chown d1/c4 4 1 2024-04-24T23:13:50.213 INFO:tasks.workunit.client.0.smithi031.stdout:5/551: creat d0/d6/d3b/d4b/d1d/d2e/fba x:0 0 0 2024-04-24T23:13:50.213 INFO:tasks.workunit.client.1.smithi040.stdout:7/328: mkdir db/d12/d73 0 2024-04-24T23:13:50.216 INFO:tasks.workunit.client.1.smithi040.stdout:6/286: creat d7/d2c/d52/f5a x:0 0 0 2024-04-24T23:13:50.216 INFO:tasks.workunit.client.1.smithi040.stdout:7/329: dread f5 [0,4194304] 0 2024-04-24T23:13:50.217 INFO:tasks.workunit.client.0.smithi031.stdout:9/687: mkdir d2/dae/d14/d78/d84/d4a/dab/dd1 0 2024-04-24T23:13:50.220 INFO:tasks.workunit.client.0.smithi031.stdout:9/688: chown d2/d16/dbe/d7d/fac 112627 1 2024-04-24T23:13:50.220 INFO:tasks.workunit.client.0.smithi031.stdout:4/728: dwrite d3/d20/d25/fc3 [0,4194304] 0 2024-04-24T23:13:50.221 INFO:tasks.workunit.client.0.smithi031.stdout:9/689: chown d2/l37 5354 1 2024-04-24T23:13:50.225 INFO:tasks.workunit.client.1.smithi040.stdout:3/292: symlink d1/d2d/d44/l6b 0 2024-04-24T23:13:50.226 INFO:tasks.workunit.client.1.smithi040.stdout:3/293: write d1/d9/d18/f5e [339103,84945] 0 2024-04-24T23:13:50.230 INFO:tasks.workunit.client.1.smithi040.stdout:6/287: creat d7/d1a/d20/d51/d2f/f5b x:0 0 0 2024-04-24T23:13:50.231 INFO:tasks.workunit.client.0.smithi031.stdout:5/552: mknod d0/d6/d3b/d4b/dc/d86/cbb 0 2024-04-24T23:13:50.233 INFO:tasks.workunit.client.1.smithi040.stdout:7/330: rename db/d43/f4e to db/d12/d3f/f74 0 2024-04-24T23:13:50.233 INFO:tasks.workunit.client.1.smithi040.stdout:7/331: read db/d12/f16 [1415080,57125] 0 2024-04-24T23:13:50.235 INFO:tasks.workunit.client.1.smithi040.stdout:3/294: creat d1/d2d/d44/f6c x:0 0 0 2024-04-24T23:13:50.239 INFO:tasks.workunit.client.0.smithi031.stdout:4/729: rmdir d3/d23/d3b 39 2024-04-24T23:13:50.239 INFO:tasks.workunit.client.1.smithi040.stdout:6/288: dread d7/de/ff [0,4194304] 0 2024-04-24T23:13:50.240 INFO:tasks.workunit.client.1.smithi040.stdout:9/311: dread d7/da/d17/d40/d55/d2f/d1f/d31/d4d/f43 [0,4194304] 0 2024-04-24T23:13:50.242 INFO:tasks.workunit.client.1.smithi040.stdout:7/332: read db/f24 [103788,113106] 0 2024-04-24T23:13:50.242 INFO:tasks.workunit.client.1.smithi040.stdout:9/312: write d7/da/d17/d40/d55/d2f/d1f/f5a [5164301,5417] 0 2024-04-24T23:13:50.243 INFO:tasks.workunit.client.0.smithi031.stdout:0/659: link da/d18/d1a/c8f da/d20/cda 0 2024-04-24T23:13:50.243 INFO:tasks.workunit.client.1.smithi040.stdout:3/295: creat d1/d2a/f6d x:0 0 0 2024-04-24T23:13:50.244 INFO:tasks.workunit.client.1.smithi040.stdout:3/296: chown d1/d2d/d44/f52 144479 1 2024-04-24T23:13:50.244 INFO:tasks.workunit.client.0.smithi031.stdout:2/493: getdents d5/d6/d29/d4f 0 2024-04-24T23:13:50.247 INFO:tasks.workunit.client.0.smithi031.stdout:5/553: symlink d0/d6/d3b/d4b/d1d/d2e/d28/lbc 0 2024-04-24T23:13:50.249 INFO:tasks.workunit.client.1.smithi040.stdout:4/247: sync 2024-04-24T23:13:50.249 INFO:tasks.workunit.client.1.smithi040.stdout:4/248: readlink da/l17 0 2024-04-24T23:13:50.249 INFO:tasks.workunit.client.0.smithi031.stdout:4/730: rename d3/d23/d37/d86/l8e to d3/d20/d25/d60/d63/db9/lee 0 2024-04-24T23:13:50.250 INFO:tasks.workunit.client.0.smithi031.stdout:4/731: chown d3/d20/cac 62 1 2024-04-24T23:13:50.250 INFO:tasks.workunit.client.0.smithi031.stdout:1/561: truncate d1/dc/f70 2142486 0 2024-04-24T23:13:50.251 INFO:tasks.workunit.client.1.smithi040.stdout:7/333: symlink db/d12/d20/d56/l75 0 2024-04-24T23:13:50.252 INFO:tasks.workunit.client.0.smithi031.stdout:1/562: truncate d1/da/d12/d7e/f9e 906290 0 2024-04-24T23:13:50.252 INFO:tasks.workunit.client.0.smithi031.stdout:9/690: mknod d2/dae/d14/d78/d84/d4a/cd2 0 2024-04-24T23:13:50.253 INFO:tasks.workunit.client.1.smithi040.stdout:9/313: creat d7/da/d17/d40/d55/d2f/d28/f6b x:0 0 0 2024-04-24T23:13:50.255 INFO:tasks.workunit.client.0.smithi031.stdout:0/660: truncate da/fe 4944778 0 2024-04-24T23:13:50.256 INFO:tasks.workunit.client.1.smithi040.stdout:6/289: rename d7/d1a/d20/d51/f1f to d7/d1a/d20/d51/d2f/f5c 0 2024-04-24T23:13:50.258 INFO:tasks.workunit.client.0.smithi031.stdout:5/554: mkdir d0/d7/d74/dbd 0 2024-04-24T23:13:50.260 INFO:tasks.workunit.client.0.smithi031.stdout:4/732: fdatasync d3/d23/d37/f7b 0 2024-04-24T23:13:50.263 INFO:tasks.workunit.client.0.smithi031.stdout:9/691: read - d2/dae/d14/f88 zero size 2024-04-24T23:13:50.266 INFO:tasks.workunit.client.0.smithi031.stdout:0/661: mkdir da/d18/d1a/dd6/d82/ddb 0 2024-04-24T23:13:50.267 INFO:tasks.workunit.client.0.smithi031.stdout:0/662: stat da/d18/d1a/dd6/d82/fc7 0 2024-04-24T23:13:50.269 INFO:tasks.workunit.client.1.smithi040.stdout:7/334: unlink db/f3c 0 2024-04-24T23:13:50.269 INFO:tasks.workunit.client.1.smithi040.stdout:7/335: fdatasync db/d12/f3e 0 2024-04-24T23:13:50.271 INFO:tasks.workunit.client.0.smithi031.stdout:2/494: write d5/d6/f1a [6075116,123613] 0 2024-04-24T23:13:50.273 INFO:tasks.workunit.client.1.smithi040.stdout:9/314: mkdir d7/da/d17/d40/d55/d61/d6c 0 2024-04-24T23:13:50.274 INFO:tasks.workunit.client.1.smithi040.stdout:9/315: stat d7/da/d17/d40/d55/l5d 0 2024-04-24T23:13:50.274 INFO:tasks.workunit.client.0.smithi031.stdout:1/563: write d1/f66 [85617,18880] 0 2024-04-24T23:13:50.277 INFO:tasks.workunit.client.0.smithi031.stdout:9/692: rmdir d2/dae/d14/d78 39 2024-04-24T23:13:50.281 INFO:tasks.workunit.client.0.smithi031.stdout:4/733: write d3/d23/d69/d3f/f6d [1520968,100617] 0 2024-04-24T23:13:50.281 INFO:tasks.workunit.client.1.smithi040.stdout:6/290: truncate d7/d1a/d20/f3a 2431380 0 2024-04-24T23:13:50.282 INFO:tasks.workunit.client.1.smithi040.stdout:7/336: mknod db/d12/d62/c76 0 2024-04-24T23:13:50.283 INFO:tasks.workunit.client.1.smithi040.stdout:3/297: getdents d1/d2a 0 2024-04-24T23:13:50.284 INFO:tasks.workunit.client.0.smithi031.stdout:2/495: mkdir d5/d6/df/dad 0 2024-04-24T23:13:50.285 INFO:tasks.workunit.client.0.smithi031.stdout:2/496: dread - d5/d8/d15/d56/d84/fa5 zero size 2024-04-24T23:13:50.285 INFO:tasks.workunit.client.1.smithi040.stdout:4/249: link da/d1f/c4a da/d3f/d40/d47/c5b 0 2024-04-24T23:13:50.286 INFO:tasks.workunit.client.0.smithi031.stdout:2/497: write d5/d8/d45/f4b [3538032,47616] 0 2024-04-24T23:13:50.288 INFO:tasks.workunit.client.1.smithi040.stdout:7/337: mknod db/d12/d20/c77 0 2024-04-24T23:13:50.289 INFO:tasks.workunit.client.0.smithi031.stdout:9/693: fsync d2/dae/d14/d1d/d45/d56/f66 0 2024-04-24T23:13:50.290 INFO:tasks.workunit.client.0.smithi031.stdout:9/694: stat d2/dae/d14/d1d/f28 0 2024-04-24T23:13:50.293 INFO:tasks.workunit.client.1.smithi040.stdout:1/286: sync 2024-04-24T23:13:50.321 INFO:tasks.workunit.client.1.smithi040.stdout:3/298: symlink d1/d2a/d6a/l6e 0 2024-04-24T23:13:50.341 INFO:tasks.workunit.client.0.smithi031.stdout:3/546: sync 2024-04-24T23:13:50.341 INFO:tasks.workunit.client.0.smithi031.stdout:7/601: sync 2024-04-24T23:13:50.341 INFO:tasks.workunit.client.0.smithi031.stdout:6/699: sync 2024-04-24T23:13:50.344 INFO:tasks.workunit.client.0.smithi031.stdout:7/602: write d1/d2/dd/d33/d5e/d64/d6d/dae/db1/fc6 [550138,31513] 0 2024-04-24T23:13:50.350 INFO:tasks.workunit.client.1.smithi040.stdout:8/261: sync 2024-04-24T23:13:50.350 INFO:tasks.workunit.client.1.smithi040.stdout:2/351: sync 2024-04-24T23:13:50.350 INFO:tasks.workunit.client.1.smithi040.stdout:0/336: sync 2024-04-24T23:13:50.350 INFO:tasks.workunit.client.1.smithi040.stdout:5/253: sync 2024-04-24T23:13:50.369 INFO:tasks.workunit.client.1.smithi040.stdout:4/250: fsync da/d1d/d2b/f55 0 2024-04-24T23:13:50.373 INFO:tasks.workunit.client.0.smithi031.stdout:1/564: creat d1/d3e/da2/fb0 x:0 0 0 2024-04-24T23:13:50.382 INFO:tasks.workunit.client.0.smithi031.stdout:2/498: dwrite d5/d6/d29/f12 [0,4194304] 0 2024-04-24T23:13:50.386 INFO:tasks.workunit.client.0.smithi031.stdout:9/695: unlink d2/dae/d14/d1d/d45/f73 0 2024-04-24T23:13:50.394 INFO:tasks.workunit.client.1.smithi040.stdout:8/262: symlink d0/d16/d1b/d3f/l59 0 2024-04-24T23:13:50.396 INFO:tasks.workunit.client.0.smithi031.stdout:6/700: creat d13/d1b/d32/d9c/fed x:0 0 0 2024-04-24T23:13:50.401 INFO:tasks.workunit.client.1.smithi040.stdout:9/316: creat d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d x:0 0 0 2024-04-24T23:13:50.402 INFO:tasks.workunit.client.1.smithi040.stdout:9/317: readlink d7/da/d17/d40/l4f 0 2024-04-24T23:13:50.403 INFO:tasks.workunit.client.0.smithi031.stdout:0/663: mkdir da/d86/da3/dcd/ddc 0 2024-04-24T23:13:50.403 INFO:tasks.workunit.client.0.smithi031.stdout:0/664: stat da/f99 0 2024-04-24T23:13:50.410 INFO:tasks.workunit.client.0.smithi031.stdout:5/555: getdents d0/d6/d3b/d4b/dc/d86 0 2024-04-24T23:13:50.412 INFO:tasks.workunit.client.1.smithi040.stdout:9/318: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [0,4194304] 0 2024-04-24T23:13:50.420 INFO:tasks.workunit.client.1.smithi040.stdout:1/287: symlink da/d12/d14/d58/l63 0 2024-04-24T23:13:50.421 INFO:tasks.workunit.client.0.smithi031.stdout:2/499: truncate d5/d8/f98 832431 0 2024-04-24T23:13:50.421 INFO:tasks.workunit.client.1.smithi040.stdout:1/288: write da/f1f [948786,105235] 0 2024-04-24T23:13:50.423 INFO:tasks.workunit.client.1.smithi040.stdout:0/337: creat df/d12/d4b/d62/f65 x:0 0 0 2024-04-24T23:13:50.426 INFO:tasks.workunit.client.1.smithi040.stdout:2/352: getdents da/d1c/d7e 0 2024-04-24T23:13:50.429 INFO:tasks.workunit.client.0.smithi031.stdout:2/500: dwrite d5/d6/f16 [4194304,4194304] 0 2024-04-24T23:13:50.431 INFO:tasks.workunit.client.0.smithi031.stdout:2/501: fdatasync d5/d8/d45/d5b/d49/f9f 0 2024-04-24T23:13:50.441 INFO:tasks.workunit.client.0.smithi031.stdout:5/556: truncate d0/d6/d12/f6f 1587202 0 2024-04-24T23:13:50.453 INFO:tasks.workunit.client.1.smithi040.stdout:0/338: truncate df/d14/d1b/f36 1623422 0 2024-04-24T23:13:50.455 INFO:tasks.workunit.client.0.smithi031.stdout:2/502: fdatasync d5/d6/d68/f6b 0 2024-04-24T23:13:50.455 INFO:tasks.workunit.client.0.smithi031.stdout:2/503: fdatasync d5/d8/f10 0 2024-04-24T23:13:50.458 INFO:tasks.workunit.client.1.smithi040.stdout:2/353: creat da/d1c/d37/d39/d3f/d52/f87 x:0 0 0 2024-04-24T23:13:50.461 INFO:tasks.workunit.client.1.smithi040.stdout:8/263: symlink d0/d16/d1b/d3f/d44/l5a 0 2024-04-24T23:13:50.462 INFO:tasks.workunit.client.1.smithi040.stdout:8/264: readlink d0/d8/l9 0 2024-04-24T23:13:50.463 INFO:tasks.workunit.client.1.smithi040.stdout:5/254: rmdir d2/d7/d8/d1f/d57/d27/d3c/d52 0 2024-04-24T23:13:50.467 INFO:tasks.workunit.client.1.smithi040.stdout:5/255: dread d2/d7/d8/d1f/d57/d59/f5a [0,4194304] 0 2024-04-24T23:13:50.468 INFO:tasks.workunit.client.1.smithi040.stdout:1/289: mknod da/d12/d18/d2a/d41/d4f/c64 0 2024-04-24T23:13:50.469 INFO:tasks.workunit.client.1.smithi040.stdout:1/290: write f2 [1945107,93589] 0 2024-04-24T23:13:50.472 INFO:tasks.workunit.client.1.smithi040.stdout:5/256: dread d2/d7/d8/f13 [0,4194304] 0 2024-04-24T23:13:50.473 INFO:tasks.workunit.client.1.smithi040.stdout:2/354: creat da/d1c/d37/d39/d3f/f88 x:0 0 0 2024-04-24T23:13:50.478 INFO:tasks.workunit.client.1.smithi040.stdout:4/251: getdents da 0 2024-04-24T23:13:50.480 INFO:tasks.workunit.client.0.smithi031.stdout:5/557: getdents d0/d6/d3b/d4b/dc/d25 0 2024-04-24T23:13:50.480 INFO:tasks.workunit.client.0.smithi031.stdout:2/504: link d5/d6/d29/l78 d5/d8/d45/d5b/lae 0 2024-04-24T23:13:50.485 INFO:tasks.workunit.client.0.smithi031.stdout:2/505: symlink d5/d6/d29/laf 0 2024-04-24T23:13:50.486 INFO:tasks.workunit.client.1.smithi040.stdout:0/339: mknod df/d1e/d20/c66 0 2024-04-24T23:13:50.488 INFO:tasks.workunit.client.1.smithi040.stdout:0/340: dread df/d1e/d20/d21/f32 [0,4194304] 0 2024-04-24T23:13:50.489 INFO:tasks.workunit.client.0.smithi031.stdout:8/715: sync 2024-04-24T23:13:50.491 INFO:tasks.workunit.client.1.smithi040.stdout:5/257: symlink d2/d7/d8/d1f/d57/d27/d3c/l5c 0 2024-04-24T23:13:50.496 INFO:tasks.workunit.client.0.smithi031.stdout:8/716: dwrite d8/d1a/d1d/d50/dc0/d9d/fbe [0,4194304] 0 2024-04-24T23:13:50.506 INFO:tasks.workunit.client.1.smithi040.stdout:1/291: symlink da/d12/d55/l65 0 2024-04-24T23:13:50.506 INFO:tasks.workunit.client.1.smithi040.stdout:1/292: stat da/cb 0 2024-04-24T23:13:50.508 INFO:tasks.workunit.client.1.smithi040.stdout:0/341: readlink lc 0 2024-04-24T23:13:50.511 INFO:tasks.workunit.client.0.smithi031.stdout:2/506: getdents d5/d8/d15/d65 0 2024-04-24T23:13:50.512 INFO:tasks.workunit.client.1.smithi040.stdout:2/355: mknod da/d1c/d37/d39/c89 0 2024-04-24T23:13:50.514 INFO:tasks.workunit.client.0.smithi031.stdout:8/717: mknod d8/d15/d37/d43/d5c/df4/cf7 0 2024-04-24T23:13:50.515 INFO:tasks.workunit.client.0.smithi031.stdout:8/718: truncate d8/d15/d37/d62/f7f 5123417 0 2024-04-24T23:13:50.517 INFO:tasks.workunit.client.0.smithi031.stdout:8/719: read - d8/d15/d37/d43/d7b/d87/fa0 zero size 2024-04-24T23:13:50.517 INFO:tasks.workunit.client.0.smithi031.stdout:2/507: mkdir d5/d6/d29/d26/db0 0 2024-04-24T23:13:50.519 INFO:tasks.workunit.client.1.smithi040.stdout:2/356: dwrite da/d1c/d37/d39/d3f/d52/f87 [0,4194304] 0 2024-04-24T23:13:50.520 INFO:tasks.workunit.client.1.smithi040.stdout:0/342: chown df/d12/d49/l56 84 1 2024-04-24T23:13:50.523 INFO:tasks.workunit.client.1.smithi040.stdout:5/258: symlink d2/d7/d8/d1f/d57/d3e/d48/l5d 0 2024-04-24T23:13:50.523 INFO:tasks.workunit.client.0.smithi031.stdout:8/720: dread - d8/d15/d37/d62/fbb zero size 2024-04-24T23:13:50.524 INFO:tasks.workunit.client.1.smithi040.stdout:5/259: dread - d2/d7/d8/d1f/d57/d3e/f58 zero size 2024-04-24T23:13:50.524 INFO:tasks.workunit.client.1.smithi040.stdout:2/357: dread da/d10/d15/f2c [0,4194304] 0 2024-04-24T23:13:50.527 INFO:tasks.workunit.client.1.smithi040.stdout:1/293: link da/d12/d23/f45 da/d31/f66 0 2024-04-24T23:13:50.531 INFO:tasks.workunit.client.0.smithi031.stdout:8/721: mknod d8/d15/d37/d43/d7b/dc8/cf8 0 2024-04-24T23:13:50.532 INFO:tasks.workunit.client.1.smithi040.stdout:2/358: dwrite da/d29/f7f [0,4194304] 0 2024-04-24T23:13:50.538 INFO:tasks.workunit.client.1.smithi040.stdout:2/359: mkdir da/d1c/d37/d8a 0 2024-04-24T23:13:50.539 INFO:tasks.workunit.client.0.smithi031.stdout:2/508: dread d5/d28/f3a [0,4194304] 0 2024-04-24T23:13:50.541 INFO:tasks.workunit.client.0.smithi031.stdout:8/722: getdents d8/d15/d37/d43/dbd/dc4 0 2024-04-24T23:13:50.545 INFO:tasks.workunit.client.1.smithi040.stdout:6/291: chown d7/d1a/d20/d51/d1e/f49 7 1 2024-04-24T23:13:50.550 INFO:tasks.workunit.client.1.smithi040.stdout:4/252: dwrite da/d1d/d2b/f55 [0,4194304] 0 2024-04-24T23:13:50.554 INFO:tasks.workunit.client.1.smithi040.stdout:5/260: dread d2/d7/d8/d1f/d57/f44 [0,4194304] 0 2024-04-24T23:13:50.555 INFO:tasks.workunit.client.1.smithi040.stdout:5/261: fsync d2/d7/d8/db/d1e/f55 0 2024-04-24T23:13:50.556 INFO:tasks.workunit.client.1.smithi040.stdout:5/262: write d2/d7/fe [1835238,93258] 0 2024-04-24T23:13:50.557 INFO:tasks.workunit.client.1.smithi040.stdout:5/263: dread - d2/d7/d8/db/d1e/f55 zero size 2024-04-24T23:13:50.558 INFO:tasks.workunit.client.1.smithi040.stdout:5/264: chown d2/d7/d8/d1f/d57/d59 279609 1 2024-04-24T23:13:50.559 INFO:tasks.workunit.client.1.smithi040.stdout:5/265: chown d2/d7/d8/d1f/d57/d59/f5a 1 1 2024-04-24T23:13:50.565 INFO:tasks.workunit.client.0.smithi031.stdout:8/723: dread d8/d15/d37/d43/d5c/de1/f53 [0,4194304] 0 2024-04-24T23:13:50.566 INFO:tasks.workunit.client.0.smithi031.stdout:8/724: chown d8/d1a/d1d/d35/d58/d5b/fe0 218109 1 2024-04-24T23:13:50.592 INFO:tasks.workunit.client.0.smithi031.stdout:7/603: sync 2024-04-24T23:13:50.593 INFO:tasks.workunit.client.0.smithi031.stdout:7/604: chown d1/d2/dd/f15 110342 1 2024-04-24T23:13:50.594 INFO:tasks.workunit.client.1.smithi040.stdout:0/343: dread df/f39 [0,4194304] 0 2024-04-24T23:13:50.601 INFO:tasks.workunit.client.1.smithi040.stdout:7/338: write db/d12/f16 [3141871,52523] 0 2024-04-24T23:13:50.602 INFO:tasks.workunit.client.0.smithi031.stdout:7/605: creat d1/d45/d84/fcd x:0 0 0 2024-04-24T23:13:50.602 INFO:tasks.workunit.client.1.smithi040.stdout:4/253: creat da/d1d/d3c/f5c x:0 0 0 2024-04-24T23:13:50.605 INFO:tasks.workunit.client.1.smithi040.stdout:5/266: mknod d2/d7/d8/d1f/d57/d27/d3c/d43/c5e 0 2024-04-24T23:13:50.609 INFO:tasks.workunit.client.1.smithi040.stdout:8/265: sync 2024-04-24T23:13:50.609 INFO:tasks.workunit.client.1.smithi040.stdout:4/254: dwrite da/d1d/d3c/f5c [0,4194304] 0 2024-04-24T23:13:50.609 INFO:tasks.workunit.client.1.smithi040.stdout:8/266: chown d0/d52 74217 1 2024-04-24T23:13:50.610 INFO:tasks.workunit.client.1.smithi040.stdout:4/255: stat da/d45 0 2024-04-24T23:13:50.611 INFO:tasks.workunit.client.1.smithi040.stdout:8/267: write d0/d16/d1b/f54 [770865,125230] 0 2024-04-24T23:13:50.616 INFO:tasks.workunit.client.1.smithi040.stdout:5/267: mkdir d2/d7/d8/d1f/d57/d27/d3d/d5f 0 2024-04-24T23:13:50.619 INFO:tasks.workunit.client.1.smithi040.stdout:4/256: mkdir da/d3f/d40/d47/d5d 0 2024-04-24T23:13:50.619 INFO:tasks.workunit.client.1.smithi040.stdout:5/268: fdatasync d2/d7/d8/d1f/d31/f4e 0 2024-04-24T23:13:50.621 INFO:tasks.workunit.client.1.smithi040.stdout:8/268: dwrite d0/d16/d1b/f54 [0,4194304] 0 2024-04-24T23:13:50.623 INFO:tasks.workunit.client.1.smithi040.stdout:5/269: symlink d2/d7/d8/db/d1e/l60 0 2024-04-24T23:13:50.624 INFO:tasks.workunit.client.1.smithi040.stdout:5/270: chown d2/d7/d8 19232689 1 2024-04-24T23:13:50.628 INFO:tasks.workunit.client.0.smithi031.stdout:4/734: write d3/d23/d69/d3f/fd9 [1598783,92592] 0 2024-04-24T23:13:50.629 INFO:tasks.workunit.client.1.smithi040.stdout:5/271: dread d2/d7/d8/db/d1e/f2e [0,4194304] 0 2024-04-24T23:13:50.630 INFO:tasks.workunit.client.1.smithi040.stdout:5/272: read d2/d7/d8/d1f/d57/f44 [309457,7800] 0 2024-04-24T23:13:50.631 INFO:tasks.workunit.client.0.smithi031.stdout:4/735: write d3/d20/d25/d60/d63/fbf [727870,116562] 0 2024-04-24T23:13:50.632 INFO:tasks.workunit.client.0.smithi031.stdout:4/736: chown d3/d23/d69/d3f/d9e/ce0 382069879 1 2024-04-24T23:13:50.639 INFO:tasks.workunit.client.1.smithi040.stdout:0/344: rmdir df 39 2024-04-24T23:13:50.641 INFO:tasks.workunit.client.1.smithi040.stdout:0/345: creat df/d12/d1a/f67 x:0 0 0 2024-04-24T23:13:50.652 INFO:tasks.workunit.client.1.smithi040.stdout:9/319: dwrite d7/da/d17/d40/d55/d2f/f46 [0,4194304] 0 2024-04-24T23:13:50.657 INFO:tasks.workunit.client.1.smithi040.stdout:3/299: rename d1/c4 to d1/d2a/c6f 0 2024-04-24T23:13:50.657 INFO:tasks.workunit.client.1.smithi040.stdout:9/320: dwrite d7/da/f39 [0,4194304] 0 2024-04-24T23:13:50.677 INFO:tasks.workunit.client.0.smithi031.stdout:5/558: truncate d0/d6/d12/f6f 530377 0 2024-04-24T23:13:50.678 INFO:tasks.workunit.client.0.smithi031.stdout:1/565: truncate d1/d31/f39 3970413 0 2024-04-24T23:13:50.679 INFO:tasks.workunit.client.0.smithi031.stdout:9/696: truncate d2/dae/f8f 2019280 0 2024-04-24T23:13:50.681 INFO:tasks.workunit.client.0.smithi031.stdout:0/665: rename da/f2b to da/d11/fdd 0 2024-04-24T23:13:50.681 INFO:tasks.workunit.client.0.smithi031.stdout:1/566: read - d1/d3e/da2/fb0 zero size 2024-04-24T23:13:50.686 INFO:tasks.workunit.client.0.smithi031.stdout:5/559: fsync d0/d6/d3b/d4b/dc/f84 0 2024-04-24T23:13:50.688 INFO:tasks.workunit.client.0.smithi031.stdout:4/737: getdents d3/d23/d69/d3f 0 2024-04-24T23:13:50.689 INFO:tasks.workunit.client.0.smithi031.stdout:1/567: rmdir d1/da/d10/d52 39 2024-04-24T23:13:50.689 INFO:tasks.workunit.client.0.smithi031.stdout:0/666: read da/d18/d1a/dd6/d82/fc7 [7860976,26942] 0 2024-04-24T23:13:50.690 INFO:tasks.workunit.client.0.smithi031.stdout:3/547: dwrite d6/df/d16/d2a/d95/d9b/d1a/d25/f54 [0,4194304] 0 2024-04-24T23:13:50.690 INFO:tasks.workunit.client.0.smithi031.stdout:6/701: dwrite d13/d15/d1d/d23/d24/f50 [0,4194304] 0 2024-04-24T23:13:50.690 INFO:tasks.workunit.client.0.smithi031.stdout:9/697: rename d2/l37 to d2/dae/d14/d1d/d45/d99/ld3 0 2024-04-24T23:13:50.691 INFO:tasks.workunit.client.0.smithi031.stdout:5/560: stat d0/d7/c8f 0 2024-04-24T23:13:50.692 INFO:tasks.workunit.client.0.smithi031.stdout:4/738: stat d3/da3/lcf 0 2024-04-24T23:13:50.696 INFO:tasks.workunit.client.0.smithi031.stdout:1/568: mknod d1/cb1 0 2024-04-24T23:13:50.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/561: chown d0/d6/d32/d6b/f7b 1112197 1 2024-04-24T23:13:50.700 INFO:tasks.workunit.client.0.smithi031.stdout:5/562: chown d0/d6/d32/f39 45083 1 2024-04-24T23:13:50.702 INFO:tasks.workunit.client.0.smithi031.stdout:8/725: dwrite d8/d1a/d1d/d35/f8f [0,4194304] 0 2024-04-24T23:13:50.702 INFO:tasks.workunit.client.0.smithi031.stdout:4/739: read d3/d23/d3b/f56 [1535296,114624] 0 2024-04-24T23:13:50.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/606: dwrite d1/d2/dd/d33/d5e/d64/f91 [0,4194304] 0 2024-04-24T23:13:50.707 INFO:tasks.workunit.client.0.smithi031.stdout:4/740: fdatasync d3/d20/d2d/fd2 0 2024-04-24T23:13:50.707 INFO:tasks.workunit.client.0.smithi031.stdout:6/702: rmdir d13/d15/d1d/d23/d44/d47 39 2024-04-24T23:13:50.709 INFO:tasks.workunit.client.0.smithi031.stdout:0/667: symlink da/d86/da3/dcd/ddc/lde 0 2024-04-24T23:13:50.711 INFO:tasks.workunit.client.0.smithi031.stdout:8/726: unlink d8/d15/d37/d43/daf/lce 0 2024-04-24T23:13:50.713 INFO:tasks.workunit.client.0.smithi031.stdout:7/607: creat d1/d2/dd/d33/d5e/d64/fce x:0 0 0 2024-04-24T23:13:50.715 INFO:tasks.workunit.client.0.smithi031.stdout:6/703: dread d13/d15/d1d/d23/d24/f50 [0,4194304] 0 2024-04-24T23:13:50.716 INFO:tasks.workunit.client.0.smithi031.stdout:6/704: fsync d13/f3f 0 2024-04-24T23:13:50.717 INFO:tasks.workunit.client.0.smithi031.stdout:0/668: symlink da/d18/d1a/db7/ldf 0 2024-04-24T23:13:50.718 INFO:tasks.workunit.client.0.smithi031.stdout:8/727: fsync d8/d15/d37/d43/d5c/de1/f53 0 2024-04-24T23:13:50.719 INFO:tasks.workunit.client.1.smithi040.stdout:1/294: rename da/d12/d18/f22 to da/d12/d18/d2a/d32/f67 0 2024-04-24T23:13:50.719 INFO:tasks.workunit.client.1.smithi040.stdout:7/339: mknod db/c78 0 2024-04-24T23:13:50.719 INFO:tasks.workunit.client.0.smithi031.stdout:9/698: symlink d2/dae/d14/d78/d84/d4a/dab/db7/ld4 0 2024-04-24T23:13:50.720 INFO:tasks.workunit.client.1.smithi040.stdout:6/292: write d7/d1a/d20/f47 [4490442,107768] 0 2024-04-24T23:13:50.722 INFO:tasks.workunit.client.1.smithi040.stdout:5/273: write d2/d7/d8/db/d1e/f2e [4485521,55555] 0 2024-04-24T23:13:50.722 INFO:tasks.workunit.client.1.smithi040.stdout:6/293: chown d7/d1a/d20/d51/l30 0 1 2024-04-24T23:13:50.723 INFO:tasks.workunit.client.1.smithi040.stdout:3/300: creat d1/d2d/de/f70 x:0 0 0 2024-04-24T23:13:50.724 INFO:tasks.workunit.client.1.smithi040.stdout:2/360: rename da/d10/d15/l3b to da/d1c/d37/d39/d3f/l8b 0 2024-04-24T23:13:50.730 INFO:tasks.workunit.client.1.smithi040.stdout:5/274: chown d2/d7/d8/d1f/d57/d27/d3c/d43/c5e 1727669477 1 2024-04-24T23:13:50.732 INFO:tasks.workunit.client.0.smithi031.stdout:0/669: mknod da/d86/da3/dcd/ddc/ce0 0 2024-04-24T23:13:50.732 INFO:tasks.workunit.client.1.smithi040.stdout:4/257: dwrite da/d1d/d2b/f39 [0,4194304] 0 2024-04-24T23:13:50.734 INFO:tasks.workunit.client.0.smithi031.stdout:0/670: write da/d18/f9f [830564,40760] 0 2024-04-24T23:13:50.735 INFO:tasks.workunit.client.1.smithi040.stdout:5/275: read d2/d7/d8/f1b [528979,85461] 0 2024-04-24T23:13:50.737 INFO:tasks.workunit.client.0.smithi031.stdout:0/671: truncate da/d20/d8e/fd4 253992 0 2024-04-24T23:13:50.737 INFO:tasks.workunit.client.1.smithi040.stdout:6/294: dwrite d7/d1a/d20/d51/f4b [0,4194304] 0 2024-04-24T23:13:50.738 INFO:tasks.workunit.client.0.smithi031.stdout:4/741: dread d3/f5c [0,4194304] 0 2024-04-24T23:13:50.739 INFO:tasks.workunit.client.0.smithi031.stdout:9/699: fdatasync d2/d43/f58 0 2024-04-24T23:13:50.741 INFO:tasks.workunit.client.0.smithi031.stdout:9/700: fsync d2/d16/f20 0 2024-04-24T23:13:50.742 INFO:tasks.workunit.client.1.smithi040.stdout:2/361: rename da/d1c/c33 to da/d1c/d37/d39/d3f/d52/d78/c8c 0 2024-04-24T23:13:50.744 INFO:tasks.workunit.client.0.smithi031.stdout:8/728: symlink d8/d15/d37/d43/d7b/d86/lf9 0 2024-04-24T23:13:50.746 INFO:tasks.workunit.client.0.smithi031.stdout:5/563: link d0/d6/d3b/d4b/d1d/c2f d0/d6/d3b/d4b/d1d/d2e/d28/cbe 0 2024-04-24T23:13:50.746 INFO:tasks.workunit.client.1.smithi040.stdout:6/295: dwrite f2 [4194304,4194304] 0 2024-04-24T23:13:50.747 INFO:tasks.workunit.client.1.smithi040.stdout:6/296: write d7/d1a/d20/f28 [492322,92676] 0 2024-04-24T23:13:50.750 INFO:tasks.workunit.client.1.smithi040.stdout:6/297: chown d7/d1a/d20/d51/d1e/f3d 23 1 2024-04-24T23:13:50.752 INFO:tasks.workunit.client.1.smithi040.stdout:7/340: creat db/d12/d20/d52/f79 x:0 0 0 2024-04-24T23:13:50.760 INFO:tasks.workunit.client.0.smithi031.stdout:3/548: dwrite d6/fc [0,4194304] 0 2024-04-24T23:13:50.760 INFO:tasks.workunit.client.1.smithi040.stdout:4/258: rename da/c14 to da/d1f/c5e 0 2024-04-24T23:13:50.762 INFO:tasks.workunit.client.0.smithi031.stdout:5/564: dread d0/d6/d3b/d4b/dc/d25/f69 [0,4194304] 0 2024-04-24T23:13:50.763 INFO:tasks.workunit.client.1.smithi040.stdout:8/269: sync 2024-04-24T23:13:50.764 INFO:tasks.workunit.client.1.smithi040.stdout:8/270: readlink d0/d16/l25 0 2024-04-24T23:13:50.764 INFO:tasks.workunit.client.0.smithi031.stdout:9/701: mkdir d2/d43/d9d/dc0/dd5 0 2024-04-24T23:13:50.766 INFO:tasks.workunit.client.0.smithi031.stdout:8/729: readlink d8/l3f 0 2024-04-24T23:13:50.767 INFO:tasks.workunit.client.0.smithi031.stdout:5/565: read d0/d6/d3b/f68 [3466912,84092] 0 2024-04-24T23:13:50.768 INFO:tasks.workunit.client.0.smithi031.stdout:5/566: stat d0/d6/d3b/d4b/dc/d8b/d9b 0 2024-04-24T23:13:50.771 INFO:tasks.workunit.client.1.smithi040.stdout:7/341: mkdir db/d12/d3f/d7a 0 2024-04-24T23:13:50.774 INFO:tasks.workunit.client.0.smithi031.stdout:7/608: getdents d1/d2/dd/d33 0 2024-04-24T23:13:50.777 INFO:tasks.workunit.client.1.smithi040.stdout:1/295: write da/d31/f66 [18507,33523] 0 2024-04-24T23:13:50.779 INFO:tasks.workunit.client.1.smithi040.stdout:2/362: creat da/d10/d7a/f8d x:0 0 0 2024-04-24T23:13:50.780 INFO:tasks.workunit.client.1.smithi040.stdout:8/271: chown d0/l13 53661 1 2024-04-24T23:13:50.782 INFO:tasks.workunit.client.1.smithi040.stdout:6/298: truncate d7/d1a/d20/d51/d1e/f49 1109113 0 2024-04-24T23:13:50.786 INFO:tasks.workunit.client.0.smithi031.stdout:6/705: write d13/d15/d1d/d23/d44/d47/d69/fd4 [111172,43863] 0 2024-04-24T23:13:50.787 INFO:tasks.workunit.client.0.smithi031.stdout:6/706: chown d13/d1b/d32/fc3 11 1 2024-04-24T23:13:50.787 INFO:tasks.workunit.client.1.smithi040.stdout:1/296: creat da/d12/d14/f68 x:0 0 0 2024-04-24T23:13:50.789 INFO:tasks.workunit.client.1.smithi040.stdout:2/363: creat da/d1c/d37/f8e x:0 0 0 2024-04-24T23:13:50.789 INFO:tasks.workunit.client.1.smithi040.stdout:6/299: dwrite d7/d1a/d20/f23 [0,4194304] 0 2024-04-24T23:13:50.790 INFO:tasks.workunit.client.0.smithi031.stdout:5/567: write d0/d6/d3b/f8d [1347792,33328] 0 2024-04-24T23:13:50.790 INFO:tasks.workunit.client.1.smithi040.stdout:2/364: stat da/d10/l1f 0 2024-04-24T23:13:50.791 INFO:tasks.workunit.client.1.smithi040.stdout:8/272: read d0/d8/f29 [408933,27809] 0 2024-04-24T23:13:50.792 INFO:tasks.workunit.client.0.smithi031.stdout:3/549: symlink d6/df/d16/d2a/d95/d9b/lac 0 2024-04-24T23:13:50.793 INFO:tasks.workunit.client.0.smithi031.stdout:4/742: link d3/d23/d3b/lb3 d3/d23/d69/d3f/dea/lef 0 2024-04-24T23:13:50.793 INFO:tasks.workunit.client.0.smithi031.stdout:3/550: chown d6/df/d16/d2a/caa 2064918 1 2024-04-24T23:13:50.794 INFO:tasks.workunit.client.0.smithi031.stdout:4/743: chown d3/d23/d69/d3f/d9e/ce0 4450 1 2024-04-24T23:13:50.795 INFO:tasks.workunit.client.1.smithi040.stdout:1/297: mknod da/d12/d18/d2a/d41/d4f/c69 0 2024-04-24T23:13:50.797 INFO:tasks.workunit.client.1.smithi040.stdout:2/365: symlink da/d1c/d4f/l8f 0 2024-04-24T23:13:50.798 INFO:tasks.workunit.client.1.smithi040.stdout:8/273: rename d0/d8/l9 to d0/d28/l5b 0 2024-04-24T23:13:50.798 INFO:tasks.workunit.client.0.smithi031.stdout:6/707: mknod d13/d15/d1d/d23/d24/d43/cee 0 2024-04-24T23:13:50.799 INFO:tasks.workunit.client.1.smithi040.stdout:1/298: creat da/d12/d18/d2a/d41/f6a x:0 0 0 2024-04-24T23:13:50.800 INFO:tasks.workunit.client.0.smithi031.stdout:5/568: truncate d0/d6/d32/f17 2362590 0 2024-04-24T23:13:50.801 INFO:tasks.workunit.client.0.smithi031.stdout:5/569: write d0/d6/d3b/d4b/d1d/fb2 [577502,115650] 0 2024-04-24T23:13:50.802 INFO:tasks.workunit.client.0.smithi031.stdout:4/744: truncate d3/d23/d3b/f41 838417 0 2024-04-24T23:13:50.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/299: rename da/d12/d18/d2a/d32/f67 to da/d12/d55/f6b 0 2024-04-24T23:13:50.805 INFO:tasks.workunit.client.0.smithi031.stdout:3/551: mkdir d6/d14/d71/dad 0 2024-04-24T23:13:50.806 INFO:tasks.workunit.client.0.smithi031.stdout:5/570: rmdir d0/d7/d56/d7c/d7e 39 2024-04-24T23:13:50.807 INFO:tasks.workunit.client.1.smithi040.stdout:2/366: truncate da/f72 1605843 0 2024-04-24T23:13:50.808 INFO:tasks.workunit.client.0.smithi031.stdout:4/745: fsync d3/d23/d69/f31 0 2024-04-24T23:13:50.809 INFO:tasks.workunit.client.1.smithi040.stdout:8/274: symlink d0/d16/d1b/d47/l5c 0 2024-04-24T23:13:50.810 INFO:tasks.workunit.client.1.smithi040.stdout:1/300: unlink da/l13 0 2024-04-24T23:13:50.810 INFO:tasks.workunit.client.1.smithi040.stdout:1/301: chown da/d12/d14/f24 2 1 2024-04-24T23:13:50.811 INFO:tasks.workunit.client.0.smithi031.stdout:5/571: readlink d0/d7/d56/d7c/d7e/l66 0 2024-04-24T23:13:50.814 INFO:tasks.workunit.client.0.smithi031.stdout:6/708: creat d13/d15/fef x:0 0 0 2024-04-24T23:13:50.818 INFO:tasks.workunit.client.1.smithi040.stdout:1/302: getdents da/d31 0 2024-04-24T23:13:50.820 INFO:tasks.workunit.client.1.smithi040.stdout:1/303: creat da/d12/d18/f6c x:0 0 0 2024-04-24T23:13:50.820 INFO:tasks.workunit.client.0.smithi031.stdout:4/746: dread d3/d23/d69/f31 [0,4194304] 0 2024-04-24T23:13:50.830 INFO:tasks.workunit.client.1.smithi040.stdout:1/304: dread da/d35/d38/f3e [0,4194304] 0 2024-04-24T23:13:50.833 INFO:tasks.workunit.client.1.smithi040.stdout:1/305: unlink da/d12/d18/d2a/d41/f50 0 2024-04-24T23:13:50.834 INFO:tasks.workunit.client.1.smithi040.stdout:1/306: mknod da/d35/d38/c6d 0 2024-04-24T23:13:50.834 INFO:tasks.workunit.client.1.smithi040.stdout:1/307: readlink da/d12/d18/l29 0 2024-04-24T23:13:50.836 INFO:tasks.workunit.client.1.smithi040.stdout:1/308: creat da/d35/d38/f6e x:0 0 0 2024-04-24T23:13:50.838 INFO:tasks.workunit.client.1.smithi040.stdout:1/309: creat da/d12/d14/d25/f6f x:0 0 0 2024-04-24T23:13:50.839 INFO:tasks.workunit.client.1.smithi040.stdout:1/310: creat da/d12/d18/d2a/d41/f70 x:0 0 0 2024-04-24T23:13:50.861 INFO:tasks.workunit.client.1.smithi040.stdout:5/276: truncate d2/d7/d8/db/d1e/f2e 451816 0 2024-04-24T23:13:50.870 INFO:tasks.workunit.client.0.smithi031.stdout:9/702: dwrite d2/dae/d85/f9b [0,4194304] 0 2024-04-24T23:13:50.870 INFO:tasks.workunit.client.0.smithi031.stdout:8/730: dwrite d8/d15/d37/f2d [0,4194304] 0 2024-04-24T23:13:50.872 INFO:tasks.workunit.client.1.smithi040.stdout:7/342: dwrite db/fd [0,4194304] 0 2024-04-24T23:13:50.874 INFO:tasks.workunit.client.1.smithi040.stdout:7/343: truncate db/d43/f5e 1970964 0 2024-04-24T23:13:50.878 INFO:tasks.workunit.client.1.smithi040.stdout:5/277: dread d2/d7/d8/d1f/f38 [0,4194304] 0 2024-04-24T23:13:50.892 INFO:tasks.workunit.client.1.smithi040.stdout:0/346: sync 2024-04-24T23:13:50.893 INFO:tasks.workunit.client.0.smithi031.stdout:9/703: creat d2/dae/d14/d78/d84/d7e/dc7/fd6 x:0 0 0 2024-04-24T23:13:50.897 INFO:tasks.workunit.client.0.smithi031.stdout:7/609: dread d1/d2/f4 [0,4194304] 0 2024-04-24T23:13:50.898 INFO:tasks.workunit.client.1.smithi040.stdout:7/344: mknod db/d12/d20/d52/c7b 0 2024-04-24T23:13:50.899 INFO:tasks.workunit.client.1.smithi040.stdout:0/347: dwrite df/d1e/d2b/f29 [0,4194304] 0 2024-04-24T23:13:50.904 INFO:tasks.workunit.client.0.smithi031.stdout:8/731: mkdir d8/d15/d37/d62/dfa 0 2024-04-24T23:13:50.905 INFO:tasks.workunit.client.1.smithi040.stdout:7/345: rmdir db/d2f 39 2024-04-24T23:13:50.906 INFO:tasks.workunit.client.0.smithi031.stdout:7/610: dwrite d1/d2/dd/d33/d5e/d64/f91 [0,4194304] 0 2024-04-24T23:13:50.906 INFO:tasks.workunit.client.0.smithi031.stdout:8/732: stat d8/d15/d37/d43/d7b/d86 0 2024-04-24T23:13:50.906 INFO:tasks.workunit.client.1.smithi040.stdout:7/346: chown db/d12/d3f/f41 17934 1 2024-04-24T23:13:50.907 INFO:tasks.workunit.client.0.smithi031.stdout:8/733: truncate d8/fb 4704184 0 2024-04-24T23:13:50.908 INFO:tasks.workunit.client.0.smithi031.stdout:3/552: write f0 [272316,23150] 0 2024-04-24T23:13:50.910 INFO:tasks.workunit.client.0.smithi031.stdout:3/553: readlink d6/df/d16/d2a/d95/d9b/d29/l50 0 2024-04-24T23:13:50.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:50 smithi040 ceph-mon[33128]: pgmap v28: 65 pgs: 65 active+clean; 1.5 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 29 MiB/s rd, 139 MiB/s wr, 304 op/s 2024-04-24T23:13:50.918 INFO:tasks.workunit.client.0.smithi031.stdout:7/611: creat d1/d2/dd/d33/d5e/d64/d6d/da1/fcf x:0 0 0 2024-04-24T23:13:50.918 INFO:tasks.workunit.client.0.smithi031.stdout:7/612: chown d1/d2 3431955 1 2024-04-24T23:13:50.918 INFO:tasks.workunit.client.0.smithi031.stdout:4/747: write d3/d23/d69/d3f/f61 [3989217,30996] 0 2024-04-24T23:13:50.919 INFO:tasks.workunit.client.0.smithi031.stdout:8/734: mkdir d8/d15/d37/d43/d7b/d86/dfb 0 2024-04-24T23:13:50.919 INFO:tasks.workunit.client.1.smithi040.stdout:8/275: dwrite d0/d8/dc/f3a [0,4194304] 0 2024-04-24T23:13:50.919 INFO:tasks.workunit.client.1.smithi040.stdout:9/321: sync 2024-04-24T23:13:50.919 INFO:tasks.workunit.client.1.smithi040.stdout:3/301: sync 2024-04-24T23:13:50.920 INFO:tasks.workunit.client.0.smithi031.stdout:5/572: dwrite d0/d6/d12/f6f [0,4194304] 0 2024-04-24T23:13:50.920 INFO:tasks.workunit.client.1.smithi040.stdout:3/302: write d1/d2d/de/d30/f41 [70530,37614] 0 2024-04-24T23:13:50.922 INFO:tasks.workunit.client.1.smithi040.stdout:8/276: dread d0/d16/d1b/d47/f48 [0,4194304] 0 2024-04-24T23:13:50.925 INFO:tasks.workunit.client.1.smithi040.stdout:7/347: mknod db/d12/d20/d52/c7c 0 2024-04-24T23:13:50.926 INFO:tasks.workunit.client.0.smithi031.stdout:9/704: link d2/d16/l1e d2/dae/d14/d78/d84/da6/ld7 0 2024-04-24T23:13:50.929 INFO:tasks.workunit.client.0.smithi031.stdout:7/613: dwrite d1/d45/d75/d9f/d66/f94 [0,4194304] 0 2024-04-24T23:13:50.929 INFO:tasks.workunit.client.0.smithi031.stdout:4/748: dread d3/d23/d69/f13 [0,4194304] 0 2024-04-24T23:13:50.930 INFO:tasks.workunit.client.1.smithi040.stdout:1/311: truncate da/d12/d23/f30 2474325 0 2024-04-24T23:13:50.931 INFO:tasks.workunit.client.1.smithi040.stdout:1/312: fsync da/d12/d18/f27 0 2024-04-24T23:13:50.933 INFO:tasks.workunit.client.1.smithi040.stdout:1/313: truncate da/d12/d18/f6c 536584 0 2024-04-24T23:13:50.938 INFO:tasks.workunit.client.1.smithi040.stdout:4/259: truncate da/d3f/f5a 3641464 0 2024-04-24T23:13:50.940 INFO:tasks.workunit.client.1.smithi040.stdout:0/348: dread df/d12/d1a/f2e [0,4194304] 0 2024-04-24T23:13:50.940 INFO:tasks.workunit.client.1.smithi040.stdout:9/322: dread d7/da/d17/d40/d55/d2f/d1f/d34/f49 [0,4194304] 0 2024-04-24T23:13:50.942 INFO:tasks.workunit.client.1.smithi040.stdout:8/277: unlink d0/d16/c2e 0 2024-04-24T23:13:50.946 INFO:tasks.workunit.client.1.smithi040.stdout:7/348: chown db/d2f/c40 15 1 2024-04-24T23:13:50.947 INFO:tasks.workunit.client.1.smithi040.stdout:7/349: write db/d12/d20/d52/d4b/f4f [1735331,101093] 0 2024-04-24T23:13:50.950 INFO:tasks.workunit.client.0.smithi031.stdout:7/614: symlink d1/d45/da0/d4f/ld0 0 2024-04-24T23:13:50.951 INFO:tasks.workunit.client.1.smithi040.stdout:5/278: write d2/f6 [799344,111535] 0 2024-04-24T23:13:50.953 INFO:tasks.workunit.client.1.smithi040.stdout:3/303: getdents d1/d2d/de/d30/d43 0 2024-04-24T23:13:50.954 INFO:tasks.workunit.client.1.smithi040.stdout:2/367: dwrite da/f72 [0,4194304] 0 2024-04-24T23:13:50.955 INFO:tasks.workunit.client.1.smithi040.stdout:3/304: write d1/d9/d15/f1b [2001607,3293] 0 2024-04-24T23:13:50.960 INFO:tasks.workunit.client.1.smithi040.stdout:2/368: write da/d1c/f5c [1788212,65673] 0 2024-04-24T23:13:50.961 INFO:tasks.workunit.client.1.smithi040.stdout:0/349: mkdir df/d12/d58/d68 0 2024-04-24T23:13:50.961 INFO:tasks.workunit.client.1.smithi040.stdout:4/260: write da/d1d/d3c/f3e [279933,70675] 0 2024-04-24T23:13:50.964 INFO:tasks.workunit.client.1.smithi040.stdout:6/300: truncate d7/d1a/d20/f3a 745645 0 2024-04-24T23:13:50.964 INFO:tasks.workunit.client.1.smithi040.stdout:0/350: dread - df/d12/d1a/f67 zero size 2024-04-24T23:13:50.965 INFO:tasks.workunit.client.1.smithi040.stdout:5/279: dwrite d2/d7/d8/d1f/d57/d3e/f58 [0,4194304] 0 2024-04-24T23:13:50.965 INFO:tasks.workunit.client.0.smithi031.stdout:4/749: getdents d3/d5e/dc4 0 2024-04-24T23:13:50.967 INFO:tasks.workunit.client.1.smithi040.stdout:6/301: chown d7/d1a/d20/d51/d1e 9245055 1 2024-04-24T23:13:50.968 INFO:tasks.workunit.client.1.smithi040.stdout:5/280: truncate d2/d7/d8/d1f/f4b 1371010 0 2024-04-24T23:13:50.968 INFO:tasks.workunit.client.0.smithi031.stdout:7/615: rename d1/d2/d83/faf to d1/d45/d75/d9f/d66/fd1 0 2024-04-24T23:13:50.968 INFO:tasks.workunit.client.1.smithi040.stdout:6/302: read - d7/d1a/d20/d51/d2f/f5b zero size 2024-04-24T23:13:50.971 INFO:tasks.workunit.client.0.smithi031.stdout:8/735: creat d8/d15/d37/ffc x:0 0 0 2024-04-24T23:13:50.974 INFO:tasks.workunit.client.1.smithi040.stdout:6/303: read d7/d2c/f31 [2209306,5900] 0 2024-04-24T23:13:50.976 INFO:tasks.workunit.client.1.smithi040.stdout:8/278: dread d0/d28/f39 [0,4194304] 0 2024-04-24T23:13:50.976 INFO:tasks.workunit.client.1.smithi040.stdout:8/279: chown d0/d8/dc/d34/f3c 3 1 2024-04-24T23:13:50.980 INFO:tasks.workunit.client.0.smithi031.stdout:3/554: write d6/df/d16/d2a/d95/d9b/d60/f64 [796932,21073] 0 2024-04-24T23:13:50.982 INFO:tasks.workunit.client.0.smithi031.stdout:3/555: chown d6/df/d16/d2a/d95/d9b/d1a 95104234 1 2024-04-24T23:13:50.983 INFO:tasks.workunit.client.0.smithi031.stdout:5/573: dwrite d0/d6/d3b/d4b/dc/f23 [0,4194304] 0 2024-04-24T23:13:50.984 INFO:tasks.workunit.client.1.smithi040.stdout:2/369: dread da/d10/f2b [0,4194304] 0 2024-04-24T23:13:50.989 INFO:tasks.workunit.client.1.smithi040.stdout:7/350: mkdir db/d12/d7d 0 2024-04-24T23:13:50.992 INFO:tasks.workunit.client.1.smithi040.stdout:0/351: creat df/d12/d4b/d62/f69 x:0 0 0 2024-04-24T23:13:50.993 INFO:tasks.workunit.client.1.smithi040.stdout:0/352: write df/d12/f1f [3190876,80816] 0 2024-04-24T23:13:50.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:50 smithi031 ceph-mon[26411]: pgmap v28: 65 pgs: 65 active+clean; 1.5 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 29 MiB/s rd, 139 MiB/s wr, 304 op/s 2024-04-24T23:13:51.005 INFO:tasks.workunit.client.1.smithi040.stdout:3/305: mknod d1/d9/d39/c71 0 2024-04-24T23:13:51.010 INFO:tasks.workunit.client.1.smithi040.stdout:3/306: dwrite d1/d9/f27 [0,4194304] 0 2024-04-24T23:13:51.010 INFO:tasks.workunit.client.1.smithi040.stdout:3/307: dread - d1/d2d/d44/f52 zero size 2024-04-24T23:13:51.011 INFO:tasks.workunit.client.1.smithi040.stdout:9/323: rename d7/da/d17/d40/d55/d2f/c32 to d7/da/d17/d40/d55/d2f/d1f/d31/d4d/c6e 0 2024-04-24T23:13:51.018 INFO:tasks.workunit.client.1.smithi040.stdout:0/353: creat df/d12/d4b/f6a x:0 0 0 2024-04-24T23:13:51.025 INFO:tasks.workunit.client.0.smithi031.stdout:9/705: creat d2/dae/d14/d1d/d45/fd8 x:0 0 0 2024-04-24T23:13:51.027 INFO:tasks.workunit.client.1.smithi040.stdout:2/370: creat da/d1c/d37/d8a/f90 x:0 0 0 2024-04-24T23:13:51.028 INFO:tasks.workunit.client.0.smithi031.stdout:9/706: truncate d2/dae/d14/d78/d84/fd0 175840 0 2024-04-24T23:13:51.028 INFO:tasks.workunit.client.1.smithi040.stdout:2/371: stat da/d29 0 2024-04-24T23:13:51.030 INFO:tasks.workunit.client.1.smithi040.stdout:9/324: creat d7/da/d17/d40/d55/d2f/f6f x:0 0 0 2024-04-24T23:13:51.031 INFO:tasks.workunit.client.0.smithi031.stdout:8/736: creat d8/d15/d37/d43/ffd x:0 0 0 2024-04-24T23:13:51.034 INFO:tasks.workunit.client.0.smithi031.stdout:5/574: mknod d0/d6/cbf 0 2024-04-24T23:13:51.035 INFO:tasks.workunit.client.0.smithi031.stdout:8/737: dread d8/f70 [4194304,4194304] 0 2024-04-24T23:13:51.035 INFO:tasks.workunit.client.1.smithi040.stdout:1/314: sync 2024-04-24T23:13:51.036 INFO:tasks.workunit.client.0.smithi031.stdout:2/509: sync 2024-04-24T23:13:51.037 INFO:tasks.workunit.client.1.smithi040.stdout:0/354: mkdir df/d12/d4b/d62/d6b 0 2024-04-24T23:13:51.038 INFO:tasks.workunit.client.0.smithi031.stdout:9/707: rmdir d2/dae/d14/d1d/d45/d56 39 2024-04-24T23:13:51.042 INFO:tasks.workunit.client.0.smithi031.stdout:7/616: mkdir d1/d45/d75/d9f/dd2 0 2024-04-24T23:13:51.048 INFO:tasks.workunit.client.1.smithi040.stdout:2/372: creat da/d1c/d37/d8a/f91 x:0 0 0 2024-04-24T23:13:51.051 INFO:tasks.workunit.client.1.smithi040.stdout:9/325: unlink d7/da/d17/d40/d55/d2f/d1f/d34/f52 0 2024-04-24T23:13:51.052 INFO:tasks.workunit.client.1.smithi040.stdout:9/326: read d7/da/d17/d40/f42 [64812,43655] 0 2024-04-24T23:13:51.055 INFO:tasks.workunit.client.1.smithi040.stdout:5/281: creat d2/d2f/d14/f61 x:0 0 0 2024-04-24T23:13:51.056 INFO:tasks.workunit.client.1.smithi040.stdout:5/282: read d2/d7/d8/db/d1e/f46 [19070,106972] 0 2024-04-24T23:13:51.057 INFO:tasks.workunit.client.0.smithi031.stdout:9/708: stat d2/dae/d14/d78/d84/d4a/f91 0 2024-04-24T23:13:51.058 INFO:tasks.workunit.client.1.smithi040.stdout:4/261: stat da/d3f/f5a 0 2024-04-24T23:13:51.059 INFO:tasks.workunit.client.0.smithi031.stdout:9/709: write d2/dae/d14/d78/d84/d4a/f8d [2549833,6045] 0 2024-04-24T23:13:51.061 INFO:tasks.workunit.client.0.smithi031.stdout:7/617: rename d1/d2/dd/d33/c52 to d1/d2/d83/dc1/cd3 0 2024-04-24T23:13:51.062 INFO:tasks.workunit.client.1.smithi040.stdout:6/304: write d7/d1a/d20/d51/d1e/f49 [74248,11711] 0 2024-04-24T23:13:51.063 INFO:tasks.workunit.client.0.smithi031.stdout:0/672: sync 2024-04-24T23:13:51.063 INFO:tasks.workunit.client.0.smithi031.stdout:1/569: sync 2024-04-24T23:13:51.064 INFO:tasks.workunit.client.1.smithi040.stdout:8/280: getdents d0/d16/d1b/d3f 0 2024-04-24T23:13:51.065 INFO:tasks.workunit.client.0.smithi031.stdout:3/556: creat d6/df/fae x:0 0 0 2024-04-24T23:13:51.066 INFO:tasks.workunit.client.1.smithi040.stdout:7/351: truncate f2 4518060 0 2024-04-24T23:13:51.068 INFO:tasks.workunit.client.0.smithi031.stdout:4/750: write d3/d23/d37/d86/fde [335827,118744] 0 2024-04-24T23:13:51.068 INFO:tasks.workunit.client.0.smithi031.stdout:3/557: dread d6/df/d16/d2a/f68 [0,4194304] 0 2024-04-24T23:13:51.071 INFO:tasks.workunit.client.1.smithi040.stdout:2/373: creat da/d1c/d37/d39/d3f/f92 x:0 0 0 2024-04-24T23:13:51.072 INFO:tasks.workunit.client.1.smithi040.stdout:2/374: dread - da/d10/f71 zero size 2024-04-24T23:13:51.076 INFO:tasks.workunit.client.0.smithi031.stdout:4/751: dwrite d3/d23/d3b/fe2 [0,4194304] 0 2024-04-24T23:13:51.079 INFO:tasks.workunit.client.0.smithi031.stdout:4/752: write d3/d20/d25/d7e/fc9 [240021,83819] 0 2024-04-24T23:13:51.081 INFO:tasks.workunit.client.0.smithi031.stdout:4/753: dread - d3/d23/d37/fcb zero size 2024-04-24T23:13:51.081 INFO:tasks.workunit.client.0.smithi031.stdout:8/738: dwrite d8/d15/d37/d43/d5c/de1/f53 [0,4194304] 0 2024-04-24T23:13:51.083 INFO:tasks.workunit.client.0.smithi031.stdout:7/618: creat d1/d2/d60/fd4 x:0 0 0 2024-04-24T23:13:51.084 INFO:tasks.workunit.client.0.smithi031.stdout:4/754: chown d3/d20/d25/d60/cab 1347993 1 2024-04-24T23:13:51.085 INFO:tasks.workunit.client.0.smithi031.stdout:8/739: chown d8/d15/d37/d43/dbd 61 1 2024-04-24T23:13:51.086 INFO:tasks.workunit.client.0.smithi031.stdout:1/570: stat d1/dc/f70 0 2024-04-24T23:13:51.086 INFO:tasks.workunit.client.0.smithi031.stdout:7/619: stat d1/d45/d75/d81/lb9 0 2024-04-24T23:13:51.087 INFO:tasks.workunit.client.0.smithi031.stdout:5/575: write d0/d6/d12/f31 [1166973,100554] 0 2024-04-24T23:13:51.087 INFO:tasks.workunit.client.1.smithi040.stdout:0/355: symlink df/d12/d58/d68/l6c 0 2024-04-24T23:13:51.089 INFO:tasks.workunit.client.0.smithi031.stdout:0/673: symlink da/d86/dbf/le1 0 2024-04-24T23:13:51.091 INFO:tasks.workunit.client.1.smithi040.stdout:4/262: creat da/d1d/d2b/f5f x:0 0 0 2024-04-24T23:13:51.094 INFO:tasks.workunit.client.0.smithi031.stdout:1/571: dwrite d1/d3e/da3/f77 [0,4194304] 0 2024-04-24T23:13:51.094 INFO:tasks.workunit.client.1.smithi040.stdout:1/315: truncate da/d12/d14/f3f 3956722 0 2024-04-24T23:13:51.095 INFO:tasks.workunit.client.1.smithi040.stdout:1/316: write da/d12/d18/f6c [377507,119532] 0 2024-04-24T23:13:51.096 INFO:tasks.workunit.client.0.smithi031.stdout:9/710: write d2/f8 [4165113,115369] 0 2024-04-24T23:13:51.097 INFO:tasks.workunit.client.1.smithi040.stdout:1/317: truncate da/d12/f42 3079182 0 2024-04-24T23:13:51.098 INFO:tasks.workunit.client.1.smithi040.stdout:1/318: write da/d12/f42 [1939913,115279] 0 2024-04-24T23:13:51.099 INFO:tasks.workunit.client.1.smithi040.stdout:5/283: write d2/d7/d8/f13 [2902448,86408] 0 2024-04-24T23:13:51.099 INFO:tasks.workunit.client.1.smithi040.stdout:3/308: getdents d1/d2d/d2b 0 2024-04-24T23:13:51.100 INFO:tasks.workunit.client.0.smithi031.stdout:9/711: truncate d2/dae/d14/d78/d84/d4a/fb1 419026 0 2024-04-24T23:13:51.103 INFO:tasks.workunit.client.1.smithi040.stdout:7/352: unlink db/d12/d62/c76 0 2024-04-24T23:13:51.104 INFO:tasks.workunit.client.1.smithi040.stdout:7/353: chown db/d12/d20/c50 165077 1 2024-04-24T23:13:51.105 INFO:tasks.workunit.client.0.smithi031.stdout:1/572: dread d1/f66 [0,4194304] 0 2024-04-24T23:13:51.106 INFO:tasks.workunit.client.1.smithi040.stdout:9/327: truncate d7/f37 414702 0 2024-04-24T23:13:51.113 INFO:tasks.workunit.client.0.smithi031.stdout:1/573: dwrite d1/d33/d36/f91 [0,4194304] 0 2024-04-24T23:13:51.115 INFO:tasks.workunit.client.1.smithi040.stdout:0/356: rmdir df/d1e 39 2024-04-24T23:13:51.116 INFO:tasks.workunit.client.0.smithi031.stdout:8/740: truncate d8/d15/d37/d62/fb3 4955292 0 2024-04-24T23:13:51.117 INFO:tasks.workunit.client.0.smithi031.stdout:3/558: dwrite d6/df/d16/d2a/f68 [0,4194304] 0 2024-04-24T23:13:51.125 INFO:tasks.workunit.client.1.smithi040.stdout:8/281: mkdir d0/d38/d4c/d5d 0 2024-04-24T23:13:51.129 INFO:tasks.workunit.client.0.smithi031.stdout:9/712: rmdir d2/d16/dbe 39 2024-04-24T23:13:51.130 INFO:tasks.workunit.client.1.smithi040.stdout:5/284: chown d2/d2f/l5b 0 1 2024-04-24T23:13:51.130 INFO:tasks.workunit.client.0.smithi031.stdout:6/709: sync 2024-04-24T23:13:51.131 INFO:tasks.workunit.client.0.smithi031.stdout:2/510: getdents d5/d8/d45 0 2024-04-24T23:13:51.132 INFO:tasks.workunit.client.1.smithi040.stdout:7/354: mkdir db/d2f/d37/d7e 0 2024-04-24T23:13:51.132 INFO:tasks.workunit.client.0.smithi031.stdout:2/511: dread - d5/d8/d45/d5b/d49/f9f zero size 2024-04-24T23:13:51.133 INFO:tasks.workunit.client.1.smithi040.stdout:7/355: truncate db/f61 833138 0 2024-04-24T23:13:51.133 INFO:tasks.workunit.client.1.smithi040.stdout:9/328: readlink l5 0 2024-04-24T23:13:51.134 INFO:tasks.workunit.client.1.smithi040.stdout:5/285: dread d2/d7/d8/f39 [0,4194304] 0 2024-04-24T23:13:51.136 INFO:tasks.workunit.client.1.smithi040.stdout:2/375: rename da/d1c/d37/d39/d77/c6f to da/d10/d4d/d6c/c93 0 2024-04-24T23:13:51.136 INFO:tasks.workunit.client.1.smithi040.stdout:9/329: write d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [3364218,69207] 0 2024-04-24T23:13:51.137 INFO:tasks.workunit.client.0.smithi031.stdout:4/755: symlink d3/d20/d25/lf0 0 2024-04-24T23:13:51.137 INFO:tasks.workunit.client.1.smithi040.stdout:2/376: stat da/d1c/f5c 0 2024-04-24T23:13:51.138 INFO:tasks.workunit.client.0.smithi031.stdout:4/756: write d3/d20/d2d/d44/f84 [3218488,38716] 0 2024-04-24T23:13:51.144 INFO:tasks.workunit.client.1.smithi040.stdout:2/377: dread da/f20 [0,4194304] 0 2024-04-24T23:13:51.145 INFO:tasks.workunit.client.1.smithi040.stdout:5/286: dwrite d2/d7/d8/db/d1e/f28 [0,4194304] 0 2024-04-24T23:13:51.145 INFO:tasks.workunit.client.0.smithi031.stdout:3/559: rename d6/fc to d6/df/d16/d2a/d95/d9b/d1a/d25/faf 0 2024-04-24T23:13:51.146 INFO:tasks.workunit.client.1.smithi040.stdout:6/305: dwrite d7/d1a/d20/d51/d2f/f5c [0,4194304] 0 2024-04-24T23:13:51.149 INFO:tasks.workunit.client.0.smithi031.stdout:0/674: dwrite da/d18/d1a/f63 [0,4194304] 0 2024-04-24T23:13:51.155 INFO:tasks.workunit.client.0.smithi031.stdout:2/512: dread d5/d8/d15/f22 [0,4194304] 0 2024-04-24T23:13:51.156 INFO:tasks.workunit.client.0.smithi031.stdout:3/560: dwrite d6/df/d16/d2a/f5a [0,4194304] 0 2024-04-24T23:13:51.156 INFO:tasks.workunit.client.0.smithi031.stdout:8/741: write d8/f22 [1791114,24012] 0 2024-04-24T23:13:51.157 INFO:tasks.workunit.client.1.smithi040.stdout:5/287: dwrite d2/d7/f3b [0,4194304] 0 2024-04-24T23:13:51.158 INFO:tasks.workunit.client.0.smithi031.stdout:8/742: chown d8/d1a/d74/fcb 7493 1 2024-04-24T23:13:51.160 INFO:tasks.workunit.client.0.smithi031.stdout:9/713: creat d2/d43/d6e/fd9 x:0 0 0 2024-04-24T23:13:51.161 INFO:tasks.workunit.client.0.smithi031.stdout:3/561: write d6/df/d16/d2a/d95/d9b/d1a/f62 [2701458,42841] 0 2024-04-24T23:13:51.164 INFO:tasks.workunit.client.0.smithi031.stdout:3/562: truncate d6/d14/d1d/f9d 898433 0 2024-04-24T23:13:51.164 INFO:tasks.workunit.client.0.smithi031.stdout:2/513: dread d5/d6/d29/f5a [0,4194304] 0 2024-04-24T23:13:51.164 INFO:tasks.workunit.client.0.smithi031.stdout:6/710: creat d13/d15/d1d/d23/d2d/d9e/ff0 x:0 0 0 2024-04-24T23:13:51.167 INFO:tasks.workunit.client.1.smithi040.stdout:9/330: dread d7/f59 [0,4194304] 0 2024-04-24T23:13:51.170 INFO:tasks.workunit.client.1.smithi040.stdout:1/319: sync 2024-04-24T23:13:51.170 INFO:tasks.workunit.client.0.smithi031.stdout:4/757: dread d3/f7 [0,4194304] 0 2024-04-24T23:13:51.171 INFO:tasks.workunit.client.1.smithi040.stdout:1/320: truncate da/d35/d38/f6e 433620 0 2024-04-24T23:13:51.172 INFO:tasks.workunit.client.1.smithi040.stdout:4/263: rename da/d1d/d1e to da/d3f/d40/d41/d60 0 2024-04-24T23:13:51.173 INFO:tasks.workunit.client.1.smithi040.stdout:1/321: truncate da/d12/d18/d2a/d41/f6a 400190 0 2024-04-24T23:13:51.174 INFO:tasks.workunit.client.0.smithi031.stdout:1/574: link d1/d1d/f8e d1/d3e/da2/fb2 0 2024-04-24T23:13:51.175 INFO:tasks.workunit.client.1.smithi040.stdout:0/357: creat df/d1e/d20/d21/d41/f6d x:0 0 0 2024-04-24T23:13:51.175 INFO:tasks.workunit.client.1.smithi040.stdout:0/358: chown df/l10 432704 1 2024-04-24T23:13:51.176 INFO:tasks.workunit.client.0.smithi031.stdout:3/563: dwrite d6/df/d16/d2a/d95/d9b/d29/fa0 [0,4194304] 0 2024-04-24T23:13:51.177 INFO:tasks.workunit.client.0.smithi031.stdout:3/564: chown d6/df/f34 9608 1 2024-04-24T23:13:51.178 INFO:tasks.workunit.client.0.smithi031.stdout:7/620: link d1/d45/d84/l8f d1/d45/d75/d81/ld5 0 2024-04-24T23:13:51.180 INFO:tasks.workunit.client.0.smithi031.stdout:7/621: write d1/d45/d75/d9f/fa5 [4732363,27375] 0 2024-04-24T23:13:51.182 INFO:tasks.workunit.client.1.smithi040.stdout:0/359: dwrite f2 [4194304,4194304] 0 2024-04-24T23:13:51.184 INFO:tasks.workunit.client.1.smithi040.stdout:5/288: unlink d2/d2f/c15 0 2024-04-24T23:13:51.186 INFO:tasks.workunit.client.1.smithi040.stdout:7/356: creat db/d2f/d37/d7e/f7f x:0 0 0 2024-04-24T23:13:51.187 INFO:tasks.workunit.client.0.smithi031.stdout:8/743: chown d8/d15/d37/d62/f81 103 1 2024-04-24T23:13:51.189 INFO:tasks.workunit.client.0.smithi031.stdout:9/714: rename d2/f64 to d2/d43/d9d/dbd/fda 0 2024-04-24T23:13:51.189 INFO:tasks.workunit.client.1.smithi040.stdout:7/357: dread db/d12/d20/d52/f23 [0,4194304] 0 2024-04-24T23:13:51.190 INFO:tasks.workunit.client.1.smithi040.stdout:4/264: mknod da/d1d/d2b/c61 0 2024-04-24T23:13:51.192 INFO:tasks.workunit.client.0.smithi031.stdout:6/711: chown d13/d15/d1d/d23/d44/c81 15937206 1 2024-04-24T23:13:51.193 INFO:tasks.workunit.client.1.smithi040.stdout:1/322: rmdir da/d12/d23 39 2024-04-24T23:13:51.194 INFO:tasks.workunit.client.1.smithi040.stdout:5/289: dwrite d2/d7/d8/d1f/d57/d3e/f58 [0,4194304] 0 2024-04-24T23:13:51.198 INFO:tasks.workunit.client.0.smithi031.stdout:4/758: creat d3/d20/d25/d60/d63/db9/ff1 x:0 0 0 2024-04-24T23:13:51.199 INFO:tasks.workunit.client.1.smithi040.stdout:1/323: dread da/d12/d18/f6c [0,4194304] 0 2024-04-24T23:13:51.200 INFO:tasks.workunit.client.0.smithi031.stdout:4/759: read - d3/d20/d25/d60/d63/db9/ff1 zero size 2024-04-24T23:13:51.201 INFO:tasks.workunit.client.0.smithi031.stdout:4/760: chown d3/d20/d2d/d44/f84 30476353 1 2024-04-24T23:13:51.201 INFO:tasks.workunit.client.0.smithi031.stdout:1/575: mknod d1/d33/d41/d9b/cb3 0 2024-04-24T23:13:51.202 INFO:tasks.workunit.client.0.smithi031.stdout:4/761: chown d3/d20/c76 200 1 2024-04-24T23:13:51.204 INFO:tasks.workunit.client.0.smithi031.stdout:4/762: chown d3/d20/d25/d7e/faa 3912 1 2024-04-24T23:13:51.207 INFO:tasks.workunit.client.1.smithi040.stdout:3/309: getdents d1/d9/d39 0 2024-04-24T23:13:51.208 INFO:tasks.workunit.client.0.smithi031.stdout:5/576: link d0/d6/d3b/d4b/d1d/f9d d0/d6/d3b/d4b/dc/fc0 0 2024-04-24T23:13:51.208 INFO:tasks.workunit.client.0.smithi031.stdout:2/514: dread d5/d6/d29/d26/f3f [0,4194304] 0 2024-04-24T23:13:51.211 INFO:tasks.workunit.client.1.smithi040.stdout:0/360: unlink f2 0 2024-04-24T23:13:51.214 INFO:tasks.workunit.client.0.smithi031.stdout:0/675: symlink da/d18/d1a/dd6/d82/ddb/le2 0 2024-04-24T23:13:51.215 INFO:tasks.workunit.client.1.smithi040.stdout:9/331: symlink d7/da/d17/d40/d55/d2f/d1f/d31/l70 0 2024-04-24T23:13:51.225 INFO:tasks.workunit.client.1.smithi040.stdout:5/290: mkdir d2/d62 0 2024-04-24T23:13:51.228 INFO:tasks.workunit.client.1.smithi040.stdout:1/324: unlink da/d12/d14/f4e 0 2024-04-24T23:13:51.229 INFO:tasks.workunit.client.1.smithi040.stdout:8/282: link d0/l1a d0/d8/dc/d34/l5e 0 2024-04-24T23:13:51.230 INFO:tasks.workunit.client.0.smithi031.stdout:4/763: dread - d3/d20/d25/d7e/f83 zero size 2024-04-24T23:13:51.231 INFO:tasks.workunit.client.1.smithi040.stdout:4/265: dread da/f1a [4194304,4194304] 0 2024-04-24T23:13:51.232 INFO:tasks.workunit.client.1.smithi040.stdout:3/310: unlink d1/f8 0 2024-04-24T23:13:51.232 INFO:tasks.workunit.client.1.smithi040.stdout:6/306: truncate f2 6404340 0 2024-04-24T23:13:51.233 INFO:tasks.workunit.client.0.smithi031.stdout:3/565: symlink d6/df/d16/d2a/d95/d9b/d1a/lb0 0 2024-04-24T23:13:51.234 INFO:tasks.workunit.client.1.smithi040.stdout:8/283: dread d0/d8/f29 [0,4194304] 0 2024-04-24T23:13:51.235 INFO:tasks.workunit.client.0.smithi031.stdout:3/566: write d6/df/d16/d2a/f31 [1635373,117378] 0 2024-04-24T23:13:51.237 INFO:tasks.workunit.client.1.smithi040.stdout:0/361: mkdir df/d1e/d2b/d6e 0 2024-04-24T23:13:51.243 INFO:tasks.workunit.client.0.smithi031.stdout:3/567: dwrite d6/df/d16/d2a/d95/d9b/d1a/d25/f74 [0,4194304] 0 2024-04-24T23:13:51.243 INFO:tasks.workunit.client.1.smithi040.stdout:7/358: mknod db/d12/c80 0 2024-04-24T23:13:51.244 INFO:tasks.workunit.client.1.smithi040.stdout:7/359: chown db/d12/d73 40467 1 2024-04-24T23:13:51.245 INFO:tasks.workunit.client.0.smithi031.stdout:3/568: dread d6/d14/d1d/f9d [0,4194304] 0 2024-04-24T23:13:51.247 INFO:tasks.workunit.client.0.smithi031.stdout:8/744: write d8/d15/f40 [2078062,8289] 0 2024-04-24T23:13:51.250 INFO:tasks.workunit.client.0.smithi031.stdout:0/676: unlink da/d11/l3a 0 2024-04-24T23:13:51.252 INFO:tasks.workunit.client.1.smithi040.stdout:2/378: getdents da/d10/d4d 0 2024-04-24T23:13:51.252 INFO:tasks.workunit.client.0.smithi031.stdout:8/745: dread d8/d1a/d1d/d50/dc0/d9d/fea [0,4194304] 0 2024-04-24T23:13:51.253 INFO:tasks.workunit.client.0.smithi031.stdout:1/576: write d1/da/d12/f83 [479703,23147] 0 2024-04-24T23:13:51.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/577: fdatasync d1/d33/d36/f8d 0 2024-04-24T23:13:51.257 INFO:tasks.workunit.client.0.smithi031.stdout:9/715: link d2/dae/d14/d1d/f47 d2/dae/d14/d1d/d45/d65/fdb 0 2024-04-24T23:13:51.259 INFO:tasks.workunit.client.0.smithi031.stdout:6/712: mkdir d13/d1b/d32/d9c/ddb/df1 0 2024-04-24T23:13:51.262 INFO:tasks.workunit.client.1.smithi040.stdout:4/266: mkdir da/d3f/d40/d41/d62 0 2024-04-24T23:13:51.263 INFO:tasks.workunit.client.1.smithi040.stdout:4/267: write da/d1d/f32 [413962,41617] 0 2024-04-24T23:13:51.264 INFO:tasks.workunit.client.1.smithi040.stdout:4/268: truncate da/d3f/d40/d41/d60/f23 1627455 0 2024-04-24T23:13:51.264 INFO:tasks.workunit.client.0.smithi031.stdout:4/764: unlink d3/d20/d2d/d44/c1e 0 2024-04-24T23:13:51.265 INFO:tasks.workunit.client.1.smithi040.stdout:4/269: truncate da/d1d/d2b/f5f 923319 0 2024-04-24T23:13:51.266 INFO:tasks.workunit.client.1.smithi040.stdout:4/270: fdatasync da/d3f/d40/d41/d60/f23 0 2024-04-24T23:13:51.266 INFO:tasks.workunit.client.0.smithi031.stdout:2/515: dwrite d5/d8/d45/d5b/d49/f6d [0,4194304] 0 2024-04-24T23:13:51.266 INFO:tasks.workunit.client.0.smithi031.stdout:7/622: write d1/f53 [4698166,36422] 0 2024-04-24T23:13:51.267 INFO:tasks.workunit.client.0.smithi031.stdout:7/623: dread - d1/d2/d60/fd4 zero size 2024-04-24T23:13:51.270 INFO:tasks.workunit.client.1.smithi040.stdout:8/284: creat d0/d16/d19/f5f x:0 0 0 2024-04-24T23:13:51.275 INFO:tasks.workunit.client.1.smithi040.stdout:9/332: creat d7/da/d17/d40/d55/d61/d6c/f71 x:0 0 0 2024-04-24T23:13:51.277 INFO:tasks.workunit.client.1.smithi040.stdout:7/360: mknod db/d12/d20/d52/d4b/c81 0 2024-04-24T23:13:51.277 INFO:tasks.workunit.client.1.smithi040.stdout:7/361: stat db/d2f/f30 0 2024-04-24T23:13:51.278 INFO:tasks.workunit.client.1.smithi040.stdout:2/379: mknod da/d10/d4d/c94 0 2024-04-24T23:13:51.279 INFO:tasks.workunit.client.0.smithi031.stdout:3/569: unlink d6/d14/d1d/d69/l8a 0 2024-04-24T23:13:51.280 INFO:tasks.workunit.client.0.smithi031.stdout:3/570: readlink d6/df/d16/d2a/l44 0 2024-04-24T23:13:51.281 INFO:tasks.workunit.client.0.smithi031.stdout:3/571: read d6/df/d16/d2a/f5a [1067668,42788] 0 2024-04-24T23:13:51.282 INFO:tasks.workunit.client.0.smithi031.stdout:3/572: readlink d6/df/d16/d2a/d95/d9b/d1a/l5d 0 2024-04-24T23:13:51.283 INFO:tasks.workunit.client.0.smithi031.stdout:8/746: creat d8/d15/d9a/ffe x:0 0 0 2024-04-24T23:13:51.284 INFO:tasks.workunit.client.1.smithi040.stdout:2/380: dwrite da/d10/d15/f63 [0,4194304] 0 2024-04-24T23:13:51.290 INFO:tasks.workunit.client.0.smithi031.stdout:5/577: dwrite d0/d6/d3b/d4b/dc/f26 [0,4194304] 0 2024-04-24T23:13:51.292 INFO:tasks.workunit.client.0.smithi031.stdout:6/713: rename d13/d15/d1d/d23/d24/d43/d68/c86 to d13/d15/d1d/d23/cf2 0 2024-04-24T23:13:51.294 INFO:tasks.workunit.client.1.smithi040.stdout:4/271: mkdir da/d3f/d40/d63 0 2024-04-24T23:13:51.297 INFO:tasks.workunit.client.1.smithi040.stdout:8/285: symlink d0/d16/d19/l60 0 2024-04-24T23:13:51.299 INFO:tasks.workunit.client.0.smithi031.stdout:4/765: mkdir d3/d23/d3b/df2 0 2024-04-24T23:13:51.300 INFO:tasks.workunit.client.1.smithi040.stdout:0/362: creat df/d12/d4b/d62/d6b/f6f x:0 0 0 2024-04-24T23:13:51.304 INFO:tasks.workunit.client.0.smithi031.stdout:0/677: dwrite da/d22/d62/d9c/fb6 [4194304,4194304] 0 2024-04-24T23:13:51.305 INFO:tasks.workunit.client.0.smithi031.stdout:0/678: chown da/d18/d1a/l7a 1 1 2024-04-24T23:13:51.308 INFO:tasks.workunit.client.1.smithi040.stdout:7/362: dread - db/f2e zero size 2024-04-24T23:13:51.309 INFO:tasks.workunit.client.1.smithi040.stdout:5/291: rename d2/d7/d8/d1f/d57/d27/l4f to d2/d7/d8/d1f/d57/l63 0 2024-04-24T23:13:51.310 INFO:tasks.workunit.client.1.smithi040.stdout:7/363: stat db/d12/d20/d52/f79 0 2024-04-24T23:13:51.310 INFO:tasks.workunit.client.0.smithi031.stdout:5/578: dread d0/d6/d3b/d4b/dc/f84 [0,4194304] 0 2024-04-24T23:13:51.311 INFO:tasks.workunit.client.0.smithi031.stdout:5/579: chown d0/d6 11074282 1 2024-04-24T23:13:51.314 INFO:tasks.workunit.client.1.smithi040.stdout:9/333: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/d4d/f43 [0,4194304] 0 2024-04-24T23:13:51.314 INFO:tasks.workunit.client.1.smithi040.stdout:3/311: rmdir d1/d2d/de/d30/d43 0 2024-04-24T23:13:51.315 INFO:tasks.workunit.client.0.smithi031.stdout:3/573: mkdir d6/d14/d71/db1 0 2024-04-24T23:13:51.317 INFO:tasks.workunit.client.1.smithi040.stdout:9/334: truncate d7/da/d17/d40/d55/d2f/f6f 778340 0 2024-04-24T23:13:51.317 INFO:tasks.workunit.client.0.smithi031.stdout:7/624: dwrite d1/d2/dd/f23 [0,4194304] 0 2024-04-24T23:13:51.318 INFO:tasks.workunit.client.1.smithi040.stdout:9/335: dread - d7/da/d17/d40/d55/d2f/d1f/d34/f5b zero size 2024-04-24T23:13:51.321 INFO:tasks.workunit.client.0.smithi031.stdout:8/747: creat d8/d15/dcf/fff x:0 0 0 2024-04-24T23:13:51.326 INFO:tasks.workunit.client.0.smithi031.stdout:8/748: chown d8/d1a 16280473 1 2024-04-24T23:13:51.326 INFO:tasks.workunit.client.1.smithi040.stdout:8/286: dwrite d0/d8/f29 [0,4194304] 0 2024-04-24T23:13:51.327 INFO:tasks.workunit.client.1.smithi040.stdout:8/287: fsync d0/d16/d19/f5f 0 2024-04-24T23:13:51.327 INFO:tasks.workunit.client.1.smithi040.stdout:8/288: dread - d0/d8/dc/d34/f3c zero size 2024-04-24T23:13:51.327 INFO:tasks.workunit.client.1.smithi040.stdout:8/289: write d0/d16/d19/f5f [271989,126402] 0 2024-04-24T23:13:51.332 INFO:tasks.workunit.client.1.smithi040.stdout:3/312: mkdir d1/d2d/d72 0 2024-04-24T23:13:51.335 INFO:tasks.workunit.client.1.smithi040.stdout:9/336: mknod d7/da/d17/d40/d55/d61/c72 0 2024-04-24T23:13:51.337 INFO:tasks.workunit.client.0.smithi031.stdout:4/766: creat d3/d20/d25/d95/ff3 x:0 0 0 2024-04-24T23:13:51.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/574: dwrite d6/df/fae [0,4194304] 0 2024-04-24T23:13:51.337 INFO:tasks.workunit.client.0.smithi031.stdout:4/767: stat d3/d23/d69/f31 0 2024-04-24T23:13:51.338 INFO:tasks.workunit.client.1.smithi040.stdout:7/364: read db/f1c [2350425,34908] 0 2024-04-24T23:13:51.338 INFO:tasks.workunit.client.0.smithi031.stdout:0/679: creat da/d11/d6a/dba/dc0/fe3 x:0 0 0 2024-04-24T23:13:51.338 INFO:tasks.workunit.client.1.smithi040.stdout:0/363: mkdir df/d1e/d70 0 2024-04-24T23:13:51.339 INFO:tasks.workunit.client.1.smithi040.stdout:0/364: write df/d12/d58/f59 [521758,23041] 0 2024-04-24T23:13:51.342 INFO:tasks.workunit.client.1.smithi040.stdout:8/290: mknod d0/d16/d1b/d3f/c61 0 2024-04-24T23:13:51.344 INFO:tasks.workunit.client.0.smithi031.stdout:1/578: mknod d1/dc/d11/cb4 0 2024-04-24T23:13:51.345 INFO:tasks.workunit.client.0.smithi031.stdout:3/575: dread d6/df/d16/d2a/d95/d9b/d29/fa0 [0,4194304] 0 2024-04-24T23:13:51.349 INFO:tasks.workunit.client.0.smithi031.stdout:8/749: rename d8/d15/d37/d43/d7b/d89/fdf to d8/d15/d37/d43/d7b/dd3/f100 0 2024-04-24T23:13:51.353 INFO:tasks.workunit.client.1.smithi040.stdout:1/325: rename da/d12/d18/f27 to da/f71 0 2024-04-24T23:13:51.354 INFO:tasks.workunit.client.0.smithi031.stdout:9/716: creat d2/fdc x:0 0 0 2024-04-24T23:13:51.355 INFO:tasks.workunit.client.1.smithi040.stdout:7/365: mknod db/d12/d20/d52/d4b/c82 0 2024-04-24T23:13:51.355 INFO:tasks.workunit.client.1.smithi040.stdout:3/313: dwrite d1/d9/d18/d36/d55/f67 [0,4194304] 0 2024-04-24T23:13:51.356 INFO:tasks.workunit.client.0.smithi031.stdout:9/717: write d2/dae/d14/d78/d84/d4a/d4b/d96/fa8 [4990903,56672] 0 2024-04-24T23:13:51.357 INFO:tasks.workunit.client.1.smithi040.stdout:8/291: creat d0/d8/d22/f62 x:0 0 0 2024-04-24T23:13:51.357 INFO:tasks.workunit.client.0.smithi031.stdout:2/516: link d5/d8/d45/d5b/d49/l47 d5/d8/d45/d5b/d49/d2f/lb1 0 2024-04-24T23:13:51.365 INFO:tasks.workunit.client.0.smithi031.stdout:4/768: dread d3/d23/d69/d3f/f57 [0,4194304] 0 2024-04-24T23:13:51.367 INFO:tasks.workunit.client.0.smithi031.stdout:4/769: write d3/d20/d2d/d44/d5b/d5d/f96 [1757663,127487] 0 2024-04-24T23:13:51.371 INFO:tasks.workunit.client.1.smithi040.stdout:4/272: truncate da/d1d/d3c/f5c 1861120 0 2024-04-24T23:13:51.378 INFO:tasks.workunit.client.1.smithi040.stdout:6/307: rename d7/d2c to d7/d1a/d20/d51/d1e/d5d 0 2024-04-24T23:13:51.379 INFO:tasks.workunit.client.0.smithi031.stdout:3/576: fsync d6/df/d16/d53/fa8 0 2024-04-24T23:13:51.381 INFO:tasks.workunit.client.0.smithi031.stdout:3/577: read d6/df/d16/d2a/d95/d9b/d60/f64 [29372,108369] 0 2024-04-24T23:13:51.387 INFO:tasks.workunit.client.1.smithi040.stdout:5/292: getdents d2/d7/d8/d1f/d57/d27/d3d 0 2024-04-24T23:13:51.390 INFO:tasks.workunit.client.0.smithi031.stdout:0/680: rename da/d22/c2e to da/d86/da3/dcd/ce4 0 2024-04-24T23:13:51.391 INFO:tasks.workunit.client.0.smithi031.stdout:0/681: write da/d11/f59 [2684165,110807] 0 2024-04-24T23:13:51.391 INFO:tasks.workunit.client.1.smithi040.stdout:4/273: symlink da/d3f/d40/l64 0 2024-04-24T23:13:51.395 INFO:tasks.workunit.client.0.smithi031.stdout:5/580: write d0/d6/d3b/d4b/d1d/f9d [1197597,67811] 0 2024-04-24T23:13:51.399 INFO:tasks.workunit.client.1.smithi040.stdout:2/381: rename da/d10/d15/f5e to da/d1c/d37/d39/d3f/d52/f95 0 2024-04-24T23:13:51.400 INFO:tasks.workunit.client.1.smithi040.stdout:2/382: write da/d1c/d37/d8a/f91 [549639,28930] 0 2024-04-24T23:13:51.401 INFO:tasks.workunit.client.1.smithi040.stdout:7/366: dread db/f1c [0,4194304] 0 2024-04-24T23:13:51.403 INFO:tasks.workunit.client.0.smithi031.stdout:5/581: dwrite d0/d6/d3b/d4b/d1d/fb2 [0,4194304] 0 2024-04-24T23:13:51.414 INFO:tasks.workunit.client.1.smithi040.stdout:5/293: unlink d2/f6 0 2024-04-24T23:13:51.419 INFO:tasks.workunit.client.1.smithi040.stdout:9/337: rename d7/da/d17/d40/d55/l5d to d7/da/d17/d40/d55/d2f/d1f/d34/l73 0 2024-04-24T23:13:51.420 INFO:tasks.workunit.client.0.smithi031.stdout:7/625: link d1/d2/f7 d1/d45/d75/d9f/d66/fd6 0 2024-04-24T23:13:51.421 INFO:tasks.workunit.client.0.smithi031.stdout:7/626: write d1/d2/dd/f95 [30520,122162] 0 2024-04-24T23:13:51.424 INFO:tasks.workunit.client.1.smithi040.stdout:7/367: mknod db/d12/d20/d56/c83 0 2024-04-24T23:13:51.424 INFO:tasks.workunit.client.0.smithi031.stdout:9/718: write d2/dae/d14/f19 [1521119,47107] 0 2024-04-24T23:13:51.425 INFO:tasks.workunit.client.0.smithi031.stdout:9/719: chown d2/dae/d14/f62 1 1 2024-04-24T23:13:51.426 INFO:tasks.workunit.client.0.smithi031.stdout:1/579: creat d1/d3e/d46/d74/fb5 x:0 0 0 2024-04-24T23:13:51.427 INFO:tasks.workunit.client.0.smithi031.stdout:1/580: chown d1 5963 1 2024-04-24T23:13:51.427 INFO:tasks.workunit.client.0.smithi031.stdout:9/720: stat d2/d43/d9d/dc0/fcc 0 2024-04-24T23:13:51.431 INFO:tasks.workunit.client.0.smithi031.stdout:6/714: rename d13/d1b/d32/db4/fe8 to d13/d15/d1d/d23/d24/d8b/ff3 0 2024-04-24T23:13:51.432 INFO:tasks.workunit.client.1.smithi040.stdout:5/294: dread d2/d7/d8/d1f/d57/d27/f32 [0,4194304] 0 2024-04-24T23:13:51.432 INFO:tasks.workunit.client.1.smithi040.stdout:3/314: truncate d1/d9/d2e/f5f 358005 0 2024-04-24T23:13:51.433 INFO:tasks.workunit.client.1.smithi040.stdout:5/295: write d2/d7/f3b [2200889,7881] 0 2024-04-24T23:13:51.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/682: truncate da/fb 66409 0 2024-04-24T23:13:51.436 INFO:tasks.workunit.client.0.smithi031.stdout:0/683: chown da/d22/d62/fb4 1457203 1 2024-04-24T23:13:51.437 INFO:tasks.workunit.client.1.smithi040.stdout:3/315: dread d1/d2d/de/f2f [0,4194304] 0 2024-04-24T23:13:51.437 INFO:tasks.workunit.client.1.smithi040.stdout:3/316: stat d1/d9/d18/d36/d55/d62/f65 0 2024-04-24T23:13:51.444 INFO:tasks.workunit.client.0.smithi031.stdout:2/517: mknod d5/d8/cb2 0 2024-04-24T23:13:51.446 INFO:tasks.workunit.client.0.smithi031.stdout:2/518: truncate d5/d28/d4e/fa1 763227 0 2024-04-24T23:13:51.451 INFO:tasks.workunit.client.1.smithi040.stdout:0/365: rename df/d14/d1b to df/d12/d1a/d71 0 2024-04-24T23:13:51.452 INFO:tasks.workunit.client.1.smithi040.stdout:0/366: write df/d12/d4b/d62/d6b/f6f [609676,51012] 0 2024-04-24T23:13:51.454 INFO:tasks.workunit.client.0.smithi031.stdout:1/581: creat d1/da/d12/d65/fb6 x:0 0 0 2024-04-24T23:13:51.459 INFO:tasks.workunit.client.1.smithi040.stdout:1/326: link da/d35/d38/c5f da/c72 0 2024-04-24T23:13:51.463 INFO:tasks.workunit.client.0.smithi031.stdout:7/627: rename d1/d2/dd/d33/d5e/d64/d6d/dbd to d1/d2/dd/d33/d5e/d64/d6d/dae/dd7 0 2024-04-24T23:13:51.465 INFO:tasks.workunit.client.1.smithi040.stdout:1/327: dwrite da/d12/d14/f68 [0,4194304] 0 2024-04-24T23:13:51.465 INFO:tasks.workunit.client.0.smithi031.stdout:7/628: truncate d1/d2/dd/d33/d5e/d64/fca 988859 0 2024-04-24T23:13:51.466 INFO:tasks.workunit.client.0.smithi031.stdout:8/750: creat d8/d1a/d1d/d35/d58/f101 x:0 0 0 2024-04-24T23:13:51.470 INFO:tasks.workunit.client.0.smithi031.stdout:0/684: creat da/d18/d1a/dd6/d82/ddb/fe5 x:0 0 0 2024-04-24T23:13:51.472 INFO:tasks.workunit.client.1.smithi040.stdout:3/317: fdatasync d1/d9/f20 0 2024-04-24T23:13:51.473 INFO:tasks.workunit.client.0.smithi031.stdout:5/582: mknod d0/d6/d3b/d4b/d1d/cc1 0 2024-04-24T23:13:51.473 INFO:tasks.workunit.client.1.smithi040.stdout:5/296: dread d2/d7/d8/f1b [0,4194304] 0 2024-04-24T23:13:51.476 INFO:tasks.workunit.client.0.smithi031.stdout:2/519: symlink d5/d6/df/lb3 0 2024-04-24T23:13:51.478 INFO:tasks.workunit.client.0.smithi031.stdout:4/770: creat d3/d23/d37/ddc/ff4 x:0 0 0 2024-04-24T23:13:51.479 INFO:tasks.workunit.client.0.smithi031.stdout:2/520: read d5/d8/d15/d56/d84/d75/f91 [410575,71155] 0 2024-04-24T23:13:51.481 INFO:tasks.workunit.client.1.smithi040.stdout:4/274: rename da/d1d/d2b/l2d to da/d3f/d40/d47/l65 0 2024-04-24T23:13:51.482 INFO:tasks.workunit.client.0.smithi031.stdout:4/771: truncate d3/d23/d69/d3f/fd9 5017443 0 2024-04-24T23:13:51.483 INFO:tasks.workunit.client.0.smithi031.stdout:9/721: dwrite d2/dae/d14/f46 [0,4194304] 0 2024-04-24T23:13:51.486 INFO:tasks.workunit.client.1.smithi040.stdout:2/383: link da/d1c/d37/d39/c6a da/d10/d4d/c96 0 2024-04-24T23:13:51.489 INFO:tasks.workunit.client.0.smithi031.stdout:3/578: truncate d6/df/d16/d57/f65 1513639 0 2024-04-24T23:13:51.493 INFO:tasks.workunit.client.1.smithi040.stdout:2/384: dwrite da/d29/f69 [0,4194304] 0 2024-04-24T23:13:51.494 INFO:tasks.workunit.client.0.smithi031.stdout:0/685: rename da/d20/d8e/cce to da/d86/dbf/ce6 0 2024-04-24T23:13:51.494 INFO:tasks.workunit.client.0.smithi031.stdout:0/686: readlink l9 0 2024-04-24T23:13:51.495 INFO:tasks.workunit.client.1.smithi040.stdout:3/318: dread d1/d2d/d2b/f42 [0,4194304] 0 2024-04-24T23:13:51.495 INFO:tasks.workunit.client.1.smithi040.stdout:3/319: readlink d1/d2d/de/d30/l3d 0 2024-04-24T23:13:51.497 INFO:tasks.workunit.client.1.smithi040.stdout:1/328: creat da/d12/d14/d58/f73 x:0 0 0 2024-04-24T23:13:51.502 INFO:tasks.workunit.client.0.smithi031.stdout:2/521: read d5/d6/df/f44 [731371,114186] 0 2024-04-24T23:13:51.503 INFO:tasks.workunit.client.1.smithi040.stdout:7/368: rename db/f55 to db/d12/d7d/f84 0 2024-04-24T23:13:51.504 INFO:tasks.workunit.client.0.smithi031.stdout:1/582: getdents d1/d33/d36/d81 0 2024-04-24T23:13:51.506 INFO:tasks.workunit.client.1.smithi040.stdout:4/275: symlink da/d3f/d40/d41/d60/l66 0 2024-04-24T23:13:51.507 INFO:tasks.workunit.client.1.smithi040.stdout:4/276: dread - da/d3f/d40/d41/d60/f4d zero size 2024-04-24T23:13:51.508 INFO:tasks.workunit.client.0.smithi031.stdout:4/772: fdatasync d3/d20/da7/fed 0 2024-04-24T23:13:51.509 INFO:tasks.workunit.client.1.smithi040.stdout:9/338: readlink d7/da/d17/d40/d55/d2f/d1f/d34/l73 0 2024-04-24T23:13:51.510 INFO:tasks.workunit.client.0.smithi031.stdout:9/722: symlink d2/dae/d14/d1d/d45/d65/ldd 0 2024-04-24T23:13:51.513 INFO:tasks.workunit.client.1.smithi040.stdout:7/369: dread db/d2f/f30 [0,4194304] 0 2024-04-24T23:13:51.515 INFO:tasks.workunit.client.0.smithi031.stdout:3/579: dread d6/df/d16/d2a/f2e [0,4194304] 0 2024-04-24T23:13:51.518 INFO:tasks.workunit.client.0.smithi031.stdout:8/751: truncate d8/d15/d9a/fc2 1389167 0 2024-04-24T23:13:51.520 INFO:tasks.workunit.client.1.smithi040.stdout:3/320: truncate d1/d9/f21 981252 0 2024-04-24T23:13:51.521 INFO:tasks.workunit.client.1.smithi040.stdout:3/321: truncate d1/d2d/d2b/f4d 76454 0 2024-04-24T23:13:51.522 INFO:tasks.workunit.client.1.smithi040.stdout:1/329: mknod da/d12/d14/d25/c74 0 2024-04-24T23:13:51.523 INFO:tasks.workunit.client.1.smithi040.stdout:3/322: write d1/d2d/de/f70 [780985,45149] 0 2024-04-24T23:13:51.523 INFO:tasks.workunit.client.1.smithi040.stdout:9/339: dread d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f50 [0,4194304] 0 2024-04-24T23:13:51.524 INFO:tasks.workunit.client.0.smithi031.stdout:0/687: unlink da/d11/d6a/dba/dc0/cc4 0 2024-04-24T23:13:51.528 INFO:tasks.workunit.client.0.smithi031.stdout:9/723: dread d2/f8 [0,4194304] 0 2024-04-24T23:13:51.530 INFO:tasks.workunit.client.1.smithi040.stdout:8/292: sync 2024-04-24T23:13:51.530 INFO:tasks.workunit.client.1.smithi040.stdout:8/293: truncate d0/d16/d19/f5f 763850 0 2024-04-24T23:13:51.530 INFO:tasks.workunit.client.1.smithi040.stdout:8/294: chown d0/d8/d22/l2f 192 1 2024-04-24T23:13:51.531 INFO:tasks.workunit.client.1.smithi040.stdout:8/295: chown d0/d16/d19/l60 13272 1 2024-04-24T23:13:51.531 INFO:tasks.workunit.client.1.smithi040.stdout:0/367: rename df/d12/d1a/f55 to df/d12/d58/f72 0 2024-04-24T23:13:51.534 INFO:tasks.workunit.client.0.smithi031.stdout:9/724: dwrite d2/dae/d14/d78/d84/d4a/f8d [0,4194304] 0 2024-04-24T23:13:51.535 INFO:tasks.workunit.client.1.smithi040.stdout:8/296: dread d0/d16/f3b [0,4194304] 0 2024-04-24T23:13:51.535 INFO:tasks.workunit.client.0.smithi031.stdout:9/725: fsync d2/dae/d14/f19 0 2024-04-24T23:13:51.536 INFO:tasks.workunit.client.1.smithi040.stdout:4/277: creat da/d3f/d40/d41/f67 x:0 0 0 2024-04-24T23:13:51.536 INFO:tasks.workunit.client.1.smithi040.stdout:8/297: chown d0/d16/l18 895832 1 2024-04-24T23:13:51.541 INFO:tasks.workunit.client.1.smithi040.stdout:2/385: creat da/d81/f97 x:0 0 0 2024-04-24T23:13:51.543 INFO:tasks.workunit.client.0.smithi031.stdout:3/580: dread - d6/df/d16/d2a/d95/d9b/d29/f79 zero size 2024-04-24T23:13:51.544 INFO:tasks.workunit.client.0.smithi031.stdout:3/581: stat d6/df/c52 0 2024-04-24T23:13:51.545 INFO:tasks.workunit.client.1.smithi040.stdout:1/330: creat da/d12/d18/d2a/d4d/f75 x:0 0 0 2024-04-24T23:13:51.546 INFO:tasks.workunit.client.0.smithi031.stdout:8/752: mknod d8/d1a/d1d/d50/c102 0 2024-04-24T23:13:51.552 INFO:tasks.workunit.client.0.smithi031.stdout:0/688: rename da/d18/d1a/dd6/d82 to da/d86/da3/dcd/de7 0 2024-04-24T23:13:51.552 INFO:tasks.workunit.client.0.smithi031.stdout:8/753: rename d8/d15/d37/d43 to d8/d15/d37/d43/dbd/dc4/d103 22 2024-04-24T23:13:51.554 INFO:tasks.workunit.client.0.smithi031.stdout:8/754: chown d8/d15/d37/d43/c8a 102 1 2024-04-24T23:13:51.554 INFO:tasks.workunit.client.0.smithi031.stdout:2/522: creat d5/d28/d4e/d5e/fb4 x:0 0 0 2024-04-24T23:13:51.558 INFO:tasks.workunit.client.0.smithi031.stdout:1/583: creat d1/da/d10/d52/d62/fb7 x:0 0 0 2024-04-24T23:13:51.560 INFO:tasks.workunit.client.1.smithi040.stdout:0/368: creat df/d12/d4b/d62/d6b/f73 x:0 0 0 2024-04-24T23:13:51.560 INFO:tasks.workunit.client.0.smithi031.stdout:1/584: truncate d1/d3e/d46/d74/fb5 610483 0 2024-04-24T23:13:51.562 INFO:tasks.workunit.client.1.smithi040.stdout:0/369: write df/d12/d4b/d62/f65 [287846,122554] 0 2024-04-24T23:13:51.565 INFO:tasks.workunit.client.1.smithi040.stdout:2/386: symlink da/d1c/d4f/l98 0 2024-04-24T23:13:51.565 INFO:tasks.workunit.client.1.smithi040.stdout:3/323: dwrite d1/d2d/d2b/f42 [4194304,4194304] 0 2024-04-24T23:13:51.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/582: mkdir d6/d14/db2 0 2024-04-24T23:13:51.566 INFO:tasks.workunit.client.0.smithi031.stdout:5/583: link d0/d6/d3b/d4b/dc/d8b/lb0 d0/lc2 0 2024-04-24T23:13:51.567 INFO:tasks.workunit.client.1.smithi040.stdout:7/370: symlink db/d12/d73/l85 0 2024-04-24T23:13:51.568 INFO:tasks.workunit.client.0.smithi031.stdout:0/689: mkdir da/d11/d6a/de8 0 2024-04-24T23:13:51.569 INFO:tasks.workunit.client.1.smithi040.stdout:1/331: rmdir da/d12/d18/d2a/d41 39 2024-04-24T23:13:51.569 INFO:tasks.workunit.client.1.smithi040.stdout:0/370: dwrite df/d12/f22 [4194304,4194304] 0 2024-04-24T23:13:51.570 INFO:tasks.workunit.client.1.smithi040.stdout:1/332: chown da/d12/d18/d2a/d32 16628636 1 2024-04-24T23:13:51.572 INFO:tasks.workunit.client.1.smithi040.stdout:9/340: mknod d7/da/c74 0 2024-04-24T23:13:51.573 INFO:tasks.workunit.client.0.smithi031.stdout:2/523: symlink d5/d6/d29/lb5 0 2024-04-24T23:13:51.574 INFO:tasks.workunit.client.0.smithi031.stdout:2/524: write d5/d8/f6c [164906,10875] 0 2024-04-24T23:13:51.578 INFO:tasks.workunit.client.0.smithi031.stdout:4/773: link d3/d20/d25/d95/cb5 d3/d20/d2d/d44/d5b/cf5 0 2024-04-24T23:13:51.579 INFO:tasks.workunit.client.1.smithi040.stdout:1/333: dwrite da/d12/f54 [0,4194304] 0 2024-04-24T23:13:51.579 INFO:tasks.workunit.client.0.smithi031.stdout:4/774: stat d3/d23/d69/d3f/d9e/ddb 0 2024-04-24T23:13:51.580 INFO:tasks.workunit.client.1.smithi040.stdout:1/334: fdatasync da/d12/d18/f49 0 2024-04-24T23:13:51.581 INFO:tasks.workunit.client.1.smithi040.stdout:4/278: getdents da/d3f/d40/d47/d5d 0 2024-04-24T23:13:51.586 INFO:tasks.workunit.client.0.smithi031.stdout:4/775: dwrite d3/f9 [0,4194304] 0 2024-04-24T23:13:51.588 INFO:tasks.workunit.client.0.smithi031.stdout:4/776: rename d3/d20/d25/d60 to d3/d20/d25/d60/dc8/df6 22 2024-04-24T23:13:51.589 INFO:tasks.workunit.client.0.smithi031.stdout:0/690: unlink da/d18/d1a/dd6/l84 0 2024-04-24T23:13:51.590 INFO:tasks.workunit.client.0.smithi031.stdout:0/691: chown da/d11/d6a/dba/fca 184 1 2024-04-24T23:13:51.591 INFO:tasks.workunit.client.1.smithi040.stdout:2/387: creat da/d10/d7a/f99 x:0 0 0 2024-04-24T23:13:51.591 INFO:tasks.workunit.client.0.smithi031.stdout:8/755: mknod d8/d1a/c104 0 2024-04-24T23:13:51.592 INFO:tasks.workunit.client.1.smithi040.stdout:3/324: read - d1/d9/d15/f1d zero size 2024-04-24T23:13:51.592 INFO:tasks.workunit.client.0.smithi031.stdout:8/756: fsync d8/d1a/f9b 0 2024-04-24T23:13:51.656 INFO:tasks.workunit.client.0.smithi031.stdout:1/585: write d1/d3e/da2/fb2 [1969818,66279] 0 2024-04-24T23:13:51.659 INFO:tasks.workunit.client.0.smithi031.stdout:3/583: write d6/d14/f58 [445503,125859] 0 2024-04-24T23:13:51.661 INFO:tasks.workunit.client.1.smithi040.stdout:6/308: rename d7/d1a/d20/d51/f4b to d7/de/f5e 0 2024-04-24T23:13:51.663 INFO:tasks.workunit.client.0.smithi031.stdout:4/777: unlink d3/d20/fa4 0 2024-04-24T23:13:51.667 INFO:tasks.workunit.client.1.smithi040.stdout:4/279: read da/f18 [272478,68280] 0 2024-04-24T23:13:51.667 INFO:tasks.workunit.client.1.smithi040.stdout:2/388: write da/d1c/d37/d39/d3f/d52/f95 [119096,109914] 0 2024-04-24T23:13:51.670 INFO:tasks.workunit.client.0.smithi031.stdout:8/757: mknod d8/d15/d37/d43/d7b/d86/c105 0 2024-04-24T23:13:51.671 INFO:tasks.workunit.client.0.smithi031.stdout:8/758: chown d8/d15/d37/d43/d5c/df4/cf7 3 1 2024-04-24T23:13:51.671 INFO:tasks.workunit.client.1.smithi040.stdout:7/371: link db/d2f/d37/d7e/f7f db/d2f/d37/d7e/f86 0 2024-04-24T23:13:51.672 INFO:tasks.workunit.client.1.smithi040.stdout:9/341: dwrite d7/da/d17/d40/d55/d2f/d1f/d34/f49 [0,4194304] 0 2024-04-24T23:13:51.672 INFO:tasks.workunit.client.0.smithi031.stdout:8/759: fsync d8/d1a/d1d/d50/dc0/d9d/fc6 0 2024-04-24T23:13:51.673 INFO:tasks.workunit.client.1.smithi040.stdout:0/371: fdatasync df/d12/d1a/d71/f36 0 2024-04-24T23:13:51.675 INFO:tasks.workunit.client.1.smithi040.stdout:0/372: chown df/d12/d58/f72 13052 1 2024-04-24T23:13:51.675 INFO:tasks.workunit.client.0.smithi031.stdout:9/726: link d2/dae/d85/fc4 d2/dae/d14/d78/d84/d4a/fde 0 2024-04-24T23:13:51.676 INFO:tasks.workunit.client.0.smithi031.stdout:3/584: mknod d6/d14/cb3 0 2024-04-24T23:13:51.677 INFO:tasks.workunit.client.0.smithi031.stdout:9/727: write d2/dae/f10 [2702560,112832] 0 2024-04-24T23:13:51.678 INFO:tasks.workunit.client.1.smithi040.stdout:5/297: rename d2/d7/d8/d1f/d57/d27/d3c/d4a to d2/d7/d8/d64 0 2024-04-24T23:13:51.679 INFO:tasks.workunit.client.1.smithi040.stdout:5/298: chown d2/d2f/l29 337434 1 2024-04-24T23:13:51.680 INFO:tasks.workunit.client.0.smithi031.stdout:9/728: readlink d2/dae/d14/d78/d84/d4a/dab/db7/ld4 0 2024-04-24T23:13:51.682 INFO:tasks.workunit.client.0.smithi031.stdout:1/586: dwrite d1/da/d12/f23 [4194304,4194304] 0 2024-04-24T23:13:51.684 INFO:tasks.workunit.client.0.smithi031.stdout:1/587: write d1/d1d/f7a [1381245,10320] 0 2024-04-24T23:13:51.684 INFO:tasks.workunit.client.0.smithi031.stdout:4/778: stat d3/d20/d2d/d44/c38 0 2024-04-24T23:13:51.685 INFO:tasks.workunit.client.1.smithi040.stdout:4/280: chown da/d45/c51 8558 1 2024-04-24T23:13:51.693 INFO:tasks.workunit.client.0.smithi031.stdout:2/525: link d5/d6/df/fab d5/d8/d45/d5b/d49/d6f/fb6 0 2024-04-24T23:13:51.694 INFO:tasks.workunit.client.1.smithi040.stdout:7/372: chown db/c42 49493362 1 2024-04-24T23:13:51.702 INFO:tasks.workunit.client.0.smithi031.stdout:2/526: dwrite d5/d28/d4e/d5e/fb4 [0,4194304] 0 2024-04-24T23:13:51.703 INFO:tasks.workunit.client.1.smithi040.stdout:3/325: rename d1/d9/l4f to d1/d9/d18/d36/d55/l73 0 2024-04-24T23:13:51.703 INFO:tasks.workunit.client.0.smithi031.stdout:2/527: chown d5/d8/d15/d56/f8f 1 1 2024-04-24T23:13:51.706 INFO:tasks.workunit.client.0.smithi031.stdout:1/588: creat d1/da/d12/d7e/d7b/fb8 x:0 0 0 2024-04-24T23:13:51.706 INFO:tasks.workunit.client.0.smithi031.stdout:0/692: write da/d18/d5a/d7d/dbd/fd1 [1012220,4005] 0 2024-04-24T23:13:51.707 INFO:tasks.workunit.client.0.smithi031.stdout:1/589: dread - d1/d3e/da2/fb0 zero size 2024-04-24T23:13:51.710 INFO:tasks.workunit.client.0.smithi031.stdout:8/760: symlink d8/d1a/d1d/d35/l106 0 2024-04-24T23:13:51.711 INFO:tasks.workunit.client.1.smithi040.stdout:4/281: symlink da/d3f/l68 0 2024-04-24T23:13:51.712 INFO:tasks.workunit.client.1.smithi040.stdout:8/298: sync 2024-04-24T23:13:51.715 INFO:tasks.workunit.client.1.smithi040.stdout:2/389: link da/d1c/f5c da/d1c/d37/d8a/f9a 0 2024-04-24T23:13:51.716 INFO:tasks.workunit.client.0.smithi031.stdout:2/528: symlink d5/d8/d15/d56/d84/d75/lb7 0 2024-04-24T23:13:51.717 INFO:tasks.workunit.client.1.smithi040.stdout:4/282: dwrite da/d1d/d3c/f3e [0,4194304] 0 2024-04-24T23:13:51.718 INFO:tasks.workunit.client.1.smithi040.stdout:9/342: fdatasync d7/f37 0 2024-04-24T23:13:51.720 INFO:tasks.workunit.client.0.smithi031.stdout:0/693: rename da/f9e to da/d22/fe9 0 2024-04-24T23:13:51.721 INFO:tasks.workunit.client.0.smithi031.stdout:0/694: readlink da/l21 0 2024-04-24T23:13:51.722 INFO:tasks.workunit.client.0.smithi031.stdout:0/695: chown da/d86/da3/dcd/de7/ddb/le2 4 1 2024-04-24T23:13:51.724 INFO:tasks.workunit.client.1.smithi040.stdout:5/299: dwrite d2/d7/d8/f1b [0,4194304] 0 2024-04-24T23:13:51.724 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:51 smithi031 ceph-mon[26411]: pgmap v29: 65 pgs: 65 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 233 MiB/s wr, 494 op/s 2024-04-24T23:13:51.728 INFO:tasks.workunit.client.1.smithi040.stdout:3/326: creat d1/d2d/de/d30/d37/f74 x:0 0 0 2024-04-24T23:13:51.733 INFO:tasks.workunit.client.1.smithi040.stdout:6/309: rename d7/de/l14 to d7/de/l5f 0 2024-04-24T23:13:51.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/590: symlink d1/da/d12/d7e/d7b/d4e/d98/lb9 0 2024-04-24T23:13:51.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/761: dwrite d8/fc [0,4194304] 0 2024-04-24T23:13:51.740 INFO:tasks.workunit.client.1.smithi040.stdout:4/283: mkdir da/d1d/d3c/d69 0 2024-04-24T23:13:51.745 INFO:tasks.workunit.client.1.smithi040.stdout:5/300: rename d2/d7/d8/d1f/d57/d59 to d2/d7/d8/d1f/d57/d27/d65 0 2024-04-24T23:13:51.748 INFO:tasks.workunit.client.1.smithi040.stdout:0/373: creat df/d1e/f74 x:0 0 0 2024-04-24T23:13:51.749 INFO:tasks.workunit.client.0.smithi031.stdout:0/696: write da/d86/fb1 [305544,106572] 0 2024-04-24T23:13:51.749 INFO:tasks.workunit.client.1.smithi040.stdout:0/374: write df/d12/f1f [2584042,45419] 0 2024-04-24T23:13:51.752 INFO:tasks.workunit.client.0.smithi031.stdout:1/591: creat d1/dc/d11/fba x:0 0 0 2024-04-24T23:13:51.753 INFO:tasks.workunit.client.0.smithi031.stdout:1/592: write d1/da/d12/d7e/d7b/f6a [4163395,4135] 0 2024-04-24T23:13:51.754 INFO:tasks.workunit.client.1.smithi040.stdout:5/301: dwrite d2/d7/d8/d1f/f4b [0,4194304] 0 2024-04-24T23:13:51.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/762: getdents d8/d15/d37/d43/d5c/de1 0 2024-04-24T23:13:51.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/763: chown d8/d1a/d74/l93 48179540 1 2024-04-24T23:13:51.759 INFO:tasks.workunit.client.0.smithi031.stdout:2/529: truncate d5/d6/f35 1066845 0 2024-04-24T23:13:51.759 INFO:tasks.workunit.client.1.smithi040.stdout:5/302: dread d2/d7/d8/d1f/f38 [0,4194304] 0 2024-04-24T23:13:51.761 INFO:tasks.workunit.client.1.smithi040.stdout:9/343: creat d7/da/d17/d40/d55/d61/f75 x:0 0 0 2024-04-24T23:13:51.761 INFO:tasks.workunit.client.0.smithi031.stdout:1/593: rmdir d1/da/d10/d52 39 2024-04-24T23:13:51.763 INFO:tasks.workunit.client.1.smithi040.stdout:6/310: dread d7/f9 [0,4194304] 0 2024-04-24T23:13:51.764 INFO:tasks.workunit.client.1.smithi040.stdout:9/344: truncate d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d 5113804 0 2024-04-24T23:13:51.764 INFO:tasks.workunit.client.0.smithi031.stdout:8/764: mkdir d8/d1a/d1d/d35/d58/d107 0 2024-04-24T23:13:51.766 INFO:tasks.workunit.client.0.smithi031.stdout:2/530: dwrite d5/d28/d4e/d7f/fa9 [0,4194304] 0 2024-04-24T23:13:51.766 INFO:tasks.workunit.client.0.smithi031.stdout:1/594: mkdir d1/dc/d11/d21/d5a/dbb 0 2024-04-24T23:13:51.767 INFO:tasks.workunit.client.1.smithi040.stdout:8/299: link d0/d16/d1b/d47/f48 d0/d8/dc/f63 0 2024-04-24T23:13:51.768 INFO:tasks.workunit.client.0.smithi031.stdout:2/531: chown d5/d28/d43/f97 1292460333 1 2024-04-24T23:13:51.770 INFO:tasks.workunit.client.1.smithi040.stdout:2/390: creat da/f9b x:0 0 0 2024-04-24T23:13:51.771 INFO:tasks.workunit.client.0.smithi031.stdout:8/765: readlink d8/d1a/d1d/d35/d58/d5b/lf2 0 2024-04-24T23:13:51.773 INFO:tasks.workunit.client.1.smithi040.stdout:5/303: mkdir d2/d7/d8/d1f/d57/d27/d66 0 2024-04-24T23:13:51.773 INFO:tasks.workunit.client.1.smithi040.stdout:5/304: stat d2/d7 0 2024-04-24T23:13:51.775 INFO:tasks.workunit.client.1.smithi040.stdout:5/305: write d2/d7/d8/d1f/d57/d27/f54 [14740,42242] 0 2024-04-24T23:13:51.776 INFO:tasks.workunit.client.1.smithi040.stdout:5/306: fsync d2/d7/f3b 0 2024-04-24T23:13:51.776 INFO:tasks.workunit.client.1.smithi040.stdout:3/327: write d1/fd [7826118,23471] 0 2024-04-24T23:13:51.777 INFO:tasks.workunit.client.1.smithi040.stdout:0/375: mknod df/d1e/d2b/d6e/c75 0 2024-04-24T23:13:51.779 INFO:tasks.workunit.client.0.smithi031.stdout:1/595: dwrite d1/dc/f2b [0,4194304] 0 2024-04-24T23:13:51.784 INFO:tasks.workunit.client.1.smithi040.stdout:2/391: rename da/d29/d55/f62 to da/d10/d4d/d6c/f9c 0 2024-04-24T23:13:51.785 INFO:tasks.workunit.client.1.smithi040.stdout:5/307: dwrite d2/d7/d8/d1f/d57/f25 [0,4194304] 0 2024-04-24T23:13:51.786 INFO:tasks.workunit.client.1.smithi040.stdout:0/376: dwrite df/d12/d1a/f4e [4194304,4194304] 0 2024-04-24T23:13:51.787 INFO:tasks.workunit.client.0.smithi031.stdout:0/697: dwrite da/d11/fa0 [0,4194304] 0 2024-04-24T23:13:51.790 INFO:tasks.workunit.client.0.smithi031.stdout:1/596: fdatasync d1/da/d10/d52/d62/fb7 0 2024-04-24T23:13:51.797 INFO:tasks.workunit.client.1.smithi040.stdout:5/308: dread d2/d7/d8/d1f/f51 [0,4194304] 0 2024-04-24T23:13:51.799 INFO:tasks.workunit.client.1.smithi040.stdout:3/328: mknod d1/d2d/d44/c75 0 2024-04-24T23:13:51.799 INFO:tasks.workunit.client.1.smithi040.stdout:0/377: dwrite df/d12/d1a/f67 [0,4194304] 0 2024-04-24T23:13:51.800 INFO:tasks.workunit.client.1.smithi040.stdout:5/309: dread - d2/d7/d8/db/f56 zero size 2024-04-24T23:13:51.804 INFO:tasks.workunit.client.0.smithi031.stdout:0/698: symlink da/d20/lea 0 2024-04-24T23:13:51.808 INFO:tasks.workunit.client.1.smithi040.stdout:1/335: sync 2024-04-24T23:13:51.812 INFO:tasks.workunit.client.0.smithi031.stdout:2/532: link d5/d6/d29/l2e d5/d8/d15/lb8 0 2024-04-24T23:13:51.814 INFO:tasks.workunit.client.0.smithi031.stdout:1/597: dread d1/d3e/da2/fb2 [0,4194304] 0 2024-04-24T23:13:51.815 INFO:tasks.workunit.client.0.smithi031.stdout:2/533: write d5/d8/d45/d5b/d49/f6d [887074,42678] 0 2024-04-24T23:13:51.817 INFO:tasks.workunit.client.0.smithi031.stdout:8/766: dwrite d8/d1a/d1d/f7d [0,4194304] 0 2024-04-24T23:13:51.821 INFO:tasks.workunit.client.1.smithi040.stdout:6/311: creat d7/f60 x:0 0 0 2024-04-24T23:13:51.825 INFO:tasks.workunit.client.0.smithi031.stdout:2/534: mknod d5/d8/d45/d5b/cb9 0 2024-04-24T23:13:51.826 INFO:tasks.workunit.client.1.smithi040.stdout:3/329: mknod d1/d2a/d6a/c76 0 2024-04-24T23:13:51.831 INFO:tasks.workunit.client.1.smithi040.stdout:1/336: dread da/d12/d18/d2a/f3d [0,4194304] 0 2024-04-24T23:13:51.832 INFO:tasks.workunit.client.1.smithi040.stdout:0/378: symlink df/d14/l76 0 2024-04-24T23:13:51.835 INFO:tasks.workunit.client.1.smithi040.stdout:3/330: rename d1/d2d/d44/l6b to d1/d9/d15/d45/l77 0 2024-04-24T23:13:51.835 INFO:tasks.workunit.client.1.smithi040.stdout:0/379: rename df to df/d1e/d20/d21/d41/d52/d77 22 2024-04-24T23:13:51.835 INFO:tasks.workunit.client.1.smithi040.stdout:9/345: link d7/da/d17/d40/c45 d7/da/d17/d40/d55/d61/d6c/c76 0 2024-04-24T23:13:51.835 INFO:tasks.workunit.client.1.smithi040.stdout:6/312: dread d7/d1a/d20/f23 [0,4194304] 0 2024-04-24T23:13:51.836 INFO:tasks.workunit.client.1.smithi040.stdout:5/310: creat d2/d7/d8/d1f/f67 x:0 0 0 2024-04-24T23:13:51.837 INFO:tasks.workunit.client.1.smithi040.stdout:0/380: chown df/d12/d1a/d71/c5a 1752151296 1 2024-04-24T23:13:51.837 INFO:tasks.workunit.client.1.smithi040.stdout:3/331: write d1/d2d/de/d30/d37/f74 [639894,90947] 0 2024-04-24T23:13:51.838 INFO:tasks.workunit.client.1.smithi040.stdout:1/337: mknod da/d35/d61/c76 0 2024-04-24T23:13:51.840 INFO:tasks.workunit.client.1.smithi040.stdout:6/313: dread d7/f9 [0,4194304] 0 2024-04-24T23:13:51.840 INFO:tasks.workunit.client.1.smithi040.stdout:5/311: creat d2/d7/d8/d64/f68 x:0 0 0 2024-04-24T23:13:51.840 INFO:tasks.workunit.client.1.smithi040.stdout:3/332: dread - d1/d9/d15/d45/f4c zero size 2024-04-24T23:13:51.841 INFO:tasks.workunit.client.1.smithi040.stdout:1/338: fsync da/f71 0 2024-04-24T23:13:51.841 INFO:tasks.workunit.client.1.smithi040.stdout:0/381: dread df/d1e/d20/d21/f32 [0,4194304] 0 2024-04-24T23:13:51.843 INFO:tasks.workunit.client.1.smithi040.stdout:6/314: rmdir d7/d1a 39 2024-04-24T23:13:51.846 INFO:tasks.workunit.client.1.smithi040.stdout:0/382: symlink df/d12/d1a/l78 0 2024-04-24T23:13:51.846 INFO:tasks.workunit.client.1.smithi040.stdout:3/333: chown d1/d2a/d6a/c69 1104 1 2024-04-24T23:13:51.847 INFO:tasks.workunit.client.1.smithi040.stdout:9/346: dread d7/da/d17/d40/d55/d2f/f6f [0,4194304] 0 2024-04-24T23:13:51.847 INFO:tasks.workunit.client.1.smithi040.stdout:5/312: dread d2/d7/d8/d1f/f4b [0,4194304] 0 2024-04-24T23:13:51.849 INFO:tasks.workunit.client.1.smithi040.stdout:0/383: mkdir df/d1e/d20/d21/d51/d79 0 2024-04-24T23:13:51.850 INFO:tasks.workunit.client.1.smithi040.stdout:9/347: fsync d7/da/d17/d40/d55/d2f/d1f/d31/f35 0 2024-04-24T23:13:51.851 INFO:tasks.workunit.client.1.smithi040.stdout:0/384: write df/d12/d4b/d62/d6b/f73 [999258,45188] 0 2024-04-24T23:13:51.851 INFO:tasks.workunit.client.1.smithi040.stdout:5/313: dread d2/d7/d8/db/d1e/f46 [0,4194304] 0 2024-04-24T23:13:51.852 INFO:tasks.workunit.client.1.smithi040.stdout:0/385: fdatasync df/d12/d4b/d62/f69 0 2024-04-24T23:13:51.853 INFO:tasks.workunit.client.1.smithi040.stdout:3/334: link d1/d9/d18/d36/d61/f54 d1/d9/d39/f78 0 2024-04-24T23:13:51.853 INFO:tasks.workunit.client.1.smithi040.stdout:9/348: dread - d7/da/d17/d40/d55/d61/d6c/f71 zero size 2024-04-24T23:13:51.854 INFO:tasks.workunit.client.1.smithi040.stdout:5/314: write d2/d7/d8/d1f/f67 [883592,115742] 0 2024-04-24T23:13:51.857 INFO:tasks.workunit.client.1.smithi040.stdout:5/315: truncate d2/d7/d8/d1f/f38 4593516 0 2024-04-24T23:13:51.858 INFO:tasks.workunit.client.1.smithi040.stdout:2/392: write da/d10/f84 [1553743,121625] 0 2024-04-24T23:13:51.858 INFO:tasks.workunit.client.1.smithi040.stdout:3/335: truncate d1/d9/f21 455649 0 2024-04-24T23:13:51.861 INFO:tasks.workunit.client.0.smithi031.stdout:8/767: dread d8/d1a/f2f [0,4194304] 0 2024-04-24T23:13:51.861 INFO:tasks.workunit.client.1.smithi040.stdout:9/349: dread d7/da/d17/d40/d55/d2f/d1f/d34/f49 [0,4194304] 0 2024-04-24T23:13:51.863 INFO:tasks.workunit.client.1.smithi040.stdout:2/393: mknod da/d10/d7a/c9d 0 2024-04-24T23:13:51.864 INFO:tasks.workunit.client.1.smithi040.stdout:0/386: dread df/d12/d1a/f3a [0,4194304] 0 2024-04-24T23:13:51.866 INFO:tasks.workunit.client.1.smithi040.stdout:5/316: dwrite d2/d7/d8/d1f/d31/f4e [0,4194304] 0 2024-04-24T23:13:51.866 INFO:tasks.workunit.client.1.smithi040.stdout:5/317: chown d2/d7/d8/d64 17228579 1 2024-04-24T23:13:51.867 INFO:tasks.workunit.client.1.smithi040.stdout:3/336: creat d1/f79 x:0 0 0 2024-04-24T23:13:51.870 INFO:tasks.workunit.client.1.smithi040.stdout:1/339: dread da/d12/d14/f24 [0,4194304] 0 2024-04-24T23:13:51.872 INFO:tasks.workunit.client.1.smithi040.stdout:9/350: symlink d7/d5e/l77 0 2024-04-24T23:13:51.875 INFO:tasks.workunit.client.0.smithi031.stdout:2/535: dwrite d5/d8/d15/d56/d84/d75/f9d [0,4194304] 0 2024-04-24T23:13:51.875 INFO:tasks.workunit.client.1.smithi040.stdout:0/387: mknod df/d1e/d20/d21/c7a 0 2024-04-24T23:13:51.876 INFO:tasks.workunit.client.0.smithi031.stdout:2/536: stat d5/d6/d29/d4f/c8a 0 2024-04-24T23:13:51.876 INFO:tasks.workunit.client.1.smithi040.stdout:5/318: dread d2/d7/d8/d1f/d57/d27/d65/f5a [0,4194304] 0 2024-04-24T23:13:51.877 INFO:tasks.workunit.client.1.smithi040.stdout:9/351: readlink d7/da/d17/d40/d55/d2f/d1f/d31/l70 0 2024-04-24T23:13:51.879 INFO:tasks.workunit.client.1.smithi040.stdout:0/388: write df/d12/d4b/d62/f69 [1030193,85532] 0 2024-04-24T23:13:51.880 INFO:tasks.workunit.client.1.smithi040.stdout:3/337: dwrite d1/d2d/d44/f6c [0,4194304] 0 2024-04-24T23:13:51.881 INFO:tasks.workunit.client.1.smithi040.stdout:1/340: symlink da/d35/d38/d47/l77 0 2024-04-24T23:13:51.881 INFO:tasks.workunit.client.1.smithi040.stdout:9/352: dread - d7/da/d17/d40/d55/d61/f75 zero size 2024-04-24T23:13:51.881 INFO:tasks.workunit.client.0.smithi031.stdout:8/768: dread d8/d15/d37/f5e [0,4194304] 0 2024-04-24T23:13:51.883 INFO:tasks.workunit.client.1.smithi040.stdout:0/389: read - df/d12/f4c zero size 2024-04-24T23:13:51.886 INFO:tasks.workunit.client.0.smithi031.stdout:2/537: creat d5/fba x:0 0 0 2024-04-24T23:13:51.892 INFO:tasks.workunit.client.1.smithi040.stdout:1/341: creat da/d12/d18/f78 x:0 0 0 2024-04-24T23:13:51.893 INFO:tasks.workunit.client.0.smithi031.stdout:8/769: creat d8/d15/f108 x:0 0 0 2024-04-24T23:13:51.897 INFO:tasks.workunit.client.1.smithi040.stdout:5/319: unlink d2/cd 0 2024-04-24T23:13:51.899 INFO:tasks.workunit.client.1.smithi040.stdout:2/394: dwrite da/d1c/d37/d39/f76 [0,4194304] 0 2024-04-24T23:13:51.900 INFO:tasks.workunit.client.0.smithi031.stdout:8/770: symlink d8/d15/d37/d43/d7b/dbf/l109 0 2024-04-24T23:13:51.902 INFO:tasks.workunit.client.1.smithi040.stdout:0/390: getdents df/d1e/d20/d21/d41/d52 0 2024-04-24T23:13:51.906 INFO:tasks.workunit.client.0.smithi031.stdout:8/771: mknod d8/d1a/d1d/d50/dc0/d9d/c10a 0 2024-04-24T23:13:51.906 INFO:tasks.workunit.client.1.smithi040.stdout:7/373: sync 2024-04-24T23:13:51.907 INFO:tasks.workunit.client.0.smithi031.stdout:8/772: write d8/d1a/d1d/d35/fcd [230907,86206] 0 2024-04-24T23:13:51.909 INFO:tasks.workunit.client.1.smithi040.stdout:9/353: link d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f50 d7/da/d17/d40/d55/d2f/d1f/d34/f78 0 2024-04-24T23:13:51.911 INFO:tasks.workunit.client.1.smithi040.stdout:2/395: rename da/d1c/d4f/l98 to da/d1c/d4f/l9e 0 2024-04-24T23:13:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:51 smithi040 ceph-mon[33128]: pgmap v29: 65 pgs: 65 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 58 MiB/s rd, 233 MiB/s wr, 494 op/s 2024-04-24T23:13:51.915 INFO:tasks.workunit.client.1.smithi040.stdout:0/391: rmdir df/d12/d4b/d62/d6b 39 2024-04-24T23:13:51.916 INFO:tasks.workunit.client.0.smithi031.stdout:2/538: dwrite d5/d8/f94 [0,4194304] 0 2024-04-24T23:13:51.917 INFO:tasks.workunit.client.1.smithi040.stdout:3/338: dwrite d1/d2d/de/f2f [0,4194304] 0 2024-04-24T23:13:51.917 INFO:tasks.workunit.client.1.smithi040.stdout:5/320: chown d2/d2f/d14/c19 10737427 1 2024-04-24T23:13:51.917 INFO:tasks.workunit.client.1.smithi040.stdout:0/392: stat df/d12/f4c 0 2024-04-24T23:13:51.920 INFO:tasks.workunit.client.0.smithi031.stdout:2/539: creat d5/d6/df/fbb x:0 0 0 2024-04-24T23:13:51.921 INFO:tasks.workunit.client.1.smithi040.stdout:5/321: write d2/d7/d8/d1f/d57/d27/d3c/f47 [2323569,5463] 0 2024-04-24T23:13:51.921 INFO:tasks.workunit.client.1.smithi040.stdout:9/354: dwrite d7/da/d17/d40/d55/d2f/d1f/d34/f47 [0,4194304] 0 2024-04-24T23:13:51.922 INFO:tasks.workunit.client.0.smithi031.stdout:2/540: fsync d5/d8/d15/f93 0 2024-04-24T23:13:51.926 INFO:tasks.workunit.client.0.smithi031.stdout:8/773: creat d8/d15/d37/f10b x:0 0 0 2024-04-24T23:13:51.927 INFO:tasks.workunit.client.1.smithi040.stdout:9/355: write d7/da/d17/d40/d55/d2f/d1f/d31/f35 [344263,117241] 0 2024-04-24T23:13:51.931 INFO:tasks.workunit.client.1.smithi040.stdout:1/342: creat da/d12/d18/d2a/f79 x:0 0 0 2024-04-24T23:13:51.932 INFO:tasks.workunit.client.0.smithi031.stdout:8/774: getdents d8/d15/d37/d43/d5c/de1 0 2024-04-24T23:13:51.934 INFO:tasks.workunit.client.0.smithi031.stdout:2/541: rename d5/d28/d4e/f82 to d5/d8/d15/fbc 0 2024-04-24T23:13:51.934 INFO:tasks.workunit.client.1.smithi040.stdout:1/343: readlink da/d12/d14/d58/l63 0 2024-04-24T23:13:51.934 INFO:tasks.workunit.client.1.smithi040.stdout:3/339: symlink d1/d2d/de/d30/d37/l7a 0 2024-04-24T23:13:51.936 INFO:tasks.workunit.client.1.smithi040.stdout:1/344: chown da/d12/d18/f49 20701 1 2024-04-24T23:13:51.937 INFO:tasks.workunit.client.0.smithi031.stdout:2/542: truncate d5/f25 3343099 0 2024-04-24T23:13:51.937 INFO:tasks.workunit.client.1.smithi040.stdout:9/356: dwrite d7/da/d17/d40/d55/d2f/d28/f6b [0,4194304] 0 2024-04-24T23:13:51.938 INFO:tasks.workunit.client.0.smithi031.stdout:2/543: write d5/d8/d45/d5b/d49/f6d [3294762,121075] 0 2024-04-24T23:13:51.939 INFO:tasks.workunit.client.1.smithi040.stdout:2/396: mkdir da/d9f 0 2024-04-24T23:13:51.940 INFO:tasks.workunit.client.0.smithi031.stdout:8/775: creat d8/d1a/f10c x:0 0 0 2024-04-24T23:13:51.946 INFO:tasks.workunit.client.1.smithi040.stdout:0/393: dread df/d12/d1a/d71/f28 [0,4194304] 0 2024-04-24T23:13:51.961 INFO:tasks.workunit.client.0.smithi031.stdout:8/776: dread d8/d15/d37/d43/d7b/dd3/f100 [0,4194304] 0 2024-04-24T23:13:51.968 INFO:tasks.workunit.client.0.smithi031.stdout:8/777: dread d8/d1a/f2f [0,4194304] 0 2024-04-24T23:13:51.969 INFO:tasks.workunit.client.0.smithi031.stdout:8/778: fdatasync d8/d15/d37/d43/d5c/db5/fe6 0 2024-04-24T23:13:51.987 INFO:tasks.workunit.client.1.smithi040.stdout:8/300: sync 2024-04-24T23:13:51.987 INFO:tasks.workunit.client.1.smithi040.stdout:4/284: sync 2024-04-24T23:13:51.988 INFO:tasks.workunit.client.1.smithi040.stdout:4/285: write da/d3f/f4b [822127,69159] 0 2024-04-24T23:13:51.988 INFO:tasks.workunit.client.1.smithi040.stdout:8/301: write d0/d16/d19/d4f/f4e [299415,86324] 0 2024-04-24T23:13:52.048 INFO:tasks.workunit.client.1.smithi040.stdout:6/315: sync 2024-04-24T23:13:52.063 INFO:tasks.workunit.client.1.smithi040.stdout:8/302: sync 2024-04-24T23:13:52.064 INFO:tasks.workunit.client.1.smithi040.stdout:2/397: sync 2024-04-24T23:13:52.064 INFO:tasks.workunit.client.1.smithi040.stdout:8/303: chown d0/l13 11798 1 2024-04-24T23:13:52.065 INFO:tasks.workunit.client.1.smithi040.stdout:2/398: write da/d1c/d37/d39/d3f/f7b [5237787,75370] 0 2024-04-24T23:13:52.065 INFO:tasks.workunit.client.1.smithi040.stdout:8/304: chown d0/d16/l15 0 1 2024-04-24T23:13:52.109 INFO:tasks.workunit.client.0.smithi031.stdout:6/715: sync 2024-04-24T23:13:52.182 INFO:tasks.workunit.client.0.smithi031.stdout:3/585: sync 2024-04-24T23:13:52.182 INFO:tasks.workunit.client.0.smithi031.stdout:5/584: sync 2024-04-24T23:13:52.182 INFO:tasks.workunit.client.0.smithi031.stdout:7/629: sync 2024-04-24T23:13:52.184 INFO:tasks.workunit.client.0.smithi031.stdout:7/630: read d1/d45/d75/d9f/d2c/f4c [126075,57119] 0 2024-04-24T23:13:52.187 INFO:tasks.workunit.client.1.smithi040.stdout:1/345: rmdir da/d12 39 2024-04-24T23:13:52.189 INFO:tasks.workunit.client.1.smithi040.stdout:5/322: chown d2/d7/d8/d1f/d57/l63 19 1 2024-04-24T23:13:52.192 INFO:tasks.workunit.client.1.smithi040.stdout:5/323: dread d2/d7/d8/f39 [0,4194304] 0 2024-04-24T23:13:52.202 INFO:tasks.workunit.client.1.smithi040.stdout:7/374: getdents db/d2f 0 2024-04-24T23:13:52.205 INFO:tasks.workunit.client.1.smithi040.stdout:4/286: rmdir da/d3f/d40/d47 39 2024-04-24T23:13:52.208 INFO:tasks.workunit.client.1.smithi040.stdout:6/316: write d7/d1a/d20/d51/d2f/f5b [549502,76467] 0 2024-04-24T23:13:52.210 INFO:tasks.workunit.client.1.smithi040.stdout:2/399: mknod da/d10/ca0 0 2024-04-24T23:13:52.210 INFO:tasks.workunit.client.1.smithi040.stdout:2/400: readlink da/d1c/d37/d39/d3f/l49 0 2024-04-24T23:13:52.211 INFO:tasks.workunit.client.1.smithi040.stdout:2/401: fdatasync da/d10/d7a/f8d 0 2024-04-24T23:13:52.216 INFO:tasks.workunit.client.1.smithi040.stdout:9/357: mknod d7/da/c79 0 2024-04-24T23:13:52.216 INFO:tasks.workunit.client.1.smithi040.stdout:1/346: dread da/d12/d14/f68 [0,4194304] 0 2024-04-24T23:13:52.218 INFO:tasks.workunit.client.1.smithi040.stdout:7/375: symlink db/d12/d7d/l87 0 2024-04-24T23:13:52.218 INFO:tasks.workunit.client.1.smithi040.stdout:7/376: chown db/d12/d3f/f74 5842492 1 2024-04-24T23:13:52.219 INFO:tasks.workunit.client.1.smithi040.stdout:7/377: write db/d12/f29 [502971,108708] 0 2024-04-24T23:13:52.223 INFO:tasks.workunit.client.1.smithi040.stdout:3/340: write d1/d9/f21 [1229514,39738] 0 2024-04-24T23:13:52.224 INFO:tasks.workunit.client.1.smithi040.stdout:8/305: creat d0/d8/d55/f64 x:0 0 0 2024-04-24T23:13:52.226 INFO:tasks.workunit.client.1.smithi040.stdout:4/287: dwrite da/d3f/f56 [0,4194304] 0 2024-04-24T23:13:52.229 INFO:tasks.workunit.client.1.smithi040.stdout:9/358: creat d7/da/d17/d40/d55/d61/d6c/f7a x:0 0 0 2024-04-24T23:13:52.230 INFO:tasks.workunit.client.1.smithi040.stdout:9/359: readlink d7/da/d17/d40/d55/d2f/d1f/l4b 0 2024-04-24T23:13:52.233 INFO:tasks.workunit.client.1.smithi040.stdout:0/394: link df/d1e/d20/f3f df/d1e/f7b 0 2024-04-24T23:13:52.234 INFO:tasks.workunit.client.1.smithi040.stdout:0/395: stat df/d12/d1a/d71 0 2024-04-24T23:13:52.237 INFO:tasks.workunit.client.1.smithi040.stdout:7/378: creat db/d12/d20/d52/d44/f88 x:0 0 0 2024-04-24T23:13:52.241 INFO:tasks.workunit.client.1.smithi040.stdout:0/396: dread df/d1e/f40 [0,4194304] 0 2024-04-24T23:13:52.242 INFO:tasks.workunit.client.1.smithi040.stdout:0/397: write df/f63 [849059,16600] 0 2024-04-24T23:13:52.242 INFO:tasks.workunit.client.0.smithi031.stdout:8/779: mkdir d8/d15/d37/d43/d7b/d87/d10d 0 2024-04-24T23:13:52.244 INFO:tasks.workunit.client.0.smithi031.stdout:6/716: chown d13/d1b/d32/ca2 37145295 1 2024-04-24T23:13:52.245 INFO:tasks.workunit.client.1.smithi040.stdout:0/398: truncate df/d1e/d20/f3f 540183 0 2024-04-24T23:13:52.247 INFO:tasks.workunit.client.1.smithi040.stdout:4/288: rename da/d45/c51 to da/d45/d48/c6a 0 2024-04-24T23:13:52.248 INFO:tasks.workunit.client.1.smithi040.stdout:2/402: dwrite da/d29/f42 [0,4194304] 0 2024-04-24T23:13:52.249 INFO:tasks.workunit.client.1.smithi040.stdout:5/324: link d2/d7/d8/d1f/d57/d27/d3c/f47 d2/f69 0 2024-04-24T23:13:52.250 INFO:tasks.workunit.client.0.smithi031.stdout:3/586: mknod d6/df/d16/d2a/d95/d9b/d1a/d25/cb4 0 2024-04-24T23:13:52.255 INFO:tasks.workunit.client.1.smithi040.stdout:2/403: dwrite da/d1c/d37/f41 [0,4194304] 0 2024-04-24T23:13:52.255 INFO:tasks.workunit.client.1.smithi040.stdout:7/379: truncate db/f24 2353063 0 2024-04-24T23:13:52.256 INFO:tasks.workunit.client.1.smithi040.stdout:2/404: fdatasync da/f40 0 2024-04-24T23:13:52.257 INFO:tasks.workunit.client.1.smithi040.stdout:2/405: stat da/d1c/d7e 0 2024-04-24T23:13:52.259 INFO:tasks.workunit.client.1.smithi040.stdout:6/317: creat d7/d1a/d20/f61 x:0 0 0 2024-04-24T23:13:52.259 INFO:tasks.workunit.client.1.smithi040.stdout:2/406: read da/d10/d15/f2e [281722,49523] 0 2024-04-24T23:13:52.260 INFO:tasks.workunit.client.1.smithi040.stdout:7/380: dread db/d2f/f35 [0,4194304] 0 2024-04-24T23:13:52.272 INFO:tasks.workunit.client.0.smithi031.stdout:5/585: dwrite d0/d6/d12/f63 [0,4194304] 0 2024-04-24T23:13:52.272 INFO:tasks.workunit.client.0.smithi031.stdout:8/780: unlink d8/d15/ca9 0 2024-04-24T23:13:52.272 INFO:tasks.workunit.client.0.smithi031.stdout:3/587: dread d6/df/d16/d2a/f31 [0,4194304] 0 2024-04-24T23:13:52.274 INFO:tasks.workunit.client.0.smithi031.stdout:3/588: dread - d6/df/d16/d2a/d95/d9b/d29/f79 zero size 2024-04-24T23:13:52.275 INFO:tasks.workunit.client.0.smithi031.stdout:6/717: creat d13/d15/d1d/d23/d4d/ff4 x:0 0 0 2024-04-24T23:13:52.277 INFO:tasks.workunit.client.0.smithi031.stdout:3/589: chown d6/df/d16/d2a/d95/d9b/d1a/d25/f54 976687491 1 2024-04-24T23:13:52.277 INFO:tasks.workunit.client.1.smithi040.stdout:5/325: fsync d2/d7/d8/d1f/d57/d27/d3c/f47 0 2024-04-24T23:13:52.282 INFO:tasks.workunit.client.0.smithi031.stdout:7/631: creat d1/d45/d75/d9f/d2c/fd8 x:0 0 0 2024-04-24T23:13:52.282 INFO:tasks.workunit.client.1.smithi040.stdout:5/326: dread d2/d7/d8/d1f/d57/d3e/f58 [0,4194304] 0 2024-04-24T23:13:52.283 INFO:tasks.workunit.client.0.smithi031.stdout:7/632: chown d1/d45/d75/d81/db8 996377 1 2024-04-24T23:13:52.283 INFO:tasks.workunit.client.0.smithi031.stdout:3/590: dwrite d6/fa1 [0,4194304] 0 2024-04-24T23:13:52.285 INFO:tasks.workunit.client.0.smithi031.stdout:7/633: chown d1/d2/dd/f97 41220643 1 2024-04-24T23:13:52.286 INFO:tasks.workunit.client.1.smithi040.stdout:3/341: write d1/d9/d2e/f5f [193824,10271] 0 2024-04-24T23:13:52.290 INFO:tasks.workunit.client.0.smithi031.stdout:8/781: mknod d8/d15/dcf/c10e 0 2024-04-24T23:13:52.291 INFO:tasks.workunit.client.1.smithi040.stdout:2/407: chown da/l13 5923 1 2024-04-24T23:13:52.292 INFO:tasks.workunit.client.1.smithi040.stdout:2/408: chown da/d29/d55 10609827 1 2024-04-24T23:13:52.299 INFO:tasks.workunit.client.0.smithi031.stdout:9/729: sync 2024-04-24T23:13:52.299 INFO:tasks.workunit.client.1.smithi040.stdout:7/381: mknod db/d12/d20/d52/d4b/c89 0 2024-04-24T23:13:52.300 INFO:tasks.workunit.client.1.smithi040.stdout:8/306: link d0/d8/c3e d0/d8/dc/d34/c65 0 2024-04-24T23:13:52.300 INFO:tasks.workunit.client.1.smithi040.stdout:7/382: chown db/d12/d20/d56/c83 492 1 2024-04-24T23:13:52.301 INFO:tasks.workunit.client.1.smithi040.stdout:8/307: dread - d0/d8/dc/d34/f3c zero size 2024-04-24T23:13:52.301 INFO:tasks.workunit.client.0.smithi031.stdout:8/782: mknod d8/d15/dcf/c10f 0 2024-04-24T23:13:52.302 INFO:tasks.workunit.client.1.smithi040.stdout:8/308: write d0/d8/d22/f62 [674999,4263] 0 2024-04-24T23:13:52.303 INFO:tasks.workunit.client.0.smithi031.stdout:8/783: stat d8/d15/d37/d62/fb3 0 2024-04-24T23:13:52.303 INFO:tasks.workunit.client.1.smithi040.stdout:8/309: write d0/d16/f57 [585486,14600] 0 2024-04-24T23:13:52.307 INFO:tasks.workunit.client.0.smithi031.stdout:5/586: dwrite d0/d6/d32/f4a [0,4194304] 0 2024-04-24T23:13:52.310 INFO:tasks.workunit.client.0.smithi031.stdout:6/718: link d13/d15/d1d/d23/d44/d47/d69/faa d13/d1b/de3/ff5 0 2024-04-24T23:13:52.311 INFO:tasks.workunit.client.0.smithi031.stdout:3/591: dwrite d6/df/d16/d2a/d95/d9b/d1a/d25/f3d [4194304,4194304] 0 2024-04-24T23:13:52.316 INFO:tasks.workunit.client.0.smithi031.stdout:3/592: dread d6/df/d16/d2a/f68 [0,4194304] 0 2024-04-24T23:13:52.317 INFO:tasks.workunit.client.0.smithi031.stdout:8/784: fdatasync d8/d15/d37/d43/d5c/fba 0 2024-04-24T23:13:52.318 INFO:tasks.workunit.client.1.smithi040.stdout:5/327: chown d2/d2f/d14/c18 28207 1 2024-04-24T23:13:52.319 INFO:tasks.workunit.client.0.smithi031.stdout:3/593: write d6/df/d16/d2a/d95/d9b/f7c [2842081,2560] 0 2024-04-24T23:13:52.322 INFO:tasks.workunit.client.1.smithi040.stdout:3/342: rename d1/d9/d15/d45/f49 to d1/d9/d18/d36/d55/d5a/f7b 0 2024-04-24T23:13:52.324 INFO:tasks.workunit.client.1.smithi040.stdout:2/409: creat da/d10/fa1 x:0 0 0 2024-04-24T23:13:52.325 INFO:tasks.workunit.client.0.smithi031.stdout:8/785: rmdir d8/d15/d9a 39 2024-04-24T23:13:52.329 INFO:tasks.workunit.client.0.smithi031.stdout:6/719: symlink d13/d15/d1d/lf6 0 2024-04-24T23:13:52.333 INFO:tasks.workunit.client.0.smithi031.stdout:8/786: creat d8/d15/d37/d43/daf/f110 x:0 0 0 2024-04-24T23:13:52.334 INFO:tasks.workunit.client.0.smithi031.stdout:9/730: getdents d2 0 2024-04-24T23:13:52.335 INFO:tasks.workunit.client.1.smithi040.stdout:9/360: link l1 d7/da/d17/d40/d55/d2f/d1f/d31/l7b 0 2024-04-24T23:13:52.337 INFO:tasks.workunit.client.1.smithi040.stdout:5/328: readlink d2/d7/d8/d1f/d57/d27/l4c 0 2024-04-24T23:13:52.340 INFO:tasks.workunit.client.0.smithi031.stdout:9/731: dwrite d2/f54 [0,4194304] 0 2024-04-24T23:13:52.340 INFO:tasks.workunit.client.0.smithi031.stdout:5/587: write d0/d6/d32/f5f [3912854,81547] 0 2024-04-24T23:13:52.344 INFO:tasks.workunit.client.1.smithi040.stdout:3/343: mkdir d1/d9/d18/d36/d55/d5a/d7c 0 2024-04-24T23:13:52.344 INFO:tasks.workunit.client.1.smithi040.stdout:3/344: dread - d1/f59 zero size 2024-04-24T23:13:52.350 INFO:tasks.workunit.client.1.smithi040.stdout:2/410: mknod da/d81/ca2 0 2024-04-24T23:13:52.350 INFO:tasks.workunit.client.0.smithi031.stdout:6/720: symlink d13/d15/d1d/d23/d2d/d9e/lf7 0 2024-04-24T23:13:52.350 INFO:tasks.workunit.client.0.smithi031.stdout:6/721: readlink d13/d15/l64 0 2024-04-24T23:13:52.350 INFO:tasks.workunit.client.0.smithi031.stdout:6/722: write d13/f3f [5159623,48849] 0 2024-04-24T23:13:52.351 INFO:tasks.workunit.client.0.smithi031.stdout:8/787: symlink d8/d15/d37/d43/d7b/dbf/dcc/df1/l111 0 2024-04-24T23:13:52.352 INFO:tasks.workunit.client.0.smithi031.stdout:6/723: fsync d13/d15/d1d/d23/d24/d8b/fcd 0 2024-04-24T23:13:52.352 INFO:tasks.workunit.client.1.smithi040.stdout:7/383: dwrite db/d2f/d37/d7e/f7f [0,4194304] 0 2024-04-24T23:13:52.353 INFO:tasks.workunit.client.0.smithi031.stdout:9/732: creat d2/fdf x:0 0 0 2024-04-24T23:13:52.354 INFO:tasks.workunit.client.1.smithi040.stdout:0/399: getdents df/d12/d1a 0 2024-04-24T23:13:52.355 INFO:tasks.workunit.client.1.smithi040.stdout:1/347: sync 2024-04-24T23:13:52.356 INFO:tasks.workunit.client.0.smithi031.stdout:5/588: mkdir d0/d6/d3b/d4b/dc/d86/dc3 0 2024-04-24T23:13:52.356 INFO:tasks.workunit.client.1.smithi040.stdout:1/348: dread - da/d12/d14/d58/f73 zero size 2024-04-24T23:13:52.357 INFO:tasks.workunit.client.1.smithi040.stdout:8/310: mkdir d0/d8/d66 0 2024-04-24T23:13:52.358 INFO:tasks.workunit.client.0.smithi031.stdout:3/594: getdents d6/d14/d1d 0 2024-04-24T23:13:52.360 INFO:tasks.workunit.client.0.smithi031.stdout:3/595: readlink d6/df/d16/d2a/d95/d9b/d29/l50 0 2024-04-24T23:13:52.361 INFO:tasks.workunit.client.1.smithi040.stdout:4/289: rmdir da/d3f/d40/d47/d5d 0 2024-04-24T23:13:52.363 INFO:tasks.workunit.client.1.smithi040.stdout:9/361: creat d7/da/d17/d40/d55/d2f/f7c x:0 0 0 2024-04-24T23:13:52.364 INFO:tasks.workunit.client.1.smithi040.stdout:1/349: dwrite da/d12/d18/d2a/d4d/f75 [0,4194304] 0 2024-04-24T23:13:52.365 INFO:tasks.workunit.client.1.smithi040.stdout:1/350: readlink da/d35/l36 0 2024-04-24T23:13:52.367 INFO:tasks.workunit.client.1.smithi040.stdout:3/345: symlink d1/d2d/de/l7d 0 2024-04-24T23:13:52.367 INFO:tasks.workunit.client.1.smithi040.stdout:5/329: creat d2/d7/d8/d1f/d57/d27/d3c/d43/f6a x:0 0 0 2024-04-24T23:13:52.367 INFO:tasks.workunit.client.0.smithi031.stdout:6/724: unlink d13/d15/d1d/lf6 0 2024-04-24T23:13:52.369 INFO:tasks.workunit.client.0.smithi031.stdout:3/596: symlink d6/df/d16/d57/lb5 0 2024-04-24T23:13:52.371 INFO:tasks.workunit.client.1.smithi040.stdout:6/318: link d7/d1a/d20/d51/d2f/c37 d7/d1a/d20/d51/d2f/c62 0 2024-04-24T23:13:52.372 INFO:tasks.workunit.client.0.smithi031.stdout:5/589: read d0/d6/d3b/d4b/dc/f6e [1089452,12055] 0 2024-04-24T23:13:52.374 INFO:tasks.workunit.client.1.smithi040.stdout:6/319: fdatasync d7/d1a/d20/d51/d1e/f49 0 2024-04-24T23:13:52.377 INFO:tasks.workunit.client.1.smithi040.stdout:4/290: chown da/cd 1740 1 2024-04-24T23:13:52.378 INFO:tasks.workunit.client.1.smithi040.stdout:1/351: mknod da/d12/d14/d25/c7a 0 2024-04-24T23:13:52.378 INFO:tasks.workunit.client.1.smithi040.stdout:3/346: dwrite d1/d9/d2e/f5f [0,4194304] 0 2024-04-24T23:13:52.378 INFO:tasks.workunit.client.0.smithi031.stdout:5/590: truncate d0/d6/d32/f17 2237061 0 2024-04-24T23:13:52.380 INFO:tasks.workunit.client.1.smithi040.stdout:0/400: dread df/d1e/f2f [0,4194304] 0 2024-04-24T23:13:52.380 INFO:tasks.workunit.client.1.smithi040.stdout:2/411: mkdir da/d1c/d37/d39/d79/da3 0 2024-04-24T23:13:52.381 INFO:tasks.workunit.client.1.smithi040.stdout:4/291: write da/d1d/f32 [4805394,66302] 0 2024-04-24T23:13:52.382 INFO:tasks.workunit.client.1.smithi040.stdout:5/330: mkdir d2/d7/d8/d64/d6b 0 2024-04-24T23:13:52.382 INFO:tasks.workunit.client.1.smithi040.stdout:8/311: dread d0/d16/d1b/d47/f48 [0,4194304] 0 2024-04-24T23:13:52.384 INFO:tasks.workunit.client.1.smithi040.stdout:5/331: dread - d2/d7/d8/f50 zero size 2024-04-24T23:13:52.385 INFO:tasks.workunit.client.1.smithi040.stdout:8/312: dread - d0/d8/dc/d34/f3c zero size 2024-04-24T23:13:52.385 INFO:tasks.workunit.client.1.smithi040.stdout:8/313: chown d0 213295 1 2024-04-24T23:13:52.387 INFO:tasks.workunit.client.1.smithi040.stdout:9/362: dread d7/f57 [0,4194304] 0 2024-04-24T23:13:52.394 INFO:tasks.workunit.client.0.smithi031.stdout:9/733: dread d2/dae/d14/f19 [0,4194304] 0 2024-04-24T23:13:52.394 INFO:tasks.workunit.client.1.smithi040.stdout:6/320: mknod d7/d1a/d20/d51/d1e/d5d/d52/c63 0 2024-04-24T23:13:52.395 INFO:tasks.workunit.client.0.smithi031.stdout:8/788: write d8/d15/d37/d43/f4b [1986178,126221] 0 2024-04-24T23:13:52.395 INFO:tasks.workunit.client.1.smithi040.stdout:6/321: chown d7/d1a/d20/d51/d1e/d46 119183 1 2024-04-24T23:13:52.396 INFO:tasks.workunit.client.0.smithi031.stdout:5/591: getdents d0/d6/d3b/d4b 0 2024-04-24T23:13:52.397 INFO:tasks.workunit.client.1.smithi040.stdout:1/352: symlink da/d12/d14/l7b 0 2024-04-24T23:13:52.399 INFO:tasks.workunit.client.1.smithi040.stdout:4/292: stat da/d1f/c5e 0 2024-04-24T23:13:52.399 INFO:tasks.workunit.client.0.smithi031.stdout:8/789: rmdir d8/d1a/d1d/d35/d58/d5b 39 2024-04-24T23:13:52.402 INFO:tasks.workunit.client.1.smithi040.stdout:3/347: mkdir d1/d9/d18/d36/d55/d7e 0 2024-04-24T23:13:52.403 INFO:tasks.workunit.client.1.smithi040.stdout:3/348: dread - d1/d9/d18/f19 zero size 2024-04-24T23:13:52.404 INFO:tasks.workunit.client.1.smithi040.stdout:3/349: write d1/d2d/f24 [4871904,64779] 0 2024-04-24T23:13:52.405 INFO:tasks.workunit.client.0.smithi031.stdout:8/790: mknod d8/d15/d37/d43/dbd/c112 0 2024-04-24T23:13:52.405 INFO:tasks.workunit.client.0.smithi031.stdout:9/734: dwrite d2/f32 [4194304,4194304] 0 2024-04-24T23:13:52.405 INFO:tasks.workunit.client.1.smithi040.stdout:3/350: write d1/d9/d18/f5e [1349946,86437] 0 2024-04-24T23:13:52.406 INFO:tasks.workunit.client.0.smithi031.stdout:6/725: dread d13/d15/d1d/d23/d2d/f4c [0,4194304] 0 2024-04-24T23:13:52.407 INFO:tasks.workunit.client.1.smithi040.stdout:5/332: mknod d2/d7/d8/d1f/d57/d27/c6c 0 2024-04-24T23:13:52.408 INFO:tasks.workunit.client.1.smithi040.stdout:8/314: unlink d0/l12 0 2024-04-24T23:13:52.410 INFO:tasks.workunit.client.1.smithi040.stdout:9/363: truncate d7/da/d17/d40/f42 1337117 0 2024-04-24T23:13:52.410 INFO:tasks.workunit.client.1.smithi040.stdout:8/315: chown d0/d16/d19/f5f 35 1 2024-04-24T23:13:52.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/726: read d13/d15/d1d/d23/d24/d43/d68/f72 [1053943,52547] 0 2024-04-24T23:13:52.417 INFO:tasks.workunit.client.0.smithi031.stdout:5/592: link d0/d7/c90 d0/d7/cc4 0 2024-04-24T23:13:52.417 INFO:tasks.workunit.client.1.smithi040.stdout:1/353: rename da/d12/d18/d2a/f5d to da/d31/f7c 0 2024-04-24T23:13:52.418 INFO:tasks.workunit.client.1.smithi040.stdout:0/401: symlink df/d12/d58/d5b/l7c 0 2024-04-24T23:13:52.418 INFO:tasks.workunit.client.0.smithi031.stdout:9/735: mkdir d2/d16/de0 0 2024-04-24T23:13:52.419 INFO:tasks.workunit.client.1.smithi040.stdout:3/351: dwrite d1/d2d/de/f70 [0,4194304] 0 2024-04-24T23:13:52.419 INFO:tasks.workunit.client.0.smithi031.stdout:6/727: creat d13/d15/d1d/d23/d2d/d9e/ff8 x:0 0 0 2024-04-24T23:13:52.421 INFO:tasks.workunit.client.1.smithi040.stdout:6/322: dread d7/d1a/d20/f47 [0,4194304] 0 2024-04-24T23:13:52.421 INFO:tasks.workunit.client.1.smithi040.stdout:2/412: truncate da/d1c/f5c 1205684 0 2024-04-24T23:13:52.423 INFO:tasks.workunit.client.1.smithi040.stdout:7/384: getdents db/d12/d20/d52/d6c 0 2024-04-24T23:13:52.424 INFO:tasks.workunit.client.0.smithi031.stdout:9/736: creat d2/dae/d14/d78/d84/d4a/d4b/d96/fe1 x:0 0 0 2024-04-24T23:13:52.424 INFO:tasks.workunit.client.1.smithi040.stdout:3/352: write d1/d2d/d44/f6c [2488254,81216] 0 2024-04-24T23:13:52.426 INFO:tasks.workunit.client.1.smithi040.stdout:7/385: stat db/d12/d20/c2a 0 2024-04-24T23:13:52.426 INFO:tasks.workunit.client.0.smithi031.stdout:8/791: getdents d8/d15/d37/d43 0 2024-04-24T23:13:52.426 INFO:tasks.workunit.client.1.smithi040.stdout:6/323: read d7/d1a/d20/d51/d2f/f5b [344500,44886] 0 2024-04-24T23:13:52.428 INFO:tasks.workunit.client.1.smithi040.stdout:8/316: dwrite d0/d38/f50 [0,4194304] 0 2024-04-24T23:13:52.429 INFO:tasks.workunit.client.1.smithi040.stdout:7/386: fdatasync db/d12/d20/d52/f70 0 2024-04-24T23:13:52.429 INFO:tasks.workunit.client.0.smithi031.stdout:6/728: creat d13/d1b/dd0/ff9 x:0 0 0 2024-04-24T23:13:52.429 INFO:tasks.workunit.client.1.smithi040.stdout:8/317: chown d0/d8/dc/f3a 1 1 2024-04-24T23:13:52.430 INFO:tasks.workunit.client.0.smithi031.stdout:6/729: chown d13/d15/fb9 975821820 1 2024-04-24T23:13:52.430 INFO:tasks.workunit.client.1.smithi040.stdout:7/387: dread - db/d2f/f5b zero size 2024-04-24T23:13:52.431 INFO:tasks.workunit.client.0.smithi031.stdout:5/593: mknod d0/d7/cc5 0 2024-04-24T23:13:52.435 INFO:tasks.workunit.client.0.smithi031.stdout:9/737: creat d2/dae/d14/d78/d84/d4a/dab/dad/db5/fe2 x:0 0 0 2024-04-24T23:13:52.437 INFO:tasks.workunit.client.0.smithi031.stdout:8/792: chown d8/d15/d37/d43/dbd/cc3 330220287 1 2024-04-24T23:13:52.438 INFO:tasks.workunit.client.0.smithi031.stdout:8/793: chown d8/d1a/ff0 21 1 2024-04-24T23:13:52.441 INFO:tasks.workunit.client.0.smithi031.stdout:9/738: dwrite d2/d16/f20 [0,4194304] 0 2024-04-24T23:13:52.447 INFO:tasks.workunit.client.0.smithi031.stdout:8/794: dread d8/d1a/d1d/d50/f88 [0,4194304] 0 2024-04-24T23:13:52.448 INFO:tasks.workunit.client.1.smithi040.stdout:9/364: dread d7/da/d17/d40/d55/d2f/f64 [0,4194304] 0 2024-04-24T23:13:52.450 INFO:tasks.workunit.client.0.smithi031.stdout:5/594: rename d0/d6/d12 to d0/d6/d3b/d4b/d1d/d2e/dc6 0 2024-04-24T23:13:52.451 INFO:tasks.workunit.client.0.smithi031.stdout:5/595: stat d0/d6/d3b/d4b/f96 0 2024-04-24T23:13:52.452 INFO:tasks.workunit.client.0.smithi031.stdout:9/739: mknod d2/d43/d6e/ce3 0 2024-04-24T23:13:52.456 INFO:tasks.workunit.client.1.smithi040.stdout:4/293: fsync da/d1d/d3c/f5c 0 2024-04-24T23:13:52.458 INFO:tasks.workunit.client.0.smithi031.stdout:8/795: creat d8/d15/d37/d43/f113 x:0 0 0 2024-04-24T23:13:52.459 INFO:tasks.workunit.client.0.smithi031.stdout:8/796: write d8/d15/d37/d43/d7b/f8c [2545645,7544] 0 2024-04-24T23:13:52.461 INFO:tasks.workunit.client.0.smithi031.stdout:5/596: unlink d0/d7/d56/d7c/d7e/lae 0 2024-04-24T23:13:52.465 INFO:tasks.workunit.client.1.smithi040.stdout:0/402: creat df/d12/d58/f7d x:0 0 0 2024-04-24T23:13:52.470 INFO:tasks.workunit.client.1.smithi040.stdout:2/413: mkdir da/d10/d4d/da4 0 2024-04-24T23:13:52.470 INFO:tasks.workunit.client.0.smithi031.stdout:8/797: dread d8/d15/d37/d43/d7b/d89/f92 [0,4194304] 0 2024-04-24T23:13:52.473 INFO:tasks.workunit.client.1.smithi040.stdout:3/353: symlink d1/d2d/d44/l7f 0 2024-04-24T23:13:52.475 INFO:tasks.workunit.client.1.smithi040.stdout:2/414: dwrite da/d1c/f57 [0,4194304] 0 2024-04-24T23:13:52.479 INFO:tasks.workunit.client.1.smithi040.stdout:7/388: truncate db/d12/d20/d52/f23 300178 0 2024-04-24T23:13:52.485 INFO:tasks.workunit.client.1.smithi040.stdout:1/354: truncate da/d12/d18/f1d 2595974 0 2024-04-24T23:13:52.486 INFO:tasks.workunit.client.1.smithi040.stdout:0/403: mknod df/d12/d1a/d71/c7e 0 2024-04-24T23:13:52.486 INFO:tasks.workunit.client.0.smithi031.stdout:5/597: link d0/d7/cc4 d0/d6/d3b/d4b/dc/cc7 0 2024-04-24T23:13:52.486 INFO:tasks.workunit.client.0.smithi031.stdout:8/798: dread d8/d15/d37/d43/d7b/d86/fb6 [0,4194304] 0 2024-04-24T23:13:52.487 INFO:tasks.workunit.client.0.smithi031.stdout:4/779: sync 2024-04-24T23:13:52.487 INFO:tasks.workunit.client.0.smithi031.stdout:1/598: sync 2024-04-24T23:13:52.487 INFO:tasks.workunit.client.0.smithi031.stdout:0/699: sync 2024-04-24T23:13:52.489 INFO:tasks.workunit.client.0.smithi031.stdout:4/780: fsync d3/d20/d25/d60/fc0 0 2024-04-24T23:13:52.490 INFO:tasks.workunit.client.0.smithi031.stdout:0/700: dread da/d20/d8e/fd4 [0,4194304] 0 2024-04-24T23:13:52.491 INFO:tasks.workunit.client.0.smithi031.stdout:4/781: chown d3/d20/d2d/f39 1 1 2024-04-24T23:13:52.492 INFO:tasks.workunit.client.0.smithi031.stdout:4/782: stat d3/d20/d25/d7e 0 2024-04-24T23:13:52.492 INFO:tasks.workunit.client.1.smithi040.stdout:3/354: unlink d1/d2d/de/d30/l3d 0 2024-04-24T23:13:52.493 INFO:tasks.workunit.client.0.smithi031.stdout:5/598: dwrite d0/d6/d3b/d4b/d1d/fa4 [0,4194304] 0 2024-04-24T23:13:52.497 INFO:tasks.workunit.client.1.smithi040.stdout:3/355: dwrite d1/f59 [0,4194304] 0 2024-04-24T23:13:52.503 INFO:tasks.workunit.client.1.smithi040.stdout:7/389: mkdir db/d12/d62/d8a 0 2024-04-24T23:13:52.506 INFO:tasks.workunit.client.1.smithi040.stdout:9/365: rename d7/da/d17/d40/d55/d2f/f64 to d7/d5e/f7d 0 2024-04-24T23:13:52.510 INFO:tasks.workunit.client.1.smithi040.stdout:1/355: mkdir da/d12/d18/d2a/d4d/d7d 0 2024-04-24T23:13:52.511 INFO:tasks.workunit.client.1.smithi040.stdout:6/324: write d7/d1a/d20/d51/d1e/d5d/f31 [2247857,14498] 0 2024-04-24T23:13:52.513 INFO:tasks.workunit.client.1.smithi040.stdout:8/318: write d0/d16/f3b [118887,22252] 0 2024-04-24T23:13:52.516 INFO:tasks.workunit.client.1.smithi040.stdout:0/404: dread df/d12/d4b/d62/d6b/f73 [0,4194304] 0 2024-04-24T23:13:52.519 INFO:tasks.workunit.client.1.smithi040.stdout:4/294: rename da/d1d to da/d3f/d6b 0 2024-04-24T23:13:52.522 INFO:tasks.workunit.client.1.smithi040.stdout:5/333: sync 2024-04-24T23:13:52.522 INFO:tasks.workunit.client.1.smithi040.stdout:0/405: dwrite df/d12/d4b/d62/f65 [0,4194304] 0 2024-04-24T23:13:52.524 INFO:tasks.workunit.client.1.smithi040.stdout:8/319: symlink d0/d16/d1b/d3f/d44/l67 0 2024-04-24T23:13:52.524 INFO:tasks.workunit.client.1.smithi040.stdout:8/320: stat d0/d16/d1b/d47 0 2024-04-24T23:13:52.525 INFO:tasks.workunit.client.1.smithi040.stdout:3/356: mkdir d1/d80 0 2024-04-24T23:13:52.531 INFO:tasks.workunit.client.1.smithi040.stdout:7/390: rename db/fd to db/d12/d62/d8a/f8b 0 2024-04-24T23:13:52.531 INFO:tasks.workunit.client.1.smithi040.stdout:9/366: rename d7/da to d7/da/d17/d40/d55/d2f/d7e 22 2024-04-24T23:13:52.537 INFO:tasks.workunit.client.1.smithi040.stdout:8/321: rmdir d0/d38 39 2024-04-24T23:13:52.538 INFO:tasks.workunit.client.1.smithi040.stdout:3/357: unlink d1/d2d/de/l7d 0 2024-04-24T23:13:52.540 INFO:tasks.workunit.client.1.smithi040.stdout:2/415: link da/d1c/d4f/l9e da/d10/d4d/d6c/la5 0 2024-04-24T23:13:52.547 INFO:tasks.workunit.client.1.smithi040.stdout:1/356: link da/d35/d38/d47/f51 da/f7e 0 2024-04-24T23:13:52.550 INFO:tasks.workunit.client.1.smithi040.stdout:5/334: rename d2/d7/d8/d1f/d57/d27/d3d/d5f to d2/d7/d8/d6d 0 2024-04-24T23:13:52.554 INFO:tasks.workunit.client.1.smithi040.stdout:6/325: creat d7/de/d4e/f64 x:0 0 0 2024-04-24T23:13:52.554 INFO:tasks.workunit.client.1.smithi040.stdout:6/326: stat d7/d1a/d20/d51/d1e/l40 0 2024-04-24T23:13:52.555 INFO:tasks.workunit.client.1.smithi040.stdout:9/367: write f2 [1263560,79235] 0 2024-04-24T23:13:52.557 INFO:tasks.workunit.client.1.smithi040.stdout:1/357: unlink da/d12/d14/f68 0 2024-04-24T23:13:52.558 INFO:tasks.workunit.client.1.smithi040.stdout:6/327: write d7/de/f5e [650528,60232] 0 2024-04-24T23:13:52.559 INFO:tasks.workunit.client.1.smithi040.stdout:0/406: link df/f34 df/d12/d4b/d62/f7f 0 2024-04-24T23:13:52.559 INFO:tasks.workunit.client.1.smithi040.stdout:6/328: chown d7/d1a/d20/d51/d1e/d5d/l43 6243 1 2024-04-24T23:13:52.560 INFO:tasks.workunit.client.1.smithi040.stdout:8/322: creat d0/d16/d1b/d58/f68 x:0 0 0 2024-04-24T23:13:52.560 INFO:tasks.workunit.client.1.smithi040.stdout:3/358: mknod d1/d9/d18/d36/d55/d5a/d7c/c81 0 2024-04-24T23:13:52.562 INFO:tasks.workunit.client.1.smithi040.stdout:7/391: creat db/d12/f8c x:0 0 0 2024-04-24T23:13:52.562 INFO:tasks.workunit.client.1.smithi040.stdout:8/323: write d0/d16/d19/d4f/f4e [1255185,9589] 0 2024-04-24T23:13:52.562 INFO:tasks.workunit.client.1.smithi040.stdout:9/368: symlink d7/da/d17/d40/d55/d2f/d1f/l7f 0 2024-04-24T23:13:52.563 INFO:tasks.workunit.client.1.smithi040.stdout:1/358: mknod da/d12/d14/d58/c7f 0 2024-04-24T23:13:52.565 INFO:tasks.workunit.client.1.smithi040.stdout:6/329: symlink d7/d1a/d56/l65 0 2024-04-24T23:13:52.565 INFO:tasks.workunit.client.1.smithi040.stdout:9/369: stat d7/da/d17/d40/l4f 0 2024-04-24T23:13:52.566 INFO:tasks.workunit.client.1.smithi040.stdout:4/295: mknod da/d3f/d40/d63/c6c 0 2024-04-24T23:13:52.568 INFO:tasks.workunit.client.1.smithi040.stdout:5/335: creat d2/d7/d8/d1f/f6e x:0 0 0 2024-04-24T23:13:52.568 INFO:tasks.workunit.client.1.smithi040.stdout:0/407: dwrite df/d12/f22 [8388608,4194304] 0 2024-04-24T23:13:52.573 INFO:tasks.workunit.client.1.smithi040.stdout:2/416: getdents da/d1c/d37/d39/d77 0 2024-04-24T23:13:52.574 INFO:tasks.workunit.client.1.smithi040.stdout:2/417: fsync da/d1c/f57 0 2024-04-24T23:13:52.576 INFO:tasks.workunit.client.1.smithi040.stdout:4/296: creat da/d3f/d6b/d2b/f6d x:0 0 0 2024-04-24T23:13:52.578 INFO:tasks.workunit.client.1.smithi040.stdout:9/370: symlink d7/da/d17/d40/d55/d2f/d1f/l80 0 2024-04-24T23:13:52.579 INFO:tasks.workunit.client.1.smithi040.stdout:0/408: mknod df/d1e/d20/d21/d51/c80 0 2024-04-24T23:13:52.581 INFO:tasks.workunit.client.1.smithi040.stdout:8/324: rename d0/d16/d19/c4a to d0/d38/c69 0 2024-04-24T23:13:52.582 INFO:tasks.workunit.client.1.smithi040.stdout:8/325: write d0/d8/dc/d34/f3c [690214,106726] 0 2024-04-24T23:13:52.583 INFO:tasks.workunit.client.0.smithi031.stdout:9/740: write d2/d16/f77 [1829644,6986] 0 2024-04-24T23:13:52.583 INFO:tasks.workunit.client.0.smithi031.stdout:0/701: mknod da/d18/ceb 0 2024-04-24T23:13:52.584 INFO:tasks.workunit.client.0.smithi031.stdout:5/599: rmdir d0 39 2024-04-24T23:13:52.586 INFO:tasks.workunit.client.1.smithi040.stdout:3/359: link d1/f79 d1/d9/d18/d36/f82 0 2024-04-24T23:13:52.587 INFO:tasks.workunit.client.1.smithi040.stdout:7/392: write db/f1c [1652180,26482] 0 2024-04-24T23:13:52.588 INFO:tasks.workunit.client.1.smithi040.stdout:9/371: mknod d7/da/d17/d40/d55/d2f/d1f/d31/d3b/c81 0 2024-04-24T23:13:52.590 INFO:tasks.workunit.client.1.smithi040.stdout:6/330: write d7/d1a/d20/d51/f26 [2892725,21470] 0 2024-04-24T23:13:52.594 INFO:tasks.workunit.client.1.smithi040.stdout:8/326: dwrite d0/d8/dc/f3a [4194304,4194304] 0 2024-04-24T23:13:52.594 INFO:tasks.workunit.client.1.smithi040.stdout:7/393: chown db/d12/d20/d52/d44/f88 12001090 1 2024-04-24T23:13:52.595 INFO:tasks.workunit.client.1.smithi040.stdout:6/331: stat d7/d1a/d20/d51/d2f 0 2024-04-24T23:13:52.596 INFO:tasks.workunit.client.1.smithi040.stdout:4/297: mkdir da/d45/d48/d6e 0 2024-04-24T23:13:52.596 INFO:tasks.workunit.client.0.smithi031.stdout:0/702: fdatasync da/d18/d1a/f72 0 2024-04-24T23:13:52.600 INFO:tasks.workunit.client.0.smithi031.stdout:1/599: truncate d1/d3e/da3/f77 1890315 0 2024-04-24T23:13:52.601 INFO:tasks.workunit.client.1.smithi040.stdout:1/359: dwrite da/d12/d18/f6c [0,4194304] 0 2024-04-24T23:13:52.601 INFO:tasks.workunit.client.1.smithi040.stdout:2/418: dread da/d10/f2b [0,4194304] 0 2024-04-24T23:13:52.601 INFO:tasks.workunit.client.0.smithi031.stdout:1/600: write d1/d33/d36/fa7 [497038,26821] 0 2024-04-24T23:13:52.602 INFO:tasks.workunit.client.1.smithi040.stdout:3/360: mknod d1/d9/d18/d36/d55/c83 0 2024-04-24T23:13:52.602 INFO:tasks.workunit.client.1.smithi040.stdout:0/409: write df/d1e/d20/d21/f32 [1910866,78779] 0 2024-04-24T23:13:52.602 INFO:tasks.workunit.client.1.smithi040.stdout:8/327: chown d0/d16/d1b/f54 57137 1 2024-04-24T23:13:52.605 INFO:tasks.workunit.client.0.smithi031.stdout:5/600: fsync d0/d6/d32/d6b/f7b 0 2024-04-24T23:13:52.607 INFO:tasks.workunit.client.1.smithi040.stdout:0/410: write df/d12/f1f [3755895,789] 0 2024-04-24T23:13:52.608 INFO:tasks.workunit.client.1.smithi040.stdout:7/394: creat db/d12/d20/d52/d44/f8d x:0 0 0 2024-04-24T23:13:52.608 INFO:tasks.workunit.client.1.smithi040.stdout:2/419: mkdir da/d10/d4d/da6 0 2024-04-24T23:13:52.609 INFO:tasks.workunit.client.0.smithi031.stdout:9/741: dread d2/dae/d14/d78/d84/d4a/dab/dad/fc3 [0,4194304] 0 2024-04-24T23:13:52.610 INFO:tasks.workunit.client.0.smithi031.stdout:0/703: creat da/d86/da3/dcd/fec x:0 0 0 2024-04-24T23:13:52.612 INFO:tasks.workunit.client.1.smithi040.stdout:8/328: dwrite d0/d8/dc/f3a [4194304,4194304] 0 2024-04-24T23:13:52.614 INFO:tasks.workunit.client.1.smithi040.stdout:1/360: write da/d12/d14/f24 [7840911,92949] 0 2024-04-24T23:13:52.614 INFO:tasks.workunit.client.1.smithi040.stdout:8/329: stat d0/d16/d19/f35 0 2024-04-24T23:13:52.616 INFO:tasks.workunit.client.1.smithi040.stdout:3/361: creat d1/d9/d2e/f84 x:0 0 0 2024-04-24T23:13:52.616 INFO:tasks.workunit.client.1.smithi040.stdout:1/361: dread - da/d12/d18/d2a/f56 zero size 2024-04-24T23:13:52.616 INFO:tasks.workunit.client.1.smithi040.stdout:8/330: chown d0/d38/c46 14007898 1 2024-04-24T23:13:52.619 INFO:tasks.workunit.client.1.smithi040.stdout:1/362: write da/d12/d18/d2a/f79 [91751,44334] 0 2024-04-24T23:13:52.622 INFO:tasks.workunit.client.0.smithi031.stdout:5/601: rename d0/d7/d56/d7c/d7e/fb5 to d0/d6/d3b/d4b/d1d/d2e/dc6/fc8 0 2024-04-24T23:13:52.623 INFO:tasks.workunit.client.0.smithi031.stdout:5/602: chown d0/d7/d56/d7c/d7e/l66 46 1 2024-04-24T23:13:52.626 INFO:tasks.workunit.client.1.smithi040.stdout:7/395: symlink db/d12/d62/d8a/l8e 0 2024-04-24T23:13:52.628 INFO:tasks.workunit.client.1.smithi040.stdout:2/420: readlink da/d1c/d37/d39/d3f/l8b 0 2024-04-24T23:13:52.628 INFO:tasks.workunit.client.0.smithi031.stdout:4/783: dread d3/d23/d69/d3f/f61 [0,4194304] 0 2024-04-24T23:13:52.629 INFO:tasks.workunit.client.1.smithi040.stdout:2/421: write da/d1c/d37/d39/d3f/f67 [326022,126133] 0 2024-04-24T23:13:52.630 INFO:tasks.workunit.client.1.smithi040.stdout:3/362: chown d1/d9/d15/d45/l77 3 1 2024-04-24T23:13:52.631 INFO:tasks.workunit.client.0.smithi031.stdout:0/704: rename da/d18 to da/d18/d1a/d37/ded 22 2024-04-24T23:13:52.632 INFO:tasks.workunit.client.1.smithi040.stdout:8/331: stat d0/d8/dc/d34/l5e 0 2024-04-24T23:13:52.634 INFO:tasks.workunit.client.0.smithi031.stdout:9/742: dwrite d2/d16/dbe/d7d/fc9 [0,4194304] 0 2024-04-24T23:13:52.634 INFO:tasks.workunit.client.0.smithi031.stdout:1/601: dwrite d1/d33/d41/d9b/da8/fa9 [0,4194304] 0 2024-04-24T23:13:52.634 INFO:tasks.workunit.client.0.smithi031.stdout:0/705: chown da/d20/d4d/fcf 31890 1 2024-04-24T23:13:52.636 INFO:tasks.workunit.client.0.smithi031.stdout:5/603: truncate d0/d6/f40 150363 0 2024-04-24T23:13:52.637 INFO:tasks.workunit.client.1.smithi040.stdout:9/372: rename d7/da/d17/d40/d55/d61/d6c/f71 to d7/f82 0 2024-04-24T23:13:52.637 INFO:tasks.workunit.client.1.smithi040.stdout:6/332: rename d7/d1a to d7/d1a/d20/d51/d2f/d66 22 2024-04-24T23:13:52.638 INFO:tasks.workunit.client.0.smithi031.stdout:9/743: write d2/dae/d14/d78/d84/d4a/d4b/d96/fe1 [506515,92758] 0 2024-04-24T23:13:52.640 INFO:tasks.workunit.client.1.smithi040.stdout:9/373: read - d7/da/d17/d40/d55/d2f/d1f/d34/f5b zero size 2024-04-24T23:13:52.640 INFO:tasks.workunit.client.1.smithi040.stdout:4/298: creat da/d3f/d40/d41/d62/f6f x:0 0 0 2024-04-24T23:13:52.641 INFO:tasks.workunit.client.0.smithi031.stdout:4/784: fdatasync d3/d20/d2d/f52 0 2024-04-24T23:13:52.642 INFO:tasks.workunit.client.1.smithi040.stdout:1/363: creat da/d35/d38/d47/f80 x:0 0 0 2024-04-24T23:13:52.643 INFO:tasks.workunit.client.1.smithi040.stdout:1/364: stat da/ff 0 2024-04-24T23:13:52.644 INFO:tasks.workunit.client.1.smithi040.stdout:1/365: read - da/d12/d18/d2a/d4d/f59 zero size 2024-04-24T23:13:52.645 INFO:tasks.workunit.client.1.smithi040.stdout:7/396: mkdir db/d12/d20/d52/d6c/d8f 0 2024-04-24T23:13:52.645 INFO:tasks.workunit.client.0.smithi031.stdout:0/706: dwrite da/d22/f54 [0,4194304] 0 2024-04-24T23:13:52.646 INFO:tasks.workunit.client.1.smithi040.stdout:9/374: truncate d7/da/d17/d40/d55/d2f/d1f/d34/f5b 582749 0 2024-04-24T23:13:52.646 INFO:tasks.workunit.client.1.smithi040.stdout:7/397: stat db/d12/d20/d52/f70 0 2024-04-24T23:13:52.647 INFO:tasks.workunit.client.0.smithi031.stdout:2/544: sync 2024-04-24T23:13:52.647 INFO:tasks.workunit.client.0.smithi031.stdout:3/597: sync 2024-04-24T23:13:52.647 INFO:tasks.workunit.client.0.smithi031.stdout:7/634: sync 2024-04-24T23:13:52.651 INFO:tasks.workunit.client.0.smithi031.stdout:4/785: read d3/d20/d25/fb0 [559859,130895] 0 2024-04-24T23:13:52.651 INFO:tasks.workunit.client.0.smithi031.stdout:1/602: rmdir d1/dc/d11/d21/d5a 39 2024-04-24T23:13:52.653 INFO:tasks.workunit.client.1.smithi040.stdout:3/363: mknod d1/d2d/c85 0 2024-04-24T23:13:52.653 INFO:tasks.workunit.client.0.smithi031.stdout:4/786: truncate d3/d23/d3b/fcc 570470 0 2024-04-24T23:13:52.656 INFO:tasks.workunit.client.1.smithi040.stdout:5/336: sync 2024-04-24T23:13:52.658 INFO:tasks.workunit.client.1.smithi040.stdout:2/422: dread da/d10/d4d/f64 [0,4194304] 0 2024-04-24T23:13:52.661 INFO:tasks.workunit.client.1.smithi040.stdout:0/411: rename df/d12/d1a/d71/c30 to df/d1e/d20/d21/d51/d79/c81 0 2024-04-24T23:13:52.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/635: mkdir d1/d45/d75/d81/dd9 0 2024-04-24T23:13:52.667 INFO:tasks.workunit.client.1.smithi040.stdout:1/366: mknod da/d12/d14/d25/c81 0 2024-04-24T23:13:52.669 INFO:tasks.workunit.client.0.smithi031.stdout:9/744: write d2/dae/d14/f88 [461562,51871] 0 2024-04-24T23:13:52.674 INFO:tasks.workunit.client.1.smithi040.stdout:8/332: truncate d0/d16/d19/f35 1549126 0 2024-04-24T23:13:52.674 INFO:tasks.workunit.client.1.smithi040.stdout:4/299: write da/d3f/d6b/d2b/f39 [975959,112508] 0 2024-04-24T23:13:52.677 INFO:tasks.workunit.client.1.smithi040.stdout:2/423: rmdir da/d1c 39 2024-04-24T23:13:52.680 INFO:tasks.workunit.client.1.smithi040.stdout:6/333: dwrite d7/f9 [0,4194304] 0 2024-04-24T23:13:52.683 INFO:tasks.workunit.client.1.smithi040.stdout:9/375: rename d7/da/d17/d40/d55/d2f/d1f/d31/d4d to d7/da/d17/d40/d55/d2f/d1f/d4e/d83 0 2024-04-24T23:13:52.683 INFO:tasks.workunit.client.1.smithi040.stdout:3/364: rename d1/d2d/de/d30 to d1/d2d/de/d30/d37/d86 22 2024-04-24T23:13:52.684 INFO:tasks.workunit.client.1.smithi040.stdout:4/300: dwrite da/d3f/f56 [4194304,4194304] 0 2024-04-24T23:13:52.685 INFO:tasks.workunit.client.0.smithi031.stdout:6/730: sync 2024-04-24T23:13:52.686 INFO:tasks.workunit.client.1.smithi040.stdout:9/376: dread - d7/da/d17/d40/d55/d2f/f7c zero size 2024-04-24T23:13:52.686 INFO:tasks.workunit.client.1.smithi040.stdout:4/301: chown da/d3f/d6b/d2b/f31 9 1 2024-04-24T23:13:52.687 INFO:tasks.workunit.client.0.smithi031.stdout:3/598: link d6/d14/d1d/d69/f94 d6/d14/d1d/fb6 0 2024-04-24T23:13:52.688 INFO:tasks.workunit.client.0.smithi031.stdout:3/599: chown d6/d14/d91 19729041 1 2024-04-24T23:13:52.690 INFO:tasks.workunit.client.0.smithi031.stdout:1/603: mkdir d1/da/dbc 0 2024-04-24T23:13:52.694 INFO:tasks.workunit.client.0.smithi031.stdout:9/745: truncate d2/dae/d14/d1d/f28 739776 0 2024-04-24T23:13:52.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/604: creat d0/d7/fc9 x:0 0 0 2024-04-24T23:13:52.699 INFO:tasks.workunit.client.1.smithi040.stdout:0/412: dwrite df/d12/d1a/f3a [0,4194304] 0 2024-04-24T23:13:52.700 INFO:tasks.workunit.client.1.smithi040.stdout:8/333: creat d0/d8/d22/f6a x:0 0 0 2024-04-24T23:13:52.700 INFO:tasks.workunit.client.0.smithi031.stdout:3/600: dwrite d6/df/d16/d2a/d95/d9b/d1a/d25/f54 [0,4194304] 0 2024-04-24T23:13:52.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/605: dread - d0/d6/d3b/d4b/fac zero size 2024-04-24T23:13:52.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/604: dwrite d1/da/d12/d65/fb6 [0,4194304] 0 2024-04-24T23:13:52.703 INFO:tasks.workunit.client.0.smithi031.stdout:2/545: creat d5/d8/d45/d5b/fbd x:0 0 0 2024-04-24T23:13:52.705 INFO:tasks.workunit.client.0.smithi031.stdout:1/605: readlink d1/da/l32 0 2024-04-24T23:13:52.706 INFO:tasks.workunit.client.0.smithi031.stdout:5/606: read d0/d6/d3b/d4b/d1d/fa4 [2222270,23497] 0 2024-04-24T23:13:52.707 INFO:tasks.workunit.client.1.smithi040.stdout:2/424: mkdir da/d29/d55/da7 0 2024-04-24T23:13:52.713 INFO:tasks.workunit.client.1.smithi040.stdout:2/425: dwrite da/d29/f69 [0,4194304] 0 2024-04-24T23:13:52.716 INFO:tasks.workunit.client.1.smithi040.stdout:3/365: unlink d1/d9/d18/f5e 0 2024-04-24T23:13:52.718 INFO:tasks.workunit.client.0.smithi031.stdout:5/607: dread d0/d6/d3b/d4b/d1d/d2e/dc6/f31 [0,4194304] 0 2024-04-24T23:13:52.720 INFO:tasks.workunit.client.0.smithi031.stdout:4/787: link d3/d23/d69/d3f/fd9 d3/d5e/dc4/ff7 0 2024-04-24T23:13:52.721 INFO:tasks.workunit.client.0.smithi031.stdout:5/608: write d0/d6/d3b/d4b/f51 [1945370,24256] 0 2024-04-24T23:13:52.722 INFO:tasks.workunit.client.0.smithi031.stdout:4/788: write d3/d23/d3b/fcc [460594,105532] 0 2024-04-24T23:13:52.724 INFO:tasks.workunit.client.1.smithi040.stdout:4/302: read da/f18 [266801,25643] 0 2024-04-24T23:13:52.728 INFO:tasks.workunit.client.0.smithi031.stdout:6/731: creat d13/d15/d1d/d23/d44/da0/da6/de6/ffa x:0 0 0 2024-04-24T23:13:52.729 INFO:tasks.workunit.client.0.smithi031.stdout:7/636: truncate d1/d2/dd/d33/d5e/d64/d6d/fac 1092482 0 2024-04-24T23:13:52.729 INFO:tasks.workunit.client.1.smithi040.stdout:1/367: getdents da/d12/d18/d2a/d4d/d7d 0 2024-04-24T23:13:52.731 INFO:tasks.workunit.client.1.smithi040.stdout:7/398: link db/d12/d20/c2a db/d12/d62/c90 0 2024-04-24T23:13:52.734 INFO:tasks.workunit.client.0.smithi031.stdout:3/601: mkdir d6/df/d16/d2a/d95/d9b/d29/db7 0 2024-04-24T23:13:52.735 INFO:tasks.workunit.client.0.smithi031.stdout:1/606: mkdir d1/d3e/d46/dbd 0 2024-04-24T23:13:52.736 INFO:tasks.workunit.client.1.smithi040.stdout:8/334: symlink d0/d38/d4c/l6b 0 2024-04-24T23:13:52.737 INFO:tasks.workunit.client.0.smithi031.stdout:7/637: dwrite d1/d45/da0/d4f/fb5 [0,4194304] 0 2024-04-24T23:13:52.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/607: chown d1/da/l32 31801884 1 2024-04-24T23:13:52.739 INFO:tasks.workunit.client.0.smithi031.stdout:8/799: sync 2024-04-24T23:13:52.740 INFO:tasks.workunit.client.0.smithi031.stdout:8/800: readlink d8/d15/d37/d43/d7b/l96 0 2024-04-24T23:13:52.746 INFO:tasks.workunit.client.0.smithi031.stdout:5/609: unlink d0/d6/d3b/d4b/d1d/c35 0 2024-04-24T23:13:52.749 INFO:tasks.workunit.client.0.smithi031.stdout:8/801: dwrite d8/d1a/d1d/d35/f6d [0,4194304] 0 2024-04-24T23:13:52.749 INFO:tasks.workunit.client.1.smithi040.stdout:9/377: dwrite d7/da/d17/d40/d55/d2f/f6f [0,4194304] 0 2024-04-24T23:13:52.751 INFO:tasks.workunit.client.1.smithi040.stdout:5/337: rename d2/d7/d8/d1f/f4b to d2/d7/d8/d1f/d57/d27/d3c/d43/f6f 0 2024-04-24T23:13:52.753 INFO:tasks.workunit.client.0.smithi031.stdout:0/707: getdents da/d20/d4e/dc9 0 2024-04-24T23:13:52.758 INFO:tasks.workunit.client.0.smithi031.stdout:5/610: dwrite d0/d6/d3b/d4b/f51 [0,4194304] 0 2024-04-24T23:13:52.762 INFO:tasks.workunit.client.0.smithi031.stdout:4/789: rename d3/f9 to d3/d20/d25/d60/ff8 0 2024-04-24T23:13:52.764 INFO:tasks.workunit.client.0.smithi031.stdout:6/732: read - d13/d15/d1d/d23/d24/d43/fcb zero size 2024-04-24T23:13:52.765 INFO:tasks.workunit.client.0.smithi031.stdout:3/602: dread d6/df/d16/d2a/d95/d9b/f7c [0,4194304] 0 2024-04-24T23:13:52.766 INFO:tasks.workunit.client.0.smithi031.stdout:3/603: chown d6/df/d16/d53/fa5 12744780 1 2024-04-24T23:13:52.768 INFO:tasks.workunit.client.1.smithi040.stdout:6/334: truncate d7/f9 3300890 0 2024-04-24T23:13:52.770 INFO:tasks.workunit.client.1.smithi040.stdout:6/335: write d7/d1a/d20/f3a [950691,93752] 0 2024-04-24T23:13:52.770 INFO:tasks.workunit.client.0.smithi031.stdout:2/546: mkdir d5/d6/df/dad/dbe 0 2024-04-24T23:13:52.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/547: dread - d5/d6/d29/d26/fa8 zero size 2024-04-24T23:13:52.774 INFO:tasks.workunit.client.1.smithi040.stdout:7/399: mknod db/d43/c91 0 2024-04-24T23:13:52.775 INFO:tasks.workunit.client.1.smithi040.stdout:0/413: mknod df/d1e/d20/d61/c82 0 2024-04-24T23:13:52.777 INFO:tasks.workunit.client.0.smithi031.stdout:1/608: rmdir d1/da/d61/d80 39 2024-04-24T23:13:52.778 INFO:tasks.workunit.client.1.smithi040.stdout:9/378: mknod d7/da/d17/c84 0 2024-04-24T23:13:52.779 INFO:tasks.workunit.client.0.smithi031.stdout:7/638: mknod d1/d45/d75/d81/cda 0 2024-04-24T23:13:52.780 INFO:tasks.workunit.client.1.smithi040.stdout:3/366: rename d1/d2d/de/c29 to d1/d9/d18/d36/d55/d7e/c87 0 2024-04-24T23:13:52.781 INFO:tasks.workunit.client.0.smithi031.stdout:9/746: link d2/fdf d2/d16/fe4 0 2024-04-24T23:13:52.784 INFO:tasks.workunit.client.1.smithi040.stdout:7/400: dwrite db/d12/d20/d52/d44/f88 [0,4194304] 0 2024-04-24T23:13:52.784 INFO:tasks.workunit.client.1.smithi040.stdout:0/414: dwrite df/d1e/d20/f43 [0,4194304] 0 2024-04-24T23:13:52.784 INFO:tasks.workunit.client.1.smithi040.stdout:9/379: write d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [6005718,5298] 0 2024-04-24T23:13:52.784 INFO:tasks.workunit.client.1.smithi040.stdout:5/338: unlink d2/f69 0 2024-04-24T23:13:52.787 INFO:tasks.workunit.client.1.smithi040.stdout:5/339: fdatasync d2/d2f/d14/f61 0 2024-04-24T23:13:52.789 INFO:tasks.workunit.client.1.smithi040.stdout:4/303: mkdir da/d45/d48/d6e/d70 0 2024-04-24T23:13:52.789 INFO:tasks.workunit.client.1.smithi040.stdout:5/340: fsync d2/d7/fe 0 2024-04-24T23:13:52.792 INFO:tasks.workunit.client.1.smithi040.stdout:6/336: creat d7/d1a/f67 x:0 0 0 2024-04-24T23:13:52.793 INFO:tasks.workunit.client.0.smithi031.stdout:8/802: symlink d8/d15/d37/d43/da5/l114 0 2024-04-24T23:13:52.796 INFO:tasks.workunit.client.1.smithi040.stdout:5/341: dwrite d2/d7/d8/db/d1e/f28 [0,4194304] 0 2024-04-24T23:13:52.797 INFO:tasks.workunit.client.1.smithi040.stdout:5/342: chown d2/d7/d8/d64 1 1 2024-04-24T23:13:52.802 INFO:tasks.workunit.client.0.smithi031.stdout:4/790: mknod d3/d20/d2d/d44/d5b/cf9 0 2024-04-24T23:13:52.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/368: rename da/d35/d38/l40 to da/d12/d18/d2a/d41/d4f/l82 0 2024-04-24T23:13:52.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/369: chown da/d12/d14/d58 498925 1 2024-04-24T23:13:52.806 INFO:tasks.workunit.client.1.smithi040.stdout:7/401: mknod db/d2f/d37/c92 0 2024-04-24T23:13:52.807 INFO:tasks.workunit.client.1.smithi040.stdout:7/402: write db/d12/f3e [547235,11788] 0 2024-04-24T23:13:52.809 INFO:tasks.workunit.client.0.smithi031.stdout:1/609: write d1/d1d/f8e [16196,9081] 0 2024-04-24T23:13:52.811 INFO:tasks.workunit.client.0.smithi031.stdout:1/610: write d1/da/d12/d7e/f9e [1677737,96690] 0 2024-04-24T23:13:52.812 INFO:tasks.workunit.client.0.smithi031.stdout:7/639: fsync d1/f13 0 2024-04-24T23:13:52.813 INFO:tasks.workunit.client.0.smithi031.stdout:7/640: write d1/f9 [5699565,74569] 0 2024-04-24T23:13:52.814 INFO:tasks.workunit.client.0.smithi031.stdout:9/747: fsync d2/dae/d14/d78/d84/f7a 0 2024-04-24T23:13:52.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/803: creat d8/d15/d37/d43/daf/f115 x:0 0 0 2024-04-24T23:13:52.822 INFO:tasks.workunit.client.0.smithi031.stdout:4/791: symlink d3/d20/d25/d60/d63/lfa 0 2024-04-24T23:13:52.823 INFO:tasks.workunit.client.1.smithi040.stdout:8/335: sync 2024-04-24T23:13:52.824 INFO:tasks.workunit.client.1.smithi040.stdout:8/336: read d0/d8/f29 [6350057,59936] 0 2024-04-24T23:13:52.827 INFO:tasks.workunit.client.1.smithi040.stdout:6/337: rename d7/d1a/d20/d51/d1e/l40 to d7/d1a/d20/d51/d2f/l68 0 2024-04-24T23:13:52.827 INFO:tasks.workunit.client.0.smithi031.stdout:0/708: dread da/d11/f90 [0,4194304] 0 2024-04-24T23:13:52.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/548: mknod d5/cbf 0 2024-04-24T23:13:52.829 INFO:tasks.workunit.client.1.smithi040.stdout:1/370: creat da/d12/d14/f83 x:0 0 0 2024-04-24T23:13:52.830 INFO:tasks.workunit.client.1.smithi040.stdout:0/415: mknod df/d1e/d20/c83 0 2024-04-24T23:13:52.835 INFO:tasks.workunit.client.0.smithi031.stdout:1/611: creat d1/d3e/fbe x:0 0 0 2024-04-24T23:13:52.839 INFO:tasks.workunit.client.0.smithi031.stdout:8/804: dread - d8/d1a/d1d/d50/dc0/d9d/fa3 zero size 2024-04-24T23:13:52.843 INFO:tasks.workunit.client.1.smithi040.stdout:3/367: creat d1/f88 x:0 0 0 2024-04-24T23:13:52.843 INFO:tasks.workunit.client.1.smithi040.stdout:2/426: dwrite da/d1c/f5c [0,4194304] 0 2024-04-24T23:13:52.844 INFO:tasks.workunit.client.0.smithi031.stdout:5/611: truncate d0/d6/d3b/d4b/d1d/fb2 833167 0 2024-04-24T23:13:52.844 INFO:tasks.workunit.client.1.smithi040.stdout:9/380: dwrite d7/da/d17/d40/d55/d2f/d1f/d34/f78 [0,4194304] 0 2024-04-24T23:13:52.845 INFO:tasks.workunit.client.0.smithi031.stdout:5/612: chown d0/d6/d3b/d4b/dc/d25/d55 3113 1 2024-04-24T23:13:52.845 INFO:tasks.workunit.client.1.smithi040.stdout:3/368: fsync d1/d9/d15/f53 0 2024-04-24T23:13:52.845 INFO:tasks.workunit.client.0.smithi031.stdout:9/748: dread d2/fc [0,4194304] 0 2024-04-24T23:13:52.846 INFO:tasks.workunit.client.0.smithi031.stdout:0/709: rmdir da/d18/d1a/d37/d41 39 2024-04-24T23:13:52.846 INFO:tasks.workunit.client.0.smithi031.stdout:6/733: write d13/d15/d1d/d23/d2d/fd2 [251067,89854] 0 2024-04-24T23:13:52.847 INFO:tasks.workunit.client.0.smithi031.stdout:6/734: readlink l0 0 2024-04-24T23:13:52.847 INFO:tasks.workunit.client.0.smithi031.stdout:9/749: chown d2/dae/d14/d78/d84/d4a/dab/dad 6701610 1 2024-04-24T23:13:52.850 INFO:tasks.workunit.client.0.smithi031.stdout:2/549: unlink d5/d6/f3c 0 2024-04-24T23:13:52.851 INFO:tasks.workunit.client.0.smithi031.stdout:2/550: stat d5/d28/d43 0 2024-04-24T23:13:52.851 INFO:tasks.workunit.client.0.smithi031.stdout:9/750: truncate d2/dae/d14/d78/d84/d7e/dc7/fd6 11965 0 2024-04-24T23:13:52.855 INFO:tasks.workunit.client.1.smithi040.stdout:7/403: unlink db/cf 0 2024-04-24T23:13:52.855 INFO:tasks.workunit.client.1.smithi040.stdout:7/404: readlink db/d12/d20/d56/l67 0 2024-04-24T23:13:52.860 INFO:tasks.workunit.client.0.smithi031.stdout:8/805: symlink d8/d15/d37/d43/daf/dca/l116 0 2024-04-24T23:13:52.861 INFO:tasks.workunit.client.1.smithi040.stdout:4/304: truncate f9 3848126 0 2024-04-24T23:13:52.861 INFO:tasks.workunit.client.1.smithi040.stdout:4/305: fsync da/d3f/d6b/d2b/f6d 0 2024-04-24T23:13:52.862 INFO:tasks.workunit.client.0.smithi031.stdout:4/792: link d3/d5e/fda d3/d5e/dc4/ffb 0 2024-04-24T23:13:52.863 INFO:tasks.workunit.client.1.smithi040.stdout:4/306: dread - da/d3f/d40/d41/d60/f4d zero size 2024-04-24T23:13:52.863 INFO:tasks.workunit.client.1.smithi040.stdout:6/338: mkdir d7/d1a/d20/d51/d1e/d46/d69 0 2024-04-24T23:13:52.865 INFO:tasks.workunit.client.1.smithi040.stdout:6/339: truncate d7/d1a/f53 556733 0 2024-04-24T23:13:52.866 INFO:tasks.workunit.client.0.smithi031.stdout:6/735: mkdir d13/d1b/d32/db4/d79/dd7/dfb 0 2024-04-24T23:13:52.866 INFO:tasks.workunit.client.1.smithi040.stdout:7/405: dread db/d43/f5e [0,4194304] 0 2024-04-24T23:13:52.866 INFO:tasks.workunit.client.0.smithi031.stdout:6/736: dread - d13/d15/d1d/d23/d2d/d9e/ff0 zero size 2024-04-24T23:13:52.868 INFO:tasks.workunit.client.0.smithi031.stdout:9/751: write d2/f54 [4204882,14087] 0 2024-04-24T23:13:52.870 INFO:tasks.workunit.client.0.smithi031.stdout:9/752: chown d2/d16/l38 512 1 2024-04-24T23:13:52.872 INFO:tasks.workunit.client.1.smithi040.stdout:6/340: dwrite d7/d1a/d20/d51/d1e/f22 [0,4194304] 0 2024-04-24T23:13:52.876 INFO:tasks.workunit.client.0.smithi031.stdout:6/737: dwrite d13/d1b/d32/d9c/fed [0,4194304] 0 2024-04-24T23:13:52.878 INFO:tasks.workunit.client.1.smithi040.stdout:1/371: rename da/d12/d23/f3b to da/d35/d38/d47/f84 0 2024-04-24T23:13:52.879 INFO:tasks.workunit.client.1.smithi040.stdout:1/372: write da/d31/f52 [8704,96201] 0 2024-04-24T23:13:52.886 INFO:tasks.workunit.client.0.smithi031.stdout:8/806: fsync d8/d15/f57 0 2024-04-24T23:13:52.888 INFO:tasks.workunit.client.1.smithi040.stdout:5/343: creat d2/d7/d8/d1f/d57/f70 x:0 0 0 2024-04-24T23:13:52.889 INFO:tasks.workunit.client.0.smithi031.stdout:4/793: readlink d3/d23/d37/l91 0 2024-04-24T23:13:52.891 INFO:tasks.workunit.client.1.smithi040.stdout:4/307: mkdir da/d3f/d40/d47/d71 0 2024-04-24T23:13:52.891 INFO:tasks.workunit.client.1.smithi040.stdout:4/308: dread - da/d3f/d40/d41/d62/f6f zero size 2024-04-24T23:13:52.892 INFO:tasks.workunit.client.0.smithi031.stdout:0/710: dread da/d22/db0/fd7 [0,4194304] 0 2024-04-24T23:13:52.893 INFO:tasks.workunit.client.0.smithi031.stdout:8/807: dwrite d8/d1a/d1d/f7d [0,4194304] 0 2024-04-24T23:13:52.895 INFO:tasks.workunit.client.1.smithi040.stdout:3/369: write d1/f79 [46392,67810] 0 2024-04-24T23:13:52.896 INFO:tasks.workunit.client.0.smithi031.stdout:8/808: dread d8/d1a/d1d/d50/f88 [0,4194304] 0 2024-04-24T23:13:52.897 INFO:tasks.workunit.client.0.smithi031.stdout:2/551: write d5/f27 [790669,56873] 0 2024-04-24T23:13:52.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/612: creat d1/dc/fbf x:0 0 0 2024-04-24T23:13:52.900 INFO:tasks.workunit.client.1.smithi040.stdout:2/427: getdents da/d1c/d37/d39/d79/da3 0 2024-04-24T23:13:52.902 INFO:tasks.workunit.client.1.smithi040.stdout:2/428: dread - da/d1c/d37/d39/d3f/f92 zero size 2024-04-24T23:13:52.903 INFO:tasks.workunit.client.0.smithi031.stdout:6/738: creat d13/d15/d1d/d23/d44/d47/d69/ffc x:0 0 0 2024-04-24T23:13:52.903 INFO:tasks.workunit.client.1.smithi040.stdout:2/429: write da/d1c/d37/d39/d3f/d52/f95 [256654,27583] 0 2024-04-24T23:13:52.904 INFO:tasks.workunit.client.1.smithi040.stdout:6/341: symlink d7/d1a/d56/l6a 0 2024-04-24T23:13:52.904 INFO:tasks.workunit.client.0.smithi031.stdout:2/552: dread d5/d28/d4e/d7f/fa9 [0,4194304] 0 2024-04-24T23:13:52.906 INFO:tasks.workunit.client.0.smithi031.stdout:2/553: truncate d5/d8/d15/fbc 923581 0 2024-04-24T23:13:52.906 INFO:tasks.workunit.client.0.smithi031.stdout:2/554: chown d5/d6/df/dad/dbe 0 1 2024-04-24T23:13:52.907 INFO:tasks.workunit.client.0.smithi031.stdout:7/641: rename d1/d45/d75/l8d to d1/ldb 0 2024-04-24T23:13:52.909 INFO:tasks.workunit.client.1.smithi040.stdout:9/381: link d7/da/d17/d40/d55/d2f/f6f d7/da/d17/d40/d55/d61/d6c/f85 0 2024-04-24T23:13:52.911 INFO:tasks.workunit.client.1.smithi040.stdout:1/373: mknod da/d12/d18/c85 0 2024-04-24T23:13:52.913 INFO:tasks.workunit.client.1.smithi040.stdout:0/416: link df/d1e/d2b/d6e/c75 df/d1e/d20/d21/d41/c84 0 2024-04-24T23:13:52.918 INFO:tasks.workunit.client.1.smithi040.stdout:3/370: creat d1/d9/d18/d36/d55/f89 x:0 0 0 2024-04-24T23:13:52.918 INFO:tasks.workunit.client.0.smithi031.stdout:8/809: mkdir d8/d15/d37/d43/d7b/d89/d117 0 2024-04-24T23:13:52.920 INFO:tasks.workunit.client.0.smithi031.stdout:1/613: dread d1/d1d/f7a [0,4194304] 0 2024-04-24T23:13:52.922 INFO:tasks.workunit.client.1.smithi040.stdout:2/430: mkdir da/d81/da8 0 2024-04-24T23:13:52.923 INFO:tasks.workunit.client.1.smithi040.stdout:2/431: chown da/d1c/d37/d39/d77/f68 181990 1 2024-04-24T23:13:52.923 INFO:tasks.workunit.client.0.smithi031.stdout:6/739: unlink d13/d15/d1d/d23/d24/fba 0 2024-04-24T23:13:52.923 INFO:tasks.workunit.client.1.smithi040.stdout:8/337: sync 2024-04-24T23:13:52.925 INFO:tasks.workunit.client.1.smithi040.stdout:9/382: symlink d7/da/d17/d40/d55/d2f/d28/d67/l86 0 2024-04-24T23:13:52.926 INFO:tasks.workunit.client.1.smithi040.stdout:9/383: dread - d7/f82 zero size 2024-04-24T23:13:52.926 INFO:tasks.workunit.client.0.smithi031.stdout:7/642: mkdir d1/d45/da0/ddc 0 2024-04-24T23:13:52.930 INFO:tasks.workunit.client.1.smithi040.stdout:0/417: mknod df/d12/d1a/d71/c85 0 2024-04-24T23:13:52.931 INFO:tasks.workunit.client.1.smithi040.stdout:0/418: dread - df/d12/d4b/f6a zero size 2024-04-24T23:13:52.932 INFO:tasks.workunit.client.1.smithi040.stdout:5/344: symlink d2/d7/d8/d1f/d31/l71 0 2024-04-24T23:13:52.934 INFO:tasks.workunit.client.0.smithi031.stdout:4/794: mknod d3/d23/d69/cfc 0 2024-04-24T23:13:52.934 INFO:tasks.workunit.client.1.smithi040.stdout:5/345: dread - d2/d7/d8/d1f/d57/d27/d3c/d43/f6a zero size 2024-04-24T23:13:52.934 INFO:tasks.workunit.client.1.smithi040.stdout:4/309: creat da/d45/d48/d6e/d70/f72 x:0 0 0 2024-04-24T23:13:52.935 INFO:tasks.workunit.client.0.smithi031.stdout:0/711: mknod da/d18/d1a/d37/d41/d92/cee 0 2024-04-24T23:13:52.940 INFO:tasks.workunit.client.1.smithi040.stdout:3/371: mknod d1/d9/c8a 0 2024-04-24T23:13:52.942 INFO:tasks.workunit.client.0.smithi031.stdout:0/712: dwrite da/d11/fc8 [0,4194304] 0 2024-04-24T23:13:52.942 INFO:tasks.workunit.client.1.smithi040.stdout:2/432: truncate da/f19 2040313 0 2024-04-24T23:13:52.944 INFO:tasks.workunit.client.0.smithi031.stdout:9/753: rename d2/dae/d14/d78/d84/d4a/c59 to d2/d43/d9d/dc0/ce5 0 2024-04-24T23:13:52.945 INFO:tasks.workunit.client.0.smithi031.stdout:9/754: stat d2/d16/dbe/d7d/fac 0 2024-04-24T23:13:52.945 INFO:tasks.workunit.client.1.smithi040.stdout:8/338: creat d0/d16/d19/d4f/f6c x:0 0 0 2024-04-24T23:13:52.945 INFO:tasks.workunit.client.1.smithi040.stdout:8/339: readlink d0/d16/d19/d4f/l51 0 2024-04-24T23:13:52.946 INFO:tasks.workunit.client.1.smithi040.stdout:8/340: dread d0/d16/d1b/f40 [0,4194304] 0 2024-04-24T23:13:52.947 INFO:tasks.workunit.client.1.smithi040.stdout:8/341: readlink d0/d8/d22/l24 0 2024-04-24T23:13:52.948 INFO:tasks.workunit.client.0.smithi031.stdout:7/643: mknod d1/dbe/cdd 0 2024-04-24T23:13:52.949 INFO:tasks.workunit.client.0.smithi031.stdout:7/644: chown d1/d45/d84/fcd 0 1 2024-04-24T23:13:52.950 INFO:tasks.workunit.client.0.smithi031.stdout:7/645: chown d1/d45/d75/d9f/fa5 50005 1 2024-04-24T23:13:52.951 INFO:tasks.workunit.client.1.smithi040.stdout:1/374: truncate da/d12/f42 1185641 0 2024-04-24T23:13:52.952 INFO:tasks.workunit.client.0.smithi031.stdout:4/795: creat d3/d20/da7/ffd x:0 0 0 2024-04-24T23:13:52.953 INFO:tasks.workunit.client.1.smithi040.stdout:0/419: creat df/d1e/d20/d61/f86 x:0 0 0 2024-04-24T23:13:52.953 INFO:tasks.workunit.client.0.smithi031.stdout:4/796: read d3/d20/d2d/d44/d5b/d5d/f97 [1646701,79344] 0 2024-04-24T23:13:52.954 INFO:tasks.workunit.client.1.smithi040.stdout:5/346: mknod d2/d7/d8/d1f/d57/d3e/c72 0 2024-04-24T23:13:52.955 INFO:tasks.workunit.client.1.smithi040.stdout:1/375: dread da/d12/d18/f6c [0,4194304] 0 2024-04-24T23:13:52.955 INFO:tasks.workunit.client.1.smithi040.stdout:5/347: chown d2/d7/d8/f50 0 1 2024-04-24T23:13:52.956 INFO:tasks.workunit.client.0.smithi031.stdout:0/713: creat da/d18/d5a/d7d/fef x:0 0 0 2024-04-24T23:13:52.957 INFO:tasks.workunit.client.1.smithi040.stdout:7/406: getdents db/d12/d7d 0 2024-04-24T23:13:52.958 INFO:tasks.workunit.client.1.smithi040.stdout:4/310: dread da/f1a [8388608,4194304] 0 2024-04-24T23:13:52.959 INFO:tasks.workunit.client.0.smithi031.stdout:0/714: read da/d18/d1a/f63 [3102736,13171] 0 2024-04-24T23:13:52.959 INFO:tasks.workunit.client.0.smithi031.stdout:8/810: rename d8/d15/d37/d43/dbd/cc3 to d8/d15/d37/d43/d7b/d89/c118 0 2024-04-24T23:13:52.960 INFO:tasks.workunit.client.1.smithi040.stdout:3/372: write d1/d9/f17 [4809068,23464] 0 2024-04-24T23:13:52.966 INFO:tasks.workunit.client.0.smithi031.stdout:9/755: truncate d2/dae/d14/d78/d84/d4a/d4b/f87 654009 0 2024-04-24T23:13:52.966 INFO:tasks.workunit.client.0.smithi031.stdout:2/555: link d5/d8/d15/d65/c6a d5/d28/d4e/d7f/da7/cc0 0 2024-04-24T23:13:52.970 INFO:tasks.workunit.client.0.smithi031.stdout:7/646: creat d1/d45/d75/d81/fde x:0 0 0 2024-04-24T23:13:52.972 INFO:tasks.workunit.client.1.smithi040.stdout:9/384: rename l1 to d7/da/d17/d40/d55/d61/d6c/l87 0 2024-04-24T23:13:52.973 INFO:tasks.workunit.client.0.smithi031.stdout:1/614: creat d1/dc/d11/d21/d5a/dbb/fc0 x:0 0 0 2024-04-24T23:13:52.982 INFO:tasks.workunit.client.0.smithi031.stdout:0/715: creat da/d22/db0/ff0 x:0 0 0 2024-04-24T23:13:52.983 INFO:tasks.workunit.client.0.smithi031.stdout:6/740: link fe d13/d15/d1d/d23/d24/d63/ffd 0 2024-04-24T23:13:52.984 INFO:tasks.workunit.client.1.smithi040.stdout:0/420: dwrite df/d12/d1a/d71/f28 [4194304,4194304] 0 2024-04-24T23:13:52.984 INFO:tasks.workunit.client.0.smithi031.stdout:8/811: dwrite d8/d15/d9a/fb1 [0,4194304] 0 2024-04-24T23:13:52.985 INFO:tasks.workunit.client.1.smithi040.stdout:0/421: write df/f63 [1425890,24196] 0 2024-04-24T23:13:52.990 INFO:tasks.workunit.client.0.smithi031.stdout:8/812: fdatasync d8/d15/d37/d43/d7b/fe9 0 2024-04-24T23:13:52.991 INFO:tasks.workunit.client.1.smithi040.stdout:7/407: unlink db/d12/d7d/f84 0 2024-04-24T23:13:52.992 INFO:tasks.workunit.client.1.smithi040.stdout:0/422: dwrite df/d12/d1a/d71/f28 [0,4194304] 0 2024-04-24T23:13:52.993 INFO:tasks.workunit.client.0.smithi031.stdout:0/716: dwrite da/d22/db0/ff0 [0,4194304] 0 2024-04-24T23:13:52.994 INFO:tasks.workunit.client.1.smithi040.stdout:0/423: truncate df/f34 744892 0 2024-04-24T23:13:52.995 INFO:tasks.workunit.client.1.smithi040.stdout:4/311: readlink da/l17 0 2024-04-24T23:13:52.998 INFO:tasks.workunit.client.1.smithi040.stdout:3/373: creat d1/d9/d18/d36/d55/d62/f8b x:0 0 0 2024-04-24T23:13:53.001 INFO:tasks.workunit.client.1.smithi040.stdout:6/342: getdents d7/d1a/d20/d51/d2a 0 2024-04-24T23:13:53.008 INFO:tasks.workunit.client.1.smithi040.stdout:3/374: dwrite d1/d9/f17 [0,4194304] 0 2024-04-24T23:13:53.008 INFO:tasks.workunit.client.1.smithi040.stdout:2/433: dwrite da/d29/f4a [0,4194304] 0 2024-04-24T23:13:53.009 INFO:tasks.workunit.client.1.smithi040.stdout:9/385: dread - d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f48 zero size 2024-04-24T23:13:53.011 INFO:tasks.workunit.client.1.smithi040.stdout:5/348: mknod d2/d7/d8/d64/d6b/c73 0 2024-04-24T23:13:53.011 INFO:tasks.workunit.client.1.smithi040.stdout:2/434: read - da/d1c/d37/d39/d77/f68 zero size 2024-04-24T23:13:53.018 INFO:tasks.workunit.client.1.smithi040.stdout:7/408: mknod db/d12/d20/c93 0 2024-04-24T23:13:53.018 INFO:tasks.workunit.client.1.smithi040.stdout:9/386: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 [4194304,4194304] 0 2024-04-24T23:13:53.023 INFO:tasks.workunit.client.1.smithi040.stdout:9/387: write d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f50 [1922511,1142] 0 2024-04-24T23:13:53.024 INFO:tasks.workunit.client.1.smithi040.stdout:6/343: chown d7/de/l3b 0 1 2024-04-24T23:13:53.036 INFO:tasks.workunit.client.1.smithi040.stdout:2/435: fsync da/d10/f2b 0 2024-04-24T23:13:53.038 INFO:tasks.workunit.client.1.smithi040.stdout:0/424: creat df/d1e/d20/d21/d41/d52/f87 x:0 0 0 2024-04-24T23:13:53.040 INFO:tasks.workunit.client.1.smithi040.stdout:7/409: creat db/d2f/d37/f94 x:0 0 0 2024-04-24T23:13:53.041 INFO:tasks.workunit.client.1.smithi040.stdout:1/376: truncate da/d12/f4a 289246 0 2024-04-24T23:13:53.044 INFO:tasks.workunit.client.1.smithi040.stdout:0/425: dwrite df/d12/d1a/f67 [0,4194304] 0 2024-04-24T23:13:53.046 INFO:tasks.workunit.client.1.smithi040.stdout:9/388: unlink d7/da/d17/d40/d55/d2f/d28/d67/l86 0 2024-04-24T23:13:53.046 INFO:tasks.workunit.client.1.smithi040.stdout:3/375: dread d1/d9/d15/f1b [4194304,4194304] 0 2024-04-24T23:13:53.048 INFO:tasks.workunit.client.1.smithi040.stdout:9/389: fsync d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 0 2024-04-24T23:13:53.053 INFO:tasks.workunit.client.1.smithi040.stdout:2/436: rmdir da/d29/d55 39 2024-04-24T23:13:53.056 INFO:tasks.workunit.client.1.smithi040.stdout:6/344: creat d7/d1a/d20/d51/d1e/d46/d69/f6b x:0 0 0 2024-04-24T23:13:53.057 INFO:tasks.workunit.client.1.smithi040.stdout:2/437: dread da/d1c/d37/d8a/f91 [0,4194304] 0 2024-04-24T23:13:53.062 INFO:tasks.workunit.client.1.smithi040.stdout:8/342: rename d0/d16/d19/c31 to d0/d16/c6d 0 2024-04-24T23:13:53.063 INFO:tasks.workunit.client.1.smithi040.stdout:8/343: chown d0/d16/d1b/d58 9 1 2024-04-24T23:13:53.064 INFO:tasks.workunit.client.1.smithi040.stdout:8/344: dread - d0/d8/d22/f6a zero size 2024-04-24T23:13:53.065 INFO:tasks.workunit.client.1.smithi040.stdout:8/345: dread d0/d16/d1b/d47/f48 [0,4194304] 0 2024-04-24T23:13:53.066 INFO:tasks.workunit.client.1.smithi040.stdout:9/390: creat d7/da/d17/d40/d55/d2f/d28/f88 x:0 0 0 2024-04-24T23:13:53.075 INFO:tasks.workunit.client.0.smithi031.stdout:1/615: mknod d1/da/d10/d52/d62/cc1 0 2024-04-24T23:13:53.077 INFO:tasks.workunit.client.1.smithi040.stdout:4/312: rename da/d3f/f4b to da/d45/d58/f73 0 2024-04-24T23:13:53.079 INFO:tasks.workunit.client.0.smithi031.stdout:7/647: dwrite d1/d45/d75/d9f/d66/fd1 [4194304,4194304] 0 2024-04-24T23:13:53.085 INFO:tasks.workunit.client.1.smithi040.stdout:5/349: link d2/d7/d8/d1f/f51 d2/d7/f74 0 2024-04-24T23:13:53.089 INFO:tasks.workunit.client.1.smithi040.stdout:6/345: mknod d7/de/c6c 0 2024-04-24T23:13:53.090 INFO:tasks.workunit.client.0.smithi031.stdout:0/717: mknod da/d18/d1a/dd6/cf1 0 2024-04-24T23:13:53.091 INFO:tasks.workunit.client.0.smithi031.stdout:0/718: dread - da/d11/d6a/dba/fca zero size 2024-04-24T23:13:53.091 INFO:tasks.workunit.client.1.smithi040.stdout:2/438: creat da/d29/d55/fa9 x:0 0 0 2024-04-24T23:13:53.093 INFO:tasks.workunit.client.1.smithi040.stdout:0/426: creat df/d14/f88 x:0 0 0 2024-04-24T23:13:53.097 INFO:tasks.workunit.client.1.smithi040.stdout:3/376: dwrite d1/d9/d15/f1d [0,4194304] 0 2024-04-24T23:13:53.099 INFO:tasks.workunit.client.0.smithi031.stdout:6/741: creat d13/d1b/d32/d9c/ddb/df1/ffe x:0 0 0 2024-04-24T23:13:53.102 INFO:tasks.workunit.client.1.smithi040.stdout:7/410: rename db/d12/d20/d52/c7c to db/d43/c95 0 2024-04-24T23:13:53.103 INFO:tasks.workunit.client.1.smithi040.stdout:7/411: fsync db/d12/d20/d52/d4b/f57 0 2024-04-24T23:13:53.105 INFO:tasks.workunit.client.0.smithi031.stdout:4/797: write d3/d20/d2d/d44/f2c [5220499,112063] 0 2024-04-24T23:13:53.110 INFO:tasks.workunit.client.1.smithi040.stdout:2/439: unlink da/d29/f48 0 2024-04-24T23:13:53.111 INFO:tasks.workunit.client.1.smithi040.stdout:9/391: dread d7/f59 [0,4194304] 0 2024-04-24T23:13:53.112 INFO:tasks.workunit.client.0.smithi031.stdout:2/556: rename d5/d8/d15/f22 to d5/d6/fc1 0 2024-04-24T23:13:53.113 INFO:tasks.workunit.client.0.smithi031.stdout:2/557: dread - d5/d8/d45/d5b/fbd zero size 2024-04-24T23:13:53.114 INFO:tasks.workunit.client.0.smithi031.stdout:6/742: creat d13/d15/d1d/d23/d2d/fff x:0 0 0 2024-04-24T23:13:53.119 INFO:tasks.workunit.client.0.smithi031.stdout:7/648: rename d1/d2/dd/d33/c57 to d1/d2/dd/d33/d5e/d64/d6d/dae/db1/cdf 0 2024-04-24T23:13:53.119 INFO:tasks.workunit.client.0.smithi031.stdout:8/813: dwrite d8/d15/d37/d43/d7b/d86/fb6 [0,4194304] 0 2024-04-24T23:13:53.119 INFO:tasks.workunit.client.0.smithi031.stdout:9/756: write d2/dae/d14/d78/d84/d4a/d4b/f87 [423446,121867] 0 2024-04-24T23:13:53.121 INFO:tasks.workunit.client.1.smithi040.stdout:1/377: rename da/d12/d18/d2a/l2e to da/d35/d38/d47/l86 0 2024-04-24T23:13:53.126 INFO:tasks.workunit.client.0.smithi031.stdout:9/757: dread d2/dae/d14/d78/d84/d4a/fb1 [0,4194304] 0 2024-04-24T23:13:53.126 INFO:tasks.workunit.client.1.smithi040.stdout:4/313: write da/f18 [1278562,8429] 0 2024-04-24T23:13:53.128 INFO:tasks.workunit.client.0.smithi031.stdout:8/814: fsync d8/d1a/f7e 0 2024-04-24T23:13:53.131 INFO:tasks.workunit.client.1.smithi040.stdout:6/346: unlink d7/c36 0 2024-04-24T23:13:53.131 INFO:tasks.workunit.client.1.smithi040.stdout:6/347: chown d7/d1a/d20/d51/d2a 1598371 1 2024-04-24T23:13:53.135 INFO:tasks.workunit.client.0.smithi031.stdout:7/649: truncate d1/d2/f11 1558712 0 2024-04-24T23:13:53.136 INFO:tasks.workunit.client.0.smithi031.stdout:0/719: dwrite da/d11/f90 [0,4194304] 0 2024-04-24T23:13:53.139 INFO:tasks.workunit.client.1.smithi040.stdout:2/440: mkdir da/d10/d15/daa 0 2024-04-24T23:13:53.142 INFO:tasks.workunit.client.0.smithi031.stdout:4/798: rename d3/d20/d2d/d44/d5b/cf5 to d3/d5e/dc2/cfe 0 2024-04-24T23:13:53.144 INFO:tasks.workunit.client.0.smithi031.stdout:6/743: dread d13/d15/d1d/d23/d2d/f3d [0,4194304] 0 2024-04-24T23:13:53.144 INFO:tasks.workunit.client.1.smithi040.stdout:0/427: mkdir df/d1e/d89 0 2024-04-24T23:13:53.146 INFO:tasks.workunit.client.0.smithi031.stdout:6/744: truncate d13/d15/d1d/d23/d2d/d9e/ff8 62357 0 2024-04-24T23:13:53.146 INFO:tasks.workunit.client.0.smithi031.stdout:9/758: unlink d2/dae/f41 0 2024-04-24T23:13:53.150 INFO:tasks.workunit.client.0.smithi031.stdout:8/815: creat d8/d15/d37/d43/d7b/dd3/f119 x:0 0 0 2024-04-24T23:13:53.152 INFO:tasks.workunit.client.1.smithi040.stdout:8/346: rename d0/d16/d19/d4f/l2b to d0/d8/d66/l6e 0 2024-04-24T23:13:53.155 INFO:tasks.workunit.client.0.smithi031.stdout:0/720: creat da/d86/dbf/ff2 x:0 0 0 2024-04-24T23:13:53.155 INFO:tasks.workunit.client.0.smithi031.stdout:0/721: chown da/d11/f59 3 1 2024-04-24T23:13:53.159 INFO:tasks.workunit.client.1.smithi040.stdout:5/350: truncate d2/d7/d8/d1f/d57/d27/d65/f5a 4536677 0 2024-04-24T23:13:53.160 INFO:tasks.workunit.client.1.smithi040.stdout:5/351: chown d2/d7/d8/d1f/d57/l63 1972365 1 2024-04-24T23:13:53.162 INFO:tasks.workunit.client.0.smithi031.stdout:2/558: rename d5/d6/df to d5/d6/d29/d26/db0/dc2 0 2024-04-24T23:13:53.168 INFO:tasks.workunit.client.0.smithi031.stdout:9/759: write d2/f12 [5893503,73821] 0 2024-04-24T23:13:53.170 INFO:tasks.workunit.client.0.smithi031.stdout:2/559: dwrite d5/d28/d4e/d5e/fb4 [4194304,4194304] 0 2024-04-24T23:13:53.178 INFO:tasks.workunit.client.1.smithi040.stdout:8/347: mkdir d0/d16/d19/d6f 0 2024-04-24T23:13:53.186 INFO:tasks.workunit.client.1.smithi040.stdout:5/352: creat d2/d7/d8/db/f75 x:0 0 0 2024-04-24T23:13:53.187 INFO:tasks.workunit.client.1.smithi040.stdout:5/353: write d2/d7/d8/db/f75 [35585,106112] 0 2024-04-24T23:13:53.189 INFO:tasks.workunit.client.1.smithi040.stdout:6/348: unlink d7/de/l5f 0 2024-04-24T23:13:53.189 INFO:tasks.workunit.client.0.smithi031.stdout:6/745: dwrite d13/d15/d1d/d23/d44/d47/d69/f7f [0,4194304] 0 2024-04-24T23:13:53.190 INFO:tasks.workunit.client.0.smithi031.stdout:9/760: creat d2/d43/fe6 x:0 0 0 2024-04-24T23:13:53.192 INFO:tasks.workunit.client.0.smithi031.stdout:2/560: symlink d5/d6/d29/d26/lc3 0 2024-04-24T23:13:53.192 INFO:tasks.workunit.client.1.smithi040.stdout:1/378: dwrite da/d12/d18/f1d [0,4194304] 0 2024-04-24T23:13:53.193 INFO:tasks.workunit.client.1.smithi040.stdout:1/379: readlink da/d12/d14/l7b 0 2024-04-24T23:13:53.193 INFO:tasks.workunit.client.0.smithi031.stdout:2/561: write d5/d28/f40 [4296460,35613] 0 2024-04-24T23:13:53.198 INFO:tasks.workunit.client.0.smithi031.stdout:7/650: rmdir d1/d2/dd/d33/d5e/d64/d6d/dae/db1/dc5 0 2024-04-24T23:13:53.199 INFO:tasks.workunit.client.1.smithi040.stdout:3/377: getdents d1/d2d/de/d30 0 2024-04-24T23:13:53.201 INFO:tasks.workunit.client.0.smithi031.stdout:4/799: creat d3/d23/d37/fff x:0 0 0 2024-04-24T23:13:53.201 INFO:tasks.workunit.client.1.smithi040.stdout:8/348: chown d0/d38/c69 506 1 2024-04-24T23:13:53.204 INFO:tasks.workunit.client.1.smithi040.stdout:7/412: getdents db/d2f/d37 0 2024-04-24T23:13:53.204 INFO:tasks.workunit.client.1.smithi040.stdout:7/413: chown db/d12/d62 9 1 2024-04-24T23:13:53.205 INFO:tasks.workunit.client.1.smithi040.stdout:5/354: stat d2/d7/fe 0 2024-04-24T23:13:53.207 INFO:tasks.workunit.client.1.smithi040.stdout:6/349: readlink d7/l18 0 2024-04-24T23:13:53.208 INFO:tasks.workunit.client.1.smithi040.stdout:2/441: rmdir da/d10/d4d/da6 0 2024-04-24T23:13:53.210 INFO:tasks.workunit.client.0.smithi031.stdout:7/651: creat d1/dbe/fe0 x:0 0 0 2024-04-24T23:13:53.212 INFO:tasks.workunit.client.1.smithi040.stdout:0/428: write df/d12/d49/f54 [696850,96750] 0 2024-04-24T23:13:53.212 INFO:tasks.workunit.client.1.smithi040.stdout:6/350: dwrite d7/d1a/d20/d51/f26 [0,4194304] 0 2024-04-24T23:13:53.216 INFO:tasks.workunit.client.1.smithi040.stdout:9/392: getdents d7/da/d17/d40/d55/d61/d6c 0 2024-04-24T23:13:53.220 INFO:tasks.workunit.client.1.smithi040.stdout:3/378: creat d1/d2a/d6a/f8c x:0 0 0 2024-04-24T23:13:53.223 INFO:tasks.workunit.client.1.smithi040.stdout:4/314: link da/f37 da/d3f/d40/d47/d71/f74 0 2024-04-24T23:13:53.225 INFO:tasks.workunit.client.0.smithi031.stdout:8/816: getdents d8/d15/d37/d43/d5c 0 2024-04-24T23:13:53.226 INFO:tasks.workunit.client.0.smithi031.stdout:8/817: fsync d8/d15/d37/d43/d7b/f8c 0 2024-04-24T23:13:53.226 INFO:tasks.workunit.client.1.smithi040.stdout:7/414: mknod db/d12/d20/d56/c96 0 2024-04-24T23:13:53.230 INFO:tasks.workunit.client.0.smithi031.stdout:7/652: rename d1/d45 to d1/d2/dd/d33/d5e/d64/d6d/de1 0 2024-04-24T23:13:53.232 INFO:tasks.workunit.client.0.smithi031.stdout:6/746: dwrite d13/d15/d1d/d23/f2b [0,4194304] 0 2024-04-24T23:13:53.235 INFO:tasks.workunit.client.0.smithi031.stdout:2/562: write d5/f25 [739120,53728] 0 2024-04-24T23:13:53.235 INFO:tasks.workunit.client.1.smithi040.stdout:1/380: mknod da/d12/d18/d2a/c87 0 2024-04-24T23:13:53.236 INFO:tasks.workunit.client.0.smithi031.stdout:7/653: write d1/d2/dd/d33/fcb [654370,91573] 0 2024-04-24T23:13:53.236 INFO:tasks.workunit.client.1.smithi040.stdout:1/381: readlink da/d12/l2c 0 2024-04-24T23:13:53.237 INFO:tasks.workunit.client.1.smithi040.stdout:0/429: mkdir df/d1e/d20/d21/d41/d8a 0 2024-04-24T23:13:53.239 INFO:tasks.workunit.client.0.smithi031.stdout:9/761: dwrite d2/f8 [0,4194304] 0 2024-04-24T23:13:53.240 INFO:tasks.workunit.client.1.smithi040.stdout:9/393: creat d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f89 x:0 0 0 2024-04-24T23:13:53.240 INFO:tasks.workunit.client.0.smithi031.stdout:7/654: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/f80 [4546041,20187] 0 2024-04-24T23:13:53.242 INFO:tasks.workunit.client.0.smithi031.stdout:8/818: unlink d8/d15/d37/d43/d5c/de1/f77 0 2024-04-24T23:13:53.243 INFO:tasks.workunit.client.0.smithi031.stdout:7/655: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/lb9 0 2024-04-24T23:13:53.244 INFO:tasks.workunit.client.1.smithi040.stdout:4/315: mkdir da/d3f/d40/d47/d75 0 2024-04-24T23:13:53.246 INFO:tasks.workunit.client.1.smithi040.stdout:4/316: write da/d3f/d40/d41/f67 [1004614,101124] 0 2024-04-24T23:13:53.247 INFO:tasks.workunit.client.0.smithi031.stdout:6/747: read d13/d15/d1d/d23/d24/d43/d68/fa3 [574131,12227] 0 2024-04-24T23:13:53.248 INFO:tasks.workunit.client.0.smithi031.stdout:2/563: mkdir d5/d8/d45/d5b/d49/dc4 0 2024-04-24T23:13:53.250 INFO:tasks.workunit.client.0.smithi031.stdout:6/748: truncate d13/d15/d1d/d23/d24/d8b/f9b 1348197 0 2024-04-24T23:13:53.251 INFO:tasks.workunit.client.0.smithi031.stdout:9/762: creat d2/dae/d14/d1d/d45/d99/d9c/fe7 x:0 0 0 2024-04-24T23:13:53.251 INFO:tasks.workunit.client.0.smithi031.stdout:6/749: readlink d13/d15/d1d/d23/d44/d47/lce 0 2024-04-24T23:13:53.252 INFO:tasks.workunit.client.1.smithi040.stdout:1/382: mknod da/d12/d18/d2a/d32/c88 0 2024-04-24T23:13:53.252 INFO:tasks.workunit.client.1.smithi040.stdout:3/379: dread d1/d2d/de/f34 [0,4194304] 0 2024-04-24T23:13:53.253 INFO:tasks.workunit.client.0.smithi031.stdout:4/800: getdents d3/d23/d69/dce 0 2024-04-24T23:13:53.253 INFO:tasks.workunit.client.1.smithi040.stdout:3/380: read d1/d2d/de/d30/f41 [157070,25797] 0 2024-04-24T23:13:53.254 INFO:tasks.workunit.client.1.smithi040.stdout:0/430: rmdir df 39 2024-04-24T23:13:53.254 INFO:tasks.workunit.client.1.smithi040.stdout:3/381: fdatasync d1/f88 0 2024-04-24T23:13:53.256 INFO:tasks.workunit.client.0.smithi031.stdout:8/819: rename d8/d15/d37/d43/d5c/df4/cf7 to d8/d15/d37/d43/dbd/c11a 0 2024-04-24T23:13:53.256 INFO:tasks.workunit.client.0.smithi031.stdout:4/801: read d3/d20/d25/d60/ff8 [324125,11968] 0 2024-04-24T23:13:53.257 INFO:tasks.workunit.client.1.smithi040.stdout:9/394: creat d7/da/d17/d40/d55/d2f/d1f/f8a x:0 0 0 2024-04-24T23:13:53.258 INFO:tasks.workunit.client.0.smithi031.stdout:2/564: creat d5/d6/d29/d4f/fc5 x:0 0 0 2024-04-24T23:13:53.261 INFO:tasks.workunit.client.1.smithi040.stdout:3/382: dwrite d1/d2d/f24 [8388608,4194304] 0 2024-04-24T23:13:53.261 INFO:tasks.workunit.client.0.smithi031.stdout:7/656: rmdir d1/d2/dd/d33/d5e/d64/d6d/de1 39 2024-04-24T23:13:53.262 INFO:tasks.workunit.client.1.smithi040.stdout:3/383: truncate d1/d9/d2e/f84 371402 0 2024-04-24T23:13:53.263 INFO:tasks.workunit.client.1.smithi040.stdout:5/355: rename d2/d7/d8/f50 to d2/d2f/d14/f76 0 2024-04-24T23:13:53.263 INFO:tasks.workunit.client.0.smithi031.stdout:9/763: creat d2/d43/d9d/dc0/fe8 x:0 0 0 2024-04-24T23:13:53.265 INFO:tasks.workunit.client.0.smithi031.stdout:8/820: rmdir d8/d15/d37/d43/d7b/d87/dad 39 2024-04-24T23:13:53.265 INFO:tasks.workunit.client.1.smithi040.stdout:5/356: write d2/d2f/d14/f61 [956984,102423] 0 2024-04-24T23:13:53.267 INFO:tasks.workunit.client.0.smithi031.stdout:9/764: dread d2/dae/d14/d1d/f3c [0,4194304] 0 2024-04-24T23:13:53.267 INFO:tasks.workunit.client.0.smithi031.stdout:4/802: fdatasync d3/d20/d25/d60/d63/db9/fcd 0 2024-04-24T23:13:53.271 INFO:tasks.workunit.client.1.smithi040.stdout:1/383: rmdir da/d12/d14 39 2024-04-24T23:13:53.272 INFO:tasks.workunit.client.1.smithi040.stdout:7/415: write db/d2f/f35 [890750,62860] 0 2024-04-24T23:13:53.275 INFO:tasks.workunit.client.0.smithi031.stdout:4/803: dwrite d3/d20/d2d/d44/f2c [4194304,4194304] 0 2024-04-24T23:13:53.276 INFO:tasks.workunit.client.1.smithi040.stdout:6/351: write d7/f9 [2263388,87008] 0 2024-04-24T23:13:53.277 INFO:tasks.workunit.client.1.smithi040.stdout:6/352: chown d7/d1a/d20/d51/d1e/d5d/d52/c4c 239751 1 2024-04-24T23:13:53.279 INFO:tasks.workunit.client.0.smithi031.stdout:8/821: mknod d8/d15/d37/d43/da5/c11b 0 2024-04-24T23:13:53.282 INFO:tasks.workunit.client.1.smithi040.stdout:4/317: write da/d3f/d6b/f21 [2832295,16660] 0 2024-04-24T23:13:53.283 INFO:tasks.workunit.client.1.smithi040.stdout:4/318: fsync da/d3f/d40/d41/d60/f23 0 2024-04-24T23:13:53.285 INFO:tasks.workunit.client.1.smithi040.stdout:9/395: rename d7/da/d17/d40/d55/d61/f75 to d7/da/d17/d40/d55/d2f/d1f/d4e/f8b 0 2024-04-24T23:13:53.288 INFO:tasks.workunit.client.1.smithi040.stdout:1/384: symlink da/d12/d18/d2a/d41/l89 0 2024-04-24T23:13:53.289 INFO:tasks.workunit.client.1.smithi040.stdout:0/431: dread df/d12/d1a/f2e [0,4194304] 0 2024-04-24T23:13:53.290 INFO:tasks.workunit.client.1.smithi040.stdout:7/416: creat db/d2f/d37/d7e/f97 x:0 0 0 2024-04-24T23:13:53.291 INFO:tasks.workunit.client.0.smithi031.stdout:7/657: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2 0 2024-04-24T23:13:53.292 INFO:tasks.workunit.client.0.smithi031.stdout:8/822: fsync d8/d15/d37/d43/d7b/dd3/f100 0 2024-04-24T23:13:53.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/658: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/lb9 0 2024-04-24T23:13:53.293 INFO:tasks.workunit.client.0.smithi031.stdout:9/765: mknod d2/dae/d14/d78/d84/d4a/d4b/ce9 0 2024-04-24T23:13:53.294 INFO:tasks.workunit.client.1.smithi040.stdout:4/319: symlink da/d3f/d40/d41/l76 0 2024-04-24T23:13:53.295 INFO:tasks.workunit.client.0.smithi031.stdout:2/565: link d5/d28/d4e/d5e/l8b d5/d6/d29/d26/db0/dc2/dad/dbe/lc6 0 2024-04-24T23:13:53.297 INFO:tasks.workunit.client.1.smithi040.stdout:3/384: rename d1/d9/d18/d36/d55/d5a to d1/d2a/d8d 0 2024-04-24T23:13:53.297 INFO:tasks.workunit.client.0.smithi031.stdout:6/750: link d13/d1b/d32/cbd d13/d15/d1d/d23/d2d/c100 0 2024-04-24T23:13:53.301 INFO:tasks.workunit.client.1.smithi040.stdout:9/396: mkdir d7/da/d17/d40/d55/d61/d6c/d8c 0 2024-04-24T23:13:53.302 INFO:tasks.workunit.client.0.smithi031.stdout:7/659: fdatasync d1/d2/dd/f17 0 2024-04-24T23:13:53.305 INFO:tasks.workunit.client.1.smithi040.stdout:0/432: creat df/d12/d4b/f8b x:0 0 0 2024-04-24T23:13:53.308 INFO:tasks.workunit.client.0.smithi031.stdout:2/566: rename d5/d28/d4e/d7f to d5/d6/d29/d26/db0/dc7 0 2024-04-24T23:13:53.309 INFO:tasks.workunit.client.1.smithi040.stdout:7/417: mknod db/d2f/d37/c98 0 2024-04-24T23:13:53.309 INFO:tasks.workunit.client.0.smithi031.stdout:2/567: chown d5/d8/d15/fbc 117898 1 2024-04-24T23:13:53.310 INFO:tasks.workunit.client.0.smithi031.stdout:4/804: dwrite d3/d23/d69/f31 [4194304,4194304] 0 2024-04-24T23:13:53.310 INFO:tasks.workunit.client.0.smithi031.stdout:6/751: mknod d13/d1b/d32/d9c/ddb/df1/c101 0 2024-04-24T23:13:53.314 INFO:tasks.workunit.client.1.smithi040.stdout:7/418: dwrite db/d12/d3f/f41 [4194304,4194304] 0 2024-04-24T23:13:53.316 INFO:tasks.workunit.client.1.smithi040.stdout:4/320: mknod da/d45/c77 0 2024-04-24T23:13:53.316 INFO:tasks.workunit.client.1.smithi040.stdout:5/357: truncate d2/d7/d8/db/d1e/f28 1807650 0 2024-04-24T23:13:53.319 INFO:tasks.workunit.client.1.smithi040.stdout:5/358: readlink d2/d7/d8/l2c 0 2024-04-24T23:13:53.319 INFO:tasks.workunit.client.1.smithi040.stdout:3/385: creat d1/d2a/d6a/f8e x:0 0 0 2024-04-24T23:13:53.321 INFO:tasks.workunit.client.1.smithi040.stdout:0/433: fsync df/d12/d4b/f8b 0 2024-04-24T23:13:53.323 INFO:tasks.workunit.client.0.smithi031.stdout:2/568: symlink d5/d8/d15/d56/d84/d75/lc8 0 2024-04-24T23:13:53.323 INFO:tasks.workunit.client.0.smithi031.stdout:5/613: sync 2024-04-24T23:13:53.323 INFO:tasks.workunit.client.0.smithi031.stdout:3/604: sync 2024-04-24T23:13:53.324 INFO:tasks.workunit.client.1.smithi040.stdout:7/419: dwrite db/d2f/d37/f94 [0,4194304] 0 2024-04-24T23:13:53.325 INFO:tasks.workunit.client.0.smithi031.stdout:4/805: creat d3/d23/d3b/f100 x:0 0 0 2024-04-24T23:13:53.326 INFO:tasks.workunit.client.1.smithi040.stdout:6/353: creat d7/d1a/d20/f6d x:0 0 0 2024-04-24T23:13:53.327 INFO:tasks.workunit.client.0.smithi031.stdout:6/752: mkdir d13/d15/d1d/d23/d44/da0/da6/de6/d102 0 2024-04-24T23:13:53.328 INFO:tasks.workunit.client.0.smithi031.stdout:8/823: creat d8/d1a/f11c x:0 0 0 2024-04-24T23:13:53.330 INFO:tasks.workunit.client.0.smithi031.stdout:7/660: symlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/le3 0 2024-04-24T23:13:53.332 INFO:tasks.workunit.client.0.smithi031.stdout:9/766: creat d2/dae/d14/d78/fea x:0 0 0 2024-04-24T23:13:53.334 INFO:tasks.workunit.client.0.smithi031.stdout:2/569: creat d5/d6/d29/fc9 x:0 0 0 2024-04-24T23:13:53.335 INFO:tasks.workunit.client.1.smithi040.stdout:0/434: creat df/d1e/d20/d21/d51/f8c x:0 0 0 2024-04-24T23:13:53.335 INFO:tasks.workunit.client.1.smithi040.stdout:9/397: dread d7/da/d17/d40/f42 [0,4194304] 0 2024-04-24T23:13:53.336 INFO:tasks.workunit.client.1.smithi040.stdout:3/386: dread d1/d2a/d6a/f3a [0,4194304] 0 2024-04-24T23:13:53.337 INFO:tasks.workunit.client.1.smithi040.stdout:7/420: creat db/d2f/f99 x:0 0 0 2024-04-24T23:13:53.338 INFO:tasks.workunit.client.1.smithi040.stdout:9/398: truncate d7/da/d17/d40/d55/d2f/d1f/d34/f5b 1226432 0 2024-04-24T23:13:53.343 INFO:tasks.workunit.client.1.smithi040.stdout:2/442: sync 2024-04-24T23:13:53.343 INFO:tasks.workunit.client.1.smithi040.stdout:8/349: sync 2024-04-24T23:13:53.344 INFO:tasks.workunit.client.1.smithi040.stdout:4/321: mkdir da/d78 0 2024-04-24T23:13:53.346 INFO:tasks.workunit.client.0.smithi031.stdout:8/824: mkdir d8/d15/d37/d43/dbd/d11d 0 2024-04-24T23:13:53.347 INFO:tasks.workunit.client.0.smithi031.stdout:0/722: sync 2024-04-24T23:13:53.347 INFO:tasks.workunit.client.0.smithi031.stdout:1/616: sync 2024-04-24T23:13:53.349 INFO:tasks.workunit.client.1.smithi040.stdout:5/359: unlink d2/d7/d8/d1f/f38 0 2024-04-24T23:13:53.349 INFO:tasks.workunit.client.1.smithi040.stdout:2/443: dwrite da/d1c/d37/d39/d3f/f92 [0,4194304] 0 2024-04-24T23:13:53.350 INFO:tasks.workunit.client.0.smithi031.stdout:1/617: write d1/da/d12/d65/fa0 [232408,29566] 0 2024-04-24T23:13:53.351 INFO:tasks.workunit.client.0.smithi031.stdout:1/618: write d1/dc/d11/f20 [593875,127949] 0 2024-04-24T23:13:53.355 INFO:tasks.workunit.client.0.smithi031.stdout:2/570: unlink d5/d28/d4e/d5e/f60 0 2024-04-24T23:13:53.358 INFO:tasks.workunit.client.0.smithi031.stdout:7/661: dread d1/d2/f11 [0,4194304] 0 2024-04-24T23:13:53.360 INFO:tasks.workunit.client.0.smithi031.stdout:3/605: creat d6/df/d16/d2a/d95/d9b/d1a/d25/d83/fb8 x:0 0 0 2024-04-24T23:13:53.364 INFO:tasks.workunit.client.0.smithi031.stdout:5/614: write d0/d7/d74/f82 [1444684,34892] 0 2024-04-24T23:13:53.365 INFO:tasks.workunit.client.1.smithi040.stdout:8/350: creat d0/d16/d19/d4f/f70 x:0 0 0 2024-04-24T23:13:53.367 INFO:tasks.workunit.client.1.smithi040.stdout:4/322: fsync da/d3f/d40/d41/f67 0 2024-04-24T23:13:53.371 INFO:tasks.workunit.client.0.smithi031.stdout:4/806: dwrite d3/d23/d3b/f56 [0,4194304] 0 2024-04-24T23:13:53.372 INFO:tasks.workunit.client.0.smithi031.stdout:0/723: symlink da/d18/d5a/lf3 0 2024-04-24T23:13:53.377 INFO:tasks.workunit.client.0.smithi031.stdout:1/619: dread d1/dc/d11/d21/d5a/f76 [0,4194304] 0 2024-04-24T23:13:53.378 INFO:tasks.workunit.client.1.smithi040.stdout:0/435: mkdir df/d14/d8d 0 2024-04-24T23:13:53.378 INFO:tasks.workunit.client.1.smithi040.stdout:3/387: write d1/d2d/de/d30/f41 [317937,104750] 0 2024-04-24T23:13:53.380 INFO:tasks.workunit.client.1.smithi040.stdout:3/388: write d1/d9/d2e/f5f [3785944,42979] 0 2024-04-24T23:13:53.382 INFO:tasks.workunit.client.0.smithi031.stdout:9/767: fdatasync d2/dae/d14/d1d/f35 0 2024-04-24T23:13:53.384 INFO:tasks.workunit.client.0.smithi031.stdout:9/768: chown d2/dae/d14/d78/d84/da6/cb0 3 1 2024-04-24T23:13:53.384 INFO:tasks.workunit.client.0.smithi031.stdout:6/753: truncate d13/d15/f27 3820625 0 2024-04-24T23:13:53.385 INFO:tasks.workunit.client.1.smithi040.stdout:1/385: link da/d12/d14/l15 da/d12/d18/d2a/d4d/l8a 0 2024-04-24T23:13:53.386 INFO:tasks.workunit.client.0.smithi031.stdout:8/825: write d8/d1a/d1d/d50/dc0/d9d/fa3 [80385,100572] 0 2024-04-24T23:13:53.387 INFO:tasks.workunit.client.1.smithi040.stdout:6/354: dwrite f2 [0,4194304] 0 2024-04-24T23:13:53.388 INFO:tasks.workunit.client.0.smithi031.stdout:8/826: chown d8/d1a/d1d/d35/lf6 24859 1 2024-04-24T23:13:53.388 INFO:tasks.workunit.client.0.smithi031.stdout:0/724: dread da/d11/fa0 [0,4194304] 0 2024-04-24T23:13:53.393 INFO:tasks.workunit.client.1.smithi040.stdout:8/351: mkdir d0/d16/d19/d71 0 2024-04-24T23:13:53.399 INFO:tasks.workunit.client.0.smithi031.stdout:4/807: mknod d3/d20/d25/d60/d63/db9/c101 0 2024-04-24T23:13:53.402 INFO:tasks.workunit.client.1.smithi040.stdout:9/399: truncate f4 5979305 0 2024-04-24T23:13:53.403 INFO:tasks.workunit.client.1.smithi040.stdout:0/436: dread df/d12/d1a/f4e [4194304,4194304] 0 2024-04-24T23:13:53.403 INFO:tasks.workunit.client.1.smithi040.stdout:9/400: dread - d7/da/d17/d40/d55/d2f/d28/f88 zero size 2024-04-24T23:13:53.404 INFO:tasks.workunit.client.1.smithi040.stdout:4/323: unlink da/d3f/d40/d63/c6c 0 2024-04-24T23:13:53.411 INFO:tasks.workunit.client.0.smithi031.stdout:9/769: symlink d2/d16/dbe/leb 0 2024-04-24T23:13:53.412 INFO:tasks.workunit.client.0.smithi031.stdout:9/770: chown d2/dae/d14/d78/d84/d7e/dc7 588473 1 2024-04-24T23:13:53.422 INFO:tasks.workunit.client.1.smithi040.stdout:5/360: truncate d2/d7/d8/d1f/d57/f44 3824424 0 2024-04-24T23:13:53.422 INFO:tasks.workunit.client.1.smithi040.stdout:5/361: read - d2/d2f/d14/f76 zero size 2024-04-24T23:13:53.423 INFO:tasks.workunit.client.1.smithi040.stdout:5/362: dread - d2/d7/d8/d64/f68 zero size 2024-04-24T23:13:53.424 INFO:tasks.workunit.client.1.smithi040.stdout:6/355: rename d7/d1a/d20/d51/d1e/d5d/d52/c63 to d7/d1a/d20/d51/d1e/d46/c6e 0 2024-04-24T23:13:53.425 INFO:tasks.workunit.client.0.smithi031.stdout:3/606: creat d6/d14/d71/dad/fb9 x:0 0 0 2024-04-24T23:13:53.426 INFO:tasks.workunit.client.1.smithi040.stdout:8/352: creat d0/d16/d19/d4f/f72 x:0 0 0 2024-04-24T23:13:53.427 INFO:tasks.workunit.client.1.smithi040.stdout:9/401: chown d7/c9 1819081 1 2024-04-24T23:13:53.429 INFO:tasks.workunit.client.1.smithi040.stdout:5/363: dwrite d2/d7/d8/db/f75 [0,4194304] 0 2024-04-24T23:13:53.431 INFO:tasks.workunit.client.0.smithi031.stdout:0/725: rename da/d86/da3/dcd/ddc/lde to da/d22/d62/d9c/lf4 0 2024-04-24T23:13:53.433 INFO:tasks.workunit.client.0.smithi031.stdout:5/615: fdatasync d0/d6/f40 0 2024-04-24T23:13:53.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/808: mknod d3/d20/da7/c102 0 2024-04-24T23:13:53.443 INFO:tasks.workunit.client.0.smithi031.stdout:2/571: creat d5/d28/fca x:0 0 0 2024-04-24T23:13:53.444 INFO:tasks.workunit.client.0.smithi031.stdout:7/662: truncate d1/d2/dd/d33/d5e/d64/fca 705128 0 2024-04-24T23:13:53.450 INFO:tasks.workunit.client.0.smithi031.stdout:3/607: creat d6/d14/d1d/fba x:0 0 0 2024-04-24T23:13:53.453 INFO:tasks.workunit.client.0.smithi031.stdout:0/726: rmdir da/d22/d62/d9c 39 2024-04-24T23:13:53.457 INFO:tasks.workunit.client.0.smithi031.stdout:5/616: creat d0/d6/d3b/d4b/dc/d8b/d9b/fca x:0 0 0 2024-04-24T23:13:53.459 INFO:tasks.workunit.client.0.smithi031.stdout:4/809: creat d3/d20/d2d/d44/d5b/d5d/f103 x:0 0 0 2024-04-24T23:13:53.460 INFO:tasks.workunit.client.1.smithi040.stdout:3/389: mknod d1/d2d/d72/c8f 0 2024-04-24T23:13:53.461 INFO:tasks.workunit.client.0.smithi031.stdout:9/771: mknod d2/d16/cec 0 2024-04-24T23:13:53.461 INFO:tasks.workunit.client.1.smithi040.stdout:3/390: write d1/d9/d18/d36/d55/d62/f65 [96170,93871] 0 2024-04-24T23:13:53.462 INFO:tasks.workunit.client.1.smithi040.stdout:1/386: mkdir da/d12/d18/d2a/d8b 0 2024-04-24T23:13:53.463 INFO:tasks.workunit.client.1.smithi040.stdout:3/391: truncate d1/d9/d15/d45/f4c 747500 0 2024-04-24T23:13:53.463 INFO:tasks.workunit.client.0.smithi031.stdout:2/572: creat d5/d8/d45/fcb x:0 0 0 2024-04-24T23:13:53.465 INFO:tasks.workunit.client.1.smithi040.stdout:7/421: sync 2024-04-24T23:13:53.466 INFO:tasks.workunit.client.1.smithi040.stdout:8/353: chown d0/d16/d19/f35 5283529 1 2024-04-24T23:13:53.467 INFO:tasks.workunit.client.1.smithi040.stdout:9/402: chown d7/da/d17/d40/c45 1 1 2024-04-24T23:13:53.468 INFO:tasks.workunit.client.1.smithi040.stdout:5/364: stat d2/d7/d8/db/d1e/f2e 0 2024-04-24T23:13:53.469 INFO:tasks.workunit.client.1.smithi040.stdout:5/365: stat d2/d7/d8/d1f/d57/d3e 0 2024-04-24T23:13:53.471 INFO:tasks.workunit.client.0.smithi031.stdout:0/727: truncate da/d11/f5d 1152037 0 2024-04-24T23:13:53.472 INFO:tasks.workunit.client.0.smithi031.stdout:0/728: write da/d20/d4e/f51 [16109,62366] 0 2024-04-24T23:13:53.476 INFO:tasks.workunit.client.0.smithi031.stdout:0/729: dread da/d22/f30 [0,4194304] 0 2024-04-24T23:13:53.477 INFO:tasks.workunit.client.1.smithi040.stdout:2/444: link da/d10/f34 da/d81/fab 0 2024-04-24T23:13:53.477 INFO:tasks.workunit.client.1.smithi040.stdout:2/445: dread - da/d1c/d37/d39/f7c zero size 2024-04-24T23:13:53.478 INFO:tasks.workunit.client.1.smithi040.stdout:2/446: stat da/d10/d4d/f64 0 2024-04-24T23:13:53.478 INFO:tasks.workunit.client.0.smithi031.stdout:4/810: mkdir d3/da3/d104 0 2024-04-24T23:13:53.480 INFO:tasks.workunit.client.0.smithi031.stdout:9/772: dread d2/dae/d14/d78/d84/d4a/fb1 [0,4194304] 0 2024-04-24T23:13:53.481 INFO:tasks.workunit.client.0.smithi031.stdout:7/663: mkdir d1/de4 0 2024-04-24T23:13:53.483 INFO:tasks.workunit.client.1.smithi040.stdout:1/387: unlink da/c34 0 2024-04-24T23:13:53.483 INFO:tasks.workunit.client.0.smithi031.stdout:0/730: dwrite da/d18/d5a/d7d/fef [0,4194304] 0 2024-04-24T23:13:53.484 INFO:tasks.workunit.client.0.smithi031.stdout:2/573: symlink d5/d6/d29/d4f/lcc 0 2024-04-24T23:13:53.486 INFO:tasks.workunit.client.1.smithi040.stdout:3/392: creat d1/d2d/d72/f90 x:0 0 0 2024-04-24T23:13:53.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/754: getdents d13/d15/d1d/d23/d24/d43/d68/db8 0 2024-04-24T23:13:53.489 INFO:tasks.workunit.client.0.smithi031.stdout:6/755: fdatasync d13/d15/d1d/d23/f2b 0 2024-04-24T23:13:53.491 INFO:tasks.workunit.client.1.smithi040.stdout:6/356: rename d7/f54 to d7/d1a/d20/d51/d2f/f6f 0 2024-04-24T23:13:53.491 INFO:tasks.workunit.client.0.smithi031.stdout:6/756: stat d13/d15/d1d/d23/d2d 0 2024-04-24T23:13:53.493 INFO:tasks.workunit.client.0.smithi031.stdout:2/574: dwrite d5/d8/d15/f93 [0,4194304] 0 2024-04-24T23:13:53.496 INFO:tasks.workunit.client.0.smithi031.stdout:8/827: link d8/d1a/d1d/d35/d58/d5b/f91 d8/d1a/d1d/d50/f11e 0 2024-04-24T23:13:53.500 INFO:tasks.workunit.client.1.smithi040.stdout:5/366: mknod d2/d7/d8/d1f/d57/d3e/d48/c77 0 2024-04-24T23:13:53.504 INFO:tasks.workunit.client.0.smithi031.stdout:6/757: dread d13/d15/d1d/d23/d24/d43/d68/db8/fcc [0,4194304] 0 2024-04-24T23:13:53.505 INFO:tasks.workunit.client.1.smithi040.stdout:4/324: creat da/d3f/d6b/f79 x:0 0 0 2024-04-24T23:13:53.508 INFO:tasks.workunit.client.1.smithi040.stdout:2/447: rmdir da/d10/d4d/d6c 39 2024-04-24T23:13:53.511 INFO:tasks.workunit.client.0.smithi031.stdout:5/617: dwrite d0/d7/f58 [0,4194304] 0 2024-04-24T23:13:53.513 INFO:tasks.workunit.client.1.smithi040.stdout:1/388: write da/d12/d55/f6b [3422278,44563] 0 2024-04-24T23:13:53.514 INFO:tasks.workunit.client.0.smithi031.stdout:4/811: dread d3/f26 [4194304,4194304] 0 2024-04-24T23:13:53.515 INFO:tasks.workunit.client.0.smithi031.stdout:4/812: chown d3/d5e/dc4 3207 1 2024-04-24T23:13:53.517 INFO:tasks.workunit.client.0.smithi031.stdout:3/608: truncate f0 8301704 0 2024-04-24T23:13:53.518 INFO:tasks.workunit.client.0.smithi031.stdout:3/609: fdatasync d6/d14/d1d/fba 0 2024-04-24T23:13:53.519 INFO:tasks.workunit.client.1.smithi040.stdout:3/393: unlink d1/f59 0 2024-04-24T23:13:53.519 INFO:tasks.workunit.client.0.smithi031.stdout:5/618: dwrite d0/f93 [0,4194304] 0 2024-04-24T23:13:53.520 INFO:tasks.workunit.client.0.smithi031.stdout:7/664: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5 0 2024-04-24T23:13:53.524 INFO:tasks.workunit.client.0.smithi031.stdout:0/731: creat da/d18/d5a/d7d/ff5 x:0 0 0 2024-04-24T23:13:53.526 INFO:tasks.workunit.client.1.smithi040.stdout:6/357: mkdir d7/d1a/d20/d51/d70 0 2024-04-24T23:13:53.526 INFO:tasks.workunit.client.1.smithi040.stdout:8/354: dwrite d0/d16/d1b/d47/f48 [0,4194304] 0 2024-04-24T23:13:53.531 INFO:tasks.workunit.client.1.smithi040.stdout:5/367: mknod d2/d7/d8/d64/c78 0 2024-04-24T23:13:53.531 INFO:tasks.workunit.client.0.smithi031.stdout:8/828: readlink d8/d1a/d1d/ld4 0 2024-04-24T23:13:53.532 INFO:tasks.workunit.client.1.smithi040.stdout:3/394: dwrite d1/d2a/d6a/f8e [0,4194304] 0 2024-04-24T23:13:53.533 INFO:tasks.workunit.client.0.smithi031.stdout:8/829: dread - d8/d15/d37/ffc zero size 2024-04-24T23:13:53.534 INFO:tasks.workunit.client.1.smithi040.stdout:3/395: chown d1/d2d/d72/c8f 1 1 2024-04-24T23:13:53.538 INFO:tasks.workunit.client.1.smithi040.stdout:0/437: getdents df/d1e/d20/d21/d51/d79 0 2024-04-24T23:13:53.538 INFO:tasks.workunit.client.0.smithi031.stdout:6/758: write d13/d15/d1d/d23/d24/f9f [43157,31959] 0 2024-04-24T23:13:53.538 INFO:tasks.workunit.client.1.smithi040.stdout:8/355: dwrite d0/d16/d1b/d58/f68 [0,4194304] 0 2024-04-24T23:13:53.539 INFO:tasks.workunit.client.1.smithi040.stdout:2/448: mknod da/d1c/d37/d39/d79/cac 0 2024-04-24T23:13:53.539 INFO:tasks.workunit.client.0.smithi031.stdout:3/610: dread d6/df/d16/d2a/f2e [0,4194304] 0 2024-04-24T23:13:53.540 INFO:tasks.workunit.client.1.smithi040.stdout:2/449: chown da/d81/ca2 2160468 1 2024-04-24T23:13:53.540 INFO:tasks.workunit.client.1.smithi040.stdout:0/438: read - df/d12/f4c zero size 2024-04-24T23:13:53.541 INFO:tasks.workunit.client.0.smithi031.stdout:3/611: fsync d6/df/d16/d2a/d95/d9b/d1a/f62 0 2024-04-24T23:13:53.541 INFO:tasks.workunit.client.1.smithi040.stdout:8/356: stat d0/d16/d19/l60 0 2024-04-24T23:13:53.549 INFO:tasks.workunit.client.0.smithi031.stdout:9/773: creat d2/dae/d14/d1d/d45/d56/fed x:0 0 0 2024-04-24T23:13:53.551 INFO:tasks.workunit.client.1.smithi040.stdout:8/357: dwrite d0/d8/f4d [0,4194304] 0 2024-04-24T23:13:53.553 INFO:tasks.workunit.client.0.smithi031.stdout:5/619: symlink d0/d6/d3b/d4f/lcb 0 2024-04-24T23:13:53.557 INFO:tasks.workunit.client.0.smithi031.stdout:9/774: dwrite d2/dae/d14/f19 [0,4194304] 0 2024-04-24T23:13:53.559 INFO:tasks.workunit.client.0.smithi031.stdout:9/775: chown d2/dae/d14/d78/d84/f7a 455 1 2024-04-24T23:13:53.560 INFO:tasks.workunit.client.1.smithi040.stdout:9/403: creat d7/da/d17/d40/d55/d2f/d1f/d31/f8d x:0 0 0 2024-04-24T23:13:53.565 INFO:tasks.workunit.client.1.smithi040.stdout:5/368: mknod d2/d7/d8/d64/d6b/c79 0 2024-04-24T23:13:53.571 INFO:tasks.workunit.client.0.smithi031.stdout:8/830: symlink d8/d15/d37/d43/d7b/dbf/l11f 0 2024-04-24T23:13:53.574 INFO:tasks.workunit.client.0.smithi031.stdout:6/759: truncate d13/d15/d1d/d23/d44/f5b 816388 0 2024-04-24T23:13:53.574 INFO:tasks.workunit.client.0.smithi031.stdout:1/620: sync 2024-04-24T23:13:53.576 INFO:tasks.workunit.client.1.smithi040.stdout:0/439: mkdir df/d12/d58/d68/d8e 0 2024-04-24T23:13:53.579 INFO:tasks.workunit.client.0.smithi031.stdout:8/831: dwrite d8/d15/d37/f10b [0,4194304] 0 2024-04-24T23:13:53.579 INFO:tasks.workunit.client.1.smithi040.stdout:0/440: dread df/f34 [0,4194304] 0 2024-04-24T23:13:53.580 INFO:tasks.workunit.client.0.smithi031.stdout:3/612: fsync d6/f6b 0 2024-04-24T23:13:53.580 INFO:tasks.workunit.client.0.smithi031.stdout:7/665: write d1/d2/dd/f7d [4697814,127627] 0 2024-04-24T23:13:53.583 INFO:tasks.workunit.client.0.smithi031.stdout:7/666: write d1/d2/d60/fd4 [652900,55036] 0 2024-04-24T23:13:53.583 INFO:tasks.workunit.client.1.smithi040.stdout:1/389: mkdir da/d12/d18/d2a/d41/d8c 0 2024-04-24T23:13:53.583 INFO:tasks.workunit.client.0.smithi031.stdout:0/732: write da/fd [97815,112286] 0 2024-04-24T23:13:53.585 INFO:tasks.workunit.client.0.smithi031.stdout:4/813: getdents d3/d23/d69/dce 0 2024-04-24T23:13:53.587 INFO:tasks.workunit.client.0.smithi031.stdout:5/620: stat d0/d7/d56/f61 0 2024-04-24T23:13:53.588 INFO:tasks.workunit.client.1.smithi040.stdout:8/358: rmdir d0/d38 39 2024-04-24T23:13:53.591 INFO:tasks.workunit.client.0.smithi031.stdout:0/733: dwrite da/fd [8388608,4194304] 0 2024-04-24T23:13:53.591 INFO:tasks.workunit.client.1.smithi040.stdout:3/396: write d1/d9/d18/f19 [567249,102660] 0 2024-04-24T23:13:53.592 INFO:tasks.workunit.client.0.smithi031.stdout:0/734: dread - da/d11/fd3 zero size 2024-04-24T23:13:53.592 INFO:tasks.workunit.client.1.smithi040.stdout:7/422: getdents db/d12/d73 0 2024-04-24T23:13:53.593 INFO:tasks.workunit.client.1.smithi040.stdout:7/423: read - db/d2f/f6d zero size 2024-04-24T23:13:53.594 INFO:tasks.workunit.client.1.smithi040.stdout:7/424: truncate db/d12/f8c 662082 0 2024-04-24T23:13:53.597 INFO:tasks.workunit.client.0.smithi031.stdout:0/735: dread da/d18/f2f [0,4194304] 0 2024-04-24T23:13:53.600 INFO:tasks.workunit.client.1.smithi040.stdout:4/325: rmdir da/d3f/d40/d63 0 2024-04-24T23:13:53.605 INFO:tasks.workunit.client.1.smithi040.stdout:2/450: rename da/d1c/d37/f41 to da/d10/d4d/d6c/fad 0 2024-04-24T23:13:53.609 INFO:tasks.workunit.client.1.smithi040.stdout:3/397: dread d1/d2d/d44/f6c [0,4194304] 0 2024-04-24T23:13:53.609 INFO:tasks.workunit.client.1.smithi040.stdout:8/359: creat d0/d8/d22/f73 x:0 0 0 2024-04-24T23:13:53.609 INFO:tasks.workunit.client.1.smithi040.stdout:9/404: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f48 [0,4194304] 0 2024-04-24T23:13:53.610 INFO:tasks.workunit.client.0.smithi031.stdout:3/613: mkdir d6/d14/d91/dbb 0 2024-04-24T23:13:53.610 INFO:tasks.workunit.client.1.smithi040.stdout:3/398: fsync d1/d2d/de/d30/f41 0 2024-04-24T23:13:53.611 INFO:tasks.workunit.client.1.smithi040.stdout:6/358: creat d7/f71 x:0 0 0 2024-04-24T23:13:53.613 INFO:tasks.workunit.client.0.smithi031.stdout:7/667: symlink d1/d2/dd/d33/d5e/d64/le6 0 2024-04-24T23:13:53.614 INFO:tasks.workunit.client.1.smithi040.stdout:7/425: mknod db/d12/d20/d52/d4b/c9a 0 2024-04-24T23:13:53.617 INFO:tasks.workunit.client.0.smithi031.stdout:5/621: mknod d0/d6/d3b/d4b/dc/d25/ccc 0 2024-04-24T23:13:53.617 INFO:tasks.workunit.client.0.smithi031.stdout:5/622: chown d0/cb9 37 1 2024-04-24T23:13:53.619 INFO:tasks.workunit.client.1.smithi040.stdout:8/360: dwrite d0/d8/dc/f63 [0,4194304] 0 2024-04-24T23:13:53.621 INFO:tasks.workunit.client.1.smithi040.stdout:5/369: dread d2/d7/d8/d1f/d57/d27/d65/f5a [0,4194304] 0 2024-04-24T23:13:53.622 INFO:tasks.workunit.client.0.smithi031.stdout:9/776: creat d2/dae/d14/d78/d84/d4a/dab/db8/fee x:0 0 0 2024-04-24T23:13:53.622 INFO:tasks.workunit.client.1.smithi040.stdout:5/370: dread - d2/d7/d8/db/d1e/f55 zero size 2024-04-24T23:13:53.624 INFO:tasks.workunit.client.0.smithi031.stdout:2/575: getdents d5/d6/d29/d26/db0/dc2/dad 0 2024-04-24T23:13:53.625 INFO:tasks.workunit.client.0.smithi031.stdout:8/832: dwrite d8/d1a/f2f [0,4194304] 0 2024-04-24T23:13:53.625 INFO:tasks.workunit.client.1.smithi040.stdout:1/390: stat da/d12/d18/c2b 0 2024-04-24T23:13:53.626 INFO:tasks.workunit.client.1.smithi040.stdout:1/391: chown da/f7e 39692037 1 2024-04-24T23:13:53.628 INFO:tasks.workunit.client.0.smithi031.stdout:6/760: creat d13/d15/d1d/d23/d44/da0/da6/de6/d102/f103 x:0 0 0 2024-04-24T23:13:53.629 INFO:tasks.workunit.client.0.smithi031.stdout:2/576: chown d5/d8/d45/fcb 1 1 2024-04-24T23:13:53.629 INFO:tasks.workunit.client.0.smithi031.stdout:6/761: fdatasync d13/d15/d1d/d23/d4d/f88 0 2024-04-24T23:13:53.630 INFO:tasks.workunit.client.1.smithi040.stdout:2/451: mknod da/d1c/d37/d39/d77/cae 0 2024-04-24T23:13:53.630 INFO:tasks.workunit.client.0.smithi031.stdout:2/577: stat d5/cbf 0 2024-04-24T23:13:53.631 INFO:tasks.workunit.client.0.smithi031.stdout:2/578: fsync d5/d6/d29/d26/fa8 0 2024-04-24T23:13:53.632 INFO:tasks.workunit.client.0.smithi031.stdout:2/579: write d5/d28/d4e/fa1 [134466,70264] 0 2024-04-24T23:13:53.636 INFO:tasks.workunit.client.0.smithi031.stdout:3/614: fsync d6/df/d16/d2a/f68 0 2024-04-24T23:13:53.638 INFO:tasks.workunit.client.1.smithi040.stdout:6/359: creat d7/d1a/d20/d51/d1e/f72 x:0 0 0 2024-04-24T23:13:53.639 INFO:tasks.workunit.client.0.smithi031.stdout:4/814: mkdir d3/da3/d104/d105 0 2024-04-24T23:13:53.641 INFO:tasks.workunit.client.0.smithi031.stdout:7/668: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/fe7 x:0 0 0 2024-04-24T23:13:53.643 INFO:tasks.workunit.client.0.smithi031.stdout:5/623: creat d0/d6/d3b/d4b/d1d/d2e/dc6/fcd x:0 0 0 2024-04-24T23:13:53.645 INFO:tasks.workunit.client.0.smithi031.stdout:0/736: link da/d18/d5a/d7d/dbd/fd1 da/d18/ff6 0 2024-04-24T23:13:53.646 INFO:tasks.workunit.client.0.smithi031.stdout:0/737: chown da/d22/la6 28929 1 2024-04-24T23:13:53.648 INFO:tasks.workunit.client.0.smithi031.stdout:0/738: write da/d11/d6a/fc5 [378777,51973] 0 2024-04-24T23:13:53.648 INFO:tasks.workunit.client.0.smithi031.stdout:9/777: read d2/dae/d14/d78/d84/d4a/d4b/f52 [2110127,58234] 0 2024-04-24T23:13:53.649 INFO:tasks.workunit.client.0.smithi031.stdout:1/621: creat d1/dc/d11/fc2 x:0 0 0 2024-04-24T23:13:53.649 INFO:tasks.workunit.client.1.smithi040.stdout:7/426: dread db/d12/d3f/f41 [0,4194304] 0 2024-04-24T23:13:53.651 INFO:tasks.workunit.client.1.smithi040.stdout:7/427: fdatasync db/d2f/d37/d7e/f7f 0 2024-04-24T23:13:53.652 INFO:tasks.workunit.client.0.smithi031.stdout:8/833: read d8/d15/d37/f95 [492582,128571] 0 2024-04-24T23:13:53.653 INFO:tasks.workunit.client.0.smithi031.stdout:8/834: truncate d8/d1a/f10c 69745 0 2024-04-24T23:13:53.653 INFO:tasks.workunit.client.1.smithi040.stdout:0/441: rmdir df/d14/d8d 0 2024-04-24T23:13:53.656 INFO:tasks.workunit.client.1.smithi040.stdout:1/392: read da/d12/d18/f1d [1202321,125964] 0 2024-04-24T23:13:53.657 INFO:tasks.workunit.client.1.smithi040.stdout:1/393: dread da/d12/d23/f45 [0,4194304] 0 2024-04-24T23:13:53.658 INFO:tasks.workunit.client.1.smithi040.stdout:2/452: fdatasync da/d1c/d37/d39/f4e 0 2024-04-24T23:13:53.661 INFO:tasks.workunit.client.1.smithi040.stdout:9/405: mknod d7/da/d17/d40/d55/d61/d6c/d8c/c8e 0 2024-04-24T23:13:53.662 INFO:tasks.workunit.client.0.smithi031.stdout:3/615: readlink d6/df/d16/l38 0 2024-04-24T23:13:53.664 INFO:tasks.workunit.client.0.smithi031.stdout:4/815: write d3/d20/d25/d7e/faa [4783261,129012] 0 2024-04-24T23:13:53.665 INFO:tasks.workunit.client.1.smithi040.stdout:4/326: creat da/d3f/d6b/f7a x:0 0 0 2024-04-24T23:13:53.665 INFO:tasks.workunit.client.1.smithi040.stdout:6/360: dwrite d7/f3f [0,4194304] 0 2024-04-24T23:13:53.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/669: rename d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/fd1 to d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/fe8 0 2024-04-24T23:13:53.668 INFO:tasks.workunit.client.1.smithi040.stdout:6/361: read d7/d1a/d20/f3a [339122,86511] 0 2024-04-24T23:13:53.669 INFO:tasks.workunit.client.0.smithi031.stdout:7/670: truncate d1/d2/dd/d33/d5e/d64/d6d/de1/d84/fcd 520040 0 2024-04-24T23:13:53.669 INFO:tasks.workunit.client.1.smithi040.stdout:8/361: dwrite d0/d16/d19/f37 [0,4194304] 0 2024-04-24T23:13:53.670 INFO:tasks.workunit.client.1.smithi040.stdout:5/371: fdatasync d2/d2f/d14/f17 0 2024-04-24T23:13:53.671 INFO:tasks.workunit.client.1.smithi040.stdout:8/362: read - d0/d8/d55/f64 zero size 2024-04-24T23:13:53.671 INFO:tasks.workunit.client.0.smithi031.stdout:9/778: creat d2/d43/d9d/fef x:0 0 0 2024-04-24T23:13:53.675 INFO:tasks.workunit.client.1.smithi040.stdout:0/442: creat df/d1e/d20/d21/d51/f8f x:0 0 0 2024-04-24T23:13:53.682 INFO:tasks.workunit.client.0.smithi031.stdout:4/816: chown d3/d20/d2d/d44/cb8 10534 1 2024-04-24T23:13:53.682 INFO:tasks.workunit.client.1.smithi040.stdout:1/394: symlink da/d12/d14/l8d 0 2024-04-24T23:13:53.683 INFO:tasks.workunit.client.1.smithi040.stdout:1/395: truncate da/f7e 1115430 0 2024-04-24T23:13:53.687 INFO:tasks.workunit.client.0.smithi031.stdout:3/616: dwrite d6/df/d16/d2a/f5a [0,4194304] 0 2024-04-24T23:13:53.689 INFO:tasks.workunit.client.0.smithi031.stdout:3/617: write d6/f7b [1270856,21944] 0 2024-04-24T23:13:53.689 INFO:tasks.workunit.client.0.smithi031.stdout:7/671: truncate d1/d2/dd/f74 1609932 0 2024-04-24T23:13:53.691 INFO:tasks.workunit.client.0.smithi031.stdout:4/817: dwrite d3/f1b [0,4194304] 0 2024-04-24T23:13:53.692 INFO:tasks.workunit.client.0.smithi031.stdout:1/622: mknod d1/da/d12/cc3 0 2024-04-24T23:13:53.693 INFO:tasks.workunit.client.0.smithi031.stdout:2/580: write d5/d6/fc1 [241521,112499] 0 2024-04-24T23:13:53.698 INFO:tasks.workunit.client.1.smithi040.stdout:4/327: symlink da/d3f/l7b 0 2024-04-24T23:13:53.701 INFO:tasks.workunit.client.1.smithi040.stdout:3/399: sync 2024-04-24T23:13:53.704 INFO:tasks.workunit.client.0.smithi031.stdout:9/779: dread d2/dae/d14/d78/d84/d4a/f8d [0,4194304] 0 2024-04-24T23:13:53.704 INFO:tasks.workunit.client.0.smithi031.stdout:4/818: dwrite d3/d20/d2d/d44/fd3 [0,4194304] 0 2024-04-24T23:13:53.709 INFO:tasks.workunit.client.1.smithi040.stdout:8/363: unlink d0/d16/d1b/d47/f48 0 2024-04-24T23:13:53.713 INFO:tasks.workunit.client.0.smithi031.stdout:1/623: mkdir d1/da/d12/d7e/dc4 0 2024-04-24T23:13:53.714 INFO:tasks.workunit.client.0.smithi031.stdout:1/624: chown d1/da/d10/f4d 488 1 2024-04-24T23:13:53.717 INFO:tasks.workunit.client.0.smithi031.stdout:7/672: rename d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/f80 to d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/fe9 0 2024-04-24T23:13:53.719 INFO:tasks.workunit.client.1.smithi040.stdout:1/396: write da/d35/d38/f3e [3421926,46632] 0 2024-04-24T23:13:53.720 INFO:tasks.workunit.client.0.smithi031.stdout:4/819: symlink d3/d5e/dc2/l106 0 2024-04-24T23:13:53.720 INFO:tasks.workunit.client.1.smithi040.stdout:2/453: getdents da/d81/da8 0 2024-04-24T23:13:53.725 INFO:tasks.workunit.client.1.smithi040.stdout:1/397: dwrite da/d12/d18/d2a/f79 [0,4194304] 0 2024-04-24T23:13:53.726 INFO:tasks.workunit.client.1.smithi040.stdout:4/328: fsync da/d3f/d6b/d2b/f31 0 2024-04-24T23:13:53.727 INFO:tasks.workunit.client.1.smithi040.stdout:3/400: symlink d1/d2a/d6a/l91 0 2024-04-24T23:13:53.728 INFO:tasks.workunit.client.1.smithi040.stdout:6/362: mknod d7/de/c73 0 2024-04-24T23:13:53.733 INFO:tasks.workunit.client.1.smithi040.stdout:6/363: dread d7/d1a/d20/d51/d1e/f22 [0,4194304] 0 2024-04-24T23:13:53.734 INFO:tasks.workunit.client.1.smithi040.stdout:6/364: chown d7/d1a/d20/d51/d1e/f49 318651 1 2024-04-24T23:13:53.735 INFO:tasks.workunit.client.1.smithi040.stdout:7/428: dwrite db/f24 [0,4194304] 0 2024-04-24T23:13:53.736 INFO:tasks.workunit.client.1.smithi040.stdout:7/429: chown db/d2f/l58 5677 1 2024-04-24T23:13:53.738 INFO:tasks.workunit.client.1.smithi040.stdout:9/406: write d7/da/d17/d40/d55/d2f/d1f/d31/f36 [2116016,79826] 0 2024-04-24T23:13:53.744 INFO:tasks.workunit.client.1.smithi040.stdout:1/398: rmdir da/d35 39 2024-04-24T23:13:53.745 INFO:tasks.workunit.client.1.smithi040.stdout:3/401: symlink d1/d9/d15/l92 0 2024-04-24T23:13:53.748 INFO:tasks.workunit.client.1.smithi040.stdout:6/365: unlink d7/d1a/d20/d51/d1e/d5d/d52/f5a 0 2024-04-24T23:13:53.749 INFO:tasks.workunit.client.1.smithi040.stdout:7/430: rmdir db/d2f 39 2024-04-24T23:13:53.750 INFO:tasks.workunit.client.1.smithi040.stdout:8/364: creat d0/d16/d19/d6f/f74 x:0 0 0 2024-04-24T23:13:53.751 INFO:tasks.workunit.client.1.smithi040.stdout:9/407: creat d7/da/d17/d40/d55/d61/f8f x:0 0 0 2024-04-24T23:13:53.753 INFO:tasks.workunit.client.1.smithi040.stdout:1/399: fsync da/d35/d38/d47/f80 0 2024-04-24T23:13:53.753 INFO:tasks.workunit.client.1.smithi040.stdout:4/329: mkdir da/d3f/d40/d47/d75/d7c 0 2024-04-24T23:13:53.754 INFO:tasks.workunit.client.1.smithi040.stdout:4/330: truncate da/d3f/d40/d41/d62/f6f 297056 0 2024-04-24T23:13:53.755 INFO:tasks.workunit.client.1.smithi040.stdout:3/402: rename d1/d2a/d6a/l6e to d1/d2d/de/d30/l93 0 2024-04-24T23:13:53.755 INFO:tasks.workunit.client.1.smithi040.stdout:5/372: link d2/d7/d8/d1f/f51 d2/d7/d8/d1f/d57/d27/d3d/f7a 0 2024-04-24T23:13:53.756 INFO:tasks.workunit.client.1.smithi040.stdout:6/366: mknod d7/d1a/d20/d51/d2a/c74 0 2024-04-24T23:13:53.757 INFO:tasks.workunit.client.1.smithi040.stdout:7/431: mknod db/d12/d62/d8a/c9b 0 2024-04-24T23:13:53.757 INFO:tasks.workunit.client.1.smithi040.stdout:6/367: read d7/d1a/d20/d51/f26 [2495297,97151] 0 2024-04-24T23:13:53.757 INFO:tasks.workunit.client.1.smithi040.stdout:8/365: symlink d0/d8/dc/l75 0 2024-04-24T23:13:53.758 INFO:tasks.workunit.client.1.smithi040.stdout:9/408: symlink d7/d5e/l90 0 2024-04-24T23:13:53.758 INFO:tasks.workunit.client.1.smithi040.stdout:0/443: link cd df/d12/d1a/d71/c90 0 2024-04-24T23:13:53.760 INFO:tasks.workunit.client.1.smithi040.stdout:8/366: write d0/d8/d55/f64 [277322,19866] 0 2024-04-24T23:13:53.760 INFO:tasks.workunit.client.1.smithi040.stdout:1/400: fsync da/d31/f66 0 2024-04-24T23:13:53.761 INFO:tasks.workunit.client.1.smithi040.stdout:3/403: mkdir d1/d2a/d6a/d94 0 2024-04-24T23:13:53.761 INFO:tasks.workunit.client.1.smithi040.stdout:5/373: symlink d2/d7/d8/d6d/l7b 0 2024-04-24T23:13:53.762 INFO:tasks.workunit.client.1.smithi040.stdout:4/331: dread da/d3f/d40/d41/d60/f23 [0,4194304] 0 2024-04-24T23:13:53.763 INFO:tasks.workunit.client.1.smithi040.stdout:2/454: getdents da/d1c/d37/d39/d3f/d52 0 2024-04-24T23:13:53.763 INFO:tasks.workunit.client.1.smithi040.stdout:0/444: mknod df/d12/d58/c91 0 2024-04-24T23:13:53.764 INFO:tasks.workunit.client.1.smithi040.stdout:7/432: write db/d12/d3f/f74 [94358,63674] 0 2024-04-24T23:13:53.767 INFO:tasks.workunit.client.1.smithi040.stdout:8/367: mkdir d0/d16/d19/d71/d76 0 2024-04-24T23:13:53.767 INFO:tasks.workunit.client.1.smithi040.stdout:8/368: fsync d0/d16/d19/d4f/f70 0 2024-04-24T23:13:53.768 INFO:tasks.workunit.client.1.smithi040.stdout:7/433: unlink db/d2f/f30 0 2024-04-24T23:13:53.769 INFO:tasks.workunit.client.1.smithi040.stdout:4/332: rename da/d3f/d40/d41/f43 to da/d45/d48/f7d 0 2024-04-24T23:13:53.770 INFO:tasks.workunit.client.1.smithi040.stdout:8/369: mkdir d0/d16/d19/d4f/d77 0 2024-04-24T23:13:53.772 INFO:tasks.workunit.client.1.smithi040.stdout:8/370: creat d0/d8/d66/f78 x:0 0 0 2024-04-24T23:13:53.773 INFO:tasks.workunit.client.1.smithi040.stdout:0/445: write df/d1e/f60 [651702,62641] 0 2024-04-24T23:13:53.774 INFO:tasks.workunit.client.1.smithi040.stdout:5/374: truncate d2/d7/d8/d1f/d57/f70 353800 0 2024-04-24T23:13:53.774 INFO:tasks.workunit.client.1.smithi040.stdout:9/409: truncate d7/da/d17/d40/d55/d2f/d1f/f8a 886140 0 2024-04-24T23:13:53.774 INFO:tasks.workunit.client.1.smithi040.stdout:7/434: stat db/d12/d20/d52/f70 0 2024-04-24T23:13:53.775 INFO:tasks.workunit.client.1.smithi040.stdout:6/368: truncate d7/d1a/d20/d51/d2f/f5b 831341 0 2024-04-24T23:13:53.777 INFO:tasks.workunit.client.0.smithi031.stdout:9/780: rename d2/d16/l38 to d2/d16/dbe/d7d/lf0 0 2024-04-24T23:13:53.777 INFO:tasks.workunit.client.0.smithi031.stdout:4/820: dwrite d3/d20/f51 [0,4194304] 0 2024-04-24T23:13:53.777 INFO:tasks.workunit.client.0.smithi031.stdout:7/673: unlink d1/d2/dd/d33/l72 0 2024-04-24T23:13:53.780 INFO:tasks.workunit.client.0.smithi031.stdout:0/739: getdents da/d22/d62/d9c 0 2024-04-24T23:13:53.780 INFO:tasks.workunit.client.0.smithi031.stdout:3/618: truncate d6/df/d16/fa6 1159037 0 2024-04-24T23:13:53.781 INFO:tasks.workunit.client.1.smithi040.stdout:8/371: symlink d0/d16/d1b/d58/l79 0 2024-04-24T23:13:53.781 INFO:tasks.workunit.client.0.smithi031.stdout:1/625: write d1/d3e/da3/d69/f95 [7617,54668] 0 2024-04-24T23:13:53.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/781: read d2/dae/d14/d78/d84/d4a/f8d [301865,123968] 0 2024-04-24T23:13:53.783 INFO:tasks.workunit.client.1.smithi040.stdout:5/375: rmdir d2/d7/d8/d64/d6b 39 2024-04-24T23:13:53.784 INFO:tasks.workunit.client.0.smithi031.stdout:2/581: creat d5/d8/fcd x:0 0 0 2024-04-24T23:13:53.784 INFO:tasks.workunit.client.0.smithi031.stdout:4/821: mknod d3/da3/c107 0 2024-04-24T23:13:53.787 INFO:tasks.workunit.client.1.smithi040.stdout:1/401: link da/d12/d55/f6b da/d12/d55/f8e 0 2024-04-24T23:13:53.788 INFO:tasks.workunit.client.0.smithi031.stdout:0/740: creat da/d11/d6a/dba/ff7 x:0 0 0 2024-04-24T23:13:53.790 INFO:tasks.workunit.client.1.smithi040.stdout:2/455: rename da/d1c/c59 to da/d1c/d37/d39/d79/caf 0 2024-04-24T23:13:53.793 INFO:tasks.workunit.client.0.smithi031.stdout:7/674: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/fe7 [0,4194304] 0 2024-04-24T23:13:53.799 INFO:tasks.workunit.client.1.smithi040.stdout:4/333: dread da/d3f/d40/d41/d60/f42 [0,4194304] 0 2024-04-24T23:13:53.800 INFO:tasks.workunit.client.0.smithi031.stdout:2/582: symlink d5/d8/d45/lce 0 2024-04-24T23:13:53.801 INFO:tasks.workunit.client.1.smithi040.stdout:4/334: truncate da/d3f/d6b/f7a 727009 0 2024-04-24T23:13:53.803 INFO:tasks.workunit.client.0.smithi031.stdout:0/741: mkdir da/d20/d4d/df8 0 2024-04-24T23:13:53.808 INFO:tasks.workunit.client.1.smithi040.stdout:2/456: symlink da/d10/lb0 0 2024-04-24T23:13:53.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/619: getdents d6/d14/db2 0 2024-04-24T23:13:53.812 INFO:tasks.workunit.client.0.smithi031.stdout:9/782: symlink d2/dae/lf1 0 2024-04-24T23:13:53.814 INFO:tasks.workunit.client.0.smithi031.stdout:7/675: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea 0 2024-04-24T23:13:53.814 INFO:tasks.workunit.client.0.smithi031.stdout:9/783: chown d2/dae/d14/d1d/d45/d65/l80 112399 1 2024-04-24T23:13:53.817 INFO:tasks.workunit.client.0.smithi031.stdout:3/620: dwrite d6/df/d16/d2a/d95/d9b/d1a/f80 [0,4194304] 0 2024-04-24T23:13:53.817 INFO:tasks.workunit.client.1.smithi040.stdout:5/376: dread d2/d7/f74 [0,4194304] 0 2024-04-24T23:13:53.820 INFO:tasks.workunit.client.0.smithi031.stdout:2/583: symlink d5/d28/d4e/d5e/lcf 0 2024-04-24T23:13:53.821 INFO:tasks.workunit.client.1.smithi040.stdout:9/410: dread d7/f3a [0,4194304] 0 2024-04-24T23:13:53.822 INFO:tasks.workunit.client.1.smithi040.stdout:4/335: rmdir da/d3f/d6b 39 2024-04-24T23:13:53.823 INFO:tasks.workunit.client.0.smithi031.stdout:0/742: mknod da/d11/d6a/da2/cf9 0 2024-04-24T23:13:53.825 INFO:tasks.workunit.client.1.smithi040.stdout:7/435: dread db/d12/f29 [0,4194304] 0 2024-04-24T23:13:53.826 INFO:tasks.workunit.client.1.smithi040.stdout:7/436: fdatasync db/d12/d20/d52/f70 0 2024-04-24T23:13:53.828 INFO:tasks.workunit.client.1.smithi040.stdout:8/372: creat d0/d38/d4c/d5d/f7a x:0 0 0 2024-04-24T23:13:53.828 INFO:tasks.workunit.client.1.smithi040.stdout:8/373: dread - d0/d8/d66/f78 zero size 2024-04-24T23:13:53.829 INFO:tasks.workunit.client.0.smithi031.stdout:9/784: fsync d2/d43/f70 0 2024-04-24T23:13:53.829 INFO:tasks.workunit.client.1.smithi040.stdout:8/374: dread - d0/d16/d19/d4f/f70 zero size 2024-04-24T23:13:53.831 INFO:tasks.workunit.client.1.smithi040.stdout:2/457: rename da/d1c/d37/d39/d3f/l65 to da/d10/d4d/d6c/lb1 0 2024-04-24T23:13:53.832 INFO:tasks.workunit.client.1.smithi040.stdout:2/458: write da/d10/d7a/f99 [26074,105388] 0 2024-04-24T23:13:53.833 INFO:tasks.workunit.client.1.smithi040.stdout:2/459: fdatasync da/d1c/d37/d39/f7c 0 2024-04-24T23:13:53.836 INFO:tasks.workunit.client.0.smithi031.stdout:2/584: creat d5/d6/d29/d26/db0/dc2/fd0 x:0 0 0 2024-04-24T23:13:53.836 INFO:tasks.workunit.client.1.smithi040.stdout:6/369: link d7/l4a d7/d1a/d20/d51/d70/l75 0 2024-04-24T23:13:53.837 INFO:tasks.workunit.client.1.smithi040.stdout:3/404: write d1/d9/f20 [1215050,6752] 0 2024-04-24T23:13:53.838 INFO:tasks.workunit.client.1.smithi040.stdout:3/405: chown d1/d2a/d6a/c69 1381 1 2024-04-24T23:13:53.838 INFO:tasks.workunit.client.0.smithi031.stdout:0/743: creat da/d18/d5a/d7d/ffa x:0 0 0 2024-04-24T23:13:53.839 INFO:tasks.workunit.client.0.smithi031.stdout:9/785: dwrite d2/dae/d14/d78/d84/da6/fa7 [0,4194304] 0 2024-04-24T23:13:53.839 INFO:tasks.workunit.client.1.smithi040.stdout:5/377: rmdir d2/d7/d8/d1f/d57/d27 39 2024-04-24T23:13:53.841 INFO:tasks.workunit.client.0.smithi031.stdout:3/621: dwrite d6/df/d16/d2a/d95/d9b/d1a/d25/faf [0,4194304] 0 2024-04-24T23:13:53.845 INFO:tasks.workunit.client.1.smithi040.stdout:9/411: creat d7/da/d17/d40/d55/d61/d6c/f91 x:0 0 0 2024-04-24T23:13:53.846 INFO:tasks.workunit.client.1.smithi040.stdout:9/412: stat d7/da/d17/d40/d55/d61/d6c/f7a 0 2024-04-24T23:13:53.847 INFO:tasks.workunit.client.1.smithi040.stdout:9/413: chown d7/da/d17/c5f 2682745 1 2024-04-24T23:13:53.849 INFO:tasks.workunit.client.1.smithi040.stdout:4/336: read da/d3f/d6b/f21 [560094,93339] 0 2024-04-24T23:13:53.850 INFO:tasks.workunit.client.0.smithi031.stdout:2/585: rename d5/d6/f1a to d5/d99/da2/fd1 0 2024-04-24T23:13:53.850 INFO:tasks.workunit.client.1.smithi040.stdout:4/337: chown da/d3f/d40/d47/d71 3689 1 2024-04-24T23:13:53.852 INFO:tasks.workunit.client.1.smithi040.stdout:7/437: creat db/d12/d73/f9c x:0 0 0 2024-04-24T23:13:53.853 INFO:tasks.workunit.client.0.smithi031.stdout:9/786: rmdir d2/dae/d14/d78/d84/da6 39 2024-04-24T23:13:53.854 INFO:tasks.workunit.client.0.smithi031.stdout:9/787: readlink d2/dae/d14/d1d/d45/d65/l80 0 2024-04-24T23:13:53.858 INFO:tasks.workunit.client.1.smithi040.stdout:2/460: mkdir da/d1c/d37/d39/d3f/db2 0 2024-04-24T23:13:53.859 INFO:tasks.workunit.client.0.smithi031.stdout:0/744: fdatasync da/d18/fa1 0 2024-04-24T23:13:53.861 INFO:tasks.workunit.client.1.smithi040.stdout:6/370: symlink d7/d1a/l76 0 2024-04-24T23:13:53.861 INFO:tasks.workunit.client.0.smithi031.stdout:4/822: write d3/d5e/dc4/ff7 [4686767,61628] 0 2024-04-24T23:13:53.862 INFO:tasks.workunit.client.1.smithi040.stdout:6/371: dread - d7/d1a/f67 zero size 2024-04-24T23:13:53.865 INFO:tasks.workunit.client.1.smithi040.stdout:5/378: mknod d2/d7/d8/d6d/c7c 0 2024-04-24T23:13:53.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/586: rename d5/d8/d45/d5b/d49/f6d to d5/d99/da2/fd2 0 2024-04-24T23:13:53.871 INFO:tasks.workunit.client.0.smithi031.stdout:3/622: symlink d6/df/d16/d2a/d95/d9b/d1a/lbc 0 2024-04-24T23:13:53.872 INFO:tasks.workunit.client.1.smithi040.stdout:1/402: getdents da/d35/d38 0 2024-04-24T23:13:53.874 INFO:tasks.workunit.client.1.smithi040.stdout:7/438: creat db/d2f/f9d x:0 0 0 2024-04-24T23:13:53.878 INFO:tasks.workunit.client.1.smithi040.stdout:1/403: dwrite da/d35/d38/f6e [0,4194304] 0 2024-04-24T23:13:53.879 INFO:tasks.workunit.client.0.smithi031.stdout:4/823: truncate d3/da3/faf 682093 0 2024-04-24T23:13:53.881 INFO:tasks.workunit.client.1.smithi040.stdout:3/406: creat d1/d2a/d6a/d94/f95 x:0 0 0 2024-04-24T23:13:53.881 INFO:tasks.workunit.client.1.smithi040.stdout:3/407: chown d1/d9/d39/f78 66882800 1 2024-04-24T23:13:53.882 INFO:tasks.workunit.client.1.smithi040.stdout:3/408: chown d1/d2d/cc 295373 1 2024-04-24T23:13:53.882 INFO:tasks.workunit.client.1.smithi040.stdout:6/372: mknod d7/d1a/d20/d51/d1e/c77 0 2024-04-24T23:13:53.883 INFO:tasks.workunit.client.0.smithi031.stdout:4/824: dread d3/d20/d2d/d44/f2c [4194304,4194304] 0 2024-04-24T23:13:53.884 INFO:tasks.workunit.client.0.smithi031.stdout:2/587: symlink d5/d99/ld3 0 2024-04-24T23:13:53.887 INFO:tasks.workunit.client.0.smithi031.stdout:9/788: truncate d2/f5 4129096 0 2024-04-24T23:13:53.889 INFO:tasks.workunit.client.0.smithi031.stdout:3/623: fdatasync d6/df/d16/d2a/d95/d9b/d29/f79 0 2024-04-24T23:13:53.890 INFO:tasks.workunit.client.0.smithi031.stdout:3/624: chown d6/f6b 10 1 2024-04-24T23:13:53.896 INFO:tasks.workunit.client.1.smithi040.stdout:2/461: creat da/d1c/d37/d39/d79/da3/fb3 x:0 0 0 2024-04-24T23:13:53.896 INFO:tasks.workunit.client.1.smithi040.stdout:2/462: dread - da/d1c/d37/d8a/f90 zero size 2024-04-24T23:13:53.897 INFO:tasks.workunit.client.0.smithi031.stdout:0/745: write da/d20/d4e/f57 [346919,122748] 0 2024-04-24T23:13:53.898 INFO:tasks.workunit.client.1.smithi040.stdout:8/375: write d0/d8/dc/f63 [3312714,123017] 0 2024-04-24T23:13:53.898 INFO:tasks.workunit.client.0.smithi031.stdout:9/789: dread - d2/dae/d14/d78/d84/f9a zero size 2024-04-24T23:13:53.900 INFO:tasks.workunit.client.1.smithi040.stdout:8/376: truncate d0/d8/dc/d34/f3c 1761738 0 2024-04-24T23:13:53.900 INFO:tasks.workunit.client.1.smithi040.stdout:3/409: mkdir d1/d9/d15/d45/d96 0 2024-04-24T23:13:53.900 INFO:tasks.workunit.client.1.smithi040.stdout:8/377: chown d0/d16/d19/d4f/d77 5 1 2024-04-24T23:13:53.901 INFO:tasks.workunit.client.1.smithi040.stdout:3/410: read d1/d2a/d6a/f8e [3338102,117187] 0 2024-04-24T23:13:53.904 INFO:tasks.workunit.client.1.smithi040.stdout:6/373: chown d7/d1a/d20/d51/d2f/c62 16021899 1 2024-04-24T23:13:53.905 INFO:tasks.workunit.client.0.smithi031.stdout:3/625: read d6/df/d16/d57/f65 [144409,39501] 0 2024-04-24T23:13:53.905 INFO:tasks.workunit.client.1.smithi040.stdout:5/379: symlink d2/d7/d8/d64/l7d 0 2024-04-24T23:13:53.907 INFO:tasks.workunit.client.1.smithi040.stdout:0/446: sync 2024-04-24T23:13:53.913 INFO:tasks.workunit.client.0.smithi031.stdout:3/626: dwrite d6/df/d16/d2a/d95/d9b/d1a/d25/f9f [0,4194304] 0 2024-04-24T23:13:53.914 INFO:tasks.workunit.client.0.smithi031.stdout:2/588: write d5/d8/d15/d56/d84/fa5 [1026961,37105] 0 2024-04-24T23:13:53.915 INFO:tasks.workunit.client.1.smithi040.stdout:8/378: mknod d0/d16/d19/d71/c7b 0 2024-04-24T23:13:53.916 INFO:tasks.workunit.client.1.smithi040.stdout:1/404: dwrite da/d12/d18/f6c [0,4194304] 0 2024-04-24T23:13:53.918 INFO:tasks.workunit.client.0.smithi031.stdout:9/790: rmdir d2/dae/d14/d78/d84 39 2024-04-24T23:13:53.919 INFO:tasks.workunit.client.1.smithi040.stdout:6/374: symlink d7/d1a/l78 0 2024-04-24T23:13:53.925 INFO:tasks.workunit.client.0.smithi031.stdout:0/746: symlink da/d11/d6a/de8/lfb 0 2024-04-24T23:13:53.925 INFO:tasks.workunit.client.1.smithi040.stdout:5/380: dread d2/d7/d8/f34 [0,4194304] 0 2024-04-24T23:13:53.925 INFO:tasks.workunit.client.0.smithi031.stdout:4/825: truncate d3/d20/f51 1218034 0 2024-04-24T23:13:53.927 INFO:tasks.workunit.client.1.smithi040.stdout:2/463: mknod da/d9f/cb4 0 2024-04-24T23:13:53.927 INFO:tasks.workunit.client.1.smithi040.stdout:5/381: write d2/d7/d8/d64/f68 [975854,76180] 0 2024-04-24T23:13:53.928 INFO:tasks.workunit.client.0.smithi031.stdout:2/589: mknod d5/d6/d29/d26/db0/dc2/dad/cd4 0 2024-04-24T23:13:53.928 INFO:tasks.workunit.client.0.smithi031.stdout:2/590: chown d5/d3d 59216 1 2024-04-24T23:13:53.932 INFO:tasks.workunit.client.0.smithi031.stdout:9/791: rename d2/d16/f77 to d2/dae/d14/d1d/d45/d56/ff2 0 2024-04-24T23:13:53.933 INFO:tasks.workunit.client.0.smithi031.stdout:9/792: chown d2/d43/d9d/dbd/fda 87039 1 2024-04-24T23:13:53.934 INFO:tasks.workunit.client.0.smithi031.stdout:3/627: symlink d6/df/d16/lbd 0 2024-04-24T23:13:53.935 INFO:tasks.workunit.client.1.smithi040.stdout:2/464: dwrite da/d10/d7a/f8d [0,4194304] 0 2024-04-24T23:13:53.935 INFO:tasks.workunit.client.1.smithi040.stdout:1/405: symlink da/d12/d18/d2a/d4d/l8f 0 2024-04-24T23:13:53.937 INFO:tasks.workunit.client.0.smithi031.stdout:0/747: creat da/d11/ffc x:0 0 0 2024-04-24T23:13:53.940 INFO:tasks.workunit.client.0.smithi031.stdout:0/748: truncate da/d11/d6a/fd0 146478 0 2024-04-24T23:13:53.941 INFO:tasks.workunit.client.1.smithi040.stdout:6/375: creat d7/d1a/d20/d51/f79 x:0 0 0 2024-04-24T23:13:53.941 INFO:tasks.workunit.client.0.smithi031.stdout:0/749: stat da/d11/f59 0 2024-04-24T23:13:53.941 INFO:tasks.workunit.client.0.smithi031.stdout:2/591: creat d5/d6/d29/d26/fd5 x:0 0 0 2024-04-24T23:13:53.944 INFO:tasks.workunit.client.1.smithi040.stdout:5/382: dread d2/d7/d8/db/d1e/f28 [0,4194304] 0 2024-04-24T23:13:53.945 INFO:tasks.workunit.client.0.smithi031.stdout:4/826: dwrite d3/f2b [0,4194304] 0 2024-04-24T23:13:53.949 INFO:tasks.workunit.client.0.smithi031.stdout:3/628: fdatasync d6/df/d16/d2a/f31 0 2024-04-24T23:13:53.949 INFO:tasks.workunit.client.0.smithi031.stdout:4/827: dread d3/f1b [0,4194304] 0 2024-04-24T23:13:53.954 INFO:tasks.workunit.client.1.smithi040.stdout:1/406: rmdir da/d35/d38/d47 39 2024-04-24T23:13:53.955 INFO:tasks.workunit.client.1.smithi040.stdout:3/411: truncate d1/d9/f17 633074 0 2024-04-24T23:13:53.956 INFO:tasks.workunit.client.1.smithi040.stdout:3/412: write d1/d2d/d2b/f42 [9374686,121695] 0 2024-04-24T23:13:53.957 INFO:tasks.workunit.client.0.smithi031.stdout:4/828: dwrite d3/d20/d25/d60/ff8 [0,4194304] 0 2024-04-24T23:13:53.958 INFO:tasks.workunit.client.1.smithi040.stdout:9/414: sync 2024-04-24T23:13:53.958 INFO:tasks.workunit.client.0.smithi031.stdout:4/829: chown d3/d20/d2d/c42 101655751 1 2024-04-24T23:13:53.959 INFO:tasks.workunit.client.1.smithi040.stdout:6/376: creat d7/de/d4e/f7a x:0 0 0 2024-04-24T23:13:53.960 INFO:tasks.workunit.client.1.smithi040.stdout:9/415: chown d7/da/d17/d40/d55/d61/f8f 2872232 1 2024-04-24T23:13:53.960 INFO:tasks.workunit.client.1.smithi040.stdout:0/447: link df/f39 df/d12/d4b/f92 0 2024-04-24T23:13:53.960 INFO:tasks.workunit.client.1.smithi040.stdout:6/377: chown d7/d1a/d20/d51/d2f 789874908 1 2024-04-24T23:13:53.960 INFO:tasks.workunit.client.0.smithi031.stdout:4/830: write d3/d20/d25/fc3 [1321504,89668] 0 2024-04-24T23:13:53.961 INFO:tasks.workunit.client.0.smithi031.stdout:9/793: write d2/dae/d14/d78/d84/da6/fa7 [3916299,89066] 0 2024-04-24T23:13:53.961 INFO:tasks.workunit.client.0.smithi031.stdout:4/831: readlink d3/da3/lcf 0 2024-04-24T23:13:53.962 INFO:tasks.workunit.client.1.smithi040.stdout:6/378: write d7/d1a/f67 [455256,130649] 0 2024-04-24T23:13:53.962 INFO:tasks.workunit.client.0.smithi031.stdout:9/794: write d2/d43/fe6 [156200,113418] 0 2024-04-24T23:13:53.962 INFO:tasks.workunit.client.1.smithi040.stdout:9/416: fsync d7/da/d17/d40/d55/d2f/d1f/f8a 0 2024-04-24T23:13:53.976 INFO:tasks.workunit.client.1.smithi040.stdout:8/379: creat d0/d16/f7c x:0 0 0 2024-04-24T23:13:53.978 INFO:tasks.workunit.client.1.smithi040.stdout:1/407: mkdir da/d12/d14/d90 0 2024-04-24T23:13:53.979 INFO:tasks.workunit.client.0.smithi031.stdout:0/750: write da/d18/d1a/d37/f5e [370725,94684] 0 2024-04-24T23:13:53.979 INFO:tasks.workunit.client.0.smithi031.stdout:0/751: fsync da/fd 0 2024-04-24T23:13:53.983 INFO:tasks.workunit.client.1.smithi040.stdout:2/465: mknod da/d10/d4d/da4/cb5 0 2024-04-24T23:13:53.985 INFO:tasks.workunit.client.0.smithi031.stdout:3/629: write d6/df/d16/d53/f67 [1375579,118694] 0 2024-04-24T23:13:53.986 INFO:tasks.workunit.client.1.smithi040.stdout:3/413: creat d1/d2d/d2b/f97 x:0 0 0 2024-04-24T23:13:53.986 INFO:tasks.workunit.client.0.smithi031.stdout:3/630: fsync d6/df/d16/d2a/d95/d9b/d1a/d25/f54 0 2024-04-24T23:13:53.988 INFO:tasks.workunit.client.1.smithi040.stdout:0/448: truncate df/f34 810572 0 2024-04-24T23:13:53.990 INFO:tasks.workunit.client.0.smithi031.stdout:4/832: unlink d3/c1f 0 2024-04-24T23:13:53.992 INFO:tasks.workunit.client.0.smithi031.stdout:3/631: dwrite d6/df/d16/d53/fa8 [0,4194304] 0 2024-04-24T23:13:53.996 INFO:tasks.workunit.client.0.smithi031.stdout:9/795: mkdir d2/dae/d14/d78/d84/d4a/dab/db7/df3 0 2024-04-24T23:13:53.999 INFO:tasks.workunit.client.1.smithi040.stdout:8/380: unlink d0/d16/d1b/d3f/l59 0 2024-04-24T23:13:54.000 INFO:tasks.workunit.client.1.smithi040.stdout:7/439: sync 2024-04-24T23:13:54.000 INFO:tasks.workunit.client.1.smithi040.stdout:4/338: sync 2024-04-24T23:13:54.001 INFO:tasks.workunit.client.1.smithi040.stdout:8/381: write d0/d16/d19/f5f [112325,121330] 0 2024-04-24T23:13:54.001 INFO:tasks.workunit.client.1.smithi040.stdout:4/339: chown da/d3f/c49 18469075 1 2024-04-24T23:13:54.002 INFO:tasks.workunit.client.1.smithi040.stdout:7/440: stat db/d12/d20/d52 0 2024-04-24T23:13:54.002 INFO:tasks.workunit.client.1.smithi040.stdout:1/408: unlink da/d12/d18/d2a/d32/f4b 0 2024-04-24T23:13:54.005 INFO:tasks.workunit.client.1.smithi040.stdout:2/466: readlink da/d1c/d37/d39/l5d 0 2024-04-24T23:13:54.006 INFO:tasks.workunit.client.1.smithi040.stdout:3/414: creat d1/d2a/d6a/d94/f98 x:0 0 0 2024-04-24T23:13:54.008 INFO:tasks.workunit.client.1.smithi040.stdout:9/417: mkdir d7/d92 0 2024-04-24T23:13:54.011 INFO:tasks.workunit.client.1.smithi040.stdout:8/382: dwrite d0/d8/d22/f73 [0,4194304] 0 2024-04-24T23:13:54.013 INFO:tasks.workunit.client.1.smithi040.stdout:5/383: dwrite d2/d7/d8/d1f/d57/f44 [4194304,4194304] 0 2024-04-24T23:13:54.016 INFO:tasks.workunit.client.1.smithi040.stdout:4/340: stat da/f1c 0 2024-04-24T23:13:54.017 INFO:tasks.workunit.client.1.smithi040.stdout:4/341: write da/d3f/f56 [9121284,27797] 0 2024-04-24T23:13:54.021 INFO:tasks.workunit.client.1.smithi040.stdout:1/409: symlink da/d12/d18/d2a/d41/d4f/l91 0 2024-04-24T23:13:54.026 INFO:tasks.workunit.client.1.smithi040.stdout:6/379: write d7/d1a/d20/d51/d2f/f5b [298823,68905] 0 2024-04-24T23:13:54.027 INFO:tasks.workunit.client.1.smithi040.stdout:6/380: stat d7/d1a/d20/d51/d2f/f35 0 2024-04-24T23:13:54.030 INFO:tasks.workunit.client.1.smithi040.stdout:3/415: creat d1/d2a/d8d/f99 x:0 0 0 2024-04-24T23:13:54.033 INFO:tasks.workunit.client.1.smithi040.stdout:8/383: symlink d0/d16/d1b/d3f/l7d 0 2024-04-24T23:13:54.034 INFO:tasks.workunit.client.1.smithi040.stdout:6/381: dwrite d7/f60 [0,4194304] 0 2024-04-24T23:13:54.035 INFO:tasks.workunit.client.1.smithi040.stdout:8/384: read d0/d8/dc/d34/f3c [963782,15877] 0 2024-04-24T23:13:54.035 INFO:tasks.workunit.client.1.smithi040.stdout:6/382: write f2 [7143084,108746] 0 2024-04-24T23:13:54.038 INFO:tasks.workunit.client.1.smithi040.stdout:5/384: truncate d2/d7/d8/d1f/d57/d27/f54 1088675 0 2024-04-24T23:13:54.043 INFO:tasks.workunit.client.1.smithi040.stdout:0/449: write df/d12/d4b/d62/f7f [1842670,33673] 0 2024-04-24T23:13:54.045 INFO:tasks.workunit.client.1.smithi040.stdout:8/385: dread d0/d16/d19/d4f/f4e [0,4194304] 0 2024-04-24T23:13:54.046 INFO:tasks.workunit.client.1.smithi040.stdout:6/383: dwrite d7/d1a/d20/d51/f79 [0,4194304] 0 2024-04-24T23:13:54.047 INFO:tasks.workunit.client.1.smithi040.stdout:3/416: creat d1/d2a/f9a x:0 0 0 2024-04-24T23:13:54.054 INFO:tasks.workunit.client.1.smithi040.stdout:5/385: symlink d2/d7/d8/db/l7e 0 2024-04-24T23:13:54.055 INFO:tasks.workunit.client.1.smithi040.stdout:5/386: stat d2/lf 0 2024-04-24T23:13:54.060 INFO:tasks.workunit.client.1.smithi040.stdout:4/342: mknod da/d1f/c7e 0 2024-04-24T23:13:54.063 INFO:tasks.workunit.client.1.smithi040.stdout:1/410: unlink da/d12/d18/f49 0 2024-04-24T23:13:54.064 INFO:tasks.workunit.client.1.smithi040.stdout:1/411: truncate da/d12/d14/f83 833485 0 2024-04-24T23:13:54.065 INFO:tasks.workunit.client.0.smithi031.stdout:4/833: link d3/d20/d2d/d44/fd3 d3/d20/d25/d60/d63/f108 0 2024-04-24T23:13:54.069 INFO:tasks.workunit.client.0.smithi031.stdout:2/592: link d5/d6/f1f d5/d28/d4e/fd6 0 2024-04-24T23:13:54.069 INFO:tasks.workunit.client.1.smithi040.stdout:6/384: creat d7/d1a/d20/d51/d1e/d5d/f7b x:0 0 0 2024-04-24T23:13:54.070 INFO:tasks.workunit.client.1.smithi040.stdout:6/385: chown d7/d1a/d20/d51/d2f/f5b 19 1 2024-04-24T23:13:54.070 INFO:tasks.workunit.client.1.smithi040.stdout:6/386: readlink d7/d1a/l32 0 2024-04-24T23:13:54.071 INFO:tasks.workunit.client.1.smithi040.stdout:3/417: unlink d1/d2d/d2b/f97 0 2024-04-24T23:13:54.072 INFO:tasks.workunit.client.0.smithi031.stdout:0/752: creat da/ffd x:0 0 0 2024-04-24T23:13:54.074 INFO:tasks.workunit.client.0.smithi031.stdout:4/834: mkdir d3/d20/d25/d60/dc8/d109 0 2024-04-24T23:13:54.075 INFO:tasks.workunit.client.1.smithi040.stdout:6/387: dread d7/d1a/d20/d51/d2f/f5b [0,4194304] 0 2024-04-24T23:13:54.075 INFO:tasks.workunit.client.1.smithi040.stdout:3/418: dread d1/d2d/de/d30/d37/f74 [0,4194304] 0 2024-04-24T23:13:54.075 INFO:tasks.workunit.client.1.smithi040.stdout:9/418: link d7/da/d17/d40/d55/d2f/d1f/d31/d3b/c41 d7/da/d17/d40/d55/d2f/d1f/d31/d3b/c93 0 2024-04-24T23:13:54.080 INFO:tasks.workunit.client.0.smithi031.stdout:0/753: creat da/d20/d4e/dc9/ffe x:0 0 0 2024-04-24T23:13:54.081 INFO:tasks.workunit.client.1.smithi040.stdout:9/419: dwrite d7/da/d17/d40/d55/d61/d6c/f7a [0,4194304] 0 2024-04-24T23:13:54.082 INFO:tasks.workunit.client.1.smithi040.stdout:5/387: rmdir d2/d7/d8/d1f/d57/d27/d65 39 2024-04-24T23:13:54.082 INFO:tasks.workunit.client.0.smithi031.stdout:3/632: write d6/d14/d1d/d69/f9c [4880226,91932] 0 2024-04-24T23:13:54.084 INFO:tasks.workunit.client.1.smithi040.stdout:7/441: getdents db/d2f/d37 0 2024-04-24T23:13:54.085 INFO:tasks.workunit.client.0.smithi031.stdout:9/796: write d2/dae/d14/d1d/d45/fc1 [473003,59920] 0 2024-04-24T23:13:54.087 INFO:tasks.workunit.client.1.smithi040.stdout:4/343: truncate da/f2f 898238 0 2024-04-24T23:13:54.087 INFO:tasks.workunit.client.0.smithi031.stdout:0/754: fsync da/d18/f9a 0 2024-04-24T23:13:54.088 INFO:tasks.workunit.client.0.smithi031.stdout:0/755: chown da/d20/d8e/fd4 98235359 1 2024-04-24T23:13:54.088 INFO:tasks.workunit.client.0.smithi031.stdout:0/756: dread - da/d20/d4e/dc9/ffe zero size 2024-04-24T23:13:54.089 INFO:tasks.workunit.client.0.smithi031.stdout:4/835: creat d3/d5e/dc4/f10a x:0 0 0 2024-04-24T23:13:54.090 INFO:tasks.workunit.client.1.smithi040.stdout:0/450: mknod df/d12/d4b/d62/c93 0 2024-04-24T23:13:54.090 INFO:tasks.workunit.client.0.smithi031.stdout:3/633: truncate d6/d14/d1d/f24 1768521 0 2024-04-24T23:13:54.091 INFO:tasks.workunit.client.1.smithi040.stdout:1/412: write da/d12/d14/f2d [4165488,114606] 0 2024-04-24T23:13:54.092 INFO:tasks.workunit.client.0.smithi031.stdout:3/634: chown d6/df/d16/d2a/d95/d9b/d1a/f22 0 1 2024-04-24T23:13:54.093 INFO:tasks.workunit.client.1.smithi040.stdout:2/467: link da/d10/d4d/d6c/lb1 da/d1c/d37/d39/d3f/d52/d78/lb6 0 2024-04-24T23:13:54.093 INFO:tasks.workunit.client.1.smithi040.stdout:2/468: stat da/d10/d4d/d6c/f9c 0 2024-04-24T23:13:54.096 INFO:tasks.workunit.client.1.smithi040.stdout:5/388: dread d2/d7/d8/d1f/d57/d27/f54 [0,4194304] 0 2024-04-24T23:13:54.096 INFO:tasks.workunit.client.0.smithi031.stdout:9/797: read d2/dae/d14/f5f [67375,116487] 0 2024-04-24T23:13:54.097 INFO:tasks.workunit.client.1.smithi040.stdout:8/386: mkdir d0/d7e 0 2024-04-24T23:13:54.097 INFO:tasks.workunit.client.1.smithi040.stdout:8/387: fsync d0/d8/d22/f6a 0 2024-04-24T23:13:54.100 INFO:tasks.workunit.client.1.smithi040.stdout:8/388: dread d0/d8/f4d [0,4194304] 0 2024-04-24T23:13:54.102 INFO:tasks.workunit.client.0.smithi031.stdout:6/762: sync 2024-04-24T23:13:54.102 INFO:tasks.workunit.client.0.smithi031.stdout:5/624: sync 2024-04-24T23:13:54.102 INFO:tasks.workunit.client.0.smithi031.stdout:0/757: mknod da/d18/d1a/db7/cff 0 2024-04-24T23:13:54.104 INFO:tasks.workunit.client.0.smithi031.stdout:2/593: rename d5/d6/d29/d26/db0/dc2/f1e to d5/fd7 0 2024-04-24T23:13:54.105 INFO:tasks.workunit.client.0.smithi031.stdout:2/594: stat d5/d28/d43/c81 0 2024-04-24T23:13:54.106 INFO:tasks.workunit.client.1.smithi040.stdout:8/389: dwrite d0/d38/d4c/d5d/f7a [0,4194304] 0 2024-04-24T23:13:54.109 INFO:tasks.workunit.client.1.smithi040.stdout:6/388: creat d7/d1a/d20/d51/d2a/f7c x:0 0 0 2024-04-24T23:13:54.110 INFO:tasks.workunit.client.0.smithi031.stdout:0/758: creat da/d11/f100 x:0 0 0 2024-04-24T23:13:54.112 INFO:tasks.workunit.client.1.smithi040.stdout:6/389: dread - d7/d1a/d20/d51/d1e/d5d/f7b zero size 2024-04-24T23:13:54.112 INFO:tasks.workunit.client.1.smithi040.stdout:6/390: fsync d7/d1a/f67 0 2024-04-24T23:13:54.114 INFO:tasks.workunit.client.0.smithi031.stdout:6/763: symlink d13/d15/d1d/d23/d24/d43/d68/db8/l104 0 2024-04-24T23:13:54.117 INFO:tasks.workunit.client.1.smithi040.stdout:9/420: mknod d7/da/d17/d40/d55/d2f/d1f/d31/d3b/c94 0 2024-04-24T23:13:54.119 INFO:tasks.workunit.client.1.smithi040.stdout:4/344: fdatasync da/d45/d58/f73 0 2024-04-24T23:13:54.121 INFO:tasks.workunit.client.0.smithi031.stdout:0/759: truncate da/d86/faa 649822 0 2024-04-24T23:13:54.122 INFO:tasks.workunit.client.0.smithi031.stdout:0/760: chown da/d11/d6a/dba/dc0 28 1 2024-04-24T23:13:54.123 INFO:tasks.workunit.client.1.smithi040.stdout:2/469: mknod da/d1c/cb7 0 2024-04-24T23:13:54.123 INFO:tasks.workunit.client.0.smithi031.stdout:3/635: write d6/df/d16/d2a/d95/d9b/d1a/d25/f87 [465177,56042] 0 2024-04-24T23:13:54.123 INFO:tasks.workunit.client.1.smithi040.stdout:2/470: fdatasync da/d1c/d37/d39/d3f/f88 0 2024-04-24T23:13:54.124 INFO:tasks.workunit.client.1.smithi040.stdout:2/471: chown da/d10/d15/c82 103821973 1 2024-04-24T23:13:54.125 INFO:tasks.workunit.client.0.smithi031.stdout:0/761: truncate da/d86/da3/dcd/de7/ddb/fe5 195774 0 2024-04-24T23:13:54.125 INFO:tasks.workunit.client.1.smithi040.stdout:2/472: dread - da/d10/f71 zero size 2024-04-24T23:13:54.125 INFO:tasks.workunit.client.1.smithi040.stdout:5/389: unlink d2/d7/d8/d1f/d57/l49 0 2024-04-24T23:13:54.127 INFO:tasks.workunit.client.0.smithi031.stdout:6/764: stat d13/d15/d1d/l20 0 2024-04-24T23:13:54.128 INFO:tasks.workunit.client.0.smithi031.stdout:4/836: dwrite d3/d5e/f8d [0,4194304] 0 2024-04-24T23:13:54.130 INFO:tasks.workunit.client.0.smithi031.stdout:6/765: write d13/d15/d1d/d23/d44/d47/d69/f7f [4752784,61837] 0 2024-04-24T23:13:54.136 INFO:tasks.workunit.client.0.smithi031.stdout:5/625: dwrite d0/d6/d3b/d4b/dc/d25/f60 [0,4194304] 0 2024-04-24T23:13:54.141 INFO:tasks.workunit.client.1.smithi040.stdout:3/419: symlink d1/d9/d18/d36/l9b 0 2024-04-24T23:13:54.148 INFO:tasks.workunit.client.1.smithi040.stdout:9/421: mknod d7/da/d17/d40/d55/d2f/d1f/d34/c95 0 2024-04-24T23:13:54.149 INFO:tasks.workunit.client.0.smithi031.stdout:0/762: rename da/d11/fdd to da/d18/d5a/d7d/f101 0 2024-04-24T23:13:54.150 INFO:tasks.workunit.client.1.smithi040.stdout:7/442: unlink db/d12/d20/d52/c3d 0 2024-04-24T23:13:54.152 INFO:tasks.workunit.client.1.smithi040.stdout:4/345: rename da/d1f/c3a to da/d78/c7f 0 2024-04-24T23:13:54.153 INFO:tasks.workunit.client.0.smithi031.stdout:4/837: fdatasync d3/d5e/f8d 0 2024-04-24T23:13:54.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/595: getdents d5/d6/d29/d4f 0 2024-04-24T23:13:54.155 INFO:tasks.workunit.client.1.smithi040.stdout:2/473: rmdir da/d9f 39 2024-04-24T23:13:54.156 INFO:tasks.workunit.client.0.smithi031.stdout:3/636: creat d6/fbe x:0 0 0 2024-04-24T23:13:54.157 INFO:tasks.workunit.client.0.smithi031.stdout:3/637: fdatasync d6/d14/d1d/d59/f9a 0 2024-04-24T23:13:54.159 INFO:tasks.workunit.client.0.smithi031.stdout:0/763: symlink da/d11/d6a/l102 0 2024-04-24T23:13:54.159 INFO:tasks.workunit.client.0.smithi031.stdout:0/764: fsync da/d11/d6a/fc5 0 2024-04-24T23:13:54.163 INFO:tasks.workunit.client.1.smithi040.stdout:8/390: symlink d0/d52/l7f 0 2024-04-24T23:13:54.163 INFO:tasks.workunit.client.1.smithi040.stdout:8/391: fsync d0/d38/d4c/d5d/f7a 0 2024-04-24T23:13:54.164 INFO:tasks.workunit.client.1.smithi040.stdout:3/420: creat d1/d9/d18/f9c x:0 0 0 2024-04-24T23:13:54.168 INFO:tasks.workunit.client.1.smithi040.stdout:9/422: symlink d7/da/d17/d40/d55/d61/d6c/l96 0 2024-04-24T23:13:54.169 INFO:tasks.workunit.client.0.smithi031.stdout:5/626: truncate d0/d6/d3b/d4b/d1d/d2e/dc6/f6f 188414 0 2024-04-24T23:13:54.170 INFO:tasks.workunit.client.0.smithi031.stdout:2/596: rename d5/f41 to d5/d8/d15/d56/d84/fd8 0 2024-04-24T23:13:54.171 INFO:tasks.workunit.client.1.smithi040.stdout:7/443: fsync db/d12/d62/d8a/f8b 0 2024-04-24T23:13:54.172 INFO:tasks.workunit.client.0.smithi031.stdout:6/766: creat d13/d15/d1d/d23/d24/f105 x:0 0 0 2024-04-24T23:13:54.173 INFO:tasks.workunit.client.1.smithi040.stdout:4/346: mknod da/d1f/c80 0 2024-04-24T23:13:54.174 INFO:tasks.workunit.client.1.smithi040.stdout:0/451: link df/d12/d58/c91 df/d12/c94 0 2024-04-24T23:13:54.177 INFO:tasks.workunit.client.1.smithi040.stdout:2/474: unlink f7 0 2024-04-24T23:13:54.177 INFO:tasks.workunit.client.1.smithi040.stdout:1/413: sync 2024-04-24T23:13:54.177 INFO:tasks.workunit.client.1.smithi040.stdout:1/414: read - da/d12/d18/d2a/f56 zero size 2024-04-24T23:13:54.178 INFO:tasks.workunit.client.0.smithi031.stdout:0/765: mknod da/d20/d8e/c103 0 2024-04-24T23:13:54.180 INFO:tasks.workunit.client.1.smithi040.stdout:5/390: mkdir d2/d2f/d14/d7f 0 2024-04-24T23:13:54.181 INFO:tasks.workunit.client.1.smithi040.stdout:5/391: truncate d2/d7/d8/d1f/f6e 851023 0 2024-04-24T23:13:54.186 INFO:tasks.workunit.client.0.smithi031.stdout:2/597: creat d5/d28/d4e/d5e/fd9 x:0 0 0 2024-04-24T23:13:54.188 INFO:tasks.workunit.client.0.smithi031.stdout:3/638: dread d6/df/d16/d2a/d95/d9b/d60/f84 [0,4194304] 0 2024-04-24T23:13:54.189 INFO:tasks.workunit.client.0.smithi031.stdout:6/767: mkdir d13/d1b/d32/db4/d106 0 2024-04-24T23:13:54.189 INFO:tasks.workunit.client.0.smithi031.stdout:3/639: write d6/d14/d1d/f1e [3046957,24156] 0 2024-04-24T23:13:54.191 INFO:tasks.workunit.client.0.smithi031.stdout:3/640: read d6/d14/d1d/f9d [598721,111870] 0 2024-04-24T23:13:54.195 INFO:tasks.workunit.client.0.smithi031.stdout:5/627: write d0/d6/d3b/d4b/d1d/f9e [686943,77338] 0 2024-04-24T23:13:54.199 INFO:tasks.workunit.client.1.smithi040.stdout:0/452: mknod df/d1e/d20/d21/d41/d52/c95 0 2024-04-24T23:13:54.201 INFO:tasks.workunit.client.1.smithi040.stdout:4/347: dwrite da/d3f/d40/d41/d60/f34 [0,4194304] 0 2024-04-24T23:13:54.201 INFO:tasks.workunit.client.1.smithi040.stdout:2/475: mknod da/d81/cb8 0 2024-04-24T23:13:54.201 INFO:tasks.workunit.client.0.smithi031.stdout:0/766: dwrite da/d20/d4e/f48 [0,4194304] 0 2024-04-24T23:13:54.205 INFO:tasks.workunit.client.0.smithi031.stdout:4/838: dwrite d3/d23/d37/d86/fde [0,4194304] 0 2024-04-24T23:13:54.207 INFO:tasks.workunit.client.1.smithi040.stdout:3/421: symlink d1/d9/d18/d36/l9d 0 2024-04-24T23:13:54.208 INFO:tasks.workunit.client.1.smithi040.stdout:6/391: getdents d7/d1a/d20/d51/d2a 0 2024-04-24T23:13:54.212 INFO:tasks.workunit.client.1.smithi040.stdout:9/423: mkdir d7/d92/d97 0 2024-04-24T23:13:54.213 INFO:tasks.workunit.client.0.smithi031.stdout:2/598: mknod d5/d8/d45/d5b/d49/dc4/cda 0 2024-04-24T23:13:54.215 INFO:tasks.workunit.client.1.smithi040.stdout:7/444: creat db/d12/d3f/d7a/f9e x:0 0 0 2024-04-24T23:13:54.215 INFO:tasks.workunit.client.0.smithi031.stdout:6/768: write d13/d15/d1d/d23/d24/f40 [1010007,17810] 0 2024-04-24T23:13:54.217 INFO:tasks.workunit.client.0.smithi031.stdout:0/767: fsync da/d18/d1a/dd6/fbe 0 2024-04-24T23:13:54.223 INFO:tasks.workunit.client.1.smithi040.stdout:2/476: rename da/d1c/d37/d39/d3f/f92 to da/d10/d7a/fb9 0 2024-04-24T23:13:54.224 INFO:tasks.workunit.client.1.smithi040.stdout:1/415: mknod da/d35/d38/c92 0 2024-04-24T23:13:54.225 INFO:tasks.workunit.client.0.smithi031.stdout:5/628: link d0/d6/d3b/d4b/d1d/d2e/fba d0/d6/d3b/d4f/fce 0 2024-04-24T23:13:54.226 INFO:tasks.workunit.client.0.smithi031.stdout:5/629: stat d0/d6/d32/f5f 0 2024-04-24T23:13:54.228 INFO:tasks.workunit.client.1.smithi040.stdout:6/392: write d7/d1a/d20/d51/d2f/f6f [974471,11029] 0 2024-04-24T23:13:54.231 INFO:tasks.workunit.client.1.smithi040.stdout:7/445: unlink db/d2f/f99 0 2024-04-24T23:13:54.233 INFO:tasks.workunit.client.1.smithi040.stdout:0/453: truncate df/d12/d4b/f92 735021 0 2024-04-24T23:13:54.233 INFO:tasks.workunit.client.1.smithi040.stdout:0/454: chown df/d14 52841 1 2024-04-24T23:13:54.239 INFO:tasks.workunit.client.0.smithi031.stdout:4/839: rename d3/d23/d37/d86/f9a to d3/d23/d69/f10b 0 2024-04-24T23:13:54.240 INFO:tasks.workunit.client.1.smithi040.stdout:2/477: symlink da/d29/d55/lba 0 2024-04-24T23:13:54.242 INFO:tasks.workunit.client.0.smithi031.stdout:5/630: dwrite d0/d6/d3b/d4b/d1d/d2e/dc6/fc8 [0,4194304] 0 2024-04-24T23:13:54.242 INFO:tasks.workunit.client.1.smithi040.stdout:1/416: symlink da/d12/d18/d2a/d4d/l93 0 2024-04-24T23:13:54.245 INFO:tasks.workunit.client.0.smithi031.stdout:3/641: write d6/d14/f7d [1867958,107189] 0 2024-04-24T23:13:54.247 INFO:tasks.workunit.client.0.smithi031.stdout:5/631: chown d0/d6/d3b/d4b/dc/f47 3255814 1 2024-04-24T23:13:54.247 INFO:tasks.workunit.client.1.smithi040.stdout:2/478: dwrite da/d29/d55/f58 [0,4194304] 0 2024-04-24T23:13:54.248 INFO:tasks.workunit.client.0.smithi031.stdout:6/769: dread d13/d15/d1d/d23/d4d/f6a [0,4194304] 0 2024-04-24T23:13:54.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/768: rename da/d11/d6a/da2 to da/d86/da3/d104 0 2024-04-24T23:13:54.250 INFO:tasks.workunit.client.0.smithi031.stdout:2/599: dread d5/d28/d43/f97 [0,4194304] 0 2024-04-24T23:13:54.251 INFO:tasks.workunit.client.1.smithi040.stdout:8/392: getdents d0/d8 0 2024-04-24T23:13:54.252 INFO:tasks.workunit.client.0.smithi031.stdout:4/840: dwrite d3/d23/d37/d86/fde [0,4194304] 0 2024-04-24T23:13:54.252 INFO:tasks.workunit.client.1.smithi040.stdout:4/348: dread da/d3f/d6b/f21 [0,4194304] 0 2024-04-24T23:13:54.253 INFO:tasks.workunit.client.1.smithi040.stdout:8/393: write d0/d8/d22/f62 [787936,49893] 0 2024-04-24T23:13:54.254 INFO:tasks.workunit.client.0.smithi031.stdout:4/841: chown d3/d23/l3c 1660258 1 2024-04-24T23:13:54.255 INFO:tasks.workunit.client.1.smithi040.stdout:4/349: truncate da/d45/d58/f73 1096781 0 2024-04-24T23:13:54.260 INFO:tasks.workunit.client.0.smithi031.stdout:3/642: rename d6/d14/d1d/f1e to d6/d55/fbf 0 2024-04-24T23:13:54.261 INFO:tasks.workunit.client.0.smithi031.stdout:3/643: chown d6/df/d16/d2a/caa 307 1 2024-04-24T23:13:54.267 INFO:tasks.workunit.client.0.smithi031.stdout:6/770: unlink d13/d15/d1d/d23/d24/f9f 0 2024-04-24T23:13:54.270 INFO:tasks.workunit.client.1.smithi040.stdout:6/393: creat d7/d1a/d20/d51/d1e/d46/f7d x:0 0 0 2024-04-24T23:13:54.270 INFO:tasks.workunit.client.1.smithi040.stdout:6/394: chown d7/de/l19 1555477 1 2024-04-24T23:13:54.274 INFO:tasks.workunit.client.1.smithi040.stdout:7/446: rename db/d12/d3f/f41 to db/d2f/f9f 0 2024-04-24T23:13:54.275 INFO:tasks.workunit.client.1.smithi040.stdout:2/479: dread da/f40 [0,4194304] 0 2024-04-24T23:13:54.279 INFO:tasks.workunit.client.0.smithi031.stdout:5/632: rename d0/d6/l34 to d0/d6/d3b/d4b/d1d/d2e/d28/d77/lcf 0 2024-04-24T23:13:54.280 INFO:tasks.workunit.client.0.smithi031.stdout:3/644: creat d6/d14/d1d/d59/fc0 x:0 0 0 2024-04-24T23:13:54.282 INFO:tasks.workunit.client.0.smithi031.stdout:3/645: write d6/d14/d91/f99 [1266836,88887] 0 2024-04-24T23:13:54.288 INFO:tasks.workunit.client.0.smithi031.stdout:2/600: link d5/d6/d29/d26/db0/dc2/fd0 d5/d99/fdb 0 2024-04-24T23:13:54.288 INFO:tasks.workunit.client.1.smithi040.stdout:1/417: fsync da/ff 0 2024-04-24T23:13:54.289 INFO:tasks.workunit.client.1.smithi040.stdout:5/392: getdents d2/d7/d8/d1f/d31 0 2024-04-24T23:13:54.289 INFO:tasks.workunit.client.1.smithi040.stdout:4/350: fdatasync da/d3f/d40/d41/f4e 0 2024-04-24T23:13:54.289 INFO:tasks.workunit.client.1.smithi040.stdout:9/424: creat d7/da/d17/d40/d55/d2f/d1f/d31/f98 x:0 0 0 2024-04-24T23:13:54.291 INFO:tasks.workunit.client.1.smithi040.stdout:7/447: mknod db/d2f/d37/ca0 0 2024-04-24T23:13:54.292 INFO:tasks.workunit.client.0.smithi031.stdout:5/633: symlink d0/d6/d3b/d4b/dc/d8b/d9b/da8/ld0 0 2024-04-24T23:13:54.292 INFO:tasks.workunit.client.1.smithi040.stdout:5/393: dread d2/d7/d8/d1f/d57/f70 [0,4194304] 0 2024-04-24T23:13:54.293 INFO:tasks.workunit.client.1.smithi040.stdout:2/480: dwrite da/d29/d55/fa9 [0,4194304] 0 2024-04-24T23:13:54.294 INFO:tasks.workunit.client.0.smithi031.stdout:8/835: sync 2024-04-24T23:13:54.295 INFO:tasks.workunit.client.0.smithi031.stdout:6/771: dread d13/d1b/d32/db4/d79/dd7/fbc [0,4194304] 0 2024-04-24T23:13:54.295 INFO:tasks.workunit.client.0.smithi031.stdout:8/836: stat d8/d15/d37/c97 0 2024-04-24T23:13:54.296 INFO:tasks.workunit.client.1.smithi040.stdout:5/394: dread d2/d7/d8/d1f/d57/d27/d3d/f7a [0,4194304] 0 2024-04-24T23:13:54.298 INFO:tasks.workunit.client.1.smithi040.stdout:4/351: dwrite da/d3f/d40/d41/d60/f4d [0,4194304] 0 2024-04-24T23:13:54.299 INFO:tasks.workunit.client.1.smithi040.stdout:4/352: truncate da/d3f/d6b/d2b/f6d 105892 0 2024-04-24T23:13:54.300 INFO:tasks.workunit.client.0.smithi031.stdout:5/634: dread d0/d6/d3b/d4b/f7a [0,4194304] 0 2024-04-24T23:13:54.300 INFO:tasks.workunit.client.1.smithi040.stdout:4/353: write da/d3f/f5a [4454963,50415] 0 2024-04-24T23:13:54.303 INFO:tasks.workunit.client.1.smithi040.stdout:6/395: symlink d7/d1a/d20/d51/d70/l7e 0 2024-04-24T23:13:54.303 INFO:tasks.workunit.client.0.smithi031.stdout:5/635: readlink d0/d6/d3b/d4b/dc/d86/d91/l9c 0 2024-04-24T23:13:54.304 INFO:tasks.workunit.client.0.smithi031.stdout:0/769: dwrite da/d22/d62/fb4 [0,4194304] 0 2024-04-24T23:13:54.306 INFO:tasks.workunit.client.0.smithi031.stdout:0/770: chown da/d86/da3/dcd/de7/fc7 19235 1 2024-04-24T23:13:54.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/636: write d0/d6/d3b/d4b/f7a [1871039,26332] 0 2024-04-24T23:13:54.307 INFO:tasks.workunit.client.0.smithi031.stdout:2/601: mknod d5/d8/d15/d56/d84/cdc 0 2024-04-24T23:13:54.309 INFO:tasks.workunit.client.1.smithi040.stdout:4/354: read da/d3f/d6b/d2b/f55 [4748235,86058] 0 2024-04-24T23:13:54.312 INFO:tasks.workunit.client.1.smithi040.stdout:1/418: dread - da/d12/d23/f3c zero size 2024-04-24T23:13:54.314 INFO:tasks.workunit.client.1.smithi040.stdout:1/419: fdatasync da/d12/d18/d2a/f56 0 2024-04-24T23:13:54.319 INFO:tasks.workunit.client.1.smithi040.stdout:0/455: creat df/d14/f96 x:0 0 0 2024-04-24T23:13:54.329 INFO:tasks.workunit.client.0.smithi031.stdout:3/646: link d6/df/d16/d2a/d95/d9b/l2c d6/d14/d91/dbb/lc1 0 2024-04-24T23:13:54.330 INFO:tasks.workunit.client.0.smithi031.stdout:6/772: write d13/d1b/f49 [5099021,13798] 0 2024-04-24T23:13:54.333 INFO:tasks.workunit.client.0.smithi031.stdout:6/773: truncate d13/d15/d1d/d23/d24/f77 5180115 0 2024-04-24T23:13:54.333 INFO:tasks.workunit.client.1.smithi040.stdout:6/396: creat d7/d1a/d56/f7f x:0 0 0 2024-04-24T23:13:54.334 INFO:tasks.workunit.client.1.smithi040.stdout:3/422: sync 2024-04-24T23:13:54.335 INFO:tasks.workunit.client.1.smithi040.stdout:6/397: write d7/f71 [434161,237] 0 2024-04-24T23:13:54.335 INFO:tasks.workunit.client.0.smithi031.stdout:0/771: dread da/d18/d1a/d37/f3d [4194304,4194304] 0 2024-04-24T23:13:54.337 INFO:tasks.workunit.client.1.smithi040.stdout:6/398: write d7/f60 [4708794,83848] 0 2024-04-24T23:13:54.339 INFO:tasks.workunit.client.0.smithi031.stdout:3/647: rmdir d6/d14/d71/dad 39 2024-04-24T23:13:54.340 INFO:tasks.workunit.client.1.smithi040.stdout:6/399: write f2 [5752973,104680] 0 2024-04-24T23:13:54.341 INFO:tasks.workunit.client.1.smithi040.stdout:7/448: dwrite db/f2e [0,4194304] 0 2024-04-24T23:13:54.342 INFO:tasks.workunit.client.0.smithi031.stdout:5/637: dwrite d0/d7/d56/f61 [0,4194304] 0 2024-04-24T23:13:54.344 INFO:tasks.workunit.client.1.smithi040.stdout:5/395: write d2/d7/d8/db/d1e/f28 [2572625,8336] 0 2024-04-24T23:13:54.345 INFO:tasks.workunit.client.1.smithi040.stdout:9/425: mknod d7/d92/d97/c99 0 2024-04-24T23:13:54.353 INFO:tasks.workunit.client.0.smithi031.stdout:8/837: getdents d8/d15/dcf 0 2024-04-24T23:13:54.353 INFO:tasks.workunit.client.0.smithi031.stdout:5/638: dwrite d0/d6/d3b/d4b/dc/f84 [0,4194304] 0 2024-04-24T23:13:54.354 INFO:tasks.workunit.client.0.smithi031.stdout:5/639: write d0/d6/d3b/d4b/dc/fa2 [3187007,76229] 0 2024-04-24T23:13:54.354 INFO:tasks.workunit.client.1.smithi040.stdout:4/355: dwrite da/f18 [0,4194304] 0 2024-04-24T23:13:54.354 INFO:tasks.workunit.client.1.smithi040.stdout:1/420: write da/d12/d55/f8e [690084,37323] 0 2024-04-24T23:13:54.354 INFO:tasks.workunit.client.1.smithi040.stdout:5/396: dread d2/d7/d8/d1f/d57/d27/d3d/f7a [0,4194304] 0 2024-04-24T23:13:54.354 INFO:tasks.workunit.client.0.smithi031.stdout:3/648: creat d6/d14/d1d/fc2 x:0 0 0 2024-04-24T23:13:54.355 INFO:tasks.workunit.client.0.smithi031.stdout:5/640: read d0/d6/d3b/d4b/d1d/d2e/dc6/f21 [3010056,11943] 0 2024-04-24T23:13:54.357 INFO:tasks.workunit.client.1.smithi040.stdout:5/397: write d2/d7/d8/d1f/d57/f44 [1681168,108221] 0 2024-04-24T23:13:54.357 INFO:tasks.workunit.client.0.smithi031.stdout:2/602: getdents d5/d6/d68 0 2024-04-24T23:13:54.361 INFO:tasks.workunit.client.1.smithi040.stdout:9/426: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f50 [0,4194304] 0 2024-04-24T23:13:54.364 INFO:tasks.workunit.client.1.smithi040.stdout:9/427: write d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 [2368916,2051] 0 2024-04-24T23:13:54.365 INFO:tasks.workunit.client.0.smithi031.stdout:8/838: creat d8/d15/d37/d43/daf/f120 x:0 0 0 2024-04-24T23:13:54.366 INFO:tasks.workunit.client.1.smithi040.stdout:3/423: rmdir d1/d9/d18/d36/d55 39 2024-04-24T23:13:54.370 INFO:tasks.workunit.client.0.smithi031.stdout:8/839: dread - d8/d1a/f11c zero size 2024-04-24T23:13:54.370 INFO:tasks.workunit.client.0.smithi031.stdout:6/774: unlink d13/d15/d1d/d23/d99/lc0 0 2024-04-24T23:13:54.370 INFO:tasks.workunit.client.1.smithi040.stdout:3/424: chown d1/d2a/d6a/d94 261075912 1 2024-04-24T23:13:54.370 INFO:tasks.workunit.client.1.smithi040.stdout:3/425: dread - d1/d2a/d6a/d94/f95 zero size 2024-04-24T23:13:54.371 INFO:tasks.workunit.client.0.smithi031.stdout:2/603: dwrite d5/d8/d45/d5b/d49/f9f [0,4194304] 0 2024-04-24T23:13:54.373 INFO:tasks.workunit.client.0.smithi031.stdout:5/641: mknod d0/d7/d74/cd1 0 2024-04-24T23:13:54.378 INFO:tasks.workunit.client.1.smithi040.stdout:7/449: creat db/d12/d20/d52/d4b/fa1 x:0 0 0 2024-04-24T23:13:54.379 INFO:tasks.workunit.client.0.smithi031.stdout:3/649: creat d6/d14/db2/fc3 x:0 0 0 2024-04-24T23:13:54.379 INFO:tasks.workunit.client.1.smithi040.stdout:0/456: getdents df/d1e/d89 0 2024-04-24T23:13:54.383 INFO:tasks.workunit.client.0.smithi031.stdout:8/840: rmdir d8/d15/d37/d43/d5c/db5 39 2024-04-24T23:13:54.383 INFO:tasks.workunit.client.0.smithi031.stdout:3/650: write d6/df/d16/d2a/d95/d9b/d1a/d25/d83/fb8 [80789,31981] 0 2024-04-24T23:13:54.385 INFO:tasks.workunit.client.1.smithi040.stdout:7/450: dread db/d12/d20/d52/d44/f53 [0,4194304] 0 2024-04-24T23:13:54.385 INFO:tasks.workunit.client.0.smithi031.stdout:2/604: creat d5/d6/d29/fdd x:0 0 0 2024-04-24T23:13:54.386 INFO:tasks.workunit.client.1.smithi040.stdout:7/451: chown db/d12/d20/c65 4 1 2024-04-24T23:13:54.386 INFO:tasks.workunit.client.0.smithi031.stdout:0/772: dwrite da/d86/da3/d104/fab [0,4194304] 0 2024-04-24T23:13:54.387 INFO:tasks.workunit.client.0.smithi031.stdout:3/651: write d6/d14/f58 [2969087,34340] 0 2024-04-24T23:13:54.387 INFO:tasks.workunit.client.1.smithi040.stdout:7/452: write db/d2f/f5b [345267,126860] 0 2024-04-24T23:13:54.388 INFO:tasks.workunit.client.1.smithi040.stdout:5/398: symlink d2/d7/d8/d1f/d57/d27/d3d/l80 0 2024-04-24T23:13:54.391 INFO:tasks.workunit.client.0.smithi031.stdout:6/775: truncate d13/d15/d1d/d23/d2d/dbf/fec 1653742 0 2024-04-24T23:13:54.395 INFO:tasks.workunit.client.1.smithi040.stdout:3/426: fdatasync d1/d9/d15/f1b 0 2024-04-24T23:13:54.395 INFO:tasks.workunit.client.1.smithi040.stdout:3/427: stat d1/d2a/d6a/c76 0 2024-04-24T23:13:54.396 INFO:tasks.workunit.client.0.smithi031.stdout:3/652: dwrite d6/df/d16/d53/f67 [0,4194304] 0 2024-04-24T23:13:54.396 INFO:tasks.workunit.client.1.smithi040.stdout:3/428: chown d1/d9/d39/f78 65394 1 2024-04-24T23:13:54.398 INFO:tasks.workunit.client.1.smithi040.stdout:3/429: truncate d1/d2a/d6a/d94/f98 172807 0 2024-04-24T23:13:54.398 INFO:tasks.workunit.client.0.smithi031.stdout:0/773: dread da/d20/f52 [0,4194304] 0 2024-04-24T23:13:54.402 INFO:tasks.workunit.client.1.smithi040.stdout:0/457: mknod df/d1e/d2b/c97 0 2024-04-24T23:13:54.406 INFO:tasks.workunit.client.0.smithi031.stdout:6/776: dwrite d13/d15/d1d/d23/d2d/fd2 [0,4194304] 0 2024-04-24T23:13:54.406 INFO:tasks.workunit.client.1.smithi040.stdout:3/430: dwrite d1/d2d/d44/f52 [0,4194304] 0 2024-04-24T23:13:54.410 INFO:tasks.workunit.client.1.smithi040.stdout:7/453: symlink db/d12/d3f/d7a/la2 0 2024-04-24T23:13:54.413 INFO:tasks.workunit.client.1.smithi040.stdout:6/400: dread d7/d1a/d20/d51/d2f/f35 [0,4194304] 0 2024-04-24T23:13:54.417 INFO:tasks.workunit.client.0.smithi031.stdout:5/642: rename d0/d6/d3b/c42 to d0/d6/d3b/d4b/dc/cd2 0 2024-04-24T23:13:54.419 INFO:tasks.workunit.client.0.smithi031.stdout:5/643: write d0/d6/d3b/d4b/fac [353080,86649] 0 2024-04-24T23:13:54.420 INFO:tasks.workunit.client.1.smithi040.stdout:9/428: dwrite d7/da/d17/d40/d55/d61/d6c/f85 [4194304,4194304] 0 2024-04-24T23:13:54.421 INFO:tasks.workunit.client.0.smithi031.stdout:8/841: write d8/d15/d37/d43/d5c/de1/f72 [131518,26271] 0 2024-04-24T23:13:54.423 INFO:tasks.workunit.client.0.smithi031.stdout:7/676: sync 2024-04-24T23:13:54.423 INFO:tasks.workunit.client.0.smithi031.stdout:9/798: sync 2024-04-24T23:13:54.423 INFO:tasks.workunit.client.0.smithi031.stdout:1/626: sync 2024-04-24T23:13:54.424 INFO:tasks.workunit.client.0.smithi031.stdout:2/605: write d5/d6/f16 [9264904,31162] 0 2024-04-24T23:13:54.424 INFO:tasks.workunit.client.0.smithi031.stdout:7/677: stat d1/d2/dd/d33/lba 0 2024-04-24T23:13:54.426 INFO:tasks.workunit.client.1.smithi040.stdout:9/429: write d7/da/d17/d40/d55/d2f/d1f/f5a [3636196,20989] 0 2024-04-24T23:13:54.426 INFO:tasks.workunit.client.1.smithi040.stdout:0/458: unlink df/d12/d49/f54 0 2024-04-24T23:13:54.427 INFO:tasks.workunit.client.0.smithi031.stdout:3/653: fdatasync d6/d14/d1d/d59/f90 0 2024-04-24T23:13:54.429 INFO:tasks.workunit.client.1.smithi040.stdout:9/430: fdatasync d7/da/d17/d40/d55/d2f/d28/f6b 0 2024-04-24T23:13:54.429 INFO:tasks.workunit.client.1.smithi040.stdout:0/459: truncate df/d1e/d20/d21/d41/f6d 105516 0 2024-04-24T23:13:54.430 INFO:tasks.workunit.client.0.smithi031.stdout:0/774: mknod da/d20/c105 0 2024-04-24T23:13:54.433 INFO:tasks.workunit.client.0.smithi031.stdout:5/644: symlink d0/d6/d3b/d4b/d1d/d2e/d28/ld3 0 2024-04-24T23:13:54.433 INFO:tasks.workunit.client.1.smithi040.stdout:3/431: dread d1/d9/d18/d36/d55/d62/f65 [0,4194304] 0 2024-04-24T23:13:54.433 INFO:tasks.workunit.client.1.smithi040.stdout:6/401: symlink d7/d1a/d56/l80 0 2024-04-24T23:13:54.434 INFO:tasks.workunit.client.1.smithi040.stdout:6/402: chown d7/d1a/d20/d51/d1e/d46 250 1 2024-04-24T23:13:54.437 INFO:tasks.workunit.client.1.smithi040.stdout:6/403: dread d7/f71 [0,4194304] 0 2024-04-24T23:13:54.438 INFO:tasks.workunit.client.0.smithi031.stdout:6/777: dread d13/d15/d1d/d23/d24/d43/fb3 [0,4194304] 0 2024-04-24T23:13:54.438 INFO:tasks.workunit.client.0.smithi031.stdout:9/799: truncate d2/dae/d14/d78/d84/d4a/fde 615160 0 2024-04-24T23:13:54.440 INFO:tasks.workunit.client.0.smithi031.stdout:9/800: write d2/dae/d14/d78/d84/d4a/dab/db8/fee [296690,85754] 0 2024-04-24T23:13:54.443 INFO:tasks.workunit.client.1.smithi040.stdout:8/394: sync 2024-04-24T23:13:54.443 INFO:tasks.workunit.client.0.smithi031.stdout:0/775: write da/d18/d1a/f63 [457673,92183] 0 2024-04-24T23:13:54.445 INFO:tasks.workunit.client.0.smithi031.stdout:3/654: dread d6/d14/f37 [0,4194304] 0 2024-04-24T23:13:54.447 INFO:tasks.workunit.client.0.smithi031.stdout:1/627: mknod d1/dc/d11/d21/cc5 0 2024-04-24T23:13:54.449 INFO:tasks.workunit.client.0.smithi031.stdout:7/678: mkdir d1/d2/deb 0 2024-04-24T23:13:54.452 INFO:tasks.workunit.client.1.smithi040.stdout:4/356: sync 2024-04-24T23:13:54.452 INFO:tasks.workunit.client.1.smithi040.stdout:2/481: sync 2024-04-24T23:13:54.454 INFO:tasks.workunit.client.0.smithi031.stdout:9/801: creat d2/dae/d14/d78/d84/da6/ff4 x:0 0 0 2024-04-24T23:13:54.458 INFO:tasks.workunit.client.1.smithi040.stdout:1/421: getdents da/d12/d14/d58 0 2024-04-24T23:13:54.460 INFO:tasks.workunit.client.1.smithi040.stdout:0/460: unlink l1 0 2024-04-24T23:13:54.461 INFO:tasks.workunit.client.1.smithi040.stdout:9/431: chown d7/da/d17/c44 3702 1 2024-04-24T23:13:54.465 INFO:tasks.workunit.client.0.smithi031.stdout:7/679: creat d1/d2/dd/fec x:0 0 0 2024-04-24T23:13:54.468 INFO:tasks.workunit.client.0.smithi031.stdout:4/842: sync 2024-04-24T23:13:54.468 INFO:tasks.workunit.client.0.smithi031.stdout:2/606: rename d5/d6/d29/d26/f3f to d5/d6/fde 0 2024-04-24T23:13:54.468 INFO:tasks.workunit.client.0.smithi031.stdout:5/645: dread d0/d6/d32/f4a [0,4194304] 0 2024-04-24T23:13:54.470 INFO:tasks.workunit.client.0.smithi031.stdout:6/778: mkdir d13/d1b/d32/db4/d79/dd7/dde/d107 0 2024-04-24T23:13:54.475 INFO:tasks.workunit.client.1.smithi040.stdout:8/395: unlink d0/d38/d4c/d5d/f7a 0 2024-04-24T23:13:54.475 INFO:tasks.workunit.client.0.smithi031.stdout:0/776: dread da/f3f [4194304,4194304] 0 2024-04-24T23:13:54.475 INFO:tasks.workunit.client.1.smithi040.stdout:8/396: fdatasync d0/d16/d1b/f40 0 2024-04-24T23:13:54.480 INFO:tasks.workunit.client.0.smithi031.stdout:1/628: symlink d1/d33/d36/d81/lc6 0 2024-04-24T23:13:54.481 INFO:tasks.workunit.client.0.smithi031.stdout:0/777: dwrite da/d20/d4e/f48 [0,4194304] 0 2024-04-24T23:13:54.484 INFO:tasks.workunit.client.1.smithi040.stdout:0/461: symlink df/d12/d58/d5b/l98 0 2024-04-24T23:13:54.486 INFO:tasks.workunit.client.1.smithi040.stdout:9/432: mknod d7/da/d17/d40/d55/d2f/c9a 0 2024-04-24T23:13:54.487 INFO:tasks.workunit.client.0.smithi031.stdout:4/843: unlink d3/d23/l3c 0 2024-04-24T23:13:54.487 INFO:tasks.workunit.client.1.smithi040.stdout:0/462: write df/d12/d4b/f8b [627127,18122] 0 2024-04-24T23:13:54.488 INFO:tasks.workunit.client.1.smithi040.stdout:1/422: dwrite da/d12/d18/f78 [0,4194304] 0 2024-04-24T23:13:54.489 INFO:tasks.workunit.client.0.smithi031.stdout:5/646: mknod d0/d6/d3b/cd4 0 2024-04-24T23:13:54.490 INFO:tasks.workunit.client.0.smithi031.stdout:8/842: dwrite d8/d1a/d1d/d50/dc0/d9d/fe4 [4194304,4194304] 0 2024-04-24T23:13:54.490 INFO:tasks.workunit.client.0.smithi031.stdout:5/647: chown d0/f88 210376144 1 2024-04-24T23:13:54.492 INFO:tasks.workunit.client.1.smithi040.stdout:5/399: rename d2/d7/f3b to d2/d2f/d14/f81 0 2024-04-24T23:13:54.494 INFO:tasks.workunit.client.0.smithi031.stdout:6/779: unlink d13/d1b/d32/f8a 0 2024-04-24T23:13:54.496 INFO:tasks.workunit.client.0.smithi031.stdout:9/802: mkdir d2/dae/d14/d78/d84/d4a/d4b/daf/df5 0 2024-04-24T23:13:54.498 INFO:tasks.workunit.client.1.smithi040.stdout:1/423: dwrite da/d31/f52 [0,4194304] 0 2024-04-24T23:13:54.499 INFO:tasks.workunit.client.0.smithi031.stdout:3/655: creat d6/fc4 x:0 0 0 2024-04-24T23:13:54.504 INFO:tasks.workunit.client.0.smithi031.stdout:0/778: creat da/d18/d1a/f106 x:0 0 0 2024-04-24T23:13:54.505 INFO:tasks.workunit.client.1.smithi040.stdout:1/424: dread da/d12/d18/d2a/f79 [0,4194304] 0 2024-04-24T23:13:54.505 INFO:tasks.workunit.client.0.smithi031.stdout:0/779: chown da/d18/d1a/d37/c1d 295216 1 2024-04-24T23:13:54.506 INFO:tasks.workunit.client.1.smithi040.stdout:1/425: fsync da/d12/d18/d2a/f56 0 2024-04-24T23:13:54.506 INFO:tasks.workunit.client.0.smithi031.stdout:0/780: dread - da/d18/d1a/dd6/fbe zero size 2024-04-24T23:13:54.507 INFO:tasks.workunit.client.0.smithi031.stdout:0/781: stat da/d18/d5a/d7d/f9b 0 2024-04-24T23:13:54.508 INFO:tasks.workunit.client.0.smithi031.stdout:0/782: chown da/d11/fd3 6678 1 2024-04-24T23:13:54.508 INFO:tasks.workunit.client.1.smithi040.stdout:4/357: symlink da/d3f/d6b/l81 0 2024-04-24T23:13:54.511 INFO:tasks.workunit.client.1.smithi040.stdout:2/482: creat da/d10/d15/daa/fbb x:0 0 0 2024-04-24T23:13:54.514 INFO:tasks.workunit.client.1.smithi040.stdout:6/404: dwrite d7/d1a/d20/d51/d1e/f49 [0,4194304] 0 2024-04-24T23:13:54.515 INFO:tasks.workunit.client.1.smithi040.stdout:6/405: chown d7/d1a/d20/l3c 0 1 2024-04-24T23:13:54.516 INFO:tasks.workunit.client.0.smithi031.stdout:9/803: rmdir d2/d16 39 2024-04-24T23:13:54.518 INFO:tasks.workunit.client.1.smithi040.stdout:7/454: rename db/d43/f5e to db/d12/d3f/d7a/fa3 0 2024-04-24T23:13:54.519 INFO:tasks.workunit.client.1.smithi040.stdout:3/432: creat d1/d9/d18/d36/f9e x:0 0 0 2024-04-24T23:13:54.520 INFO:tasks.workunit.client.0.smithi031.stdout:0/783: dread - da/d18/d1a/f72 zero size 2024-04-24T23:13:54.520 INFO:tasks.workunit.client.1.smithi040.stdout:3/433: write d1/d9/d15/f1b [5039445,66955] 0 2024-04-24T23:13:54.522 INFO:tasks.workunit.client.1.smithi040.stdout:3/434: fsync d1/d9/d18/d36/d55/f67 0 2024-04-24T23:13:54.522 INFO:tasks.workunit.client.0.smithi031.stdout:3/656: dread d6/d14/d1d/d59/f5e [0,4194304] 0 2024-04-24T23:13:54.523 INFO:tasks.workunit.client.1.smithi040.stdout:1/426: symlink da/d35/d61/l94 0 2024-04-24T23:13:54.526 INFO:tasks.workunit.client.1.smithi040.stdout:4/358: creat da/d3f/d40/d41/d60/f82 x:0 0 0 2024-04-24T23:13:54.527 INFO:tasks.workunit.client.1.smithi040.stdout:4/359: chown da/d1f/c2e 3521 1 2024-04-24T23:13:54.528 INFO:tasks.workunit.client.0.smithi031.stdout:1/629: creat d1/d3e/da3/fc7 x:0 0 0 2024-04-24T23:13:54.530 INFO:tasks.workunit.client.0.smithi031.stdout:0/784: creat da/d18/d1a/dd6/f107 x:0 0 0 2024-04-24T23:13:54.531 INFO:tasks.workunit.client.0.smithi031.stdout:0/785: readlink da/d18/l2c 0 2024-04-24T23:13:54.531 INFO:tasks.workunit.client.1.smithi040.stdout:0/463: fsync f6 0 2024-04-24T23:13:54.532 INFO:tasks.workunit.client.0.smithi031.stdout:3/657: creat d6/d14/d1d/d69/fc5 x:0 0 0 2024-04-24T23:13:54.532 INFO:tasks.workunit.client.1.smithi040.stdout:6/406: dread d7/d1a/d20/d51/d2f/f5b [0,4194304] 0 2024-04-24T23:13:54.533 INFO:tasks.workunit.client.1.smithi040.stdout:7/455: rename db/d12/d3f/f74 to db/d12/d3f/fa4 0 2024-04-24T23:13:54.535 INFO:tasks.workunit.client.0.smithi031.stdout:8/843: creat d8/d1a/d1d/d35/d58/f121 x:0 0 0 2024-04-24T23:13:54.538 INFO:tasks.workunit.client.1.smithi040.stdout:3/435: read d1/d2a/d8d/f7b [6296551,102363] 0 2024-04-24T23:13:54.538 INFO:tasks.workunit.client.0.smithi031.stdout:3/658: dwrite d6/fbe [0,4194304] 0 2024-04-24T23:13:54.538 INFO:tasks.workunit.client.0.smithi031.stdout:9/804: readlink d2/dae/l25 0 2024-04-24T23:13:54.539 INFO:tasks.workunit.client.1.smithi040.stdout:0/464: dread df/f63 [0,4194304] 0 2024-04-24T23:13:54.540 INFO:tasks.workunit.client.1.smithi040.stdout:6/407: dwrite d7/f60 [0,4194304] 0 2024-04-24T23:13:54.541 INFO:tasks.workunit.client.0.smithi031.stdout:4/844: dread d3/d20/d25/d60/d63/f81 [0,4194304] 0 2024-04-24T23:13:54.542 INFO:tasks.workunit.client.1.smithi040.stdout:6/408: truncate d7/d1a/f53 632227 0 2024-04-24T23:13:54.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/659: dread d6/df/d16/d2a/d95/d9b/d1a/d25/f74 [0,4194304] 0 2024-04-24T23:13:54.548 INFO:tasks.workunit.client.0.smithi031.stdout:3/660: chown d6/df/d16/d57/f65 11323504 1 2024-04-24T23:13:54.552 INFO:tasks.workunit.client.0.smithi031.stdout:7/680: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f5d [0,4194304] 0 2024-04-24T23:13:54.553 INFO:tasks.workunit.client.0.smithi031.stdout:5/648: getdents d0/d6/d3b/d4f 0 2024-04-24T23:13:54.558 INFO:tasks.workunit.client.0.smithi031.stdout:9/805: symlink d2/dae/d14/d1d/d45/d65/lf6 0 2024-04-24T23:13:54.564 INFO:tasks.workunit.client.0.smithi031.stdout:0/786: mknod da/d20/d4d/df8/c108 0 2024-04-24T23:13:54.565 INFO:tasks.workunit.client.0.smithi031.stdout:0/787: write da/d11/fd3 [664143,122398] 0 2024-04-24T23:13:54.566 INFO:tasks.workunit.client.1.smithi040.stdout:3/436: symlink d1/d2a/d8d/d7c/l9f 0 2024-04-24T23:13:54.566 INFO:tasks.workunit.client.0.smithi031.stdout:0/788: chown da/d86/dbf/dd5 1 1 2024-04-24T23:13:54.567 INFO:tasks.workunit.client.1.smithi040.stdout:3/437: readlink d1/d9/d18/d36/l9d 0 2024-04-24T23:13:54.567 INFO:tasks.workunit.client.0.smithi031.stdout:0/789: write da/d20/d4e/dc9/ffe [7562,53618] 0 2024-04-24T23:13:54.571 INFO:tasks.workunit.client.1.smithi040.stdout:8/397: dwrite d0/d16/d19/d4f/f4e [0,4194304] 0 2024-04-24T23:13:54.571 INFO:tasks.workunit.client.1.smithi040.stdout:6/409: symlink d7/d1a/d20/d51/d1e/d46/d69/l81 0 2024-04-24T23:13:54.572 INFO:tasks.workunit.client.0.smithi031.stdout:8/844: truncate d8/d1a/d1d/f78 193782 0 2024-04-24T23:13:54.572 INFO:tasks.workunit.client.1.smithi040.stdout:6/410: write d7/de/d4e/f64 [921625,123935] 0 2024-04-24T23:13:54.573 INFO:tasks.workunit.client.0.smithi031.stdout:8/845: readlink d8/d15/d37/d43/daf/dca/leb 0 2024-04-24T23:13:54.574 INFO:tasks.workunit.client.1.smithi040.stdout:6/411: chown d7/d1a/d20/d51/l30 23 1 2024-04-24T23:13:54.575 INFO:tasks.workunit.client.1.smithi040.stdout:1/427: symlink da/d35/d38/d47/l95 0 2024-04-24T23:13:54.577 INFO:tasks.workunit.client.1.smithi040.stdout:9/433: sync 2024-04-24T23:13:54.578 INFO:tasks.workunit.client.1.smithi040.stdout:9/434: readlink d7/da/d17/d40/d55/d2f/d1f/l7f 0 2024-04-24T23:13:54.578 INFO:tasks.workunit.client.1.smithi040.stdout:4/360: symlink da/d3f/d6b/l83 0 2024-04-24T23:13:54.580 INFO:tasks.workunit.client.0.smithi031.stdout:9/806: write d2/f12 [6936086,23438] 0 2024-04-24T23:13:54.583 INFO:tasks.workunit.client.1.smithi040.stdout:2/483: write da/d10/f34 [2424467,105887] 0 2024-04-24T23:13:54.584 INFO:tasks.workunit.client.1.smithi040.stdout:0/465: creat df/d12/d58/d68/d8e/f99 x:0 0 0 2024-04-24T23:13:54.584 INFO:tasks.workunit.client.1.smithi040.stdout:2/484: chown da/d1c/d37/d39/c89 1386575 1 2024-04-24T23:13:54.585 INFO:tasks.workunit.client.0.smithi031.stdout:0/790: fdatasync da/d18/d5a/d7d/dbd/fd1 0 2024-04-24T23:13:54.586 INFO:tasks.workunit.client.0.smithi031.stdout:0/791: write da/fd2 [746137,115725] 0 2024-04-24T23:13:54.589 INFO:tasks.workunit.client.1.smithi040.stdout:5/400: truncate d2/d7/d8/d1f/d57/d27/d3c/d43/f6f 667490 0 2024-04-24T23:13:54.591 INFO:tasks.workunit.client.1.smithi040.stdout:6/412: truncate d7/d1a/d20/d51/d2f/f5b 922291 0 2024-04-24T23:13:54.594 INFO:tasks.workunit.client.1.smithi040.stdout:1/428: creat da/d12/d14/f96 x:0 0 0 2024-04-24T23:13:54.594 INFO:tasks.workunit.client.1.smithi040.stdout:1/429: readlink da/d12/d18/d2a/d41/l89 0 2024-04-24T23:13:54.595 INFO:tasks.workunit.client.0.smithi031.stdout:1/630: dwrite d1/d3e/d46/f59 [0,4194304] 0 2024-04-24T23:13:54.601 INFO:tasks.workunit.client.0.smithi031.stdout:4/845: write d3/d20/d25/d60/d63/f81 [2593156,65159] 0 2024-04-24T23:13:54.602 INFO:tasks.workunit.client.0.smithi031.stdout:5/649: creat d0/d6/d3b/d4b/d1d/d2e/fd5 x:0 0 0 2024-04-24T23:13:54.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/650: write d0/d7/d74/f82 [1346519,95336] 0 2024-04-24T23:13:54.605 INFO:tasks.workunit.client.1.smithi040.stdout:0/466: rename df/d1e/d2b/c97 to df/d1e/d20/d61/c9a 0 2024-04-24T23:13:54.607 INFO:tasks.workunit.client.1.smithi040.stdout:0/467: write df/d12/d58/d68/d8e/f99 [898560,89135] 0 2024-04-24T23:13:54.608 INFO:tasks.workunit.client.0.smithi031.stdout:8/846: rename d8/d1a/d1d/caa to d8/d15/d37/d43/d7b/d89/c122 0 2024-04-24T23:13:54.610 INFO:tasks.workunit.client.0.smithi031.stdout:3/661: dwrite d6/d14/f43 [0,4194304] 0 2024-04-24T23:13:54.614 INFO:tasks.workunit.client.1.smithi040.stdout:5/401: fsync d2/d2f/d14/f17 0 2024-04-24T23:13:54.617 INFO:tasks.workunit.client.0.smithi031.stdout:1/631: creat d1/d33/d41/fc8 x:0 0 0 2024-04-24T23:13:54.621 INFO:tasks.workunit.client.1.smithi040.stdout:5/402: dread d2/d7/d8/db/d1e/f46 [0,4194304] 0 2024-04-24T23:13:54.623 INFO:tasks.workunit.client.1.smithi040.stdout:4/361: unlink da/d45/d48/c6a 0 2024-04-24T23:13:54.623 INFO:tasks.workunit.client.1.smithi040.stdout:0/468: dread df/d12/d1a/f2e [0,4194304] 0 2024-04-24T23:13:54.625 INFO:tasks.workunit.client.1.smithi040.stdout:0/469: write df/d12/f1f [1405125,112778] 0 2024-04-24T23:13:54.625 INFO:tasks.workunit.client.1.smithi040.stdout:7/456: getdents db/d2f/d37/d7e 0 2024-04-24T23:13:54.626 INFO:tasks.workunit.client.1.smithi040.stdout:2/485: write da/d1c/d37/d39/f73 [379346,45448] 0 2024-04-24T23:13:54.644 INFO:tasks.workunit.client.0.smithi031.stdout:9/807: dread d2/d16/f33 [0,4194304] 0 2024-04-24T23:13:54.646 INFO:tasks.workunit.client.1.smithi040.stdout:0/470: dread df/d12/f1f [8388608,4194304] 0 2024-04-24T23:13:54.649 INFO:tasks.workunit.client.1.smithi040.stdout:0/471: dread df/d12/d1a/f2e [0,4194304] 0 2024-04-24T23:13:54.710 INFO:tasks.workunit.client.0.smithi031.stdout:5/651: creat d0/d6/d3b/d4f/fd6 x:0 0 0 2024-04-24T23:13:54.712 INFO:tasks.workunit.client.1.smithi040.stdout:6/413: rename d7/d1a/d20/d51/d2f/c62 to d7/d1a/d20/d51/d1e/d5d/d52/c82 0 2024-04-24T23:13:54.713 INFO:tasks.workunit.client.0.smithi031.stdout:7/681: getdents d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2 0 2024-04-24T23:13:54.714 INFO:tasks.workunit.client.0.smithi031.stdout:7/682: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/f98 0 1 2024-04-24T23:13:54.723 INFO:tasks.workunit.client.0.smithi031.stdout:1/632: rename d1/dc/d11/d21/fa4 to d1/da/d12/d7e/fc9 0 2024-04-24T23:13:54.725 INFO:tasks.workunit.client.1.smithi040.stdout:8/398: truncate d0/d16/d19/d4f/f4e 2703394 0 2024-04-24T23:13:54.727 INFO:tasks.workunit.client.1.smithi040.stdout:1/430: mkdir da/d12/d18/d2a/d8b/d97 0 2024-04-24T23:13:54.729 INFO:tasks.workunit.client.1.smithi040.stdout:9/435: truncate d7/d5e/f7d 1957564 0 2024-04-24T23:13:54.731 INFO:tasks.workunit.client.1.smithi040.stdout:9/436: read d7/da/d17/d40/d55/d2f/d1f/d34/f47 [1712075,29049] 0 2024-04-24T23:13:54.735 INFO:tasks.workunit.client.0.smithi031.stdout:5/652: fdatasync d0/d6/d3b/d4b/d1d/d2e/dc6/f21 0 2024-04-24T23:13:54.739 INFO:tasks.workunit.client.0.smithi031.stdout:7/683: fsync d1/d2/dd/f97 0 2024-04-24T23:13:54.742 INFO:tasks.workunit.client.0.smithi031.stdout:9/808: write d2/dae/d14/d1d/f35 [367634,80890] 0 2024-04-24T23:13:54.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:54 smithi031 ceph-mon[26411]: pgmap v30: 65 pgs: 65 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 45 MiB/s rd, 170 MiB/s wr, 331 op/s 2024-04-24T23:13:54.749 INFO:tasks.workunit.client.1.smithi040.stdout:7/457: dwrite db/d2f/f9f [4194304,4194304] 0 2024-04-24T23:13:54.749 INFO:tasks.workunit.client.0.smithi031.stdout:1/633: stat d1/dc/d11/l5e 0 2024-04-24T23:13:54.750 INFO:tasks.workunit.client.1.smithi040.stdout:6/414: symlink d7/d1a/l83 0 2024-04-24T23:13:54.751 INFO:tasks.workunit.client.0.smithi031.stdout:0/792: getdents da/d20/d4d 0 2024-04-24T23:13:54.753 INFO:tasks.workunit.client.1.smithi040.stdout:3/438: getdents d1/d9 0 2024-04-24T23:13:54.754 INFO:tasks.workunit.client.0.smithi031.stdout:4/846: creat d3/d23/d69/d3f/f10c x:0 0 0 2024-04-24T23:13:54.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/847: creat d8/d15/f123 x:0 0 0 2024-04-24T23:13:54.758 INFO:tasks.workunit.client.1.smithi040.stdout:6/415: truncate d7/d1a/d20/d51/d1e/d5d/f7b 844374 0 2024-04-24T23:13:54.758 INFO:tasks.workunit.client.1.smithi040.stdout:7/458: dwrite db/f24 [0,4194304] 0 2024-04-24T23:13:54.759 INFO:tasks.workunit.client.0.smithi031.stdout:5/653: symlink d0/d6/d3b/d4b/dc/d8b/ld7 0 2024-04-24T23:13:54.763 INFO:tasks.workunit.client.1.smithi040.stdout:9/437: creat d7/da/d17/d40/d55/d61/d6c/f9b x:0 0 0 2024-04-24T23:13:54.763 INFO:tasks.workunit.client.0.smithi031.stdout:4/847: dwrite d3/d20/d25/d7e/faa [0,4194304] 0 2024-04-24T23:13:54.765 INFO:tasks.workunit.client.1.smithi040.stdout:2/486: dread da/d10/d4d/d6c/fad [0,4194304] 0 2024-04-24T23:13:54.768 INFO:tasks.workunit.client.1.smithi040.stdout:4/362: creat da/d3f/d6b/d3c/d69/f84 x:0 0 0 2024-04-24T23:13:54.770 INFO:tasks.workunit.client.0.smithi031.stdout:3/662: rmdir d6/d14/d71/db1 0 2024-04-24T23:13:54.773 INFO:tasks.workunit.client.0.smithi031.stdout:1/634: truncate d1/f85 855400 0 2024-04-24T23:13:54.776 INFO:tasks.workunit.client.1.smithi040.stdout:0/472: creat df/d1e/d20/d21/d41/d8a/f9b x:0 0 0 2024-04-24T23:13:54.778 INFO:tasks.workunit.client.1.smithi040.stdout:6/416: dread d7/d1a/d20/f28 [0,4194304] 0 2024-04-24T23:13:54.779 INFO:tasks.workunit.client.1.smithi040.stdout:0/473: write df/d1e/d20/d21/d41/f6d [771816,41112] 0 2024-04-24T23:13:54.780 INFO:tasks.workunit.client.1.smithi040.stdout:0/474: chown df/d1e/d20/d21/d41/d52 5783 1 2024-04-24T23:13:54.781 INFO:tasks.workunit.client.0.smithi031.stdout:8/848: fsync d8/d15/fd6 0 2024-04-24T23:13:54.782 INFO:tasks.workunit.client.1.smithi040.stdout:3/439: creat d1/d9/d18/d36/d61/fa0 x:0 0 0 2024-04-24T23:13:54.783 INFO:tasks.workunit.client.1.smithi040.stdout:0/475: chown df/d1e/d20/d61/f86 7841 1 2024-04-24T23:13:54.784 INFO:tasks.workunit.client.0.smithi031.stdout:5/654: rmdir d0/d7/d56/d7c 39 2024-04-24T23:13:54.784 INFO:tasks.workunit.client.1.smithi040.stdout:8/399: link d0/d8/f4d d0/d16/d19/d4f/f80 0 2024-04-24T23:13:54.784 INFO:tasks.workunit.client.1.smithi040.stdout:3/440: stat d1/d9/d18/d36/d55/f67 0 2024-04-24T23:13:54.785 INFO:tasks.workunit.client.0.smithi031.stdout:5/655: write d0/d6/d3b/d4b/fac [1439834,71115] 0 2024-04-24T23:13:54.785 INFO:tasks.workunit.client.1.smithi040.stdout:1/431: creat da/d12/d18/d2a/d8b/d97/f98 x:0 0 0 2024-04-24T23:13:54.786 INFO:tasks.workunit.client.0.smithi031.stdout:7/684: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/ced 0 2024-04-24T23:13:54.786 INFO:tasks.workunit.client.1.smithi040.stdout:5/403: dwrite d2/d7/d8/d1f/d57/d27/d65/f5a [0,4194304] 0 2024-04-24T23:13:54.786 INFO:tasks.workunit.client.0.smithi031.stdout:5/656: truncate d0/f89 898930 0 2024-04-24T23:13:54.787 INFO:tasks.workunit.client.1.smithi040.stdout:3/441: chown d1/d9/d18/f2c 117670951 1 2024-04-24T23:13:54.789 INFO:tasks.workunit.client.1.smithi040.stdout:7/459: creat db/d2f/d37/fa5 x:0 0 0 2024-04-24T23:13:54.790 INFO:tasks.workunit.client.1.smithi040.stdout:7/460: stat db/f31 0 2024-04-24T23:13:54.791 INFO:tasks.workunit.client.0.smithi031.stdout:0/793: link da/d18/d5a/d7d/fef da/d20/d4e/dc9/f109 0 2024-04-24T23:13:54.791 INFO:tasks.workunit.client.0.smithi031.stdout:5/657: dread d0/d6/d3b/d4b/f7a [0,4194304] 0 2024-04-24T23:13:54.792 INFO:tasks.workunit.client.0.smithi031.stdout:0/794: chown da/d22/d62/fb4 2596 1 2024-04-24T23:13:54.792 INFO:tasks.workunit.client.1.smithi040.stdout:7/461: read db/d2f/d37/d7e/f7f [2709496,58004] 0 2024-04-24T23:13:54.794 INFO:tasks.workunit.client.1.smithi040.stdout:8/400: dwrite d0/d16/d1b/f40 [4194304,4194304] 0 2024-04-24T23:13:54.794 INFO:tasks.workunit.client.1.smithi040.stdout:9/438: mkdir d7/d92/d9c 0 2024-04-24T23:13:54.794 INFO:tasks.workunit.client.1.smithi040.stdout:7/462: chown db/d2f/l58 255 1 2024-04-24T23:13:54.795 INFO:tasks.workunit.client.1.smithi040.stdout:7/463: readlink db/d2f/l51 0 2024-04-24T23:13:54.798 INFO:tasks.workunit.client.1.smithi040.stdout:2/487: creat da/d81/fbc x:0 0 0 2024-04-24T23:13:54.799 INFO:tasks.workunit.client.1.smithi040.stdout:2/488: fsync da/d10/f71 0 2024-04-24T23:13:54.799 INFO:tasks.workunit.client.0.smithi031.stdout:0/795: dwrite da/d20/f44 [0,4194304] 0 2024-04-24T23:13:54.800 INFO:tasks.workunit.client.1.smithi040.stdout:4/363: chown da/c11 737 1 2024-04-24T23:13:54.807 INFO:tasks.workunit.client.0.smithi031.stdout:9/809: link d2/dae/d14/d78/d84/l8c d2/dae/d14/d78/d84/d4a/d4b/daf/df5/lf7 0 2024-04-24T23:13:54.807 INFO:tasks.workunit.client.0.smithi031.stdout:7/685: mkdir d1/d2/dd/d33/d5e/d64/d6d/dae/db1/dee 0 2024-04-24T23:13:54.807 INFO:tasks.workunit.client.1.smithi040.stdout:2/489: dwrite da/d1c/d37/d39/d3f/d52/f95 [0,4194304] 0 2024-04-24T23:13:54.808 INFO:tasks.workunit.client.0.smithi031.stdout:1/635: symlink d1/da/d61/d90/lca 0 2024-04-24T23:13:54.809 INFO:tasks.workunit.client.1.smithi040.stdout:2/490: dread - da/d1c/d37/d39/d77/f68 zero size 2024-04-24T23:13:54.809 INFO:tasks.workunit.client.0.smithi031.stdout:1/636: readlink d1/da/d10/l2e 0 2024-04-24T23:13:54.810 INFO:tasks.workunit.client.1.smithi040.stdout:0/476: creat df/d1e/d20/d21/f9c x:0 0 0 2024-04-24T23:13:54.811 INFO:tasks.workunit.client.0.smithi031.stdout:5/658: symlink d0/d7/d74/ld8 0 2024-04-24T23:13:54.814 INFO:tasks.workunit.client.0.smithi031.stdout:9/810: creat d2/dae/d14/d78/d84/d4a/dab/ff8 x:0 0 0 2024-04-24T23:13:54.815 INFO:tasks.workunit.client.1.smithi040.stdout:3/442: creat d1/d9/d2e/fa1 x:0 0 0 2024-04-24T23:13:54.816 INFO:tasks.workunit.client.0.smithi031.stdout:7/686: truncate d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/fd6 504142 0 2024-04-24T23:13:54.818 INFO:tasks.workunit.client.1.smithi040.stdout:8/401: rename d0/d16/d19/d6f to d0/d8/d22/d81 0 2024-04-24T23:13:54.819 INFO:tasks.workunit.client.0.smithi031.stdout:1/637: creat d1/da/d12/d65/fcb x:0 0 0 2024-04-24T23:13:54.820 INFO:tasks.workunit.client.1.smithi040.stdout:7/464: rmdir db/d12/d20/d52/d6c 39 2024-04-24T23:13:54.821 INFO:tasks.workunit.client.0.smithi031.stdout:9/811: dwrite d2/d16/f20 [4194304,4194304] 0 2024-04-24T23:13:54.821 INFO:tasks.workunit.client.1.smithi040.stdout:3/443: dwrite d1/d9/d18/d36/d55/f56 [0,4194304] 0 2024-04-24T23:13:54.822 INFO:tasks.workunit.client.0.smithi031.stdout:9/812: readlink d2/dae/d14/d1d/d45/l61 0 2024-04-24T23:13:54.823 INFO:tasks.workunit.client.1.smithi040.stdout:3/444: write d1/d2a/f9a [347357,390] 0 2024-04-24T23:13:54.826 INFO:tasks.workunit.client.1.smithi040.stdout:5/404: rmdir d2/d7/d8/d64/d6b 39 2024-04-24T23:13:54.827 INFO:tasks.workunit.client.1.smithi040.stdout:8/402: dwrite d0/d8/d22/f62 [0,4194304] 0 2024-04-24T23:13:54.829 INFO:tasks.workunit.client.1.smithi040.stdout:0/477: unlink df/d1e/d20/d21/c7a 0 2024-04-24T23:13:54.829 INFO:tasks.workunit.client.1.smithi040.stdout:8/403: readlink d0/d16/d1b/d3f/d44/l67 0 2024-04-24T23:13:54.831 INFO:tasks.workunit.client.1.smithi040.stdout:8/404: readlink d0/d16/d1b/d3f/d44/l67 0 2024-04-24T23:13:54.831 INFO:tasks.workunit.client.1.smithi040.stdout:1/432: mknod da/d12/d18/d2a/d41/d8c/c99 0 2024-04-24T23:13:54.832 INFO:tasks.workunit.client.0.smithi031.stdout:8/849: getdents d8/d15/d37/d43/dbd 0 2024-04-24T23:13:54.833 INFO:tasks.workunit.client.1.smithi040.stdout:9/439: unlink d7/da/d17/c66 0 2024-04-24T23:13:54.834 INFO:tasks.workunit.client.1.smithi040.stdout:9/440: truncate d7/f82 410026 0 2024-04-24T23:13:54.835 INFO:tasks.workunit.client.0.smithi031.stdout:9/813: truncate d2/dae/d14/d78/d84/d4a/f71 2758542 0 2024-04-24T23:13:54.837 INFO:tasks.workunit.client.0.smithi031.stdout:7/687: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/fef x:0 0 0 2024-04-24T23:13:54.838 INFO:tasks.workunit.client.0.smithi031.stdout:1/638: fdatasync d1/d3e/da3/f77 0 2024-04-24T23:13:54.840 INFO:tasks.workunit.client.0.smithi031.stdout:5/659: rename d0/d6/d3b/d4b/dc/d8b/ld7 to d0/d6/d3b/d4b/d1d/ld9 0 2024-04-24T23:13:54.841 INFO:tasks.workunit.client.0.smithi031.stdout:8/850: creat d8/d15/d37/d43/dbd/f124 x:0 0 0 2024-04-24T23:13:54.843 INFO:tasks.workunit.client.0.smithi031.stdout:9/814: symlink d2/dae/d14/d78/d84/da6/lf9 0 2024-04-24T23:13:54.844 INFO:tasks.workunit.client.1.smithi040.stdout:5/405: rename d2/d7/d8/db/d1e/f55 to d2/d7/d8/db/d1e/f82 0 2024-04-24T23:13:54.848 INFO:tasks.workunit.client.0.smithi031.stdout:8/851: dread - d8/d1a/d74/fcb zero size 2024-04-24T23:13:54.849 INFO:tasks.workunit.client.0.smithi031.stdout:5/660: truncate d0/d6/d3b/d4b/d1d/d2e/d28/f30 1416467 0 2024-04-24T23:13:54.850 INFO:tasks.workunit.client.0.smithi031.stdout:8/852: write d8/d15/d37/d43/daf/f120 [1010599,107880] 0 2024-04-24T23:13:54.851 INFO:tasks.workunit.client.0.smithi031.stdout:5/661: read - d0/d6/d3b/d4b/d1d/d2e/fba zero size 2024-04-24T23:13:54.852 INFO:tasks.workunit.client.1.smithi040.stdout:5/406: fsync d2/d7/d8/d1f/f51 0 2024-04-24T23:13:54.855 INFO:tasks.workunit.client.1.smithi040.stdout:1/433: symlink da/d12/d18/d2a/d4d/d7d/l9a 0 2024-04-24T23:13:54.856 INFO:tasks.workunit.client.1.smithi040.stdout:1/434: chown da/l53 994633 1 2024-04-24T23:13:54.858 INFO:tasks.workunit.client.1.smithi040.stdout:3/445: rmdir d1/d9/d18/d36/d61 39 2024-04-24T23:13:54.858 INFO:tasks.workunit.client.1.smithi040.stdout:1/435: read da/d12/d14/f24 [4556132,87943] 0 2024-04-24T23:13:54.858 INFO:tasks.workunit.client.1.smithi040.stdout:4/364: getdents da/d45/d58 0 2024-04-24T23:13:54.859 INFO:tasks.workunit.client.1.smithi040.stdout:4/365: fdatasync da/f2f 0 2024-04-24T23:13:54.860 INFO:tasks.workunit.client.0.smithi031.stdout:8/853: creat d8/d15/f125 x:0 0 0 2024-04-24T23:13:54.860 INFO:tasks.workunit.client.1.smithi040.stdout:0/478: dread df/d12/d1a/f3a [0,4194304] 0 2024-04-24T23:13:54.860 INFO:tasks.workunit.client.1.smithi040.stdout:1/436: chown da/d12/f1c 31 1 2024-04-24T23:13:54.861 INFO:tasks.workunit.client.0.smithi031.stdout:8/854: chown d8/d1a/d1d/d35/d58/d5b/l67 45217 1 2024-04-24T23:13:54.862 INFO:tasks.workunit.client.1.smithi040.stdout:4/366: dread da/d3f/d6b/d2b/f39 [4194304,4194304] 0 2024-04-24T23:13:54.862 INFO:tasks.workunit.client.1.smithi040.stdout:8/405: creat d0/f82 x:0 0 0 2024-04-24T23:13:54.863 INFO:tasks.workunit.client.0.smithi031.stdout:5/662: dread d0/d6/d3b/f8d [0,4194304] 0 2024-04-24T23:13:54.864 INFO:tasks.workunit.client.1.smithi040.stdout:5/407: unlink d2/d7/d8/d1f/d57/f25 0 2024-04-24T23:13:54.867 INFO:tasks.workunit.client.1.smithi040.stdout:7/465: getdents db/d12/d3f 0 2024-04-24T23:13:54.868 INFO:tasks.workunit.client.1.smithi040.stdout:3/446: stat d1/d2d/de/d30/l93 0 2024-04-24T23:13:54.869 INFO:tasks.workunit.client.1.smithi040.stdout:3/447: write d1/d2a/f9a [848509,24741] 0 2024-04-24T23:13:54.869 INFO:tasks.workunit.client.1.smithi040.stdout:2/491: getdents da/d9f 0 2024-04-24T23:13:54.870 INFO:tasks.workunit.client.0.smithi031.stdout:2/607: sync 2024-04-24T23:13:54.873 INFO:tasks.workunit.client.1.smithi040.stdout:1/437: rename da/d12/d14/f83 to da/d12/d18/d2a/d41/d4f/f9b 0 2024-04-24T23:13:54.874 INFO:tasks.workunit.client.0.smithi031.stdout:2/608: rename d5 to d5/d6/ddf 22 2024-04-24T23:13:54.875 INFO:tasks.workunit.client.1.smithi040.stdout:4/367: read f5 [880111,7604] 0 2024-04-24T23:13:54.875 INFO:tasks.workunit.client.0.smithi031.stdout:2/609: truncate d5/d6/fc1 5209880 0 2024-04-24T23:13:54.875 INFO:tasks.workunit.client.1.smithi040.stdout:1/438: dread - da/d12/d18/d2a/d8b/d97/f98 zero size 2024-04-24T23:13:54.876 INFO:tasks.workunit.client.1.smithi040.stdout:8/406: creat d0/d8/d55/f83 x:0 0 0 2024-04-24T23:13:54.876 INFO:tasks.workunit.client.1.smithi040.stdout:2/492: dwrite da/d29/d55/f58 [0,4194304] 0 2024-04-24T23:13:54.877 INFO:tasks.workunit.client.1.smithi040.stdout:4/368: chown da/d3f/d6b/l81 151027 1 2024-04-24T23:13:54.877 INFO:tasks.workunit.client.1.smithi040.stdout:4/369: chown da 6493520 1 2024-04-24T23:13:54.877 INFO:tasks.workunit.client.1.smithi040.stdout:4/370: readlink l1 0 2024-04-24T23:13:54.878 INFO:tasks.workunit.client.1.smithi040.stdout:2/493: write da/d10/f71 [603510,43496] 0 2024-04-24T23:13:54.879 INFO:tasks.workunit.client.0.smithi031.stdout:2/610: rmdir d5/d8/d45/d5b/d49/d6f 39 2024-04-24T23:13:54.886 INFO:tasks.workunit.client.1.smithi040.stdout:0/479: symlink df/d1e/d89/l9d 0 2024-04-24T23:13:54.887 INFO:tasks.workunit.client.0.smithi031.stdout:2/611: dwrite d5/d8/d45/d5b/fbd [0,4194304] 0 2024-04-24T23:13:54.894 INFO:tasks.workunit.client.0.smithi031.stdout:2/612: truncate d5/d6/d29/d26/f63 1766407 0 2024-04-24T23:13:54.896 INFO:tasks.workunit.client.1.smithi040.stdout:8/407: chown d0/l3 11569440 1 2024-04-24T23:13:54.896 INFO:tasks.workunit.client.0.smithi031.stdout:2/613: chown d5/d8/d45/d5b/d49/l9e 6168 1 2024-04-24T23:13:54.898 INFO:tasks.workunit.client.1.smithi040.stdout:5/408: mknod d2/d7/d8/d64/d6b/c83 0 2024-04-24T23:13:54.900 INFO:tasks.workunit.client.1.smithi040.stdout:5/409: truncate d2/d2f/d14/f76 455056 0 2024-04-24T23:13:54.900 INFO:tasks.workunit.client.1.smithi040.stdout:2/494: symlink da/d9f/lbd 0 2024-04-24T23:13:54.901 INFO:tasks.workunit.client.0.smithi031.stdout:2/614: symlink d5/d6/d68/le0 0 2024-04-24T23:13:54.902 INFO:tasks.workunit.client.1.smithi040.stdout:8/408: dwrite d0/d16/d19/f5f [0,4194304] 0 2024-04-24T23:13:54.908 INFO:tasks.workunit.client.1.smithi040.stdout:0/480: creat df/d1e/d2b/f9e x:0 0 0 2024-04-24T23:13:54.910 INFO:tasks.workunit.client.0.smithi031.stdout:2/615: fsync d5/d8/d45/d5b/d49/d6f/fb6 0 2024-04-24T23:13:54.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:54 smithi040 ceph-mon[33128]: pgmap v30: 65 pgs: 65 active+clean; 2.0 GiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 45 MiB/s rd, 170 MiB/s wr, 331 op/s 2024-04-24T23:13:54.912 INFO:tasks.workunit.client.0.smithi031.stdout:7/688: truncate d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/fe8 2350391 0 2024-04-24T23:13:54.913 INFO:tasks.workunit.client.0.smithi031.stdout:1/639: dwrite d1/f66 [0,4194304] 0 2024-04-24T23:13:54.915 INFO:tasks.workunit.client.0.smithi031.stdout:7/689: stat d1/d2/dd/d33/d5e/d64/d6d/de1/d84 0 2024-04-24T23:13:54.915 INFO:tasks.workunit.client.1.smithi040.stdout:5/410: creat d2/d2f/f84 x:0 0 0 2024-04-24T23:13:54.916 INFO:tasks.workunit.client.1.smithi040.stdout:5/411: readlink d2/d7/d8/db/d1e/l30 0 2024-04-24T23:13:54.917 INFO:tasks.workunit.client.1.smithi040.stdout:5/412: chown d2/d2f/l29 1870017 1 2024-04-24T23:13:54.920 INFO:tasks.workunit.client.1.smithi040.stdout:2/495: rename da/d1c/d37/d39/d3f/d52/l70 to da/d1c/d37/d39/d3f/d52/lbe 0 2024-04-24T23:13:54.923 INFO:tasks.workunit.client.1.smithi040.stdout:8/409: rename d0/d16/d19/f37 to d0/d52/f84 0 2024-04-24T23:13:54.925 INFO:tasks.workunit.client.1.smithi040.stdout:7/466: link db/f18 db/d2f/d37/fa6 0 2024-04-24T23:13:54.925 INFO:tasks.workunit.client.0.smithi031.stdout:5/663: write d0/d6/d3b/d4b/dc/f6e [2503607,15626] 0 2024-04-24T23:13:54.926 INFO:tasks.workunit.client.1.smithi040.stdout:7/467: write db/d12/d20/f22 [3240459,58425] 0 2024-04-24T23:13:54.926 INFO:tasks.workunit.client.0.smithi031.stdout:2/616: creat d5/d6/d29/d26/db0/dc2/dad/dbe/fe1 x:0 0 0 2024-04-24T23:13:54.929 INFO:tasks.workunit.client.1.smithi040.stdout:3/448: creat d1/d9/d18/d36/fa2 x:0 0 0 2024-04-24T23:13:54.929 INFO:tasks.workunit.client.1.smithi040.stdout:3/449: chown d1/f32 203172 1 2024-04-24T23:13:54.930 INFO:tasks.workunit.client.0.smithi031.stdout:2/617: truncate d5/d8/d15/d56/d84/d75/f91 5290530 0 2024-04-24T23:13:54.930 INFO:tasks.workunit.client.1.smithi040.stdout:6/417: sync 2024-04-24T23:13:54.931 INFO:tasks.workunit.client.1.smithi040.stdout:0/481: symlink df/d12/d58/d5b/l9f 0 2024-04-24T23:13:54.933 INFO:tasks.workunit.client.1.smithi040.stdout:1/439: creat da/d12/d18/d2a/f9c x:0 0 0 2024-04-24T23:13:54.934 INFO:tasks.workunit.client.0.smithi031.stdout:5/664: rename d0/d6/d3b/d4b/d1d/d2e/dc6/l13 to d0/d6/d3b/d4b/dc/d25/d55/lda 0 2024-04-24T23:13:54.935 INFO:tasks.workunit.client.1.smithi040.stdout:3/450: dwrite d1/d2d/de/d30/f41 [0,4194304] 0 2024-04-24T23:13:54.936 INFO:tasks.workunit.client.1.smithi040.stdout:3/451: write d1/d2d/f24 [10065497,28959] 0 2024-04-24T23:13:54.939 INFO:tasks.workunit.client.1.smithi040.stdout:8/410: stat d0/d16/c20 0 2024-04-24T23:13:54.941 INFO:tasks.workunit.client.1.smithi040.stdout:8/411: write d0/d8/d22/f73 [4362092,80369] 0 2024-04-24T23:13:54.941 INFO:tasks.workunit.client.0.smithi031.stdout:5/665: dwrite d0/d6/f1a [4194304,4194304] 0 2024-04-24T23:13:54.944 INFO:tasks.workunit.client.0.smithi031.stdout:1/640: getdents d1/d33 0 2024-04-24T23:13:54.946 INFO:tasks.workunit.client.0.smithi031.stdout:1/641: truncate d1/da/d12/d65/fa0 812053 0 2024-04-24T23:13:54.953 INFO:tasks.workunit.client.1.smithi040.stdout:0/482: creat df/d12/d58/d5b/fa0 x:0 0 0 2024-04-24T23:13:54.956 INFO:tasks.workunit.client.1.smithi040.stdout:4/371: write da/d3f/d6b/d2b/f31 [1639088,110037] 0 2024-04-24T23:13:54.957 INFO:tasks.workunit.client.1.smithi040.stdout:0/483: dread df/d12/d1a/f3a [0,4194304] 0 2024-04-24T23:13:54.959 INFO:tasks.workunit.client.1.smithi040.stdout:1/440: unlink da/d12/d18/d2a/d41/d8c/c99 0 2024-04-24T23:13:54.960 INFO:tasks.workunit.client.1.smithi040.stdout:3/452: dread d1/d2d/d2b/f42 [4194304,4194304] 0 2024-04-24T23:13:54.962 INFO:tasks.workunit.client.1.smithi040.stdout:5/413: unlink d2/d7/c41 0 2024-04-24T23:13:54.966 INFO:tasks.workunit.client.0.smithi031.stdout:2/618: creat d5/d8/fe2 x:0 0 0 2024-04-24T23:13:54.966 INFO:tasks.workunit.client.1.smithi040.stdout:4/372: dwrite da/d3f/d40/d41/d60/f34 [0,4194304] 0 2024-04-24T23:13:54.968 INFO:tasks.workunit.client.1.smithi040.stdout:0/484: dwrite df/d12/d58/d5b/fa0 [0,4194304] 0 2024-04-24T23:13:54.968 INFO:tasks.workunit.client.0.smithi031.stdout:5/666: symlink d0/d6/d3b/d4b/d1d/d2e/ldb 0 2024-04-24T23:13:54.972 INFO:tasks.workunit.client.0.smithi031.stdout:1/642: mknod d1/d3e/d46/dbd/ccc 0 2024-04-24T23:13:54.977 INFO:tasks.workunit.client.1.smithi040.stdout:8/412: unlink d0/d38/f50 0 2024-04-24T23:13:54.981 INFO:tasks.workunit.client.0.smithi031.stdout:7/690: write d1/d2/f46 [1227599,26082] 0 2024-04-24T23:13:54.987 INFO:tasks.workunit.client.1.smithi040.stdout:6/418: mkdir d7/de/d84 0 2024-04-24T23:13:54.989 INFO:tasks.workunit.client.0.smithi031.stdout:1/643: creat d1/d33/d41/d9b/fcd x:0 0 0 2024-04-24T23:13:54.999 INFO:tasks.workunit.client.1.smithi040.stdout:4/373: read da/d3f/d6b/d2b/f35 [1354883,5131] 0 2024-04-24T23:13:55.000 INFO:tasks.workunit.client.1.smithi040.stdout:4/374: write da/d3f/f56 [6041106,109157] 0 2024-04-24T23:13:55.001 INFO:tasks.workunit.client.1.smithi040.stdout:4/375: write da/d45/d58/f73 [1038406,21906] 0 2024-04-24T23:13:55.008 INFO:tasks.workunit.client.1.smithi040.stdout:6/419: mkdir d7/d1a/d20/d51/d1e/d46/d85 0 2024-04-24T23:13:55.008 INFO:tasks.workunit.client.1.smithi040.stdout:6/420: readlink d7/d1a/d56/l59 0 2024-04-24T23:13:55.008 INFO:tasks.workunit.client.1.smithi040.stdout:7/468: truncate db/d12/d20/f22 616039 0 2024-04-24T23:13:55.011 INFO:tasks.workunit.client.1.smithi040.stdout:3/453: unlink d1/d9/d18/d36/d61/f54 0 2024-04-24T23:13:55.012 INFO:tasks.workunit.client.1.smithi040.stdout:3/454: write d1/d9/d18/d36/d55/f89 [262395,87608] 0 2024-04-24T23:13:55.013 INFO:tasks.workunit.client.1.smithi040.stdout:1/441: symlink da/d12/d14/d25/d5e/l9d 0 2024-04-24T23:13:55.014 INFO:tasks.workunit.client.0.smithi031.stdout:5/667: dwrite d0/d6/d32/d6b/fa0 [0,4194304] 0 2024-04-24T23:13:55.015 INFO:tasks.workunit.client.1.smithi040.stdout:5/414: write d2/d2f/d14/f17 [2985963,17572] 0 2024-04-24T23:13:55.017 INFO:tasks.workunit.client.1.smithi040.stdout:5/415: truncate d2/d7/d8/db/f75 4514974 0 2024-04-24T23:13:55.022 INFO:tasks.workunit.client.1.smithi040.stdout:0/485: symlink df/d14/la1 0 2024-04-24T23:13:55.022 INFO:tasks.workunit.client.1.smithi040.stdout:0/486: dread - df/d12/d58/f7d zero size 2024-04-24T23:13:55.024 INFO:tasks.workunit.client.1.smithi040.stdout:3/455: dread d1/d9/d15/f53 [0,4194304] 0 2024-04-24T23:13:55.026 INFO:tasks.workunit.client.1.smithi040.stdout:6/421: creat d7/d1a/d20/d51/d1e/f86 x:0 0 0 2024-04-24T23:13:55.026 INFO:tasks.workunit.client.1.smithi040.stdout:6/422: truncate d7/d1a/d20/f23 4840177 0 2024-04-24T23:13:55.028 INFO:tasks.workunit.client.1.smithi040.stdout:9/441: sync 2024-04-24T23:13:55.029 INFO:tasks.workunit.client.1.smithi040.stdout:3/456: dread d1/f79 [0,4194304] 0 2024-04-24T23:13:55.030 INFO:tasks.workunit.client.1.smithi040.stdout:9/442: truncate d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f89 1032405 0 2024-04-24T23:13:55.031 INFO:tasks.workunit.client.1.smithi040.stdout:3/457: truncate d1/d9/d18/d36/d61/fa0 953135 0 2024-04-24T23:13:55.034 INFO:tasks.workunit.client.1.smithi040.stdout:0/487: rmdir df/d1e/d20/d21 39 2024-04-24T23:13:55.035 INFO:tasks.workunit.client.1.smithi040.stdout:0/488: write df/d12/d58/d5b/fa0 [4950755,80905] 0 2024-04-24T23:13:55.048 INFO:tasks.workunit.client.1.smithi040.stdout:3/458: symlink d1/d2d/la3 0 2024-04-24T23:13:55.048 INFO:tasks.workunit.client.1.smithi040.stdout:3/459: read d1/d2d/d44/f52 [3120024,61108] 0 2024-04-24T23:13:55.049 INFO:tasks.workunit.client.1.smithi040.stdout:3/460: readlink d1/d2d/la3 0 2024-04-24T23:13:55.049 INFO:tasks.workunit.client.1.smithi040.stdout:9/443: fdatasync d7/da/d17/d40/d55/d2f/d28/f29 0 2024-04-24T23:13:55.050 INFO:tasks.workunit.client.1.smithi040.stdout:9/444: readlink d7/da/d17/d40/d55/d2f/d1f/l80 0 2024-04-24T23:13:55.052 INFO:tasks.workunit.client.1.smithi040.stdout:7/469: mkdir db/d12/d20/d52/d6c/d8f/da7 0 2024-04-24T23:13:55.052 INFO:tasks.workunit.client.1.smithi040.stdout:8/413: truncate d0/d16/d1b/f54 885521 0 2024-04-24T23:13:55.054 INFO:tasks.workunit.client.1.smithi040.stdout:0/489: creat df/d12/d1a/d71/fa2 x:0 0 0 2024-04-24T23:13:55.055 INFO:tasks.workunit.client.1.smithi040.stdout:8/414: dread d0/d8/d55/f64 [0,4194304] 0 2024-04-24T23:13:55.056 INFO:tasks.workunit.client.1.smithi040.stdout:8/415: chown d0/d16/d1b/d3f/d44/l67 437733368 1 2024-04-24T23:13:55.056 INFO:tasks.workunit.client.1.smithi040.stdout:8/416: fsync d0/d16/d19/d4f/f72 0 2024-04-24T23:13:55.061 INFO:tasks.workunit.client.1.smithi040.stdout:4/376: link da/c50 da/d1f/c85 0 2024-04-24T23:13:55.062 INFO:tasks.workunit.client.1.smithi040.stdout:6/423: write d7/d1a/d20/d51/f17 [713351,96384] 0 2024-04-24T23:13:55.068 INFO:tasks.workunit.client.0.smithi031.stdout:7/691: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/ff0 x:0 0 0 2024-04-24T23:13:55.069 INFO:tasks.workunit.client.1.smithi040.stdout:9/445: rmdir d7/d92 39 2024-04-24T23:13:55.071 INFO:tasks.workunit.client.1.smithi040.stdout:7/470: mkdir db/d2f/da8 0 2024-04-24T23:13:55.071 INFO:tasks.workunit.client.1.smithi040.stdout:4/377: dwrite da/d3f/d40/d41/d60/f82 [0,4194304] 0 2024-04-24T23:13:55.072 INFO:tasks.workunit.client.0.smithi031.stdout:1/644: write d1/da/d12/d7e/d7b/f6a [5173022,38450] 0 2024-04-24T23:13:55.073 INFO:tasks.workunit.client.0.smithi031.stdout:1/645: fdatasync d1/d3e/d46/f59 0 2024-04-24T23:13:55.073 INFO:tasks.workunit.client.1.smithi040.stdout:1/442: dwrite f2 [0,4194304] 0 2024-04-24T23:13:55.075 INFO:tasks.workunit.client.1.smithi040.stdout:6/424: dread d7/d1a/d20/d51/f79 [0,4194304] 0 2024-04-24T23:13:55.080 INFO:tasks.workunit.client.1.smithi040.stdout:8/417: mknod d0/d16/d19/d71/c85 0 2024-04-24T23:13:55.081 INFO:tasks.workunit.client.0.smithi031.stdout:5/668: readlink d0/d6/d3b/d4b/dc/l48 0 2024-04-24T23:13:55.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/669: write d0/d7/d56/f61 [1722019,37105] 0 2024-04-24T23:13:55.083 INFO:tasks.workunit.client.1.smithi040.stdout:4/378: dwrite da/d45/d58/f73 [0,4194304] 0 2024-04-24T23:13:55.094 INFO:tasks.workunit.client.1.smithi040.stdout:3/461: write d1/d2a/d8d/f7b [8729641,39604] 0 2024-04-24T23:13:55.095 INFO:tasks.workunit.client.0.smithi031.stdout:5/670: fdatasync d0/d6/d3b/d4f/f98 0 2024-04-24T23:13:55.097 INFO:tasks.workunit.client.0.smithi031.stdout:2/619: dwrite d5/d28/d4e/fd6 [4194304,4194304] 0 2024-04-24T23:13:55.099 INFO:tasks.workunit.client.1.smithi040.stdout:7/471: write db/d12/d3f/d7a/fa3 [2319825,89498] 0 2024-04-24T23:13:55.101 INFO:tasks.workunit.client.0.smithi031.stdout:1/646: link d1/da/d12/d7e/d7b/f6a d1/d3e/d46/dbd/fce 0 2024-04-24T23:13:55.104 INFO:tasks.workunit.client.1.smithi040.stdout:9/446: creat d7/da/d17/d40/d55/d2f/d1f/d34/f9d x:0 0 0 2024-04-24T23:13:55.108 INFO:tasks.workunit.client.1.smithi040.stdout:0/490: rename df/d12/d58/d68 to df/d1e/d20/d21/d51/da3 0 2024-04-24T23:13:55.110 INFO:tasks.workunit.client.1.smithi040.stdout:0/491: chown df/d12/d1a/d71/f28 65231 1 2024-04-24T23:13:55.111 INFO:tasks.workunit.client.1.smithi040.stdout:8/418: creat d0/d8/dc/f86 x:0 0 0 2024-04-24T23:13:55.111 INFO:tasks.workunit.client.1.smithi040.stdout:4/379: mknod da/d3f/d6b/d3c/c86 0 2024-04-24T23:13:55.112 INFO:tasks.workunit.client.1.smithi040.stdout:8/419: chown d0/f82 1469 1 2024-04-24T23:13:55.112 INFO:tasks.workunit.client.1.smithi040.stdout:1/443: dwrite da/d12/d18/f1d [4194304,4194304] 0 2024-04-24T23:13:55.113 INFO:tasks.workunit.client.0.smithi031.stdout:5/671: unlink d0/d6/d3b/d4b/d1d/d2e/dc6/f5d 0 2024-04-24T23:13:55.113 INFO:tasks.workunit.client.0.smithi031.stdout:7/692: getdents d1/d2/dd/d33/d5e/d64/d6d/de1/d75 0 2024-04-24T23:13:55.114 INFO:tasks.workunit.client.1.smithi040.stdout:4/380: dread - da/d3f/d40/d47/d71/f74 zero size 2024-04-24T23:13:55.114 INFO:tasks.workunit.client.1.smithi040.stdout:2/496: sync 2024-04-24T23:13:55.115 INFO:tasks.workunit.client.1.smithi040.stdout:3/462: symlink d1/d2a/d8d/d7c/la4 0 2024-04-24T23:13:55.116 INFO:tasks.workunit.client.1.smithi040.stdout:4/381: read da/d3f/d40/d41/d60/f82 [3171979,71181] 0 2024-04-24T23:13:55.117 INFO:tasks.workunit.client.1.smithi040.stdout:2/497: fsync da/d1c/d37/d8a/f90 0 2024-04-24T23:13:55.118 INFO:tasks.workunit.client.1.smithi040.stdout:2/498: chown da/d10/d4d/da4 6989 1 2024-04-24T23:13:55.119 INFO:tasks.workunit.client.1.smithi040.stdout:2/499: readlink da/d1c/l44 0 2024-04-24T23:13:55.119 INFO:tasks.workunit.client.1.smithi040.stdout:9/447: symlink d7/da/d17/d40/d55/d2f/d1f/d4e/l9e 0 2024-04-24T23:13:55.120 INFO:tasks.workunit.client.1.smithi040.stdout:2/500: truncate da/d1c/d37/f8e 1003965 0 2024-04-24T23:13:55.121 INFO:tasks.workunit.client.0.smithi031.stdout:7/693: unlink d1/d2/dd/d33/d5e/d64/d6d/de1/da0/c1f 0 2024-04-24T23:13:55.123 INFO:tasks.workunit.client.1.smithi040.stdout:6/425: rename d7/d1a/d20/d51/d70 to d7/d1a/d20/d51/d1e/d46/d69/d87 0 2024-04-24T23:13:55.124 INFO:tasks.workunit.client.0.smithi031.stdout:5/672: rename d0/d6/d3b/d4b/d1d/d2e/d28/lbc to d0/d6/d3b/d4b/dc/d25/d55/ldc 0 2024-04-24T23:13:55.129 INFO:tasks.workunit.client.0.smithi031.stdout:5/673: symlink d0/d7/d74/ldd 0 2024-04-24T23:13:55.129 INFO:tasks.workunit.client.0.smithi031.stdout:5/674: chown d0/d7/c1f 16 1 2024-04-24T23:13:55.130 INFO:tasks.workunit.client.1.smithi040.stdout:8/420: mkdir d0/d16/d19/d4f/d87 0 2024-04-24T23:13:55.131 INFO:tasks.workunit.client.0.smithi031.stdout:5/675: truncate d0/d6/d3b/d4b/d1d/d2e/dc6/f63 4756216 0 2024-04-24T23:13:55.132 INFO:tasks.workunit.client.1.smithi040.stdout:1/444: write da/d12/d18/d2a/f5a [477473,112395] 0 2024-04-24T23:13:55.137 INFO:tasks.workunit.client.0.smithi031.stdout:2/620: truncate d5/d8/d45/d5b/d49/f9f 983798 0 2024-04-24T23:13:55.139 INFO:tasks.workunit.client.1.smithi040.stdout:9/448: dread d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f48 [0,4194304] 0 2024-04-24T23:13:55.140 INFO:tasks.workunit.client.1.smithi040.stdout:9/449: readlink d7/da/d17/d40/d55/d2f/l20 0 2024-04-24T23:13:55.141 INFO:tasks.workunit.client.0.smithi031.stdout:5/676: dread d0/d6/d3b/d4b/dc/d25/d55/f75 [0,4194304] 0 2024-04-24T23:13:55.142 INFO:tasks.workunit.client.1.smithi040.stdout:6/426: mknod d7/d1a/d20/d51/d2f/c88 0 2024-04-24T23:13:55.143 INFO:tasks.workunit.client.1.smithi040.stdout:6/427: dread - d7/d1a/d20/f61 zero size 2024-04-24T23:13:55.144 INFO:tasks.workunit.client.1.smithi040.stdout:0/492: write df/d12/f2a [7921810,76464] 0 2024-04-24T23:13:55.148 INFO:tasks.workunit.client.1.smithi040.stdout:3/463: write d1/d9/d18/d36/f82 [128422,75304] 0 2024-04-24T23:13:55.149 INFO:tasks.workunit.client.1.smithi040.stdout:3/464: chown d1/d2d/c85 90361892 1 2024-04-24T23:13:55.149 INFO:tasks.workunit.client.0.smithi031.stdout:7/694: dwrite d1/d2/f2a [0,4194304] 0 2024-04-24T23:13:55.150 INFO:tasks.workunit.client.1.smithi040.stdout:4/382: write da/d3f/d6b/f32 [1541849,8217] 0 2024-04-24T23:13:55.151 INFO:tasks.workunit.client.1.smithi040.stdout:1/445: symlink da/d12/d55/l9e 0 2024-04-24T23:13:55.155 INFO:tasks.workunit.client.0.smithi031.stdout:2/621: write d5/d6/d29/d26/db0/dc2/fab [248486,85456] 0 2024-04-24T23:13:55.162 INFO:tasks.workunit.client.0.smithi031.stdout:5/677: dwrite d0/d6/d3b/d4b/d1d/d2e/dc6/f21 [4194304,4194304] 0 2024-04-24T23:13:55.163 INFO:tasks.workunit.client.1.smithi040.stdout:9/450: mknod d7/da/d17/d40/d55/d61/d6c/c9f 0 2024-04-24T23:13:55.164 INFO:tasks.workunit.client.1.smithi040.stdout:2/501: truncate da/d10/d4d/f64 3906606 0 2024-04-24T23:13:55.164 INFO:tasks.workunit.client.1.smithi040.stdout:2/502: fdatasync da/d10/d7a/f8d 0 2024-04-24T23:13:55.165 INFO:tasks.workunit.client.1.smithi040.stdout:2/503: read - da/d1c/d37/d39/f7c zero size 2024-04-24T23:13:55.166 INFO:tasks.workunit.client.1.smithi040.stdout:8/421: mknod d0/d16/d19/d4f/d87/c88 0 2024-04-24T23:13:55.168 INFO:tasks.workunit.client.1.smithi040.stdout:3/465: creat d1/d2a/d6a/fa5 x:0 0 0 2024-04-24T23:13:55.171 INFO:tasks.workunit.client.0.smithi031.stdout:5/678: creat d0/d7/d56/fde x:0 0 0 2024-04-24T23:13:55.172 INFO:tasks.workunit.client.1.smithi040.stdout:9/451: mknod d7/da/d17/ca0 0 2024-04-24T23:13:55.173 INFO:tasks.workunit.client.0.smithi031.stdout:7/695: rename d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/ld0 to d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/lf1 0 2024-04-24T23:13:55.175 INFO:tasks.workunit.client.1.smithi040.stdout:6/428: mkdir d7/de/d89 0 2024-04-24T23:13:55.176 INFO:tasks.workunit.client.1.smithi040.stdout:6/429: dread - d7/d1a/d20/d51/d1e/f72 zero size 2024-04-24T23:13:55.178 INFO:tasks.workunit.client.0.smithi031.stdout:7/696: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/df2 0 2024-04-24T23:13:55.183 INFO:tasks.workunit.client.1.smithi040.stdout:3/466: rename d1/d2d/de/f70 to d1/d9/d18/d36/d61/fa6 0 2024-04-24T23:13:55.185 INFO:tasks.workunit.client.1.smithi040.stdout:1/446: dwrite da/d12/f21 [0,4194304] 0 2024-04-24T23:13:55.186 INFO:tasks.workunit.client.1.smithi040.stdout:9/452: readlink d7/da/d17/d40/d55/d2f/d1f/d31/l7b 0 2024-04-24T23:13:55.191 INFO:tasks.workunit.client.1.smithi040.stdout:4/383: write f9 [4641623,88179] 0 2024-04-24T23:13:55.193 INFO:tasks.workunit.client.1.smithi040.stdout:1/447: rmdir da/d12/d55 39 2024-04-24T23:13:55.194 INFO:tasks.workunit.client.1.smithi040.stdout:9/453: rmdir d7/da/d17/d40 39 2024-04-24T23:13:55.195 INFO:tasks.workunit.client.1.smithi040.stdout:0/493: getdents df/d12 0 2024-04-24T23:13:55.196 INFO:tasks.workunit.client.0.smithi031.stdout:7/697: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/f3c [4056061,66814] 0 2024-04-24T23:13:55.197 INFO:tasks.workunit.client.1.smithi040.stdout:6/430: dwrite d7/de/ff [0,4194304] 0 2024-04-24T23:13:55.198 INFO:tasks.workunit.client.0.smithi031.stdout:7/698: write d1/d2/dd/d33/fcb [1054785,78092] 0 2024-04-24T23:13:55.198 INFO:tasks.workunit.client.1.smithi040.stdout:1/448: write da/d12/d18/d2a/d4d/f59 [725014,111256] 0 2024-04-24T23:13:55.199 INFO:tasks.workunit.client.1.smithi040.stdout:6/431: write d7/f3f [490138,8182] 0 2024-04-24T23:13:55.199 INFO:tasks.workunit.client.1.smithi040.stdout:0/494: fsync df/d1e/f2f 0 2024-04-24T23:13:55.199 INFO:tasks.workunit.client.1.smithi040.stdout:4/384: symlink da/d3f/d40/l87 0 2024-04-24T23:13:55.202 INFO:tasks.workunit.client.1.smithi040.stdout:8/422: truncate d0/d16/d1b/f40 1803940 0 2024-04-24T23:13:55.207 INFO:tasks.workunit.client.1.smithi040.stdout:3/467: write d1/d2d/d44/f6c [928723,85691] 0 2024-04-24T23:13:55.208 INFO:tasks.workunit.client.1.smithi040.stdout:8/423: mkdir d0/d16/d19/d71/d89 0 2024-04-24T23:13:55.210 INFO:tasks.workunit.client.1.smithi040.stdout:0/495: dread df/d1e/d20/d21/d51/da3/d8e/f99 [0,4194304] 0 2024-04-24T23:13:55.210 INFO:tasks.workunit.client.0.smithi031.stdout:6/780: sync 2024-04-24T23:13:55.213 INFO:tasks.workunit.client.0.smithi031.stdout:7/699: write d1/d2/f12 [941874,88067] 0 2024-04-24T23:13:55.214 INFO:tasks.workunit.client.1.smithi040.stdout:6/432: truncate d7/d1a/d20/d51/f26 324874 0 2024-04-24T23:13:55.214 INFO:tasks.workunit.client.1.smithi040.stdout:1/449: dwrite da/d12/d18/f1d [0,4194304] 0 2024-04-24T23:13:55.214 INFO:tasks.workunit.client.1.smithi040.stdout:3/468: mkdir d1/d9/d18/da7 0 2024-04-24T23:13:55.214 INFO:tasks.workunit.client.1.smithi040.stdout:8/424: read d0/d28/f39 [698907,23715] 0 2024-04-24T23:13:55.215 INFO:tasks.workunit.client.0.smithi031.stdout:6/781: write d13/d15/d1d/d23/d24/f50 [2587421,51390] 0 2024-04-24T23:13:55.215 INFO:tasks.workunit.client.1.smithi040.stdout:4/385: getdents da/d3f/d40/d41 0 2024-04-24T23:13:55.216 INFO:tasks.workunit.client.1.smithi040.stdout:1/450: fdatasync da/f71 0 2024-04-24T23:13:55.216 INFO:tasks.workunit.client.0.smithi031.stdout:7/700: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/cf3 0 2024-04-24T23:13:55.222 INFO:tasks.workunit.client.1.smithi040.stdout:5/416: sync 2024-04-24T23:13:55.224 INFO:tasks.workunit.client.1.smithi040.stdout:3/469: unlink d1/d9/d18/d36/d55/d62/f65 0 2024-04-24T23:13:55.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/701: rename d1/d2/dd/d33/d5e/d64/d6d/dae/db1/dee to d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/df2/df4 0 2024-04-24T23:13:55.227 INFO:tasks.workunit.client.1.smithi040.stdout:0/496: dwrite df/d1e/d20/d21/d51/f8c [0,4194304] 0 2024-04-24T23:13:55.227 INFO:tasks.workunit.client.1.smithi040.stdout:4/386: mknod da/d3f/d6b/d3c/c88 0 2024-04-24T23:13:55.230 INFO:tasks.workunit.client.1.smithi040.stdout:9/454: truncate d7/da/d17/d40/d55/d2f/d1f/d31/f36 1075565 0 2024-04-24T23:13:55.231 INFO:tasks.workunit.client.1.smithi040.stdout:1/451: dwrite da/d35/d38/f6e [0,4194304] 0 2024-04-24T23:13:55.231 INFO:tasks.workunit.client.0.smithi031.stdout:6/782: write d13/d1b/d32/f6d [3452712,95402] 0 2024-04-24T23:13:55.232 INFO:tasks.workunit.client.1.smithi040.stdout:6/433: getdents d7/de/d84 0 2024-04-24T23:13:55.233 INFO:tasks.workunit.client.1.smithi040.stdout:9/455: fdatasync d7/da/d17/d40/d55/d2f/d1f/d31/f35 0 2024-04-24T23:13:55.233 INFO:tasks.workunit.client.1.smithi040.stdout:8/425: truncate d0/d28/f39 2505913 0 2024-04-24T23:13:55.234 INFO:tasks.workunit.client.0.smithi031.stdout:7/702: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/db8/cf5 0 2024-04-24T23:13:55.237 INFO:tasks.workunit.client.0.smithi031.stdout:6/783: creat d13/d15/d1d/d23/d2d/dbf/f108 x:0 0 0 2024-04-24T23:13:55.239 INFO:tasks.workunit.client.0.smithi031.stdout:7/703: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/ff6 x:0 0 0 2024-04-24T23:13:55.239 INFO:tasks.workunit.client.1.smithi040.stdout:1/452: dwrite da/d12/d18/d2a/d41/f6a [0,4194304] 0 2024-04-24T23:13:55.241 INFO:tasks.workunit.client.0.smithi031.stdout:6/784: truncate d13/d1b/d32/d9c/ddb/df1/ffe 466922 0 2024-04-24T23:13:55.242 INFO:tasks.workunit.client.1.smithi040.stdout:3/470: fdatasync d1/d2d/de/d30/d37/f74 0 2024-04-24T23:13:55.242 INFO:tasks.workunit.client.0.smithi031.stdout:6/785: read - d13/d15/d1d/d23/d24/f105 zero size 2024-04-24T23:13:55.244 INFO:tasks.workunit.client.0.smithi031.stdout:6/786: write d13/d15/d1d/d23/d2d/fff [241400,94719] 0 2024-04-24T23:13:55.245 INFO:tasks.workunit.client.1.smithi040.stdout:0/497: symlink df/d1e/d20/d21/d41/la4 0 2024-04-24T23:13:55.245 INFO:tasks.workunit.client.1.smithi040.stdout:4/387: symlink da/d3f/d6b/d2b/l89 0 2024-04-24T23:13:55.246 INFO:tasks.workunit.client.0.smithi031.stdout:7/704: rmdir d1 39 2024-04-24T23:13:55.247 INFO:tasks.workunit.client.1.smithi040.stdout:3/471: dread d1/d2d/f60 [0,4194304] 0 2024-04-24T23:13:55.249 INFO:tasks.workunit.client.0.smithi031.stdout:6/787: chown d13/d1b/d32/db4/c57 224 1 2024-04-24T23:13:55.253 INFO:tasks.workunit.client.1.smithi040.stdout:8/426: fsync d0/d52/f84 0 2024-04-24T23:13:55.255 INFO:tasks.workunit.client.0.smithi031.stdout:6/788: rmdir d13/d1b/d32 39 2024-04-24T23:13:55.257 INFO:tasks.workunit.client.1.smithi040.stdout:7/472: sync 2024-04-24T23:13:55.257 INFO:tasks.workunit.client.1.smithi040.stdout:7/473: stat db/d12/f3e 0 2024-04-24T23:13:55.258 INFO:tasks.workunit.client.1.smithi040.stdout:5/417: write d2/d7/f74 [764608,34038] 0 2024-04-24T23:13:55.261 INFO:tasks.workunit.client.0.smithi031.stdout:7/705: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea/cf7 0 2024-04-24T23:13:55.262 INFO:tasks.workunit.client.1.smithi040.stdout:1/453: readlink da/d12/d18/l29 0 2024-04-24T23:13:55.263 INFO:tasks.workunit.client.0.smithi031.stdout:7/706: readlink d1/l55 0 2024-04-24T23:13:55.264 INFO:tasks.workunit.client.0.smithi031.stdout:7/707: truncate d1/d2/dd/d33/d5e/d64/fce 584651 0 2024-04-24T23:13:55.266 INFO:tasks.workunit.client.1.smithi040.stdout:0/498: unlink df/l3c 0 2024-04-24T23:13:55.267 INFO:tasks.workunit.client.1.smithi040.stdout:0/499: fdatasync df/d1e/d20/d21/f9c 0 2024-04-24T23:13:55.267 INFO:tasks.workunit.client.1.smithi040.stdout:4/388: truncate da/f1c 5161785 0 2024-04-24T23:13:55.268 INFO:tasks.workunit.client.1.smithi040.stdout:0/500: truncate df/d12/d58/d5b/fa0 5078548 0 2024-04-24T23:13:55.268 INFO:tasks.workunit.client.1.smithi040.stdout:3/472: rename d1/d9/d18/f2c to d1/d9/d2e/fa8 0 2024-04-24T23:13:55.269 INFO:tasks.workunit.client.0.smithi031.stdout:7/708: mkdir d1/df8 0 2024-04-24T23:13:55.270 INFO:tasks.workunit.client.0.smithi031.stdout:6/789: dwrite d13/d15/d1d/d23/d24/d43/fcb [0,4194304] 0 2024-04-24T23:13:55.273 INFO:tasks.workunit.client.0.smithi031.stdout:7/709: dread - d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/ff0 zero size 2024-04-24T23:13:55.273 INFO:tasks.workunit.client.1.smithi040.stdout:6/434: mknod d7/d1a/d20/d51/d1e/d46/d85/c8a 0 2024-04-24T23:13:55.273 INFO:tasks.workunit.client.1.smithi040.stdout:7/474: symlink db/d2f/d37/d7e/la9 0 2024-04-24T23:13:55.275 INFO:tasks.workunit.client.1.smithi040.stdout:5/418: mknod d2/d7/d8/d1f/d57/d27/d3c/d43/c85 0 2024-04-24T23:13:55.275 INFO:tasks.workunit.client.0.smithi031.stdout:6/790: truncate d13/d15/d1d/d23/d44/d47/d69/fd4 1269678 0 2024-04-24T23:13:55.277 INFO:tasks.workunit.client.1.smithi040.stdout:1/454: rmdir da/d12/d18/d2a/d8b 39 2024-04-24T23:13:55.278 INFO:tasks.workunit.client.1.smithi040.stdout:1/455: dread - da/d12/d18/d2a/d41/f70 zero size 2024-04-24T23:13:55.283 INFO:tasks.workunit.client.0.smithi031.stdout:7/710: dwrite d1/d2/dd/f23 [4194304,4194304] 0 2024-04-24T23:13:55.288 INFO:tasks.workunit.client.0.smithi031.stdout:7/711: symlink d1/d2/dd/d33/d5e/lf9 0 2024-04-24T23:13:55.289 INFO:tasks.workunit.client.1.smithi040.stdout:8/427: symlink d0/d16/d19/d71/d76/l8a 0 2024-04-24T23:13:55.290 INFO:tasks.workunit.client.1.smithi040.stdout:7/475: chown db/f1e 1253 1 2024-04-24T23:13:55.292 INFO:tasks.workunit.client.1.smithi040.stdout:3/473: dread d1/d9/f20 [0,4194304] 0 2024-04-24T23:13:55.292 INFO:tasks.workunit.client.1.smithi040.stdout:7/476: chown db/d12/f8c 1779 1 2024-04-24T23:13:55.301 INFO:tasks.workunit.client.1.smithi040.stdout:8/428: dwrite d0/d8/d22/d81/f74 [0,4194304] 0 2024-04-24T23:13:55.305 INFO:tasks.workunit.client.1.smithi040.stdout:7/477: dwrite db/d2f/f5b [0,4194304] 0 2024-04-24T23:13:55.360 INFO:tasks.workunit.client.0.smithi031.stdout:7/712: chown d1/d2/dd/d33/d5e/d64/faa 52855978 1 2024-04-24T23:13:55.361 INFO:tasks.workunit.client.1.smithi040.stdout:6/435: rmdir d7/d1a/d20/d51/d1e/d46/d85 39 2024-04-24T23:13:55.363 INFO:tasks.workunit.client.0.smithi031.stdout:7/713: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa 0 2024-04-24T23:13:55.364 INFO:tasks.workunit.client.0.smithi031.stdout:7/714: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/l37 105518 1 2024-04-24T23:13:55.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/715: unlink d1/d2/dd/d33/d5e/d64/d6d/de1/c89 0 2024-04-24T23:13:55.370 INFO:tasks.workunit.client.0.smithi031.stdout:7/716: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/l68 0 2024-04-24T23:13:55.373 INFO:tasks.workunit.client.1.smithi040.stdout:7/478: rmdir db/d12/d20 39 2024-04-24T23:13:55.374 INFO:tasks.workunit.client.0.smithi031.stdout:7/717: mknod d1/d2/dd/d33/cfb 0 2024-04-24T23:13:55.379 INFO:tasks.workunit.client.0.smithi031.stdout:7/718: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/cfc 0 2024-04-24T23:13:55.382 INFO:tasks.workunit.client.0.smithi031.stdout:7/719: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/ffd x:0 0 0 2024-04-24T23:13:55.383 INFO:tasks.workunit.client.0.smithi031.stdout:7/720: readlink d1/d2/d60/l8c 0 2024-04-24T23:13:55.385 INFO:tasks.workunit.client.0.smithi031.stdout:7/721: dread d1/d2/dd/d33/f4e [0,4194304] 0 2024-04-24T23:13:55.386 INFO:tasks.workunit.client.1.smithi040.stdout:6/436: mkdir d7/de/d89/d8b 0 2024-04-24T23:13:55.386 INFO:tasks.workunit.client.0.smithi031.stdout:7/722: chown d1/d2/f2d 187837139 1 2024-04-24T23:13:55.387 INFO:tasks.workunit.client.0.smithi031.stdout:1/647: sync 2024-04-24T23:13:55.387 INFO:tasks.workunit.client.0.smithi031.stdout:3/663: sync 2024-04-24T23:13:55.387 INFO:tasks.workunit.client.0.smithi031.stdout:8/855: sync 2024-04-24T23:13:55.387 INFO:tasks.workunit.client.0.smithi031.stdout:0/796: sync 2024-04-24T23:13:55.387 INFO:tasks.workunit.client.0.smithi031.stdout:9/815: sync 2024-04-24T23:13:55.388 INFO:tasks.workunit.client.0.smithi031.stdout:4/848: sync 2024-04-24T23:13:55.388 INFO:tasks.workunit.client.1.smithi040.stdout:5/419: creat d2/d2f/d14/f86 x:0 0 0 2024-04-24T23:13:55.389 INFO:tasks.workunit.client.0.smithi031.stdout:9/816: chown d2/dae/d14/d1d 132786771 1 2024-04-24T23:13:55.389 INFO:tasks.workunit.client.1.smithi040.stdout:4/389: getdents da/d3f/d6b 0 2024-04-24T23:13:55.390 INFO:tasks.workunit.client.0.smithi031.stdout:1/648: truncate d1/da/d12/d7e/fc9 842354 0 2024-04-24T23:13:55.393 INFO:tasks.workunit.client.0.smithi031.stdout:3/664: dread d6/df/d16/d2a/f68 [4194304,4194304] 0 2024-04-24T23:13:55.395 INFO:tasks.workunit.client.0.smithi031.stdout:1/649: dread d1/d33/d36/fa7 [0,4194304] 0 2024-04-24T23:13:55.400 INFO:tasks.workunit.client.0.smithi031.stdout:4/849: mknod d3/da3/d104/c10d 0 2024-04-24T23:13:55.402 INFO:tasks.workunit.client.0.smithi031.stdout:1/650: dwrite d1/d33/d41/d9b/da8/fa9 [0,4194304] 0 2024-04-24T23:13:55.408 INFO:tasks.workunit.client.1.smithi040.stdout:8/429: getdents d0/d16/d1b/d3f 0 2024-04-24T23:13:55.410 INFO:tasks.workunit.client.0.smithi031.stdout:9/817: truncate d2/fdc 264635 0 2024-04-24T23:13:55.412 INFO:tasks.workunit.client.0.smithi031.stdout:9/818: chown d2/dae/d14/d1d/d45/l61 13 1 2024-04-24T23:13:55.412 INFO:tasks.workunit.client.0.smithi031.stdout:9/819: stat d2/dae/f2f 0 2024-04-24T23:13:55.418 INFO:tasks.workunit.client.1.smithi040.stdout:0/501: rename df/d1e/d20/d21/c4f to df/d1e/d20/d61/ca5 0 2024-04-24T23:13:55.418 INFO:tasks.workunit.client.1.smithi040.stdout:8/430: symlink d0/d16/d19/d4f/l8b 0 2024-04-24T23:13:55.419 INFO:tasks.workunit.client.0.smithi031.stdout:9/820: dread d2/dae/d14/d1d/f35 [0,4194304] 0 2024-04-24T23:13:55.422 INFO:tasks.workunit.client.0.smithi031.stdout:3/665: fsync d6/df/d16/d2a/d95/d9b/d29/fa3 0 2024-04-24T23:13:55.428 INFO:tasks.workunit.client.0.smithi031.stdout:4/850: dread d3/f1b [0,4194304] 0 2024-04-24T23:13:55.428 INFO:tasks.workunit.client.0.smithi031.stdout:1/651: rename d1/dc to d1/da/d12/d65/dcf 0 2024-04-24T23:13:55.430 INFO:tasks.workunit.client.1.smithi040.stdout:2/504: sync 2024-04-24T23:13:55.431 INFO:tasks.workunit.client.1.smithi040.stdout:8/431: creat d0/d8/dc/f8c x:0 0 0 2024-04-24T23:13:55.432 INFO:tasks.workunit.client.1.smithi040.stdout:2/505: truncate da/d1c/f5c 4650491 0 2024-04-24T23:13:55.432 INFO:tasks.workunit.client.1.smithi040.stdout:8/432: chown d0/d16/d19/l60 249864359 1 2024-04-24T23:13:55.433 INFO:tasks.workunit.client.1.smithi040.stdout:3/474: creat d1/d9/d18/d36/d55/fa9 x:0 0 0 2024-04-24T23:13:55.433 INFO:tasks.workunit.client.1.smithi040.stdout:8/433: chown d0/d16/d19/c1d 4 1 2024-04-24T23:13:55.434 INFO:tasks.workunit.client.1.smithi040.stdout:8/434: chown d0/d8/d22/c3d 52674875 1 2024-04-24T23:13:55.439 INFO:tasks.workunit.client.0.smithi031.stdout:9/821: readlink d2/d16/dbe/d7d/lf0 0 2024-04-24T23:13:55.446 INFO:tasks.workunit.client.0.smithi031.stdout:3/666: unlink d6/f6b 0 2024-04-24T23:13:55.449 INFO:tasks.workunit.client.1.smithi040.stdout:7/479: rmdir db/d12/d20/d52/d6c 39 2024-04-24T23:13:55.451 INFO:tasks.workunit.client.0.smithi031.stdout:7/723: getdents d1/d2 0 2024-04-24T23:13:55.452 INFO:tasks.workunit.client.0.smithi031.stdout:3/667: dread d6/d14/d91/f99 [0,4194304] 0 2024-04-24T23:13:55.453 INFO:tasks.workunit.client.0.smithi031.stdout:4/851: dread d3/d20/d2d/d44/d5b/d5d/f97 [0,4194304] 0 2024-04-24T23:13:55.458 INFO:tasks.workunit.client.0.smithi031.stdout:7/724: dwrite d1/d2/f2d [0,4194304] 0 2024-04-24T23:13:55.479 INFO:tasks.workunit.client.0.smithi031.stdout:4/852: mknod d3/c10e 0 2024-04-24T23:13:55.480 INFO:tasks.workunit.client.0.smithi031.stdout:2/622: sync 2024-04-24T23:13:55.484 INFO:tasks.workunit.client.0.smithi031.stdout:7/725: rename d1/d2/dd/lab to d1/d2/dd/d33/lfe 0 2024-04-24T23:13:55.485 INFO:tasks.workunit.client.1.smithi040.stdout:2/506: write da/d10/d4d/d6c/fad [4738025,89162] 0 2024-04-24T23:13:55.485 INFO:tasks.workunit.client.0.smithi031.stdout:1/652: link d1/da/d12/d65/dcf/f70 d1/da/d12/d65/dcf/d11/fd0 0 2024-04-24T23:13:55.486 INFO:tasks.workunit.client.0.smithi031.stdout:1/653: stat d1/d33/d41/d9b/la5 0 2024-04-24T23:13:55.490 INFO:tasks.workunit.client.0.smithi031.stdout:9/822: getdents d2/dae/d14/d78/d84/d4a/d4b/daf 0 2024-04-24T23:13:55.491 INFO:tasks.workunit.client.0.smithi031.stdout:9/823: stat d2/dae/d14/d1d/d45/d65/ldd 0 2024-04-24T23:13:55.493 INFO:tasks.workunit.client.0.smithi031.stdout:7/726: mknod d1/d2/dd/d33/d5e/d64/d6d/da1/cff 0 2024-04-24T23:13:55.495 INFO:tasks.workunit.client.0.smithi031.stdout:1/654: creat d1/d33/d41/d9b/da8/fd1 x:0 0 0 2024-04-24T23:13:55.496 INFO:tasks.workunit.client.1.smithi040.stdout:8/435: symlink d0/d16/d19/d4f/d77/l8d 0 2024-04-24T23:13:55.499 INFO:tasks.workunit.client.1.smithi040.stdout:7/480: truncate db/f18 310117 0 2024-04-24T23:13:55.502 INFO:tasks.workunit.client.1.smithi040.stdout:2/507: mknod da/d1c/d37/d39/d3f/db2/cbf 0 2024-04-24T23:13:55.504 INFO:tasks.workunit.client.1.smithi040.stdout:8/436: creat d0/d16/d19/d4f/f8e x:0 0 0 2024-04-24T23:13:55.507 INFO:tasks.workunit.client.1.smithi040.stdout:2/508: mknod da/d10/d15/daa/cc0 0 2024-04-24T23:13:55.508 INFO:tasks.workunit.client.0.smithi031.stdout:1/655: dread d1/d1d/f7a [0,4194304] 0 2024-04-24T23:13:55.508 INFO:tasks.workunit.client.1.smithi040.stdout:2/509: readlink da/l17 0 2024-04-24T23:13:55.509 INFO:tasks.workunit.client.1.smithi040.stdout:8/437: mkdir d0/d16/d1b/d58/d8f 0 2024-04-24T23:13:55.510 INFO:tasks.workunit.client.0.smithi031.stdout:7/727: mkdir d1/de4/d100 0 2024-04-24T23:13:55.511 INFO:tasks.workunit.client.0.smithi031.stdout:7/728: chown d1/f13 7 1 2024-04-24T23:13:55.514 INFO:tasks.workunit.client.1.smithi040.stdout:8/438: mkdir d0/d16/d19/d71/d89/d90 0 2024-04-24T23:13:55.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/729: mkdir d1/d2/dd/d33/d5e/d64/d6d/dae/dd7/d101 0 2024-04-24T23:13:55.519 INFO:tasks.workunit.client.0.smithi031.stdout:1/656: rename d1/da/d12/d65/dcf/d11/fd0 to d1/da/d10/d52/fd2 0 2024-04-24T23:13:55.521 INFO:tasks.workunit.client.1.smithi040.stdout:8/439: truncate d0/d16/d19/d4f/f4e 262681 0 2024-04-24T23:13:55.522 INFO:tasks.workunit.client.1.smithi040.stdout:2/510: dread da/d10/d4d/f64 [0,4194304] 0 2024-04-24T23:13:55.522 INFO:tasks.workunit.client.0.smithi031.stdout:1/657: read - d1/d33/d36/f71 zero size 2024-04-24T23:13:55.526 INFO:tasks.workunit.client.0.smithi031.stdout:7/730: rename d1/d2/dd/d33/d5e/d64/fca to d1/d2/d83/dc1/f102 0 2024-04-24T23:13:55.527 INFO:tasks.workunit.client.1.smithi040.stdout:8/440: fsync d0/d8/dc/d34/f3c 0 2024-04-24T23:13:55.528 INFO:tasks.workunit.client.0.smithi031.stdout:1/658: creat d1/d3e/d46/fd3 x:0 0 0 2024-04-24T23:13:55.528 INFO:tasks.workunit.client.0.smithi031.stdout:1/659: chown d1/f66 0 1 2024-04-24T23:13:55.529 INFO:tasks.workunit.client.1.smithi040.stdout:2/511: symlink da/d1c/d37/d8a/lc1 0 2024-04-24T23:13:55.532 INFO:tasks.workunit.client.0.smithi031.stdout:0/797: rmdir da/d20/d4e 39 2024-04-24T23:13:55.532 INFO:tasks.workunit.client.0.smithi031.stdout:1/660: read d1/d1d/f6d [464340,37671] 0 2024-04-24T23:13:55.534 INFO:tasks.workunit.client.1.smithi040.stdout:8/441: mkdir d0/d16/d19/d71/d89/d90/d91 0 2024-04-24T23:13:55.534 INFO:tasks.workunit.client.0.smithi031.stdout:8/856: write d8/d1a/ff0 [351224,124035] 0 2024-04-24T23:13:55.534 INFO:tasks.workunit.client.1.smithi040.stdout:8/442: write d0/d16/f57 [191465,21973] 0 2024-04-24T23:13:55.536 INFO:tasks.workunit.client.0.smithi031.stdout:1/661: fdatasync d1/d1d/f7a 0 2024-04-24T23:13:55.538 INFO:tasks.workunit.client.0.smithi031.stdout:1/662: write d1/da/fad [280054,45994] 0 2024-04-24T23:13:55.539 INFO:tasks.workunit.client.1.smithi040.stdout:8/443: symlink d0/d52/l92 0 2024-04-24T23:13:55.540 INFO:tasks.workunit.client.0.smithi031.stdout:1/663: chown d1/d3e/da3/fc7 352 1 2024-04-24T23:13:55.540 INFO:tasks.workunit.client.1.smithi040.stdout:1/456: rename da/d12/d14/d58/f73 to da/d35/f9f 0 2024-04-24T23:13:55.542 INFO:tasks.workunit.client.0.smithi031.stdout:8/857: dwrite d8/d15/f108 [0,4194304] 0 2024-04-24T23:13:55.543 INFO:tasks.workunit.client.1.smithi040.stdout:8/444: symlink d0/d16/d19/d71/d76/l93 0 2024-04-24T23:13:55.544 INFO:tasks.workunit.client.0.smithi031.stdout:0/798: creat da/f10a x:0 0 0 2024-04-24T23:13:55.545 INFO:tasks.workunit.client.1.smithi040.stdout:8/445: read - d0/d16/d19/d4f/f72 zero size 2024-04-24T23:13:55.546 INFO:tasks.workunit.client.0.smithi031.stdout:0/799: truncate da/d11/d6a/fd0 306270 0 2024-04-24T23:13:55.546 INFO:tasks.workunit.client.1.smithi040.stdout:5/420: dwrite d2/d7/d8/db/d1e/f46 [0,4194304] 0 2024-04-24T23:13:55.549 INFO:tasks.workunit.client.1.smithi040.stdout:4/390: rename da/c11 to da/d3f/d40/d41/d62/c8a 0 2024-04-24T23:13:55.549 INFO:tasks.workunit.client.1.smithi040.stdout:4/391: readlink l1 0 2024-04-24T23:13:55.549 INFO:tasks.workunit.client.1.smithi040.stdout:7/481: rename db/d12/d20/d52/d6c/d8f to db/d12/d20/d52/d6c/d8f/daa 22 2024-04-24T23:13:55.551 INFO:tasks.workunit.client.0.smithi031.stdout:1/664: dread d1/f85 [0,4194304] 0 2024-04-24T23:13:55.552 INFO:tasks.workunit.client.0.smithi031.stdout:8/858: truncate d8/d15/d37/d43/d5c/db5/fe6 690075 0 2024-04-24T23:13:55.553 INFO:tasks.workunit.client.0.smithi031.stdout:1/665: stat d1/da/d12/d7e/f9e 0 2024-04-24T23:13:55.553 INFO:tasks.workunit.client.0.smithi031.stdout:8/859: fsync d8/d15/d37/d43/d7b/dd3/f119 0 2024-04-24T23:13:55.554 INFO:tasks.workunit.client.0.smithi031.stdout:8/860: readlink d8/l30 0 2024-04-24T23:13:55.554 INFO:tasks.workunit.client.0.smithi031.stdout:0/800: mknod da/d86/da3/c10b 0 2024-04-24T23:13:55.556 INFO:tasks.workunit.client.0.smithi031.stdout:8/861: write d8/d15/f108 [2135181,113093] 0 2024-04-24T23:13:55.556 INFO:tasks.workunit.client.1.smithi040.stdout:8/446: creat d0/d16/d1b/f94 x:0 0 0 2024-04-24T23:13:55.557 INFO:tasks.workunit.client.1.smithi040.stdout:8/447: chown d0/lb 3 1 2024-04-24T23:13:55.561 INFO:tasks.workunit.client.1.smithi040.stdout:4/392: mknod da/d3f/d40/d41/c8b 0 2024-04-24T23:13:55.565 INFO:tasks.workunit.client.1.smithi040.stdout:8/448: creat d0/d16/d1b/f95 x:0 0 0 2024-04-24T23:13:55.570 INFO:tasks.workunit.client.1.smithi040.stdout:9/456: write d7/da/d17/d40/d55/d2f/d1f/d31/f36 [1687920,67430] 0 2024-04-24T23:13:55.572 INFO:tasks.workunit.client.1.smithi040.stdout:8/449: creat d0/d8/dc/f96 x:0 0 0 2024-04-24T23:13:55.574 INFO:tasks.workunit.client.1.smithi040.stdout:9/457: dread d7/f57 [0,4194304] 0 2024-04-24T23:13:55.574 INFO:tasks.workunit.client.1.smithi040.stdout:8/450: read d0/d8/f4d [80044,110069] 0 2024-04-24T23:13:55.575 INFO:tasks.workunit.client.1.smithi040.stdout:5/421: getdents d2/d7/d8/d1f/d57 0 2024-04-24T23:13:55.575 INFO:tasks.workunit.client.1.smithi040.stdout:7/482: getdents db/d12/d62 0 2024-04-24T23:13:55.577 INFO:tasks.workunit.client.1.smithi040.stdout:9/458: write d7/da/d17/d40/d55/d61/d6c/f7a [4382540,2385] 0 2024-04-24T23:13:55.578 INFO:tasks.workunit.client.1.smithi040.stdout:9/459: chown d7/da/l14 4897 1 2024-04-24T23:13:55.579 INFO:tasks.workunit.client.1.smithi040.stdout:5/422: mknod d2/d7/d8/db/d1e/c87 0 2024-04-24T23:13:55.579 INFO:tasks.workunit.client.1.smithi040.stdout:9/460: chown d7/d5e/l65 229153 1 2024-04-24T23:13:55.580 INFO:tasks.workunit.client.1.smithi040.stdout:7/483: creat db/d12/d20/d52/d44/fab x:0 0 0 2024-04-24T23:13:55.581 INFO:tasks.workunit.client.1.smithi040.stdout:7/484: dread - db/d12/d3f/d7a/f9e zero size 2024-04-24T23:13:55.582 INFO:tasks.workunit.client.1.smithi040.stdout:5/423: rmdir d2/d2f/d14 39 2024-04-24T23:13:55.583 INFO:tasks.workunit.client.1.smithi040.stdout:7/485: dread db/d12/f8c [0,4194304] 0 2024-04-24T23:13:55.584 INFO:tasks.workunit.client.1.smithi040.stdout:8/451: creat d0/d16/d19/d71/d89/d90/d91/f97 x:0 0 0 2024-04-24T23:13:55.585 INFO:tasks.workunit.client.1.smithi040.stdout:8/452: chown d0/d8/dc/f17 231684 1 2024-04-24T23:13:55.586 INFO:tasks.workunit.client.1.smithi040.stdout:8/453: read - d0/d8/dc/f86 zero size 2024-04-24T23:13:55.586 INFO:tasks.workunit.client.1.smithi040.stdout:8/454: chown d0/d52 3485929 1 2024-04-24T23:13:55.586 INFO:tasks.workunit.client.1.smithi040.stdout:5/424: symlink d2/d7/d8/d1f/l88 0 2024-04-24T23:13:55.590 INFO:tasks.workunit.client.1.smithi040.stdout:5/425: mkdir d2/d7/d8/d6d/d89 0 2024-04-24T23:13:55.590 INFO:tasks.workunit.client.1.smithi040.stdout:9/461: dread d7/da/d17/d40/d55/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:13:55.591 INFO:tasks.workunit.client.1.smithi040.stdout:0/502: symlink df/d1e/d20/la6 0 2024-04-24T23:13:55.592 INFO:tasks.workunit.client.0.smithi031.stdout:3/668: write d6/d14/d1d/f8b [310985,38515] 0 2024-04-24T23:13:55.593 INFO:tasks.workunit.client.1.smithi040.stdout:8/455: dwrite d0/d8/d22/f62 [0,4194304] 0 2024-04-24T23:13:55.594 INFO:tasks.workunit.client.1.smithi040.stdout:9/462: mkdir d7/d5e/da1 0 2024-04-24T23:13:55.595 INFO:tasks.workunit.client.1.smithi040.stdout:9/463: chown d7/da/d17/l1d 0 1 2024-04-24T23:13:55.596 INFO:tasks.workunit.client.1.smithi040.stdout:0/503: dread df/f63 [0,4194304] 0 2024-04-24T23:13:55.597 INFO:tasks.workunit.client.0.smithi031.stdout:3/669: creat d6/df/d16/d53/fc6 x:0 0 0 2024-04-24T23:13:55.599 INFO:tasks.workunit.client.1.smithi040.stdout:5/426: creat d2/d7/d8/d1f/d57/d3e/f8a x:0 0 0 2024-04-24T23:13:55.599 INFO:tasks.workunit.client.0.smithi031.stdout:3/670: fdatasync d6/df/d16/d2a/d95/d9b/d1a/d25/faf 0 2024-04-24T23:13:55.601 INFO:tasks.workunit.client.1.smithi040.stdout:8/456: truncate d0/d8/dc/f17 5005629 0 2024-04-24T23:13:55.601 INFO:tasks.workunit.client.1.smithi040.stdout:7/486: dread db/d12/d62/d8a/f8b [0,4194304] 0 2024-04-24T23:13:55.605 INFO:tasks.workunit.client.0.smithi031.stdout:3/671: dwrite d6/d14/f58 [0,4194304] 0 2024-04-24T23:13:55.606 INFO:tasks.workunit.client.1.smithi040.stdout:9/464: mknod d7/d5e/da1/ca2 0 2024-04-24T23:13:55.608 INFO:tasks.workunit.client.1.smithi040.stdout:8/457: dwrite d0/f82 [0,4194304] 0 2024-04-24T23:13:55.611 INFO:tasks.workunit.client.1.smithi040.stdout:7/487: creat db/d12/d20/d52/d44/fac x:0 0 0 2024-04-24T23:13:55.614 INFO:tasks.workunit.client.0.smithi031.stdout:3/672: truncate d6/f8 6051279 0 2024-04-24T23:13:55.614 INFO:tasks.workunit.client.1.smithi040.stdout:0/504: dread df/d12/d1a/f4e [0,4194304] 0 2024-04-24T23:13:55.615 INFO:tasks.workunit.client.1.smithi040.stdout:7/488: chown db/d12/d20/d56/l75 21076 1 2024-04-24T23:13:55.617 INFO:tasks.workunit.client.1.smithi040.stdout:7/489: chown db 15 1 2024-04-24T23:13:55.618 INFO:tasks.workunit.client.1.smithi040.stdout:9/465: dwrite d7/da/d17/d40/d55/d2f/d28/f88 [0,4194304] 0 2024-04-24T23:13:55.623 INFO:tasks.workunit.client.0.smithi031.stdout:3/673: symlink d6/df/d16/d2a/d95/d9b/d89/lc7 0 2024-04-24T23:13:55.624 INFO:tasks.workunit.client.1.smithi040.stdout:3/475: mkdir d1/d9/d15/d45/daa 0 2024-04-24T23:13:55.628 INFO:tasks.workunit.client.1.smithi040.stdout:6/437: mkdir d7/d1a/d20/d8c 0 2024-04-24T23:13:55.629 INFO:tasks.workunit.client.1.smithi040.stdout:6/438: read d7/f60 [3990560,27323] 0 2024-04-24T23:13:55.631 INFO:tasks.workunit.client.1.smithi040.stdout:9/466: rmdir d7/da/d17/d40 39 2024-04-24T23:13:55.631 INFO:tasks.workunit.client.1.smithi040.stdout:6/439: write d7/d1a/d20/d51/d1e/f49 [727807,6698] 0 2024-04-24T23:13:55.632 INFO:tasks.workunit.client.1.smithi040.stdout:7/490: symlink db/d12/d62/d8a/lad 0 2024-04-24T23:13:55.634 INFO:tasks.workunit.client.1.smithi040.stdout:3/476: fdatasync d1/d2d/d2b/f42 0 2024-04-24T23:13:55.638 INFO:tasks.workunit.client.0.smithi031.stdout:4/853: dwrite d3/d23/d69/d3f/fe1 [0,4194304] 0 2024-04-24T23:13:55.640 INFO:tasks.workunit.client.1.smithi040.stdout:0/505: rename df/d12/d4b/d62 to df/d1e/da7 0 2024-04-24T23:13:55.641 INFO:tasks.workunit.client.0.smithi031.stdout:9/824: write d2/dae/d14/d78/d84/d4a/d4b/f52 [721612,35187] 0 2024-04-24T23:13:55.641 INFO:tasks.workunit.client.1.smithi040.stdout:6/440: creat d7/d1a/d20/d51/d1e/d46/d69/f8d x:0 0 0 2024-04-24T23:13:55.642 INFO:tasks.workunit.client.1.smithi040.stdout:9/467: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [0,4194304] 0 2024-04-24T23:13:55.646 INFO:tasks.workunit.client.1.smithi040.stdout:9/468: read - d7/da/d17/d40/d55/d61/d6c/f91 zero size 2024-04-24T23:13:55.646 INFO:tasks.workunit.client.1.smithi040.stdout:7/491: creat db/d12/d20/d52/d6c/fae x:0 0 0 2024-04-24T23:13:55.648 INFO:tasks.workunit.client.0.smithi031.stdout:4/854: mknod d3/d23/d37/c10f 0 2024-04-24T23:13:55.649 INFO:tasks.workunit.client.1.smithi040.stdout:3/477: symlink d1/d2d/de/d30/d37/lab 0 2024-04-24T23:13:55.655 INFO:tasks.workunit.client.1.smithi040.stdout:0/506: mknod df/d1e/d20/d21/d51/da3/d8e/ca8 0 2024-04-24T23:13:55.657 INFO:tasks.workunit.client.1.smithi040.stdout:8/458: fsync d0/d16/d19/d4f/f4e 0 2024-04-24T23:13:55.668 INFO:tasks.workunit.client.0.smithi031.stdout:7/731: dwrite d1/d2/dd/d33/f77 [0,4194304] 0 2024-04-24T23:13:55.672 INFO:tasks.workunit.client.0.smithi031.stdout:7/732: creat d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea/f103 x:0 0 0 2024-04-24T23:13:55.674 INFO:tasks.workunit.client.0.smithi031.stdout:7/733: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/la9 0 2024-04-24T23:13:55.677 INFO:tasks.workunit.client.0.smithi031.stdout:8/862: write d8/d15/d37/d43/d7b/d87/dad/fda [428973,61964] 0 2024-04-24T23:13:55.682 INFO:tasks.workunit.client.0.smithi031.stdout:1/666: dwrite d1/da/d12/d65/dcf/d11/f13 [0,4194304] 0 2024-04-24T23:13:55.683 INFO:tasks.workunit.client.0.smithi031.stdout:8/863: mkdir d8/d126 0 2024-04-24T23:13:55.683 INFO:tasks.workunit.client.0.smithi031.stdout:0/801: dwrite da/d11/f25 [4194304,4194304] 0 2024-04-24T23:13:55.686 INFO:tasks.workunit.client.0.smithi031.stdout:8/864: dread - d8/d1a/d1d/d35/d58/f101 zero size 2024-04-24T23:13:55.692 INFO:tasks.workunit.client.0.smithi031.stdout:8/865: dread - d8/d1a/d1d/d35/d58/d5b/fe0 zero size 2024-04-24T23:13:55.693 INFO:tasks.workunit.client.1.smithi040.stdout:8/459: mkdir d0/d16/d19/d71/d98 0 2024-04-24T23:13:55.694 INFO:tasks.workunit.client.1.smithi040.stdout:4/393: truncate da/d3f/d6b/d3c/f3e 3063293 0 2024-04-24T23:13:55.696 INFO:tasks.workunit.client.1.smithi040.stdout:4/394: readlink da/d3f/d6b/d2b/l44 0 2024-04-24T23:13:55.697 INFO:tasks.workunit.client.0.smithi031.stdout:8/866: mkdir d8/d1a/d74/d127 0 2024-04-24T23:13:55.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/667: getdents d1/da/d12/d65/dcf/d11 0 2024-04-24T23:13:55.703 INFO:tasks.workunit.client.0.smithi031.stdout:0/802: link da/d86/da3/d104/lcc da/l10c 0 2024-04-24T23:13:55.706 INFO:tasks.workunit.client.0.smithi031.stdout:8/867: dread d8/d1a/d1d/d50/dc0/d9d/fc6 [0,4194304] 0 2024-04-24T23:13:55.709 INFO:tasks.workunit.client.0.smithi031.stdout:1/668: rename d1/da/d12/d7e/d7b/f6a to d1/d3e/d46/dbd/fd4 0 2024-04-24T23:13:55.712 INFO:tasks.workunit.client.0.smithi031.stdout:0/803: dwrite da/ffd [0,4194304] 0 2024-04-24T23:13:55.717 INFO:tasks.workunit.client.0.smithi031.stdout:8/868: dwrite d8/d1a/d1d/d35/d58/f101 [0,4194304] 0 2024-04-24T23:13:55.717 INFO:tasks.workunit.client.0.smithi031.stdout:0/804: mkdir da/d18/d5a/d7d/d10d 0 2024-04-24T23:13:55.721 INFO:tasks.workunit.client.0.smithi031.stdout:3/674: dwrite d6/df/f34 [0,4194304] 0 2024-04-24T23:13:55.725 INFO:tasks.workunit.client.0.smithi031.stdout:3/675: chown d6/d14/d1d/l49 2078 1 2024-04-24T23:13:55.726 INFO:tasks.workunit.client.0.smithi031.stdout:3/676: write d6/d14/db2/fc3 [796839,122349] 0 2024-04-24T23:13:55.729 INFO:tasks.workunit.client.0.smithi031.stdout:4/855: write d3/f71 [505018,25719] 0 2024-04-24T23:13:55.731 INFO:tasks.workunit.client.0.smithi031.stdout:3/677: rmdir d6/d14/d91 39 2024-04-24T23:13:55.733 INFO:tasks.workunit.client.0.smithi031.stdout:4/856: unlink d3/c10e 0 2024-04-24T23:13:55.734 INFO:tasks.workunit.client.0.smithi031.stdout:9/825: dwrite d2/dae/d14/d1d/d45/d56/f66 [0,4194304] 0 2024-04-24T23:13:55.737 INFO:tasks.workunit.client.0.smithi031.stdout:4/857: dread - d3/d20/d25/d60/d63/db9/ff1 zero size 2024-04-24T23:13:55.739 INFO:tasks.workunit.client.0.smithi031.stdout:9/826: mkdir d2/dae/d14/d78/d84/d4a/dab/dad/db5/dfa 0 2024-04-24T23:13:55.742 INFO:tasks.workunit.client.0.smithi031.stdout:7/734: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/f5b [0,4194304] 0 2024-04-24T23:13:55.748 INFO:tasks.workunit.client.0.smithi031.stdout:9/827: truncate d2/dae/d14/d78/d84/d4a/f91 142223 0 2024-04-24T23:13:55.751 INFO:tasks.workunit.client.0.smithi031.stdout:7/735: dwrite d1/f53 [0,4194304] 0 2024-04-24T23:13:55.755 INFO:tasks.workunit.client.0.smithi031.stdout:5/679: sync 2024-04-24T23:13:55.755 INFO:tasks.workunit.client.0.smithi031.stdout:6/791: sync 2024-04-24T23:13:55.755 INFO:tasks.workunit.client.1.smithi040.stdout:7/492: link db/d12/d20/d52/c34 db/d12/d62/d8a/caf 0 2024-04-24T23:13:55.757 INFO:tasks.workunit.client.1.smithi040.stdout:1/457: dwrite da/d12/f4a [0,4194304] 0 2024-04-24T23:13:55.758 INFO:tasks.workunit.client.1.smithi040.stdout:5/427: write d2/d7/d8/d1f/d57/d27/d3c/d43/f6f [907052,75717] 0 2024-04-24T23:13:55.759 INFO:tasks.workunit.client.0.smithi031.stdout:6/792: chown d13/d15/d1d/d23/d24/f77 2818378 1 2024-04-24T23:13:55.759 INFO:tasks.workunit.client.0.smithi031.stdout:9/828: creat d2/dae/d14/ffb x:0 0 0 2024-04-24T23:13:55.761 INFO:tasks.workunit.client.1.smithi040.stdout:8/460: dwrite d0/d16/d19/d4f/f8e [0,4194304] 0 2024-04-24T23:13:55.762 INFO:tasks.workunit.client.1.smithi040.stdout:3/478: write d1/d2d/de/d30/d37/f74 [870745,119231] 0 2024-04-24T23:13:55.763 INFO:tasks.workunit.client.0.smithi031.stdout:7/736: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/da0/ddc/c104 0 2024-04-24T23:13:55.764 INFO:tasks.workunit.client.1.smithi040.stdout:5/428: truncate d2/d7/d8/d1f/f6e 1588890 0 2024-04-24T23:13:55.764 INFO:tasks.workunit.client.1.smithi040.stdout:0/507: getdents df/d1e/d20/d21/d41/d8a 0 2024-04-24T23:13:55.765 INFO:tasks.workunit.client.1.smithi040.stdout:4/395: write da/d45/d48/f7d [170361,39083] 0 2024-04-24T23:13:55.766 INFO:tasks.workunit.client.1.smithi040.stdout:8/461: write d0/d16/f57 [679342,79984] 0 2024-04-24T23:13:55.766 INFO:tasks.workunit.client.0.smithi031.stdout:5/680: symlink d0/d6/d3b/d4f/ldf 0 2024-04-24T23:13:55.770 INFO:tasks.workunit.client.0.smithi031.stdout:1/669: dwrite d1/d3e/d46/dbd/fce [4194304,4194304] 0 2024-04-24T23:13:55.774 INFO:tasks.workunit.client.0.smithi031.stdout:0/805: dwrite da/d18/d1a/d37/d41/f47 [0,4194304] 0 2024-04-24T23:13:55.774 INFO:tasks.workunit.client.1.smithi040.stdout:3/479: creat d1/d9/d2e/fac x:0 0 0 2024-04-24T23:13:55.775 INFO:tasks.workunit.client.1.smithi040.stdout:4/396: dwrite da/d3f/d6b/d2b/f55 [0,4194304] 0 2024-04-24T23:13:55.776 INFO:tasks.workunit.client.1.smithi040.stdout:4/397: write da/d3f/d6b/f32 [1169293,68642] 0 2024-04-24T23:13:55.777 INFO:tasks.workunit.client.0.smithi031.stdout:7/737: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/f87 17 1 2024-04-24T23:13:55.779 INFO:tasks.workunit.client.0.smithi031.stdout:8/869: dwrite d8/f18 [8388608,4194304] 0 2024-04-24T23:13:55.781 INFO:tasks.workunit.client.0.smithi031.stdout:9/829: creat d2/dae/d14/d78/d84/d7e/dc7/ffc x:0 0 0 2024-04-24T23:13:55.782 INFO:tasks.workunit.client.0.smithi031.stdout:2/623: sync 2024-04-24T23:13:55.782 INFO:tasks.workunit.client.1.smithi040.stdout:5/429: creat d2/d7/d8/d64/d6b/f8b x:0 0 0 2024-04-24T23:13:55.782 INFO:tasks.workunit.client.0.smithi031.stdout:5/681: creat d0/d6/fe0 x:0 0 0 2024-04-24T23:13:55.784 INFO:tasks.workunit.client.1.smithi040.stdout:8/462: truncate d0/d8/f29 3326240 0 2024-04-24T23:13:55.788 INFO:tasks.workunit.client.1.smithi040.stdout:7/493: truncate db/d12/d62/d8a/f8b 27880 0 2024-04-24T23:13:55.789 INFO:tasks.workunit.client.1.smithi040.stdout:1/458: write da/f1f [4786105,89897] 0 2024-04-24T23:13:55.790 INFO:tasks.workunit.client.0.smithi031.stdout:2/624: chown d5/d6/f1f 1 1 2024-04-24T23:13:55.791 INFO:tasks.workunit.client.1.smithi040.stdout:1/459: stat da/d35/d38 0 2024-04-24T23:13:55.792 INFO:tasks.workunit.client.1.smithi040.stdout:2/512: sync 2024-04-24T23:13:55.793 INFO:tasks.workunit.client.0.smithi031.stdout:6/793: chown d13/d1b/d32/db4/d79/dd7/laf 16352 1 2024-04-24T23:13:55.795 INFO:tasks.workunit.client.0.smithi031.stdout:5/682: write d0/d6/d3b/d4b/f7a [2989316,3275] 0 2024-04-24T23:13:55.796 INFO:tasks.workunit.client.0.smithi031.stdout:3/678: dwrite d6/d14/d91/f99 [0,4194304] 0 2024-04-24T23:13:55.796 INFO:tasks.workunit.client.1.smithi040.stdout:0/508: fdatasync df/f39 0 2024-04-24T23:13:55.799 INFO:tasks.workunit.client.0.smithi031.stdout:4/858: dwrite d3/d20/d2d/f50 [0,4194304] 0 2024-04-24T23:13:55.799 INFO:tasks.workunit.client.1.smithi040.stdout:5/430: unlink d2/d7/f74 0 2024-04-24T23:13:55.799 INFO:tasks.workunit.client.1.smithi040.stdout:7/494: dwrite db/d12/d73/f9c [0,4194304] 0 2024-04-24T23:13:55.801 INFO:tasks.workunit.client.1.smithi040.stdout:7/495: readlink db/d2f/d37/l38 0 2024-04-24T23:13:55.801 INFO:tasks.workunit.client.0.smithi031.stdout:4/859: readlink d3/d20/l66 0 2024-04-24T23:13:55.801 INFO:tasks.workunit.client.0.smithi031.stdout:6/794: stat d13/d15/d1d/d23/d44/d47/d69/dd5/feb 0 2024-04-24T23:13:55.803 INFO:tasks.workunit.client.0.smithi031.stdout:5/683: stat d0/d6/d3b/d4b/dc/d8b/lb0 0 2024-04-24T23:13:55.805 INFO:tasks.workunit.client.0.smithi031.stdout:0/806: rename da/d18/d1a/d37/c78 to da/d11/d6a/c10e 0 2024-04-24T23:13:55.806 INFO:tasks.workunit.client.1.smithi040.stdout:3/480: creat d1/d9/d18/da7/fad x:0 0 0 2024-04-24T23:13:55.806 INFO:tasks.workunit.client.0.smithi031.stdout:3/679: dread d6/d14/d1d/f8b [0,4194304] 0 2024-04-24T23:13:55.807 INFO:tasks.workunit.client.1.smithi040.stdout:3/481: fdatasync d1/d9/d18/d36/d55/f67 0 2024-04-24T23:13:55.808 INFO:tasks.workunit.client.1.smithi040.stdout:2/513: symlink da/d10/lc2 0 2024-04-24T23:13:55.815 INFO:tasks.workunit.client.0.smithi031.stdout:4/860: read d3/f75 [4814791,82046] 0 2024-04-24T23:13:55.817 INFO:tasks.workunit.client.1.smithi040.stdout:2/514: dwrite da/d29/f6b [0,4194304] 0 2024-04-24T23:13:55.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/684: dwrite d0/d6/d3b/d4f/fd6 [0,4194304] 0 2024-04-24T23:13:55.819 INFO:tasks.workunit.client.0.smithi031.stdout:7/738: rename d1/d2/f4 to d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f105 0 2024-04-24T23:13:55.821 INFO:tasks.workunit.client.0.smithi031.stdout:5/685: write d0/d6/d3b/d4b/d1d/f9d [1285679,72294] 0 2024-04-24T23:13:55.821 INFO:tasks.workunit.client.0.smithi031.stdout:0/807: mkdir da/d86/da3/d10f 0 2024-04-24T23:13:55.823 INFO:tasks.workunit.client.1.smithi040.stdout:0/509: fdatasync df/d12/d4b/f92 0 2024-04-24T23:13:55.830 INFO:tasks.workunit.client.1.smithi040.stdout:7/496: symlink db/d12/d20/lb0 0 2024-04-24T23:13:55.831 INFO:tasks.workunit.client.1.smithi040.stdout:7/497: dread - db/d12/d20/d52/d44/f8d zero size 2024-04-24T23:13:55.831 INFO:tasks.workunit.client.0.smithi031.stdout:8/870: dread d8/d15/d37/f5e [0,4194304] 0 2024-04-24T23:13:55.833 INFO:tasks.workunit.client.0.smithi031.stdout:8/871: dread - d8/d15/d37/d43/f113 zero size 2024-04-24T23:13:55.837 INFO:tasks.workunit.client.1.smithi040.stdout:3/482: creat d1/d2a/d8d/d7c/fae x:0 0 0 2024-04-24T23:13:55.837 INFO:tasks.workunit.client.0.smithi031.stdout:1/670: write d1/d1d/f6d [1177302,130577] 0 2024-04-24T23:13:55.837 INFO:tasks.workunit.client.0.smithi031.stdout:4/861: read d3/d23/d69/d3f/f6d [313378,2224] 0 2024-04-24T23:13:55.837 INFO:tasks.workunit.client.0.smithi031.stdout:9/830: dread d2/dae/d85/f9b [0,4194304] 0 2024-04-24T23:13:55.838 INFO:tasks.workunit.client.0.smithi031.stdout:2/625: dread d5/fd7 [0,4194304] 0 2024-04-24T23:13:55.841 INFO:tasks.workunit.client.0.smithi031.stdout:5/686: creat d0/d6/d3b/d4b/d1d/d2e/d28/d77/fe1 x:0 0 0 2024-04-24T23:13:55.847 INFO:tasks.workunit.client.0.smithi031.stdout:1/671: dwrite d1/d3e/da3/d69/f95 [0,4194304] 0 2024-04-24T23:13:55.848 INFO:tasks.workunit.client.0.smithi031.stdout:1/672: readlink d1/da/d61/d90/lca 0 2024-04-24T23:13:55.848 INFO:tasks.workunit.client.1.smithi040.stdout:5/431: symlink d2/d7/d8/d6d/d89/l8c 0 2024-04-24T23:13:55.849 INFO:tasks.workunit.client.1.smithi040.stdout:5/432: fsync d2/d7/d8/d1f/d57/d3e/f8a 0 2024-04-24T23:13:55.854 INFO:tasks.workunit.client.1.smithi040.stdout:7/498: symlink db/d12/d20/d52/d44/lb1 0 2024-04-24T23:13:55.856 INFO:tasks.workunit.client.1.smithi040.stdout:3/483: symlink d1/d9/d18/d36/d55/laf 0 2024-04-24T23:13:55.857 INFO:tasks.workunit.client.1.smithi040.stdout:4/398: dwrite da/d3f/d40/d41/d60/f24 [0,4194304] 0 2024-04-24T23:13:55.858 INFO:tasks.workunit.client.0.smithi031.stdout:2/626: read - d5/f87 zero size 2024-04-24T23:13:55.866 INFO:tasks.workunit.client.0.smithi031.stdout:5/687: fsync d0/d6/d3b/d4b/dc/d25/f69 0 2024-04-24T23:13:55.870 INFO:tasks.workunit.client.1.smithi040.stdout:8/463: write d0/d16/d1b/f54 [1381911,106353] 0 2024-04-24T23:13:55.872 INFO:tasks.workunit.client.0.smithi031.stdout:6/795: link d13/c93 d13/d15/d1d/d23/d44/da0/da6/de6/c109 0 2024-04-24T23:13:55.874 INFO:tasks.workunit.client.1.smithi040.stdout:1/460: truncate da/d31/f52 4017970 0 2024-04-24T23:13:55.877 INFO:tasks.workunit.client.1.smithi040.stdout:2/515: write f8 [2864659,21658] 0 2024-04-24T23:13:55.878 INFO:tasks.workunit.client.0.smithi031.stdout:2/627: symlink d5/d8/d45/d5b/d49/d6f/le3 0 2024-04-24T23:13:55.878 INFO:tasks.workunit.client.1.smithi040.stdout:5/433: dwrite d2/d2f/d14/f81 [0,4194304] 0 2024-04-24T23:13:55.879 INFO:tasks.workunit.client.0.smithi031.stdout:2/628: dread - d5/d8/fcd zero size 2024-04-24T23:13:55.880 INFO:tasks.workunit.client.0.smithi031.stdout:3/680: link d6/d14/la2 d6/d14/lc8 0 2024-04-24T23:13:55.881 INFO:tasks.workunit.client.1.smithi040.stdout:3/484: creat d1/d9/d39/fb0 x:0 0 0 2024-04-24T23:13:55.883 INFO:tasks.workunit.client.0.smithi031.stdout:5/688: fsync d0/d6/d3b/d4b/d1d/d2e/f52 0 2024-04-24T23:13:55.885 INFO:tasks.workunit.client.1.smithi040.stdout:5/434: dwrite d2/d7/d8/d1f/d57/f44 [0,4194304] 0 2024-04-24T23:13:55.886 INFO:tasks.workunit.client.0.smithi031.stdout:5/689: write d0/d6/d3b/d4b/fac [2536347,88505] 0 2024-04-24T23:13:55.886 INFO:tasks.workunit.client.0.smithi031.stdout:6/796: symlink d13/d15/d1d/d23/d24/d43/d68/l10a 0 2024-04-24T23:13:55.887 INFO:tasks.workunit.client.1.smithi040.stdout:3/485: rmdir d1/d9 39 2024-04-24T23:13:55.888 INFO:tasks.workunit.client.0.smithi031.stdout:4/862: creat d3/d23/d37/f110 x:0 0 0 2024-04-24T23:13:55.889 INFO:tasks.workunit.client.0.smithi031.stdout:8/872: link d8/d15/d37/d43/d7b/d87/dad/cc9 d8/d15/d37/d43/d7b/d87/d10d/c128 0 2024-04-24T23:13:55.891 INFO:tasks.workunit.client.0.smithi031.stdout:2/629: creat d5/d6/d29/d26/db0/dc2/dad/fe4 x:0 0 0 2024-04-24T23:13:55.892 INFO:tasks.workunit.client.0.smithi031.stdout:3/681: fdatasync d6/d14/d1d/f8b 0 2024-04-24T23:13:55.894 INFO:tasks.workunit.client.0.smithi031.stdout:7/739: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/ca2 221666907 1 2024-04-24T23:13:55.895 INFO:tasks.workunit.client.1.smithi040.stdout:1/461: getdents da/d12 0 2024-04-24T23:13:55.895 INFO:tasks.workunit.client.1.smithi040.stdout:6/441: sync 2024-04-24T23:13:55.895 INFO:tasks.workunit.client.1.smithi040.stdout:9/469: sync 2024-04-24T23:13:55.896 INFO:tasks.workunit.client.0.smithi031.stdout:7/740: stat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/ca2 0 2024-04-24T23:13:55.896 INFO:tasks.workunit.client.1.smithi040.stdout:5/435: getdents d2/d2f/d14/d7f 0 2024-04-24T23:13:55.897 INFO:tasks.workunit.client.0.smithi031.stdout:5/690: unlink d0/d7/c1f 0 2024-04-24T23:13:55.898 INFO:tasks.workunit.client.1.smithi040.stdout:3/486: chown d1/d9/d18/d36/d55/d7e/c87 25733042 1 2024-04-24T23:13:55.901 INFO:tasks.workunit.client.1.smithi040.stdout:9/470: mknod d7/da/d17/d40/d55/d2f/d28/ca3 0 2024-04-24T23:13:55.905 INFO:tasks.workunit.client.1.smithi040.stdout:6/442: dwrite d7/d1a/d20/d51/d2f/f5c [4194304,4194304] 0 2024-04-24T23:13:55.906 INFO:tasks.workunit.client.1.smithi040.stdout:5/436: rename d2/d7/d8/d1f/d57/d27/d3c/d43/c5e to d2/d7/d8/db/c8d 0 2024-04-24T23:13:55.906 INFO:tasks.workunit.client.1.smithi040.stdout:9/471: rename d7/da/d17 to d7/da/d17/d40/d55/d61/d6c/d8c/da4 22 2024-04-24T23:13:55.907 INFO:tasks.workunit.client.1.smithi040.stdout:6/443: write d7/d1a/d20/d51/d1e/d5d/f31 [3647786,59334] 0 2024-04-24T23:13:55.909 INFO:tasks.workunit.client.1.smithi040.stdout:5/437: write d2/d7/d8/d1f/d57/f44 [3381983,3654] 0 2024-04-24T23:13:55.912 INFO:tasks.workunit.client.1.smithi040.stdout:6/444: write d7/d1a/d20/d51/d1e/d46/d69/f6b [550516,80023] 0 2024-04-24T23:13:55.912 INFO:tasks.workunit.client.0.smithi031.stdout:7/741: rename d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/db8 to d1/de4/d106 0 2024-04-24T23:13:55.913 INFO:tasks.workunit.client.1.smithi040.stdout:9/472: truncate d7/da/d17/d40/d55/d2f/d1f/d34/f9d 185377 0 2024-04-24T23:13:55.915 INFO:tasks.workunit.client.1.smithi040.stdout:3/487: dwrite d1/d2a/d6a/f8e [0,4194304] 0 2024-04-24T23:13:55.916 INFO:tasks.workunit.client.1.smithi040.stdout:3/488: dread - d1/d2d/d72/f90 zero size 2024-04-24T23:13:55.918 INFO:tasks.workunit.client.0.smithi031.stdout:4/863: mknod d3/d23/d37/c111 0 2024-04-24T23:13:55.919 INFO:tasks.workunit.client.1.smithi040.stdout:3/489: chown d1/d9/f17 5581227 1 2024-04-24T23:13:55.921 INFO:tasks.workunit.client.1.smithi040.stdout:3/490: fsync d1/d9/d2e/f5f 0 2024-04-24T23:13:55.922 INFO:tasks.workunit.client.1.smithi040.stdout:5/438: mknod d2/d2f/d14/d7f/c8e 0 2024-04-24T23:13:55.922 INFO:tasks.workunit.client.1.smithi040.stdout:9/473: dwrite d7/da/f39 [0,4194304] 0 2024-04-24T23:13:55.924 INFO:tasks.workunit.client.1.smithi040.stdout:9/474: fsync d7/da/d17/d40/d55/d2f/d1f/d34/f78 0 2024-04-24T23:13:55.928 INFO:tasks.workunit.client.1.smithi040.stdout:5/439: rename d2/d7/d8/d1f/d57/l63 to d2/d7/d8/d1f/d57/d27/d3c/l8f 0 2024-04-24T23:13:55.929 INFO:tasks.workunit.client.1.smithi040.stdout:9/475: truncate d7/da/d17/d40/d55/d61/d6c/f91 1035200 0 2024-04-24T23:13:55.931 INFO:tasks.workunit.client.0.smithi031.stdout:8/873: getdents d8/d15 0 2024-04-24T23:13:55.931 INFO:tasks.workunit.client.1.smithi040.stdout:3/491: dwrite d1/d2a/d6a/d94/f95 [0,4194304] 0 2024-04-24T23:13:55.932 INFO:tasks.workunit.client.0.smithi031.stdout:2/630: getdents d5/d6 0 2024-04-24T23:13:55.935 INFO:tasks.workunit.client.0.smithi031.stdout:5/691: getdents d0 0 2024-04-24T23:13:55.936 INFO:tasks.workunit.client.1.smithi040.stdout:3/492: mknod d1/d9/d18/da7/cb1 0 2024-04-24T23:13:55.937 INFO:tasks.workunit.client.1.smithi040.stdout:3/493: dread - d1/d2a/d8d/f99 zero size 2024-04-24T23:13:55.938 INFO:tasks.workunit.client.1.smithi040.stdout:4/399: write da/d3f/d6b/d2b/f35 [1775541,31810] 0 2024-04-24T23:13:55.939 INFO:tasks.workunit.client.1.smithi040.stdout:5/440: mkdir d2/d7/d8/d1f/d90 0 2024-04-24T23:13:55.941 INFO:tasks.workunit.client.1.smithi040.stdout:9/476: mknod d7/da/d17/ca5 0 2024-04-24T23:13:55.942 INFO:tasks.workunit.client.1.smithi040.stdout:5/441: write d2/d7/d8/f1b [2450258,35264] 0 2024-04-24T23:13:55.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/692: mkdir d0/d7/d74/de2 0 2024-04-24T23:13:55.946 INFO:tasks.workunit.client.0.smithi031.stdout:5/693: read d0/d6/d3b/d4b/d1d/d2e/d28/f30 [329234,48364] 0 2024-04-24T23:13:55.948 INFO:tasks.workunit.client.1.smithi040.stdout:4/400: link da/d3f/d40/d41/d60/f42 da/d78/f8c 0 2024-04-24T23:13:55.948 INFO:tasks.workunit.client.0.smithi031.stdout:5/694: chown d0/d6/d3b/d4b/dc/d25/d55/f75 3946697 1 2024-04-24T23:13:55.948 INFO:tasks.workunit.client.1.smithi040.stdout:4/401: chown da/d3f/d6b/f7a 5375032 1 2024-04-24T23:13:55.949 INFO:tasks.workunit.client.1.smithi040.stdout:9/477: symlink d7/da/d17/d40/d55/d2f/d1f/d4e/la6 0 2024-04-24T23:13:55.950 INFO:tasks.workunit.client.0.smithi031.stdout:1/673: dwrite d1/da/f2c [0,4194304] 0 2024-04-24T23:13:55.950 INFO:tasks.workunit.client.1.smithi040.stdout:4/402: chown da/d3f/d40/d41/d60/f4d 0 1 2024-04-24T23:13:55.952 INFO:tasks.workunit.client.1.smithi040.stdout:3/494: creat d1/fb2 x:0 0 0 2024-04-24T23:13:55.953 INFO:tasks.workunit.client.1.smithi040.stdout:4/403: write da/d3f/d40/d41/d60/f23 [895207,24081] 0 2024-04-24T23:13:55.953 INFO:tasks.workunit.client.0.smithi031.stdout:9/831: truncate d2/dae/d14/f19 128162 0 2024-04-24T23:13:55.956 INFO:tasks.workunit.client.1.smithi040.stdout:9/478: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/ca7 0 2024-04-24T23:13:55.958 INFO:tasks.workunit.client.0.smithi031.stdout:1/674: symlink d1/da/d61/ld5 0 2024-04-24T23:13:55.959 INFO:tasks.workunit.client.1.smithi040.stdout:3/495: creat d1/d2a/d8d/d7c/fb3 x:0 0 0 2024-04-24T23:13:55.963 INFO:tasks.workunit.client.0.smithi031.stdout:9/832: rename d2/dae/d14/d78/d84/d4a/d4b/f52 to d2/dae/d14/d1d/d45/d65/ffd 0 2024-04-24T23:13:55.963 INFO:tasks.workunit.client.1.smithi040.stdout:6/445: dread d7/d1a/d20/d51/d1e/f49 [0,4194304] 0 2024-04-24T23:13:55.965 INFO:tasks.workunit.client.1.smithi040.stdout:2/516: truncate da/d1c/d37/f8e 785438 0 2024-04-24T23:13:55.965 INFO:tasks.workunit.client.1.smithi040.stdout:0/510: sync 2024-04-24T23:13:55.966 INFO:tasks.workunit.client.1.smithi040.stdout:4/404: mkdir da/d3f/d8d 0 2024-04-24T23:13:55.967 INFO:tasks.workunit.client.1.smithi040.stdout:2/517: readlink da/d29/d55/l7d 0 2024-04-24T23:13:55.967 INFO:tasks.workunit.client.1.smithi040.stdout:3/496: dwrite d1/d9/d15/f1b [0,4194304] 0 2024-04-24T23:13:55.967 INFO:tasks.workunit.client.1.smithi040.stdout:0/511: chown df/d12/d58/f72 5 1 2024-04-24T23:13:55.969 INFO:tasks.workunit.client.1.smithi040.stdout:2/518: chown da/cc 3 1 2024-04-24T23:13:55.969 INFO:tasks.workunit.client.1.smithi040.stdout:6/446: truncate d7/d1a/d20/d51/d1e/d46/f7d 516421 0 2024-04-24T23:13:55.970 INFO:tasks.workunit.client.1.smithi040.stdout:6/447: read d7/d1a/f67 [13284,54732] 0 2024-04-24T23:13:55.974 INFO:tasks.workunit.client.1.smithi040.stdout:4/405: rmdir da/d1f 39 2024-04-24T23:13:55.975 INFO:tasks.workunit.client.1.smithi040.stdout:1/462: write da/d12/d14/f3f [3810882,78745] 0 2024-04-24T23:13:55.976 INFO:tasks.workunit.client.1.smithi040.stdout:1/463: read - da/d12/d14/f96 zero size 2024-04-24T23:13:55.976 INFO:tasks.workunit.client.1.smithi040.stdout:0/512: mkdir df/d12/d58/da9 0 2024-04-24T23:13:55.976 INFO:tasks.workunit.client.0.smithi031.stdout:1/675: dread d1/f2 [0,4194304] 0 2024-04-24T23:13:55.978 INFO:tasks.workunit.client.1.smithi040.stdout:2/519: creat da/d10/d15/daa/fc3 x:0 0 0 2024-04-24T23:13:55.980 INFO:tasks.workunit.client.1.smithi040.stdout:3/497: rename d1/d2d/cc to d1/d9/d18/d36/d61/cb4 0 2024-04-24T23:13:55.982 INFO:tasks.workunit.client.1.smithi040.stdout:3/498: fsync d1/d9/d18/d36/d55/f89 0 2024-04-24T23:13:55.985 INFO:tasks.workunit.client.0.smithi031.stdout:1/676: truncate d1/d33/d36/f71 124524 0 2024-04-24T23:13:55.985 INFO:tasks.workunit.client.0.smithi031.stdout:3/682: write d6/d14/d1d/d69/fa7 [71826,67362] 0 2024-04-24T23:13:55.986 INFO:tasks.workunit.client.1.smithi040.stdout:4/406: truncate da/d3f/d6b/d2b/f52 6820215 0 2024-04-24T23:13:55.988 INFO:tasks.workunit.client.1.smithi040.stdout:0/513: mkdir df/d1e/d20/d21/d51/d79/daa 0 2024-04-24T23:13:55.989 INFO:tasks.workunit.client.1.smithi040.stdout:0/514: readlink df/d12/d1a/d71/l15 0 2024-04-24T23:13:55.989 INFO:tasks.workunit.client.1.smithi040.stdout:3/499: dwrite d1/d9/d18/d36/fa2 [0,4194304] 0 2024-04-24T23:13:55.991 INFO:tasks.workunit.client.0.smithi031.stdout:6/797: dwrite d13/d15/d1d/d23/d2d/dbf/fec [0,4194304] 0 2024-04-24T23:13:55.992 INFO:tasks.workunit.client.1.smithi040.stdout:3/500: dread d1/d2d/d2b/f4d [0,4194304] 0 2024-04-24T23:13:55.992 INFO:tasks.workunit.client.1.smithi040.stdout:7/499: sync 2024-04-24T23:13:55.992 INFO:tasks.workunit.client.1.smithi040.stdout:8/464: sync 2024-04-24T23:13:55.993 INFO:tasks.workunit.client.1.smithi040.stdout:8/465: chown d0/d8/f4d 121 1 2024-04-24T23:13:55.994 INFO:tasks.workunit.client.1.smithi040.stdout:1/464: rename da/d35/l36 to da/d12/d18/d2a/d4d/d7d/la0 0 2024-04-24T23:13:56.002 INFO:tasks.workunit.client.0.smithi031.stdout:7/742: write d1/f4b [1350435,67015] 0 2024-04-24T23:13:56.002 INFO:tasks.workunit.client.0.smithi031.stdout:4/864: dwrite d3/f7 [0,4194304] 0 2024-04-24T23:13:56.004 INFO:tasks.workunit.client.1.smithi040.stdout:4/407: mkdir da/d3f/d40/d41/d8e 0 2024-04-24T23:13:56.006 INFO:tasks.workunit.client.0.smithi031.stdout:7/743: fdatasync d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/fde 0 2024-04-24T23:13:56.007 INFO:tasks.workunit.client.1.smithi040.stdout:3/501: dwrite d1/d2a/d8d/d7c/fae [0,4194304] 0 2024-04-24T23:13:56.012 INFO:tasks.workunit.client.0.smithi031.stdout:2/631: write d5/d8/d45/d5b/d49/f88 [456799,111588] 0 2024-04-24T23:13:56.013 INFO:tasks.workunit.client.0.smithi031.stdout:8/874: dwrite d8/d15/f2a [0,4194304] 0 2024-04-24T23:13:56.013 INFO:tasks.workunit.client.1.smithi040.stdout:1/465: dwrite da/d35/d38/d47/f80 [0,4194304] 0 2024-04-24T23:13:56.013 INFO:tasks.workunit.client.1.smithi040.stdout:8/466: dwrite d0/d8/d22/f6a [0,4194304] 0 2024-04-24T23:13:56.013 INFO:tasks.workunit.client.1.smithi040.stdout:2/520: link da/d10/d7a/f99 da/d1c/d37/fc4 0 2024-04-24T23:13:56.017 INFO:tasks.workunit.client.1.smithi040.stdout:8/467: fdatasync d0/d16/d19/d71/d89/d90/d91/f97 0 2024-04-24T23:13:56.021 INFO:tasks.workunit.client.1.smithi040.stdout:8/468: dread d0/d8/d22/d81/f74 [0,4194304] 0 2024-04-24T23:13:56.030 INFO:tasks.workunit.client.0.smithi031.stdout:5/695: write d0/d6/d3b/d4b/d1d/d2e/dc6/f6f [845550,20271] 0 2024-04-24T23:13:56.034 INFO:tasks.workunit.client.0.smithi031.stdout:9/833: unlink d2/dae/d14/d1d/d45/d65/ffd 0 2024-04-24T23:13:56.035 INFO:tasks.workunit.client.0.smithi031.stdout:9/834: readlink d2/dae/d14/d1d/d45/l61 0 2024-04-24T23:13:56.038 INFO:tasks.workunit.client.1.smithi040.stdout:3/502: mknod d1/d2d/de/d30/d37/cb5 0 2024-04-24T23:13:56.040 INFO:tasks.workunit.client.1.smithi040.stdout:1/466: symlink da/d12/la1 0 2024-04-24T23:13:56.041 INFO:tasks.workunit.client.1.smithi040.stdout:9/479: dwrite d7/f37 [0,4194304] 0 2024-04-24T23:13:56.049 INFO:tasks.workunit.client.1.smithi040.stdout:7/500: unlink db/c4c 0 2024-04-24T23:13:56.049 INFO:tasks.workunit.client.1.smithi040.stdout:9/480: dwrite d7/da/d17/d40/d55/d61/f8f [0,4194304] 0 2024-04-24T23:13:56.054 INFO:tasks.workunit.client.1.smithi040.stdout:6/448: dwrite d7/d1a/d20/d51/d1e/f22 [0,4194304] 0 2024-04-24T23:13:56.055 INFO:tasks.workunit.client.1.smithi040.stdout:9/481: read d7/da/d17/d40/d55/d2f/d1f/f5a [4487641,54531] 0 2024-04-24T23:13:56.058 INFO:tasks.workunit.client.1.smithi040.stdout:0/515: link df/d12/d58/c91 df/d1e/d20/d21/cab 0 2024-04-24T23:13:56.060 INFO:tasks.workunit.client.1.smithi040.stdout:2/521: mknod da/d29/cc5 0 2024-04-24T23:13:56.065 INFO:tasks.workunit.client.0.smithi031.stdout:3/683: unlink d6/df/d16/d2a/caa 0 2024-04-24T23:13:56.066 INFO:tasks.workunit.client.0.smithi031.stdout:5/696: dread d0/d6/d3b/d4b/d1d/d2e/f52 [0,4194304] 0 2024-04-24T23:13:56.067 INFO:tasks.workunit.client.0.smithi031.stdout:6/798: truncate d13/d15/d1d/d23/d2d/f90 498492 0 2024-04-24T23:13:56.067 INFO:tasks.workunit.client.0.smithi031.stdout:5/697: readlink d0/d6/d3b/d4b/d1d/d2e/d28/l97 0 2024-04-24T23:13:56.069 INFO:tasks.workunit.client.0.smithi031.stdout:7/744: dread - d1/dbe/fb7 zero size 2024-04-24T23:13:56.070 INFO:tasks.workunit.client.0.smithi031.stdout:7/745: stat d1/d2/dd/d33/d5e/l9a 0 2024-04-24T23:13:56.070 INFO:tasks.workunit.client.1.smithi040.stdout:6/449: creat d7/de/d4e/f8e x:0 0 0 2024-04-24T23:13:56.071 INFO:tasks.workunit.client.0.smithi031.stdout:2/632: mkdir d5/d8/d45/d5b/d49/dc4/de5 0 2024-04-24T23:13:56.072 INFO:tasks.workunit.client.1.smithi040.stdout:3/503: dread - d1/d9/d39/f78 zero size 2024-04-24T23:13:56.073 INFO:tasks.workunit.client.0.smithi031.stdout:8/875: symlink d8/d15/d37/d43/d7b/l129 0 2024-04-24T23:13:56.073 INFO:tasks.workunit.client.1.smithi040.stdout:3/504: stat d1/d9/d18 0 2024-04-24T23:13:56.073 INFO:tasks.workunit.client.1.smithi040.stdout:0/516: fsync df/d12/d58/f72 0 2024-04-24T23:13:56.074 INFO:tasks.workunit.client.0.smithi031.stdout:1/677: stat d1/d33/d36/f71 0 2024-04-24T23:13:56.074 INFO:tasks.workunit.client.1.smithi040.stdout:2/522: truncate da/f72 3072305 0 2024-04-24T23:13:56.075 INFO:tasks.workunit.client.1.smithi040.stdout:3/505: read d1/d9/d2e/f84 [177038,100067] 0 2024-04-24T23:13:56.077 INFO:tasks.workunit.client.1.smithi040.stdout:8/469: rmdir d0/d16/d19/d71/d98 0 2024-04-24T23:13:56.077 INFO:tasks.workunit.client.1.smithi040.stdout:6/450: dwrite d7/de/f5e [0,4194304] 0 2024-04-24T23:13:56.078 INFO:tasks.workunit.client.1.smithi040.stdout:6/451: readlink d7/d1a/l55 0 2024-04-24T23:13:56.087 INFO:tasks.workunit.client.0.smithi031.stdout:8/876: dread d8/d15/d37/d43/d7b/d86/fb6 [0,4194304] 0 2024-04-24T23:13:56.087 INFO:tasks.workunit.client.0.smithi031.stdout:4/865: rename d3/d23/d37/f7c to d3/d20/d25/d95/f112 0 2024-04-24T23:13:56.090 INFO:tasks.workunit.client.1.smithi040.stdout:9/482: rename f4 to d7/da/d17/d40/d55/d61/d6c/fa8 0 2024-04-24T23:13:56.091 INFO:tasks.workunit.client.0.smithi031.stdout:7/746: chown d1/ldb 132 1 2024-04-24T23:13:56.092 INFO:tasks.workunit.client.0.smithi031.stdout:7/747: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/ff0 [664102,126944] 0 2024-04-24T23:13:56.098 INFO:tasks.workunit.client.0.smithi031.stdout:3/684: mkdir d6/d14/d71/dc9 0 2024-04-24T23:13:56.099 INFO:tasks.workunit.client.1.smithi040.stdout:0/517: truncate df/d1e/d20/d21/d51/f57 37773 0 2024-04-24T23:13:56.102 INFO:tasks.workunit.client.1.smithi040.stdout:3/506: symlink d1/d9/d18/d36/d61/lb6 0 2024-04-24T23:13:56.103 INFO:tasks.workunit.client.1.smithi040.stdout:3/507: write d1/d2d/de/f2f [2876918,130825] 0 2024-04-24T23:13:56.103 INFO:tasks.workunit.client.0.smithi031.stdout:6/799: unlink d13/c93 0 2024-04-24T23:13:56.104 INFO:tasks.workunit.client.1.smithi040.stdout:0/518: dread df/d1e/da7/f7f [0,4194304] 0 2024-04-24T23:13:56.105 INFO:tasks.workunit.client.0.smithi031.stdout:5/698: mkdir d0/d6/d3b/d4b/dc/de3 0 2024-04-24T23:13:56.108 INFO:tasks.workunit.client.0.smithi031.stdout:8/877: creat d8/d15/dcf/f12a x:0 0 0 2024-04-24T23:13:56.111 INFO:tasks.workunit.client.0.smithi031.stdout:4/866: unlink d3/d20/d2d/d44/d5b/d5d/f87 0 2024-04-24T23:13:56.114 INFO:tasks.workunit.client.0.smithi031.stdout:7/748: dread d1/d2/dd/f95 [0,4194304] 0 2024-04-24T23:13:56.120 INFO:tasks.workunit.client.0.smithi031.stdout:3/685: mknod d6/d14/d91/cca 0 2024-04-24T23:13:56.121 INFO:tasks.workunit.client.1.smithi040.stdout:4/408: dwrite da/d3f/d6b/d3c/f5c [0,4194304] 0 2024-04-24T23:13:56.122 INFO:tasks.workunit.client.0.smithi031.stdout:6/800: mkdir d13/d15/d1d/d23/d2d/dbf/d10b 0 2024-04-24T23:13:56.123 INFO:tasks.workunit.client.1.smithi040.stdout:4/409: truncate da/d3f/d40/d41/d60/f24 4238470 0 2024-04-24T23:13:56.124 INFO:tasks.workunit.client.1.smithi040.stdout:0/519: symlink df/d12/d58/d5b/lac 0 2024-04-24T23:13:56.125 INFO:tasks.workunit.client.0.smithi031.stdout:5/699: fsync d0/d6/f40 0 2024-04-24T23:13:56.126 INFO:tasks.workunit.client.1.smithi040.stdout:0/520: dread - df/d12/d58/f7d zero size 2024-04-24T23:13:56.127 INFO:tasks.workunit.client.0.smithi031.stdout:5/700: write d0/d6/d3b/d4b/dc/f26 [5087655,99891] 0 2024-04-24T23:13:56.127 INFO:tasks.workunit.client.1.smithi040.stdout:0/521: truncate df/d12/d4b/f8b 1013176 0 2024-04-24T23:13:56.129 INFO:tasks.workunit.client.0.smithi031.stdout:8/878: mkdir d8/d1a/d1d/d50/dc0/d9d/d12b 0 2024-04-24T23:13:56.129 INFO:tasks.workunit.client.0.smithi031.stdout:8/879: fsync d8/d1a/f9b 0 2024-04-24T23:13:56.130 INFO:tasks.workunit.client.1.smithi040.stdout:9/483: symlink d7/da/d17/d40/d55/d2f/d1f/d31/la9 0 2024-04-24T23:13:56.131 INFO:tasks.workunit.client.1.smithi040.stdout:9/484: readlink d7/da/l3e 0 2024-04-24T23:13:56.132 INFO:tasks.workunit.client.0.smithi031.stdout:4/867: rmdir d3/d20/d25/d60/d63/db9 39 2024-04-24T23:13:56.133 INFO:tasks.workunit.client.1.smithi040.stdout:0/522: dwrite df/d12/f4c [0,4194304] 0 2024-04-24T23:13:56.137 INFO:tasks.workunit.client.0.smithi031.stdout:9/835: getdents d2/dae/d14/d78/d84 0 2024-04-24T23:13:56.138 INFO:tasks.workunit.client.0.smithi031.stdout:7/749: rmdir d1/d2/dd/d33/d5e/d64/d6d/dae 39 2024-04-24T23:13:56.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/686: mkdir d6/df/d16/d2a/d95/d9b/d29/dcb 0 2024-04-24T23:13:56.145 INFO:tasks.workunit.client.1.smithi040.stdout:5/442: dwrite d2/d7/d8/d1f/d57/d27/f54 [0,4194304] 0 2024-04-24T23:13:56.146 INFO:tasks.workunit.client.1.smithi040.stdout:4/410: mknod da/d3f/d40/d47/d75/c8f 0 2024-04-24T23:13:56.149 INFO:tasks.workunit.client.0.smithi031.stdout:3/687: dwrite d6/d14/db2/fc3 [0,4194304] 0 2024-04-24T23:13:56.150 INFO:tasks.workunit.client.0.smithi031.stdout:6/801: dread d13/d15/d1d/d23/d24/d43/d68/db8/fd1 [0,4194304] 0 2024-04-24T23:13:56.151 INFO:tasks.workunit.client.1.smithi040.stdout:4/411: dread da/d3f/d6b/d3c/f5c [0,4194304] 0 2024-04-24T23:13:56.154 INFO:tasks.workunit.client.0.smithi031.stdout:5/701: creat d0/d6/d3b/d4b/dc/d25/fe4 x:0 0 0 2024-04-24T23:13:56.157 INFO:tasks.workunit.client.1.smithi040.stdout:0/523: rename df/d12/d1a/f3a to df/d12/d1a/fad 0 2024-04-24T23:13:56.158 INFO:tasks.workunit.client.0.smithi031.stdout:8/880: unlink d8/d15/d37/d43/daf/dca/l116 0 2024-04-24T23:13:56.158 INFO:tasks.workunit.client.1.smithi040.stdout:4/412: dwrite da/d3f/d6b/d2b/f35 [0,4194304] 0 2024-04-24T23:13:56.159 INFO:tasks.workunit.client.1.smithi040.stdout:0/524: truncate df/d14/f88 449900 0 2024-04-24T23:13:56.160 INFO:tasks.workunit.client.0.smithi031.stdout:4/868: mkdir d3/d23/d37/d86/d113 0 2024-04-24T23:13:56.161 INFO:tasks.workunit.client.1.smithi040.stdout:7/501: write db/d12/f8c [1635504,4190] 0 2024-04-24T23:13:56.162 INFO:tasks.workunit.client.0.smithi031.stdout:7/750: dread d1/d2/dd/d33/d5e/d64/d6d/fbb [0,4194304] 0 2024-04-24T23:13:56.164 INFO:tasks.workunit.client.1.smithi040.stdout:1/467: stat da/d12/d18/d2a/d32/c5c 0 2024-04-24T23:13:56.166 INFO:tasks.workunit.client.0.smithi031.stdout:9/836: mknod d2/dae/d14/d78/d84/d7e/cfe 0 2024-04-24T23:13:56.170 INFO:tasks.workunit.client.1.smithi040.stdout:7/502: dwrite db/d12/d20/d52/d4b/fa1 [0,4194304] 0 2024-04-24T23:13:56.171 INFO:tasks.workunit.client.1.smithi040.stdout:9/485: mknod d7/d92/caa 0 2024-04-24T23:13:56.174 INFO:tasks.workunit.client.1.smithi040.stdout:8/470: getdents d0/d16/d19/d71 0 2024-04-24T23:13:56.178 INFO:tasks.workunit.client.1.smithi040.stdout:4/413: creat da/d3f/d6b/d3c/f90 x:0 0 0 2024-04-24T23:13:56.181 INFO:tasks.workunit.client.1.smithi040.stdout:0/525: creat df/d1e/d20/d21/d51/d79/fae x:0 0 0 2024-04-24T23:13:56.181 INFO:tasks.workunit.client.1.smithi040.stdout:9/486: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [4194304,4194304] 0 2024-04-24T23:13:56.187 INFO:tasks.workunit.client.0.smithi031.stdout:5/702: dread d0/d6/d3b/d4b/dc/f9a [0,4194304] 0 2024-04-24T23:13:56.190 INFO:tasks.workunit.client.0.smithi031.stdout:6/802: mkdir d13/d15/d1d/d23/d44/d47/d10c 0 2024-04-24T23:13:56.193 INFO:tasks.workunit.client.0.smithi031.stdout:7/751: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/f107 x:0 0 0 2024-04-24T23:13:56.193 INFO:tasks.workunit.client.0.smithi031.stdout:2/633: dwrite d5/d6/f35 [0,4194304] 0 2024-04-24T23:13:56.194 INFO:tasks.workunit.client.1.smithi040.stdout:7/503: creat db/d12/d20/d52/d6c/fb2 x:0 0 0 2024-04-24T23:13:56.194 INFO:tasks.workunit.client.0.smithi031.stdout:8/881: dread d8/d15/d37/f5d [0,4194304] 0 2024-04-24T23:13:56.195 INFO:tasks.workunit.client.0.smithi031.stdout:2/634: fsync d5/d8/d45/d5b/d49/f88 0 2024-04-24T23:13:56.196 INFO:tasks.workunit.client.0.smithi031.stdout:8/882: chown d8/d15/f57 115 1 2024-04-24T23:13:56.199 INFO:tasks.workunit.client.1.smithi040.stdout:2/523: dwrite da/d1c/d37/d39/f60 [0,4194304] 0 2024-04-24T23:13:56.199 INFO:tasks.workunit.client.1.smithi040.stdout:8/471: symlink d0/d16/d19/d71/d76/l99 0 2024-04-24T23:13:56.202 INFO:tasks.workunit.client.1.smithi040.stdout:6/452: write d7/f71 [410276,69359] 0 2024-04-24T23:13:56.204 INFO:tasks.workunit.client.1.smithi040.stdout:4/414: creat da/d78/f91 x:0 0 0 2024-04-24T23:13:56.205 INFO:tasks.workunit.client.1.smithi040.stdout:6/453: fsync d7/d1a/d20/d51/d2f/f6f 0 2024-04-24T23:13:56.207 INFO:tasks.workunit.client.0.smithi031.stdout:5/703: truncate d0/d6/d3b/d4b/dc/f80 363773 0 2024-04-24T23:13:56.209 INFO:tasks.workunit.client.1.smithi040.stdout:9/487: dread d7/da/d17/d40/d55/d2f/d28/f6b [0,4194304] 0 2024-04-24T23:13:56.210 INFO:tasks.workunit.client.1.smithi040.stdout:1/468: symlink da/la2 0 2024-04-24T23:13:56.210 INFO:tasks.workunit.client.0.smithi031.stdout:9/837: link d2/dae/d14/d78/d84/d4a/dab/db7/ld4 d2/dae/d14/d78/d84/d7e/lff 0 2024-04-24T23:13:56.211 INFO:tasks.workunit.client.1.smithi040.stdout:3/508: write d1/d2d/f60 [1109458,47589] 0 2024-04-24T23:13:56.213 INFO:tasks.workunit.client.1.smithi040.stdout:5/443: link d2/d2f/f84 d2/d7/d8/d1f/d57/f91 0 2024-04-24T23:13:56.218 INFO:tasks.workunit.client.1.smithi040.stdout:3/509: dwrite d1/d2a/d8d/d7c/fae [0,4194304] 0 2024-04-24T23:13:56.218 INFO:tasks.workunit.client.0.smithi031.stdout:5/704: unlink d0/d6/d3b/d4b/fb1 0 2024-04-24T23:13:56.219 INFO:tasks.workunit.client.1.smithi040.stdout:3/510: write d1/d2a/d8d/d7c/fae [4411582,112515] 0 2024-04-24T23:13:56.226 INFO:tasks.workunit.client.0.smithi031.stdout:8/883: unlink d8/d15/d37/d62/c65 0 2024-04-24T23:13:56.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/752: dread d1/f13 [0,4194304] 0 2024-04-24T23:13:56.230 INFO:tasks.workunit.client.1.smithi040.stdout:4/415: unlink da/d3f/d6b/d3c/c86 0 2024-04-24T23:13:56.234 INFO:tasks.workunit.client.1.smithi040.stdout:2/524: dread da/d29/f69 [0,4194304] 0 2024-04-24T23:13:56.236 INFO:tasks.workunit.client.0.smithi031.stdout:7/753: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/ff6 [0,4194304] 0 2024-04-24T23:13:56.236 INFO:tasks.workunit.client.0.smithi031.stdout:8/884: mknod d8/d15/d37/d43/dbd/dc4/c12c 0 2024-04-24T23:13:56.238 INFO:tasks.workunit.client.0.smithi031.stdout:7/754: stat d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/l68 0 2024-04-24T23:13:56.242 INFO:tasks.workunit.client.1.smithi040.stdout:1/469: mkdir da/d35/da3 0 2024-04-24T23:13:56.244 INFO:tasks.workunit.client.0.smithi031.stdout:8/885: write d8/d15/d37/f5e [539867,94502] 0 2024-04-24T23:13:56.248 INFO:tasks.workunit.client.0.smithi031.stdout:7/755: rename d1/dbe/cdd to d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/c108 0 2024-04-24T23:13:56.248 INFO:tasks.workunit.client.1.smithi040.stdout:1/470: dwrite da/d12/d18/f6c [4194304,4194304] 0 2024-04-24T23:13:56.256 INFO:tasks.workunit.client.0.smithi031.stdout:8/886: getdents d8/d1a/d1d/d50/dc0/d9d/d12b 0 2024-04-24T23:13:56.258 INFO:tasks.workunit.client.0.smithi031.stdout:7/756: creat d1/d2/dd/d33/d5e/d64/d6d/dae/db1/f109 x:0 0 0 2024-04-24T23:13:56.259 INFO:tasks.workunit.client.1.smithi040.stdout:8/472: fdatasync d0/d16/d1b/f40 0 2024-04-24T23:13:56.262 INFO:tasks.workunit.client.1.smithi040.stdout:4/416: creat da/d45/d48/d6e/f92 x:0 0 0 2024-04-24T23:13:56.263 INFO:tasks.workunit.client.0.smithi031.stdout:3/688: dwrite d6/df/d16/f35 [0,4194304] 0 2024-04-24T23:13:56.264 INFO:tasks.workunit.client.0.smithi031.stdout:7/757: truncate d1/d2/dd/d33/d5e/d64/d6d/de1/d84/fcd 1155926 0 2024-04-24T23:13:56.266 INFO:tasks.workunit.client.0.smithi031.stdout:7/758: write d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/fb5 [1106951,85379] 0 2024-04-24T23:13:56.267 INFO:tasks.workunit.client.1.smithi040.stdout:6/454: symlink d7/d1a/d20/d51/d1e/d46/l8f 0 2024-04-24T23:13:56.270 INFO:tasks.workunit.client.0.smithi031.stdout:8/887: symlink d8/d1a/d1d/d35/d58/d107/l12d 0 2024-04-24T23:13:56.273 INFO:tasks.workunit.client.0.smithi031.stdout:3/689: symlink d6/d14/d1d/d69/lcc 0 2024-04-24T23:13:56.273 INFO:tasks.workunit.client.0.smithi031.stdout:7/759: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/ff6 [0,4194304] 0 2024-04-24T23:13:56.275 INFO:tasks.workunit.client.1.smithi040.stdout:6/455: dread d7/de/d4e/f64 [0,4194304] 0 2024-04-24T23:13:56.294 INFO:tasks.workunit.client.1.smithi040.stdout:9/488: rename d7/da/c23 to d7/da/d17/d40/d55/d2f/d1f/d34/cab 0 2024-04-24T23:13:56.295 INFO:tasks.workunit.client.0.smithi031.stdout:2/635: write d5/d8/d15/d65/f8e [531931,1778] 0 2024-04-24T23:13:56.297 INFO:tasks.workunit.client.0.smithi031.stdout:7/760: rename d1/d2/dd/d33/d5e/d64/d6d/de1/da0/ddc/c104 to d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/c10a 0 2024-04-24T23:13:56.301 INFO:tasks.workunit.client.1.smithi040.stdout:6/456: dread d7/f3f [0,4194304] 0 2024-04-24T23:13:56.303 INFO:tasks.workunit.client.1.smithi040.stdout:6/457: dread d7/d1a/d20/d51/d1e/d46/f7d [0,4194304] 0 2024-04-24T23:13:56.305 INFO:tasks.workunit.client.1.smithi040.stdout:6/458: truncate d7/de/d4e/f7a 345219 0 2024-04-24T23:13:56.308 INFO:tasks.workunit.client.0.smithi031.stdout:3/690: dread d6/df/d16/d2a/d95/d9b/d1a/d25/f74 [0,4194304] 0 2024-04-24T23:13:56.309 INFO:tasks.workunit.client.0.smithi031.stdout:8/888: mknod d8/d1a/d1d/d35/d58/c12e 0 2024-04-24T23:13:56.312 INFO:tasks.workunit.client.0.smithi031.stdout:2/636: rename d5/d8/d15/d56/d84/c7a to d5/d8/d45/d5b/d49/d6f/ce6 0 2024-04-24T23:13:56.313 INFO:tasks.workunit.client.0.smithi031.stdout:2/637: fsync d5/d8/f6c 0 2024-04-24T23:13:56.313 INFO:tasks.workunit.client.0.smithi031.stdout:3/691: dread d6/d14/f37 [0,4194304] 0 2024-04-24T23:13:56.314 INFO:tasks.workunit.client.1.smithi040.stdout:5/444: stat d2/d7/d8/d1f/d57/d27/d3c/l8f 0 2024-04-24T23:13:56.314 INFO:tasks.workunit.client.1.smithi040.stdout:5/445: dread - d2/d7/d8/db/d1e/f82 zero size 2024-04-24T23:13:56.318 INFO:tasks.workunit.client.0.smithi031.stdout:5/705: chown d0/d6/d3b/d4b/d1d/d2e/d28/l99 552154 1 2024-04-24T23:13:56.319 INFO:tasks.workunit.client.0.smithi031.stdout:9/838: dwrite d2/dae/d14/f19 [0,4194304] 0 2024-04-24T23:13:56.320 INFO:tasks.workunit.client.1.smithi040.stdout:8/473: creat d0/d16/d19/f9a x:0 0 0 2024-04-24T23:13:56.323 INFO:tasks.workunit.client.1.smithi040.stdout:4/417: creat da/d3f/d40/d47/f93 x:0 0 0 2024-04-24T23:13:56.324 INFO:tasks.workunit.client.1.smithi040.stdout:3/511: write d1/d2d/d2b/f42 [4820778,62391] 0 2024-04-24T23:13:56.327 INFO:tasks.workunit.client.1.smithi040.stdout:0/526: getdents df/d1e/d20/d21/d51/d79 0 2024-04-24T23:13:56.328 INFO:tasks.workunit.client.0.smithi031.stdout:7/761: fsync d1/d2/dd/f97 0 2024-04-24T23:13:56.330 INFO:tasks.workunit.client.0.smithi031.stdout:7/762: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d2c/f4c 48 1 2024-04-24T23:13:56.331 INFO:tasks.workunit.client.0.smithi031.stdout:2/638: creat d5/d6/d68/fe7 x:0 0 0 2024-04-24T23:13:56.332 INFO:tasks.workunit.client.1.smithi040.stdout:9/489: symlink d7/d5e/da1/lac 0 2024-04-24T23:13:56.333 INFO:tasks.workunit.client.0.smithi031.stdout:3/692: unlink d6/d14/f37 0 2024-04-24T23:13:56.336 INFO:tasks.workunit.client.1.smithi040.stdout:2/525: dwrite da/d1c/d37/d39/d3f/d52/f95 [4194304,4194304] 0 2024-04-24T23:13:56.337 INFO:tasks.workunit.client.0.smithi031.stdout:5/706: read - d0/d6/d3b/d4b/d1d/d2e/fa1 zero size 2024-04-24T23:13:56.339 INFO:tasks.workunit.client.1.smithi040.stdout:6/459: creat d7/de/d89/f90 x:0 0 0 2024-04-24T23:13:56.340 INFO:tasks.workunit.client.0.smithi031.stdout:9/839: mkdir d2/dae/d14/d1d/d45/d99/d9c/d100 0 2024-04-24T23:13:56.342 INFO:tasks.workunit.client.1.smithi040.stdout:0/527: dread df/d12/d4b/f92 [0,4194304] 0 2024-04-24T23:13:56.343 INFO:tasks.workunit.client.1.smithi040.stdout:7/504: getdents db/d2f/d37/d7e 0 2024-04-24T23:13:56.343 INFO:tasks.workunit.client.1.smithi040.stdout:0/528: chown df/d1e/d20/c83 6 1 2024-04-24T23:13:56.343 INFO:tasks.workunit.client.0.smithi031.stdout:9/840: truncate d2/dae/d14/d1d/d45/d56/ff2 2776503 0 2024-04-24T23:13:56.344 INFO:tasks.workunit.client.0.smithi031.stdout:9/841: chown d2/dae/d85 174133 1 2024-04-24T23:13:56.346 INFO:tasks.workunit.client.1.smithi040.stdout:5/446: creat d2/d7/d8/d64/d6b/f92 x:0 0 0 2024-04-24T23:13:56.349 INFO:tasks.workunit.client.0.smithi031.stdout:8/889: write d8/d1a/d1d/d35/d58/d5b/fe0 [883954,57394] 0 2024-04-24T23:13:56.352 INFO:tasks.workunit.client.0.smithi031.stdout:3/693: truncate d6/d14/d1d/d59/f6c 1053460 0 2024-04-24T23:13:56.353 INFO:tasks.workunit.client.1.smithi040.stdout:3/512: truncate d1/d2d/de/f34 30376 0 2024-04-24T23:13:56.354 INFO:tasks.workunit.client.1.smithi040.stdout:3/513: chown d1/d9/d15/l92 27 1 2024-04-24T23:13:56.354 INFO:tasks.workunit.client.0.smithi031.stdout:7/763: dwrite d1/d2/dd/f7d [4194304,4194304] 0 2024-04-24T23:13:56.355 INFO:tasks.workunit.client.1.smithi040.stdout:3/514: stat d1/d2a/d6a/d94/f98 0 2024-04-24T23:13:56.358 INFO:tasks.workunit.client.0.smithi031.stdout:7/764: rename d1/d2/dd/d33/d5e/d64/d6d/de1 to d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dd9/d10b 22 2024-04-24T23:13:56.361 INFO:tasks.workunit.client.1.smithi040.stdout:9/490: creat d7/d5e/da1/fad x:0 0 0 2024-04-24T23:13:56.363 INFO:tasks.workunit.client.1.smithi040.stdout:1/471: link da/d35/d38/d47/f51 da/d12/d14/d25/d5e/fa4 0 2024-04-24T23:13:56.364 INFO:tasks.workunit.client.0.smithi031.stdout:2/639: truncate d5/f13 691572 0 2024-04-24T23:13:56.365 INFO:tasks.workunit.client.0.smithi031.stdout:8/890: creat d8/d15/d37/d43/daf/dca/f12f x:0 0 0 2024-04-24T23:13:56.367 INFO:tasks.workunit.client.0.smithi031.stdout:3/694: creat d6/d14/d1d/fcd x:0 0 0 2024-04-24T23:13:56.367 INFO:tasks.workunit.client.1.smithi040.stdout:9/491: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 [0,4194304] 0 2024-04-24T23:13:56.369 INFO:tasks.workunit.client.1.smithi040.stdout:2/526: rename da/l13 to da/d1c/d37/d39/d3f/lc6 0 2024-04-24T23:13:56.370 INFO:tasks.workunit.client.1.smithi040.stdout:9/492: truncate d7/da/d17/d40/d55/d2f/d1f/d31/f36 2655691 0 2024-04-24T23:13:56.371 INFO:tasks.workunit.client.0.smithi031.stdout:5/707: symlink d0/d7/d74/de2/le5 0 2024-04-24T23:13:56.374 INFO:tasks.workunit.client.0.smithi031.stdout:5/708: write d0/d6/d3b/d4b/d1d/d2e/dc6/f21 [7098464,51107] 0 2024-04-24T23:13:56.378 INFO:tasks.workunit.client.1.smithi040.stdout:7/505: mkdir db/d12/d62/d8a/db3 0 2024-04-24T23:13:56.379 INFO:tasks.workunit.client.0.smithi031.stdout:2/640: symlink d5/d8/d45/le8 0 2024-04-24T23:13:56.382 INFO:tasks.workunit.client.0.smithi031.stdout:8/891: truncate d8/d1a/f7e 3338499 0 2024-04-24T23:13:56.383 INFO:tasks.workunit.client.0.smithi031.stdout:8/892: stat d8/d15/d37/d43/daf/f120 0 2024-04-24T23:13:56.383 INFO:tasks.workunit.client.1.smithi040.stdout:5/447: chown d2/d2f/d14/c18 2261888 1 2024-04-24T23:13:56.384 INFO:tasks.workunit.client.0.smithi031.stdout:3/695: symlink d6/d14/db2/lce 0 2024-04-24T23:13:56.385 INFO:tasks.workunit.client.1.smithi040.stdout:7/506: dwrite db/d12/d20/d52/d44/f8d [0,4194304] 0 2024-04-24T23:13:56.386 INFO:tasks.workunit.client.1.smithi040.stdout:8/474: fdatasync d0/d28/f39 0 2024-04-24T23:13:56.386 INFO:tasks.workunit.client.1.smithi040.stdout:7/507: stat db/d2f/d37/l38 0 2024-04-24T23:13:56.390 INFO:tasks.workunit.client.1.smithi040.stdout:4/418: mknod da/d1f/c94 0 2024-04-24T23:13:56.393 INFO:tasks.workunit.client.1.smithi040.stdout:3/515: mknod d1/d9/d18/d36/d61/cb7 0 2024-04-24T23:13:56.393 INFO:tasks.workunit.client.0.smithi031.stdout:7/765: dread d1/d2/d83/dc1/f102 [0,4194304] 0 2024-04-24T23:13:56.396 INFO:tasks.workunit.client.0.smithi031.stdout:9/842: link d2/dae/d14/d78/d84/d7e/cfe d2/dae/d14/d78/d84/d7e/dc7/c101 0 2024-04-24T23:13:56.397 INFO:tasks.workunit.client.1.smithi040.stdout:3/516: dwrite d1/f16 [0,4194304] 0 2024-04-24T23:13:56.398 INFO:tasks.workunit.client.1.smithi040.stdout:3/517: stat d1/d2d/de/c3f 0 2024-04-24T23:13:56.401 INFO:tasks.workunit.client.1.smithi040.stdout:2/527: mkdir da/d1c/d37/d39/d3f/d52/d78/dc7 0 2024-04-24T23:13:56.406 INFO:tasks.workunit.client.1.smithi040.stdout:6/460: creat d7/de/d84/f91 x:0 0 0 2024-04-24T23:13:56.410 INFO:tasks.workunit.client.0.smithi031.stdout:2/641: dread d5/d28/f3a [0,4194304] 0 2024-04-24T23:13:56.410 INFO:tasks.workunit.client.0.smithi031.stdout:7/766: read d1/d2/f46 [3839670,967] 0 2024-04-24T23:13:56.411 INFO:tasks.workunit.client.0.smithi031.stdout:5/709: mknod d0/d6/d3b/d4b/dc/d25/d55/ce6 0 2024-04-24T23:13:56.412 INFO:tasks.workunit.client.0.smithi031.stdout:5/710: chown d0/d6/d3b/d4b/dc/l48 8412043 1 2024-04-24T23:13:56.412 INFO:tasks.workunit.client.0.smithi031.stdout:5/711: chown d0/d6/d3b 9570 1 2024-04-24T23:13:56.416 INFO:tasks.workunit.client.1.smithi040.stdout:5/448: chown d2/d7/d8/db/c8d 31894 1 2024-04-24T23:13:56.416 INFO:tasks.workunit.client.0.smithi031.stdout:3/696: symlink d6/df/d16/d2a/d95/d9b/d29/dcb/lcf 0 2024-04-24T23:13:56.418 INFO:tasks.workunit.client.1.smithi040.stdout:1/472: write da/d12/d14/f24 [337501,80706] 0 2024-04-24T23:13:56.419 INFO:tasks.workunit.client.1.smithi040.stdout:8/475: mkdir d0/d52/d9b 0 2024-04-24T23:13:56.420 INFO:tasks.workunit.client.1.smithi040.stdout:8/476: chown d0/d28/f39 0 1 2024-04-24T23:13:56.421 INFO:tasks.workunit.client.1.smithi040.stdout:7/508: rmdir db/d43 39 2024-04-24T23:13:56.422 INFO:tasks.workunit.client.0.smithi031.stdout:7/767: read d1/d2/dd/d33/d5e/d64/faa [243167,88475] 0 2024-04-24T23:13:56.422 INFO:tasks.workunit.client.1.smithi040.stdout:5/449: dwrite d2/d2f/d14/f86 [0,4194304] 0 2024-04-24T23:13:56.424 INFO:tasks.workunit.client.1.smithi040.stdout:7/509: stat db/f1c 0 2024-04-24T23:13:56.424 INFO:tasks.workunit.client.0.smithi031.stdout:2/642: creat d5/d8/d45/d5b/d49/dc4/fe9 x:0 0 0 2024-04-24T23:13:56.424 INFO:tasks.workunit.client.0.smithi031.stdout:8/893: write d8/d15/d37/d43/daf/fb9 [156178,5562] 0 2024-04-24T23:13:56.425 INFO:tasks.workunit.client.1.smithi040.stdout:5/450: chown d2/lf 30168 1 2024-04-24T23:13:56.425 INFO:tasks.workunit.client.1.smithi040.stdout:7/510: chown db/d12/d3f 0 1 2024-04-24T23:13:56.426 INFO:tasks.workunit.client.1.smithi040.stdout:7/511: stat db/f71 0 2024-04-24T23:13:56.426 INFO:tasks.workunit.client.0.smithi031.stdout:2/643: write d5/d6/f16 [5263811,62133] 0 2024-04-24T23:13:56.432 INFO:tasks.workunit.client.0.smithi031.stdout:3/697: creat d6/d55/fd0 x:0 0 0 2024-04-24T23:13:56.436 INFO:tasks.workunit.client.1.smithi040.stdout:2/528: chown da/d1c/d37/d39/d3f/lc6 8916911 1 2024-04-24T23:13:56.439 INFO:tasks.workunit.client.1.smithi040.stdout:5/451: dwrite d2/d7/d8/d64/d6b/f8b [0,4194304] 0 2024-04-24T23:13:56.439 INFO:tasks.workunit.client.0.smithi031.stdout:7/768: mkdir d1/dbe/d10c 0 2024-04-24T23:13:56.440 INFO:tasks.workunit.client.1.smithi040.stdout:0/529: rmdir df/d12/d58/da9 0 2024-04-24T23:13:56.441 INFO:tasks.workunit.client.1.smithi040.stdout:0/530: readlink df/d1e/d2b/l37 0 2024-04-24T23:13:56.441 INFO:tasks.workunit.client.0.smithi031.stdout:9/843: dwrite d2/dae/d14/d78/d84/d4a/fb1 [0,4194304] 0 2024-04-24T23:13:56.456 INFO:tasks.workunit.client.0.smithi031.stdout:5/712: dread d0/d6/d3b/d4b/dc/d8b/fa6 [0,4194304] 0 2024-04-24T23:13:56.473 INFO:tasks.workunit.client.0.smithi031.stdout:0/808: sync 2024-04-24T23:13:56.473 INFO:tasks.workunit.client.0.smithi031.stdout:1/678: sync 2024-04-24T23:13:56.476 INFO:tasks.workunit.client.1.smithi040.stdout:8/477: rmdir d0 39 2024-04-24T23:13:56.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/698: truncate d6/df/d16/d53/f6e 786755 0 2024-04-24T23:13:56.479 INFO:tasks.workunit.client.1.smithi040.stdout:4/419: truncate da/f1c 2331860 0 2024-04-24T23:13:56.481 INFO:tasks.workunit.client.1.smithi040.stdout:7/512: symlink db/d12/d3f/lb4 0 2024-04-24T23:13:56.481 INFO:tasks.workunit.client.0.smithi031.stdout:2/644: rmdir d5/d8/d45 39 2024-04-24T23:13:56.482 INFO:tasks.workunit.client.0.smithi031.stdout:7/769: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d84/l8f 3526630 1 2024-04-24T23:13:56.483 INFO:tasks.workunit.client.1.smithi040.stdout:3/518: write d1/d9/d39/f78 [910794,21335] 0 2024-04-24T23:13:56.487 INFO:tasks.workunit.client.0.smithi031.stdout:5/713: creat d0/d6/d3b/d4b/dc/d8b/d9b/fe7 x:0 0 0 2024-04-24T23:13:56.489 INFO:tasks.workunit.client.0.smithi031.stdout:8/894: write d8/d1a/d1d/d50/f88 [27771,48477] 0 2024-04-24T23:13:56.504 INFO:tasks.workunit.client.0.smithi031.stdout:5/714: symlink d0/d6/d3b/d4b/dc/d86/d91/le8 0 2024-04-24T23:13:56.504 INFO:tasks.workunit.client.0.smithi031.stdout:3/699: dread d6/df/d16/d2a/d95/d9b/f7c [0,4194304] 0 2024-04-24T23:13:56.507 INFO:tasks.workunit.client.0.smithi031.stdout:1/679: mknod d1/cd6 0 2024-04-24T23:13:56.510 INFO:tasks.workunit.client.0.smithi031.stdout:2/645: dread d5/d8/d15/f50 [0,4194304] 0 2024-04-24T23:13:56.511 INFO:tasks.workunit.client.0.smithi031.stdout:8/895: creat d8/d1a/d74/f130 x:0 0 0 2024-04-24T23:13:56.513 INFO:tasks.workunit.client.0.smithi031.stdout:9/844: creat d2/dae/d14/d78/d84/d4a/f102 x:0 0 0 2024-04-24T23:13:56.515 INFO:tasks.workunit.client.0.smithi031.stdout:9/845: truncate d2/dae/d14/d78/d84/d4a/d4b/f87 1251671 0 2024-04-24T23:13:56.516 INFO:tasks.workunit.client.1.smithi040.stdout:9/493: creat d7/da/d17/fae x:0 0 0 2024-04-24T23:13:56.521 INFO:tasks.workunit.client.0.smithi031.stdout:7/770: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/da0/f36 [0,4194304] 0 2024-04-24T23:13:56.522 INFO:tasks.workunit.client.0.smithi031.stdout:1/680: creat d1/da/d12/d7e/d7b/d4e/fd7 x:0 0 0 2024-04-24T23:13:56.523 INFO:tasks.workunit.client.1.smithi040.stdout:2/529: rmdir da/d10/d15 39 2024-04-24T23:13:56.525 INFO:tasks.workunit.client.0.smithi031.stdout:7/771: truncate d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea/f103 493641 0 2024-04-24T23:13:56.529 INFO:tasks.workunit.client.1.smithi040.stdout:0/531: rmdir df/d12/d1a/d71 39 2024-04-24T23:13:56.529 INFO:tasks.workunit.client.1.smithi040.stdout:5/452: dread d2/d7/d8/d1f/f51 [0,4194304] 0 2024-04-24T23:13:56.530 INFO:tasks.workunit.client.0.smithi031.stdout:2/646: mknod d5/d6/cea 0 2024-04-24T23:13:56.530 INFO:tasks.workunit.client.1.smithi040.stdout:1/473: getdents da/d35/da3 0 2024-04-24T23:13:56.531 INFO:tasks.workunit.client.1.smithi040.stdout:0/532: dread - df/d1e/d20/d21/d41/d52/f87 zero size 2024-04-24T23:13:56.532 INFO:tasks.workunit.client.1.smithi040.stdout:8/478: chown d0/d8/d22 19 1 2024-04-24T23:13:56.535 INFO:tasks.workunit.client.0.smithi031.stdout:5/715: rename d0/d6/d3b/d4b/dc/d25/d55/lda to d0/d7/d74/dbd/le9 0 2024-04-24T23:13:56.538 INFO:tasks.workunit.client.1.smithi040.stdout:3/519: creat d1/d2d/d72/fb8 x:0 0 0 2024-04-24T23:13:56.540 INFO:tasks.workunit.client.0.smithi031.stdout:0/809: creat da/d20/d4e/f110 x:0 0 0 2024-04-24T23:13:56.541 INFO:tasks.workunit.client.1.smithi040.stdout:9/494: dread - d7/da/d17/d40/d55/d2f/d1f/d4e/f68 zero size 2024-04-24T23:13:56.541 INFO:tasks.workunit.client.1.smithi040.stdout:9/495: readlink l5 0 2024-04-24T23:13:56.542 INFO:tasks.workunit.client.0.smithi031.stdout:3/700: dwrite d6/d14/d1d/d59/f6c [0,4194304] 0 2024-04-24T23:13:56.544 INFO:tasks.workunit.client.1.smithi040.stdout:6/461: rename d7/d1a/d20/d51/d1e/c77 to d7/d1a/d20/c92 0 2024-04-24T23:13:56.545 INFO:tasks.workunit.client.0.smithi031.stdout:8/896: creat d8/d15/d37/d62/dfa/f131 x:0 0 0 2024-04-24T23:13:56.546 INFO:tasks.workunit.client.1.smithi040.stdout:8/479: mkdir d0/d28/d9c 0 2024-04-24T23:13:56.549 INFO:tasks.workunit.client.0.smithi031.stdout:9/846: rename d2/dae/d14/d78/d84/d4a/d4b/f57 to d2/dae/d14/d78/d84/d4a/dab/db8/f103 0 2024-04-24T23:13:56.551 INFO:tasks.workunit.client.1.smithi040.stdout:7/513: truncate f2 5250623 0 2024-04-24T23:13:56.553 INFO:tasks.workunit.client.1.smithi040.stdout:3/520: fsync d1/f32 0 2024-04-24T23:13:56.554 INFO:tasks.workunit.client.1.smithi040.stdout:3/521: readlink d1/d9/d18/d36/l9b 0 2024-04-24T23:13:56.556 INFO:tasks.workunit.client.1.smithi040.stdout:9/496: rename d7/da/d17/d40/d55/d2f/d1f/d34/f47 to d7/da/d17/d40/d55/d2f/d28/d67/faf 0 2024-04-24T23:13:56.560 INFO:tasks.workunit.client.1.smithi040.stdout:6/462: creat d7/d1a/d20/d51/d1e/d5d/f93 x:0 0 0 2024-04-24T23:13:56.562 INFO:tasks.workunit.client.1.smithi040.stdout:8/480: symlink d0/d16/d19/d4f/d87/l9d 0 2024-04-24T23:13:56.563 INFO:tasks.workunit.client.0.smithi031.stdout:1/681: truncate d1/d3e/d46/d74/fb5 95994 0 2024-04-24T23:13:56.563 INFO:tasks.workunit.client.1.smithi040.stdout:4/420: dwrite da/fb [0,4194304] 0 2024-04-24T23:13:56.564 INFO:tasks.workunit.client.0.smithi031.stdout:3/701: mkdir d6/df/d16/d2a/d95/d9b/d1a/d25/dd1 0 2024-04-24T23:13:56.570 INFO:tasks.workunit.client.1.smithi040.stdout:8/481: dwrite d0/d16/f7c [0,4194304] 0 2024-04-24T23:13:56.571 INFO:tasks.workunit.client.1.smithi040.stdout:7/514: dread - db/f71 zero size 2024-04-24T23:13:56.573 INFO:tasks.workunit.client.0.smithi031.stdout:0/810: mkdir da/d18/d1a/d37/d111 0 2024-04-24T23:13:56.576 INFO:tasks.workunit.client.1.smithi040.stdout:9/497: mknod d7/d92/cb0 0 2024-04-24T23:13:56.577 INFO:tasks.workunit.client.1.smithi040.stdout:6/463: dread d7/f9 [0,4194304] 0 2024-04-24T23:13:56.583 INFO:tasks.workunit.client.1.smithi040.stdout:0/533: truncate df/d1e/d20/d21/d41/f6d 728461 0 2024-04-24T23:13:56.585 INFO:tasks.workunit.client.0.smithi031.stdout:8/897: creat d8/d1a/d1d/d50/dc0/d9d/d12b/f132 x:0 0 0 2024-04-24T23:13:56.590 INFO:tasks.workunit.client.0.smithi031.stdout:5/716: creat d0/d6/d3b/d4b/dc/fea x:0 0 0 2024-04-24T23:13:56.591 INFO:tasks.workunit.client.0.smithi031.stdout:2/647: write d5/d8/d45/d5b/d49/d2f/faa [351849,12456] 0 2024-04-24T23:13:56.593 INFO:tasks.workunit.client.0.smithi031.stdout:9/847: dwrite d2/d43/d9d/dbd/fda [0,4194304] 0 2024-04-24T23:13:56.593 INFO:tasks.workunit.client.1.smithi040.stdout:5/453: link d2/d7/d8/d1f/d57/d27/d3c/d43/f6f d2/d7/d8/d64/f93 0 2024-04-24T23:13:56.595 INFO:tasks.workunit.client.0.smithi031.stdout:7/772: getdents d1/d2/dd/d33/d5e/d64/d6d/de1/d84 0 2024-04-24T23:13:56.595 INFO:tasks.workunit.client.0.smithi031.stdout:4/869: sync 2024-04-24T23:13:56.595 INFO:tasks.workunit.client.0.smithi031.stdout:6/803: sync 2024-04-24T23:13:56.599 INFO:tasks.workunit.client.1.smithi040.stdout:7/515: truncate db/d2f/f35 1217062 0 2024-04-24T23:13:56.600 INFO:tasks.workunit.client.1.smithi040.stdout:5/454: dwrite d2/d7/d8/d1f/d31/f4e [0,4194304] 0 2024-04-24T23:13:56.601 INFO:tasks.workunit.client.1.smithi040.stdout:7/516: chown db/d12/d20/d52/d6c/fae 15732 1 2024-04-24T23:13:56.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/811: dread da/fe [4194304,4194304] 0 2024-04-24T23:13:56.603 INFO:tasks.workunit.client.0.smithi031.stdout:8/898: creat d8/d15/d37/d43/daf/f133 x:0 0 0 2024-04-24T23:13:56.604 INFO:tasks.workunit.client.1.smithi040.stdout:2/530: getdents da/d10/d4d 0 2024-04-24T23:13:56.604 INFO:tasks.workunit.client.1.smithi040.stdout:5/455: dread d2/d2f/d14/f76 [0,4194304] 0 2024-04-24T23:13:56.606 INFO:tasks.workunit.client.1.smithi040.stdout:2/531: write da/d10/d7a/f8d [1086156,20544] 0 2024-04-24T23:13:56.608 INFO:tasks.workunit.client.0.smithi031.stdout:2/648: dwrite d5/d8/fcd [0,4194304] 0 2024-04-24T23:13:56.611 INFO:tasks.workunit.client.0.smithi031.stdout:9/848: chown d2/f75 6144335 1 2024-04-24T23:13:56.612 INFO:tasks.workunit.client.0.smithi031.stdout:6/804: dwrite d13/d1b/d32/fc3 [0,4194304] 0 2024-04-24T23:13:56.613 INFO:tasks.workunit.client.0.smithi031.stdout:1/682: link d1/d3e/l60 d1/da/dbc/ld8 0 2024-04-24T23:13:56.615 INFO:tasks.workunit.client.1.smithi040.stdout:0/534: fdatasync df/d1e/da7/d6b/f73 0 2024-04-24T23:13:56.616 INFO:tasks.workunit.client.0.smithi031.stdout:6/805: chown d13/d15/d1d/d23/d24/d8b/fcd 161 1 2024-04-24T23:13:56.618 INFO:tasks.workunit.client.0.smithi031.stdout:4/870: creat d3/d20/da7/f114 x:0 0 0 2024-04-24T23:13:56.621 INFO:tasks.workunit.client.0.smithi031.stdout:8/899: dread d8/d15/d37/d43/d5c/de1/f72 [0,4194304] 0 2024-04-24T23:13:56.623 INFO:tasks.workunit.client.0.smithi031.stdout:8/900: write d8/d15/f2a [1104963,236] 0 2024-04-24T23:13:56.626 INFO:tasks.workunit.client.0.smithi031.stdout:5/717: mkdir d0/d6/d3b/d4b/d1d/d2e/d28/deb 0 2024-04-24T23:13:56.631 INFO:tasks.workunit.client.0.smithi031.stdout:3/702: dwrite d6/df/d16/d53/f6e [0,4194304] 0 2024-04-24T23:13:56.631 INFO:tasks.workunit.client.1.smithi040.stdout:9/498: write d7/da/d17/d40/d55/d2f/d1f/d4e/f68 [658996,46459] 0 2024-04-24T23:13:56.633 INFO:tasks.workunit.client.1.smithi040.stdout:9/499: write d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 [6142515,120763] 0 2024-04-24T23:13:56.634 INFO:tasks.workunit.client.0.smithi031.stdout:0/812: creat da/d86/da3/d104/f112 x:0 0 0 2024-04-24T23:13:56.635 INFO:tasks.workunit.client.1.smithi040.stdout:7/517: mknod db/d12/d20/d52/d44/cb5 0 2024-04-24T23:13:56.638 INFO:tasks.workunit.client.1.smithi040.stdout:5/456: mknod d2/d7/d8/c94 0 2024-04-24T23:13:56.642 INFO:tasks.workunit.client.1.smithi040.stdout:5/457: chown d2/d7/d8/d1f/d57/d27/d3d/f7a 5380541 1 2024-04-24T23:13:56.642 INFO:tasks.workunit.client.1.smithi040.stdout:5/458: chown d2/d7/d8/db/l45 1 1 2024-04-24T23:13:56.642 INFO:tasks.workunit.client.1.smithi040.stdout:2/532: symlink da/d1c/d37/d39/d3f/d52/d78/lc8 0 2024-04-24T23:13:56.644 INFO:tasks.workunit.client.0.smithi031.stdout:9/849: truncate d2/dae/d14/d1d/f35 220452 0 2024-04-24T23:13:56.647 INFO:tasks.workunit.client.1.smithi040.stdout:0/535: symlink df/d1e/d20/d21/d51/da3/laf 0 2024-04-24T23:13:56.647 INFO:tasks.workunit.client.1.smithi040.stdout:0/536: chown df/d1e/d2b/d6e 203 1 2024-04-24T23:13:56.648 INFO:tasks.workunit.client.1.smithi040.stdout:4/421: link da/d3f/d40/d47/d75/c8f da/d3f/d6b/c95 0 2024-04-24T23:13:56.648 INFO:tasks.workunit.client.1.smithi040.stdout:8/482: dwrite d0/d8/dc/f3a [0,4194304] 0 2024-04-24T23:13:56.650 INFO:tasks.workunit.client.1.smithi040.stdout:4/422: write da/d3f/d40/d41/f67 [721624,117842] 0 2024-04-24T23:13:56.652 INFO:tasks.workunit.client.0.smithi031.stdout:5/718: creat d0/fec x:0 0 0 2024-04-24T23:13:56.652 INFO:tasks.workunit.client.1.smithi040.stdout:3/522: getdents d1/d2d/d72 0 2024-04-24T23:13:56.657 INFO:tasks.workunit.client.1.smithi040.stdout:9/500: rmdir d7/d5e 39 2024-04-24T23:13:56.659 INFO:tasks.workunit.client.0.smithi031.stdout:0/813: rename da/d18/fb5 to da/d18/d5a/d7d/dbd/f113 0 2024-04-24T23:13:56.661 INFO:tasks.workunit.client.1.smithi040.stdout:7/518: symlink db/d2f/d37/lb6 0 2024-04-24T23:13:56.661 INFO:tasks.workunit.client.0.smithi031.stdout:0/814: truncate da/d18/d1a/dd6/f107 155230 0 2024-04-24T23:13:56.664 INFO:tasks.workunit.client.1.smithi040.stdout:5/459: write d2/d7/d8/d1f/d57/d3e/f8a [272135,77528] 0 2024-04-24T23:13:56.668 INFO:tasks.workunit.client.1.smithi040.stdout:5/460: dread d2/d7/d8/d1f/d31/f4e [0,4194304] 0 2024-04-24T23:13:56.668 INFO:tasks.workunit.client.1.smithi040.stdout:2/533: write da/d10/d15/daa/fc3 [357239,38997] 0 2024-04-24T23:13:56.670 INFO:tasks.workunit.client.0.smithi031.stdout:4/871: dwrite d3/d23/d3b/f9c [0,4194304] 0 2024-04-24T23:13:56.672 INFO:tasks.workunit.client.0.smithi031.stdout:9/850: creat d2/dae/d14/d78/d84/d4a/dab/db7/f104 x:0 0 0 2024-04-24T23:13:56.674 INFO:tasks.workunit.client.1.smithi040.stdout:0/537: creat df/d1e/d20/d61/fb0 x:0 0 0 2024-04-24T23:13:56.675 INFO:tasks.workunit.client.1.smithi040.stdout:0/538: read df/d12/d1a/f4e [3868027,77448] 0 2024-04-24T23:13:56.676 INFO:tasks.workunit.client.1.smithi040.stdout:0/539: write df/d1e/da7/d6b/f6f [884579,83908] 0 2024-04-24T23:13:56.678 INFO:tasks.workunit.client.1.smithi040.stdout:0/540: write df/d1e/d20/d61/f86 [63748,94788] 0 2024-04-24T23:13:56.683 INFO:tasks.workunit.client.1.smithi040.stdout:3/523: rename d1/d9/d18/d36/d61/fa0 to d1/d2d/de/d30/fb9 0 2024-04-24T23:13:56.691 INFO:tasks.workunit.client.1.smithi040.stdout:2/534: rmdir da/d1c/d4f 39 2024-04-24T23:13:56.691 INFO:tasks.workunit.client.1.smithi040.stdout:7/519: write db/f31 [3504954,92336] 0 2024-04-24T23:13:56.693 INFO:tasks.workunit.client.1.smithi040.stdout:2/535: write da/f40 [5233571,59790] 0 2024-04-24T23:13:56.693 INFO:tasks.workunit.client.0.smithi031.stdout:0/815: write da/d22/f96 [886025,2733] 0 2024-04-24T23:13:56.693 INFO:tasks.workunit.client.0.smithi031.stdout:6/806: dwrite d13/d15/d1d/d23/d44/f5b [0,4194304] 0 2024-04-24T23:13:56.695 INFO:tasks.workunit.client.0.smithi031.stdout:7/773: link d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/f107 d1/f10d 0 2024-04-24T23:13:56.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/816: fsync da/d86/da3/dcd/de7/ddb/fe5 0 2024-04-24T23:13:56.699 INFO:tasks.workunit.client.0.smithi031.stdout:2/649: creat d5/d6/d68/feb x:0 0 0 2024-04-24T23:13:56.701 INFO:tasks.workunit.client.1.smithi040.stdout:4/423: creat da/d3f/d40/d41/d8e/f96 x:0 0 0 2024-04-24T23:13:56.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/683: link d1/da/d12/l9d d1/d33/d36/d81/ld9 0 2024-04-24T23:13:56.704 INFO:tasks.workunit.client.1.smithi040.stdout:0/541: rename df/d1e/d20/d21/d51/c5d to df/d12/d4b/cb1 0 2024-04-24T23:13:56.705 INFO:tasks.workunit.client.1.smithi040.stdout:8/483: truncate d0/d16/d19/f5f 2048285 0 2024-04-24T23:13:56.706 INFO:tasks.workunit.client.1.smithi040.stdout:0/542: chown df/d12/d1a/fad 996 1 2024-04-24T23:13:56.706 INFO:tasks.workunit.client.1.smithi040.stdout:9/501: mknod d7/d5e/cb1 0 2024-04-24T23:13:56.708 INFO:tasks.workunit.client.0.smithi031.stdout:8/901: rmdir d8/d15/d37/d43/dbd/d11d 0 2024-04-24T23:13:56.709 INFO:tasks.workunit.client.0.smithi031.stdout:8/902: chown d8/d15/l46 44426361 1 2024-04-24T23:13:56.711 INFO:tasks.workunit.client.0.smithi031.stdout:5/719: write d0/d6/d3b/d4b/dc/f84 [2069945,67272] 0 2024-04-24T23:13:56.714 INFO:tasks.workunit.client.0.smithi031.stdout:6/807: symlink d13/d1b/d32/l10d 0 2024-04-24T23:13:56.719 INFO:tasks.workunit.client.1.smithi040.stdout:3/524: rename d1/d2d/de/d30/l93 to d1/d9/d18/d36/d55/lba 0 2024-04-24T23:13:56.721 INFO:tasks.workunit.client.0.smithi031.stdout:6/808: dwrite d13/d15/d1d/d23/d24/d43/d68/f72 [0,4194304] 0 2024-04-24T23:13:56.722 INFO:tasks.workunit.client.1.smithi040.stdout:8/484: rmdir d0/d8/d66 39 2024-04-24T23:13:56.724 INFO:tasks.workunit.client.0.smithi031.stdout:0/817: mkdir da/d86/da3/dcd/ddc/d114 0 2024-04-24T23:13:56.727 INFO:tasks.workunit.client.1.smithi040.stdout:0/543: unlink df/l10 0 2024-04-24T23:13:56.728 INFO:tasks.workunit.client.0.smithi031.stdout:2/650: symlink d5/d6/d68/lec 0 2024-04-24T23:13:56.729 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:56 smithi031 ceph-mon[26411]: pgmap v31: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 188 MiB/s wr, 363 op/s 2024-04-24T23:13:56.729 INFO:tasks.workunit.client.1.smithi040.stdout:9/502: chown d7/da/d17/d40/d55/d2f/d1f/d31/c38 417599 1 2024-04-24T23:13:56.729 INFO:tasks.workunit.client.1.smithi040.stdout:0/544: read df/d12/d1a/f67 [198164,57999] 0 2024-04-24T23:13:56.730 INFO:tasks.workunit.client.1.smithi040.stdout:1/474: sync 2024-04-24T23:13:56.731 INFO:tasks.workunit.client.1.smithi040.stdout:9/503: chown d7/da/d17/d40/d55/d2f/d1f/f8a 1819 1 2024-04-24T23:13:56.731 INFO:tasks.workunit.client.0.smithi031.stdout:4/872: rename d3/d23/c74 to d3/d5e/c115 0 2024-04-24T23:13:56.732 INFO:tasks.workunit.client.1.smithi040.stdout:9/504: write d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 [8351139,39062] 0 2024-04-24T23:13:56.733 INFO:tasks.workunit.client.1.smithi040.stdout:7/520: mkdir db/d43/db7 0 2024-04-24T23:13:56.735 INFO:tasks.workunit.client.0.smithi031.stdout:9/851: mkdir d2/dae/d105 0 2024-04-24T23:13:56.736 INFO:tasks.workunit.client.1.smithi040.stdout:2/536: symlink da/d1c/d37/d39/d3f/d52/d78/dc7/lc9 0 2024-04-24T23:13:56.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/684: symlink d1/d3e/d46/d74/lda 0 2024-04-24T23:13:56.740 INFO:tasks.workunit.client.0.smithi031.stdout:3/703: getdents d6/df/d16/d2a/d95/d9b/d29 0 2024-04-24T23:13:56.742 INFO:tasks.workunit.client.0.smithi031.stdout:7/774: dread d1/d2/dd/d33/d5e/d64/d6d/de1/da0/fc0 [0,4194304] 0 2024-04-24T23:13:56.744 INFO:tasks.workunit.client.0.smithi031.stdout:7/775: dread - d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/ffd zero size 2024-04-24T23:13:56.746 INFO:tasks.workunit.client.0.smithi031.stdout:7/776: write d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea/f103 [21330,20696] 0 2024-04-24T23:13:56.748 INFO:tasks.workunit.client.0.smithi031.stdout:7/777: stat d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea/cf7 0 2024-04-24T23:13:56.752 INFO:tasks.workunit.client.0.smithi031.stdout:6/809: rmdir d13/d15/d1d/d23/d44/d47 39 2024-04-24T23:13:56.761 INFO:tasks.workunit.client.0.smithi031.stdout:2/651: rename d5/d28/f40 to d5/d8/d15/d56/d84/d75/fed 0 2024-04-24T23:13:56.763 INFO:tasks.workunit.client.0.smithi031.stdout:4/873: creat d3/da3/f116 x:0 0 0 2024-04-24T23:13:56.765 INFO:tasks.workunit.client.0.smithi031.stdout:4/874: write d3/d23/d3b/fe2 [4552662,99890] 0 2024-04-24T23:13:56.766 INFO:tasks.workunit.client.0.smithi031.stdout:4/875: chown d3/d20/d25/fb0 2490 1 2024-04-24T23:13:56.773 INFO:tasks.workunit.client.0.smithi031.stdout:4/876: dwrite d3/d20/d2d/d44/fb2 [0,4194304] 0 2024-04-24T23:13:56.775 INFO:tasks.workunit.client.1.smithi040.stdout:3/525: symlink d1/d2d/d44/lbb 0 2024-04-24T23:13:56.776 INFO:tasks.workunit.client.1.smithi040.stdout:8/485: rename d0/d16/d19/d71/d89 to d0/d16/d19/d4f/d9e 0 2024-04-24T23:13:56.777 INFO:tasks.workunit.client.0.smithi031.stdout:0/818: dwrite da/d86/da3/dcd/de7/fc7 [4194304,4194304] 0 2024-04-24T23:13:56.778 INFO:tasks.workunit.client.1.smithi040.stdout:7/521: mkdir db/d12/d20/d52/d6c/d8f/db8 0 2024-04-24T23:13:56.778 INFO:tasks.workunit.client.1.smithi040.stdout:2/537: truncate da/d1c/d37/d39/d3f/d52/f87 1748385 0 2024-04-24T23:13:56.778 INFO:tasks.workunit.client.1.smithi040.stdout:5/461: link d2/d7/d8/d1f/d57/f70 d2/d7/d8/d1f/d57/d27/d3c/f95 0 2024-04-24T23:13:56.779 INFO:tasks.workunit.client.0.smithi031.stdout:0/819: chown l7 10819601 1 2024-04-24T23:13:56.780 INFO:tasks.workunit.client.0.smithi031.stdout:7/778: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d10e 0 2024-04-24T23:13:56.781 INFO:tasks.workunit.client.0.smithi031.stdout:0/820: fdatasync da/f10a 0 2024-04-24T23:13:56.784 INFO:tasks.workunit.client.0.smithi031.stdout:0/821: chown da/d18/ceb 14671 1 2024-04-24T23:13:56.784 INFO:tasks.workunit.client.0.smithi031.stdout:9/852: dwrite d2/d16/dbe/fb2 [0,4194304] 0 2024-04-24T23:13:56.786 INFO:tasks.workunit.client.1.smithi040.stdout:3/526: dwrite d1/d2a/d6a/d94/f95 [0,4194304] 0 2024-04-24T23:13:56.787 INFO:tasks.workunit.client.0.smithi031.stdout:6/810: symlink d13/d15/d1d/d23/d24/d63/l10e 0 2024-04-24T23:13:56.788 INFO:tasks.workunit.client.1.smithi040.stdout:3/527: chown d1/d2d/d44/f52 2 1 2024-04-24T23:13:56.788 INFO:tasks.workunit.client.1.smithi040.stdout:3/528: readlink d1/d2d/de/d30/d37/lab 0 2024-04-24T23:13:56.789 INFO:tasks.workunit.client.0.smithi031.stdout:0/822: read - da/d18/d5a/d7d/ffa zero size 2024-04-24T23:13:56.790 INFO:tasks.workunit.client.0.smithi031.stdout:6/811: chown d13/d15/d1d/cb0 130297863 1 2024-04-24T23:13:56.790 INFO:tasks.workunit.client.1.smithi040.stdout:0/545: fsync df/d1e/d20/d21/d51/f57 0 2024-04-24T23:13:56.792 INFO:tasks.workunit.client.0.smithi031.stdout:1/685: dwrite d1/d33/d36/fa1 [0,4194304] 0 2024-04-24T23:13:56.792 INFO:tasks.workunit.client.1.smithi040.stdout:9/505: symlink d7/da/d17/d40/d55/lb2 0 2024-04-24T23:13:56.793 INFO:tasks.workunit.client.0.smithi031.stdout:0/823: chown da/d20/d8e/fd4 1 1 2024-04-24T23:13:56.799 INFO:tasks.workunit.client.0.smithi031.stdout:0/824: dread da/d18/f9a [0,4194304] 0 2024-04-24T23:13:56.800 INFO:tasks.workunit.client.0.smithi031.stdout:8/903: truncate d8/d1a/d1d/d50/dc0/d9d/fea 3985719 0 2024-04-24T23:13:56.802 INFO:tasks.workunit.client.0.smithi031.stdout:3/704: mknod d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/cd2 0 2024-04-24T23:13:56.802 INFO:tasks.workunit.client.1.smithi040.stdout:5/462: rmdir d2/d7/d8/d1f/d57/d27 39 2024-04-24T23:13:56.805 INFO:tasks.workunit.client.0.smithi031.stdout:4/877: rename d3/d23/d37/d86/fde to d3/d5e/dc2/f117 0 2024-04-24T23:13:56.806 INFO:tasks.workunit.client.0.smithi031.stdout:2/652: dwrite d5/d6/d29/d26/db0/dc2/dad/fe4 [0,4194304] 0 2024-04-24T23:13:56.807 INFO:tasks.workunit.client.1.smithi040.stdout:8/486: write d0/d8/d66/f78 [964579,106955] 0 2024-04-24T23:13:56.812 INFO:tasks.workunit.client.0.smithi031.stdout:8/904: dread d8/d15/f2a [0,4194304] 0 2024-04-24T23:13:56.814 INFO:tasks.workunit.client.1.smithi040.stdout:7/522: fsync db/d12/d62/d8a/f8b 0 2024-04-24T23:13:56.815 INFO:tasks.workunit.client.1.smithi040.stdout:2/538: getdents da/d1c/d7e 0 2024-04-24T23:13:56.815 INFO:tasks.workunit.client.1.smithi040.stdout:7/523: chown db/d12/f16 220 1 2024-04-24T23:13:56.817 INFO:tasks.workunit.client.1.smithi040.stdout:5/463: creat d2/d7/d8/db/f96 x:0 0 0 2024-04-24T23:13:56.817 INFO:tasks.workunit.client.1.smithi040.stdout:8/487: mknod d0/d28/c9f 0 2024-04-24T23:13:56.820 INFO:tasks.workunit.client.1.smithi040.stdout:4/424: getdents da/d3f/d6b/d3c 0 2024-04-24T23:13:56.821 INFO:tasks.workunit.client.1.smithi040.stdout:9/506: link d7/da/d17/d40/d55/d61/d6c/f9b d7/da/d17/d40/d55/d2f/d1f/d4e/d83/fb3 0 2024-04-24T23:13:56.823 INFO:tasks.workunit.client.1.smithi040.stdout:7/524: rmdir db/d12/d20/d52 39 2024-04-24T23:13:56.823 INFO:tasks.workunit.client.0.smithi031.stdout:5/720: getdents d0/d6/d3b/d4b/d1d/d2e/d28/d77 0 2024-04-24T23:13:56.825 INFO:tasks.workunit.client.1.smithi040.stdout:9/507: chown d7/da/d17/d40/f42 7 1 2024-04-24T23:13:56.826 INFO:tasks.workunit.client.1.smithi040.stdout:0/546: creat df/d1e/d20/fb2 x:0 0 0 2024-04-24T23:13:56.827 INFO:tasks.workunit.client.1.smithi040.stdout:2/539: dwrite da/d10/d15/f2c [4194304,4194304] 0 2024-04-24T23:13:56.829 INFO:tasks.workunit.client.0.smithi031.stdout:8/905: chown d8/d1a/d1d/l6c 119574323 1 2024-04-24T23:13:56.831 INFO:tasks.workunit.client.0.smithi031.stdout:6/812: creat d13/d1b/d32/db4/d79/dd7/dfb/f10f x:0 0 0 2024-04-24T23:13:56.832 INFO:tasks.workunit.client.0.smithi031.stdout:4/878: dread d3/d20/d2d/d44/fe4 [0,4194304] 0 2024-04-24T23:13:56.833 INFO:tasks.workunit.client.1.smithi040.stdout:0/547: chown df/d1e/da7/f65 20259 1 2024-04-24T23:13:56.837 INFO:tasks.workunit.client.1.smithi040.stdout:8/488: dwrite d0/d16/d19/d4f/d9e/d90/d91/f97 [0,4194304] 0 2024-04-24T23:13:56.837 INFO:tasks.workunit.client.1.smithi040.stdout:0/548: fsync df/d12/d58/f7d 0 2024-04-24T23:13:56.837 INFO:tasks.workunit.client.1.smithi040.stdout:2/540: mknod da/d81/da8/cca 0 2024-04-24T23:13:56.837 INFO:tasks.workunit.client.1.smithi040.stdout:6/464: sync 2024-04-24T23:13:56.838 INFO:tasks.workunit.client.0.smithi031.stdout:8/906: rmdir d8/d1a/d1d/d50 39 2024-04-24T23:13:56.840 INFO:tasks.workunit.client.1.smithi040.stdout:8/489: readlink d0/d16/d19/l60 0 2024-04-24T23:13:56.842 INFO:tasks.workunit.client.1.smithi040.stdout:7/525: dwrite db/d2f/d37/f94 [4194304,4194304] 0 2024-04-24T23:13:56.846 INFO:tasks.workunit.client.0.smithi031.stdout:3/705: creat d6/d14/fd3 x:0 0 0 2024-04-24T23:13:56.846 INFO:tasks.workunit.client.0.smithi031.stdout:3/706: chown d6/df/d16/d53/f6e 78 1 2024-04-24T23:13:56.846 INFO:tasks.workunit.client.1.smithi040.stdout:2/541: rmdir da/d10/d4d/d6c 39 2024-04-24T23:13:56.848 INFO:tasks.workunit.client.0.smithi031.stdout:8/907: read - d8/d15/d37/d62/fbb zero size 2024-04-24T23:13:56.849 INFO:tasks.workunit.client.1.smithi040.stdout:0/549: creat df/d1e/d89/fb3 x:0 0 0 2024-04-24T23:13:56.850 INFO:tasks.workunit.client.1.smithi040.stdout:6/465: dwrite d7/f71 [0,4194304] 0 2024-04-24T23:13:56.851 INFO:tasks.workunit.client.1.smithi040.stdout:9/508: dread d7/da/d17/d40/d55/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:13:56.851 INFO:tasks.workunit.client.0.smithi031.stdout:0/825: getdents da/d11/d6a 0 2024-04-24T23:13:56.853 INFO:tasks.workunit.client.1.smithi040.stdout:9/509: chown d7/d92/d9c 483420 1 2024-04-24T23:13:56.855 INFO:tasks.workunit.client.1.smithi040.stdout:6/466: truncate d7/d1a/d20/d51/d1e/d46/d69/f8d 400375 0 2024-04-24T23:13:56.855 INFO:tasks.workunit.client.0.smithi031.stdout:4/879: mknod d3/d23/d69/d3f/d9e/ddb/c118 0 2024-04-24T23:13:56.859 INFO:tasks.workunit.client.1.smithi040.stdout:8/490: mkdir d0/d8/da0 0 2024-04-24T23:13:56.861 INFO:tasks.workunit.client.1.smithi040.stdout:5/464: dread d2/d7/d8/d1f/d57/d3e/f58 [0,4194304] 0 2024-04-24T23:13:56.862 INFO:tasks.workunit.client.1.smithi040.stdout:9/510: creat d7/da/d17/d40/fb4 x:0 0 0 2024-04-24T23:13:56.863 INFO:tasks.workunit.client.0.smithi031.stdout:0/826: creat da/d11/f115 x:0 0 0 2024-04-24T23:13:56.865 INFO:tasks.workunit.client.0.smithi031.stdout:0/827: write da/fd2 [765263,118908] 0 2024-04-24T23:13:56.865 INFO:tasks.workunit.client.0.smithi031.stdout:0/828: readlink l7 0 2024-04-24T23:13:56.868 INFO:tasks.workunit.client.1.smithi040.stdout:6/467: write d7/d1a/d20/d51/d1e/f49 [3447011,15957] 0 2024-04-24T23:13:56.869 INFO:tasks.workunit.client.1.smithi040.stdout:0/550: mknod df/d12/d1a/cb4 0 2024-04-24T23:13:56.869 INFO:tasks.workunit.client.0.smithi031.stdout:7/779: write d1/d2/dd/d33/d5e/d64/d6d/da1/fcf [260356,29506] 0 2024-04-24T23:13:56.871 INFO:tasks.workunit.client.0.smithi031.stdout:4/880: creat d3/d20/d2d/f119 x:0 0 0 2024-04-24T23:13:56.871 INFO:tasks.workunit.client.1.smithi040.stdout:0/551: truncate df/d1e/f60 1089376 0 2024-04-24T23:13:56.872 INFO:tasks.workunit.client.1.smithi040.stdout:0/552: write df/d1e/d20/f43 [1766804,94126] 0 2024-04-24T23:13:56.872 INFO:tasks.workunit.client.0.smithi031.stdout:7/780: read d1/d2/dd/d33/d5e/d64/d6d/de1/da0/f36 [2924757,13563] 0 2024-04-24T23:13:56.873 INFO:tasks.workunit.client.0.smithi031.stdout:1/686: write d1/da/d10/f51 [1968762,58053] 0 2024-04-24T23:13:56.874 INFO:tasks.workunit.client.1.smithi040.stdout:0/553: truncate df/d1e/d89/fb3 409569 0 2024-04-24T23:13:56.878 INFO:tasks.workunit.client.0.smithi031.stdout:0/829: creat da/d18/d5a/d7d/f116 x:0 0 0 2024-04-24T23:13:56.880 INFO:tasks.workunit.client.1.smithi040.stdout:0/554: dread df/f63 [0,4194304] 0 2024-04-24T23:13:56.881 INFO:tasks.workunit.client.0.smithi031.stdout:0/830: dread - da/d11/d6a/dba/fca zero size 2024-04-24T23:13:56.881 INFO:tasks.workunit.client.0.smithi031.stdout:3/707: dread d6/df/d16/d2a/d95/d9b/d1a/d25/f9f [0,4194304] 0 2024-04-24T23:13:56.882 INFO:tasks.workunit.client.0.smithi031.stdout:7/781: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/ff0 [0,4194304] 0 2024-04-24T23:13:56.884 INFO:tasks.workunit.client.0.smithi031.stdout:0/831: write da/d11/f100 [19829,50276] 0 2024-04-24T23:13:56.885 INFO:tasks.workunit.client.1.smithi040.stdout:8/491: fdatasync d0/d8/d55/f64 0 2024-04-24T23:13:56.890 INFO:tasks.workunit.client.1.smithi040.stdout:5/465: mknod d2/d2f/d14/c97 0 2024-04-24T23:13:56.890 INFO:tasks.workunit.client.0.smithi031.stdout:9/853: truncate d2/dae/d85/f9b 1740275 0 2024-04-24T23:13:56.894 INFO:tasks.workunit.client.0.smithi031.stdout:2/653: dwrite d5/d28/f37 [0,4194304] 0 2024-04-24T23:13:56.898 INFO:tasks.workunit.client.1.smithi040.stdout:2/542: rename da/d10/d4d/d6c/c93 to da/d1c/d37/d39/d3f/db2/ccb 0 2024-04-24T23:13:56.899 INFO:tasks.workunit.client.1.smithi040.stdout:2/543: chown da/d10/l1f 1 1 2024-04-24T23:13:56.900 INFO:tasks.workunit.client.1.smithi040.stdout:4/425: truncate da/d3f/d40/d41/d60/f82 2394047 0 2024-04-24T23:13:56.900 INFO:tasks.workunit.client.1.smithi040.stdout:2/544: write da/d10/f2b [3262361,57269] 0 2024-04-24T23:13:56.901 INFO:tasks.workunit.client.0.smithi031.stdout:5/721: write d0/d6/d3b/d4b/dc/f9a [1238805,119310] 0 2024-04-24T23:13:56.901 INFO:tasks.workunit.client.1.smithi040.stdout:4/426: dread - da/d3f/d6b/d3c/f90 zero size 2024-04-24T23:13:56.902 INFO:tasks.workunit.client.1.smithi040.stdout:0/555: symlink df/d1e/d2b/d6e/lb5 0 2024-04-24T23:13:56.906 INFO:tasks.workunit.client.1.smithi040.stdout:8/492: mknod d0/d8/d66/ca1 0 2024-04-24T23:13:56.908 INFO:tasks.workunit.client.0.smithi031.stdout:6/813: write d13/d15/d1d/d23/d2d/f41 [3446571,17550] 0 2024-04-24T23:13:56.908 INFO:tasks.workunit.client.0.smithi031.stdout:3/708: dread d6/d14/f43 [0,4194304] 0 2024-04-24T23:13:56.910 INFO:tasks.workunit.client.0.smithi031.stdout:6/814: write d13/d15/d1d/d23/d2d/fff [812094,103007] 0 2024-04-24T23:13:56.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:56 smithi040 ceph-mon[33128]: pgmap v31: 65 pgs: 65 active+clean; 2.2 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 54 MiB/s rd, 188 MiB/s wr, 363 op/s 2024-04-24T23:13:56.913 INFO:tasks.workunit.client.1.smithi040.stdout:2/545: unlink da/d1c/d37/d39/f7c 0 2024-04-24T23:13:56.913 INFO:tasks.workunit.client.1.smithi040.stdout:2/546: readlink da/l17 0 2024-04-24T23:13:56.916 INFO:tasks.workunit.client.1.smithi040.stdout:0/556: mkdir df/d1e/d2b/db6 0 2024-04-24T23:13:56.920 INFO:tasks.workunit.client.1.smithi040.stdout:4/427: rename da/d3f/d40/d47/d71/f74 to da/d3f/d40/d47/d75/f97 0 2024-04-24T23:13:56.921 INFO:tasks.workunit.client.0.smithi031.stdout:8/908: dwrite d8/d15/d37/d43/d5c/de1/f72 [0,4194304] 0 2024-04-24T23:13:56.927 INFO:tasks.workunit.client.1.smithi040.stdout:8/493: read d0/d8/f4d [280813,91469] 0 2024-04-24T23:13:56.928 INFO:tasks.workunit.client.1.smithi040.stdout:5/466: creat d2/d62/f98 x:0 0 0 2024-04-24T23:13:56.930 INFO:tasks.workunit.client.0.smithi031.stdout:0/832: mkdir da/d86/da3/dcd/ddc/d114/d117 0 2024-04-24T23:13:56.933 INFO:tasks.workunit.client.1.smithi040.stdout:9/511: dwrite d7/da/d17/d40/d55/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:13:56.933 INFO:tasks.workunit.client.1.smithi040.stdout:3/529: sync 2024-04-24T23:13:56.933 INFO:tasks.workunit.client.1.smithi040.stdout:1/475: sync 2024-04-24T23:13:56.934 INFO:tasks.workunit.client.1.smithi040.stdout:1/476: chown da/d35/d61/l94 474 1 2024-04-24T23:13:56.934 INFO:tasks.workunit.client.0.smithi031.stdout:5/722: creat d0/d6/d3b/d4b/dc/d25/d55/fed x:0 0 0 2024-04-24T23:13:56.938 INFO:tasks.workunit.client.0.smithi031.stdout:7/782: creat d1/d2/dd/d33/d5e/d64/d6d/dae/dd7/d101/f10f x:0 0 0 2024-04-24T23:13:56.939 INFO:tasks.workunit.client.1.smithi040.stdout:0/557: symlink df/d12/d58/lb7 0 2024-04-24T23:13:56.940 INFO:tasks.workunit.client.1.smithi040.stdout:3/530: dwrite d1/d9/f21 [0,4194304] 0 2024-04-24T23:13:56.940 INFO:tasks.workunit.client.0.smithi031.stdout:3/709: dread d6/df/d16/d2a/f31 [0,4194304] 0 2024-04-24T23:13:56.942 INFO:tasks.workunit.client.0.smithi031.stdout:3/710: write d6/df/fae [1806218,13993] 0 2024-04-24T23:13:56.943 INFO:tasks.workunit.client.0.smithi031.stdout:2/654: dread d5/f46 [0,4194304] 0 2024-04-24T23:13:56.944 INFO:tasks.workunit.client.0.smithi031.stdout:1/687: rmdir d1/da/d12/d7e/dc4 0 2024-04-24T23:13:56.946 INFO:tasks.workunit.client.0.smithi031.stdout:2/655: write d5/d8/d15/d65/f8e [330265,4899] 0 2024-04-24T23:13:56.946 INFO:tasks.workunit.client.0.smithi031.stdout:2/656: chown d5/d8/c66 70391714 1 2024-04-24T23:13:56.948 INFO:tasks.workunit.client.0.smithi031.stdout:2/657: write d5/d8/d45/d5b/d49/d2f/faa [564014,6355] 0 2024-04-24T23:13:56.948 INFO:tasks.workunit.client.1.smithi040.stdout:4/428: creat da/d45/d58/f98 x:0 0 0 2024-04-24T23:13:56.952 INFO:tasks.workunit.client.1.smithi040.stdout:8/494: mkdir d0/d16/d19/da2 0 2024-04-24T23:13:56.960 INFO:tasks.workunit.client.0.smithi031.stdout:8/909: rmdir d8/d15/d37/d43/da5 39 2024-04-24T23:13:56.961 INFO:tasks.workunit.client.1.smithi040.stdout:9/512: creat d7/d92/fb5 x:0 0 0 2024-04-24T23:13:56.962 INFO:tasks.workunit.client.0.smithi031.stdout:8/910: read d8/d1a/d1d/d35/d58/d5b/fe0 [556097,114240] 0 2024-04-24T23:13:56.963 INFO:tasks.workunit.client.1.smithi040.stdout:9/513: fsync d7/da/d17/d40/d55/d2f/d1f/d31/f35 0 2024-04-24T23:13:56.963 INFO:tasks.workunit.client.0.smithi031.stdout:4/881: getdents d3/d20/d25/d95 0 2024-04-24T23:13:56.964 INFO:tasks.workunit.client.1.smithi040.stdout:1/477: fsync da/d35/d38/d47/f84 0 2024-04-24T23:13:56.964 INFO:tasks.workunit.client.0.smithi031.stdout:4/882: write d3/da3/f116 [877417,23535] 0 2024-04-24T23:13:56.965 INFO:tasks.workunit.client.1.smithi040.stdout:1/478: chown da/d35/d38/f6e 3890 1 2024-04-24T23:13:56.971 INFO:tasks.workunit.client.1.smithi040.stdout:3/531: symlink d1/d9/d2e/lbc 0 2024-04-24T23:13:56.972 INFO:tasks.workunit.client.1.smithi040.stdout:3/532: dread - d1/d9/d2e/fa1 zero size 2024-04-24T23:13:56.973 INFO:tasks.workunit.client.0.smithi031.stdout:7/783: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d84/d110 0 2024-04-24T23:13:56.975 INFO:tasks.workunit.client.1.smithi040.stdout:0/558: fdatasync df/d1e/d20/f3f 0 2024-04-24T23:13:56.975 INFO:tasks.workunit.client.1.smithi040.stdout:0/559: stat df/f34 0 2024-04-24T23:13:56.978 INFO:tasks.workunit.client.1.smithi040.stdout:4/429: symlink da/d3f/d6b/d3c/d69/l99 0 2024-04-24T23:13:56.983 INFO:tasks.workunit.client.1.smithi040.stdout:5/467: mkdir d2/d7/d8/d1f/d57/d27/d3d/d99 0 2024-04-24T23:13:56.984 INFO:tasks.workunit.client.0.smithi031.stdout:7/784: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/fef [0,4194304] 0 2024-04-24T23:13:56.985 INFO:tasks.workunit.client.1.smithi040.stdout:8/495: dwrite d0/d8/dc/f3a [0,4194304] 0 2024-04-24T23:13:56.985 INFO:tasks.workunit.client.1.smithi040.stdout:9/514: fsync d7/da/d17/d40/d55/d2f/d1f/d34/f49 0 2024-04-24T23:13:56.989 INFO:tasks.workunit.client.1.smithi040.stdout:9/515: readlink d7/da/d17/d40/d55/d2f/d1f/d4e/d83/l1a 0 2024-04-24T23:13:56.990 INFO:tasks.workunit.client.1.smithi040.stdout:9/516: fdatasync d7/da/d17/d40/d55/d2f/d28/f29 0 2024-04-24T23:13:56.991 INFO:tasks.workunit.client.1.smithi040.stdout:1/479: rename da/c72 to da/d12/d18/d2a/d41/ca5 0 2024-04-24T23:13:56.992 INFO:tasks.workunit.client.1.smithi040.stdout:1/480: readlink da/d12/d14/d58/l63 0 2024-04-24T23:13:56.992 INFO:tasks.workunit.client.0.smithi031.stdout:2/658: symlink d5/d6/d29/d4f/lee 0 2024-04-24T23:13:56.993 INFO:tasks.workunit.client.1.smithi040.stdout:9/517: fsync d7/da/d17/d40/d55/d61/d6c/f7a 0 2024-04-24T23:13:56.993 INFO:tasks.workunit.client.1.smithi040.stdout:7/526: sync 2024-04-24T23:13:56.993 INFO:tasks.workunit.client.1.smithi040.stdout:2/547: truncate da/d1c/d37/d39/f76 1701974 0 2024-04-24T23:13:56.994 INFO:tasks.workunit.client.1.smithi040.stdout:3/533: symlink d1/d2a/d6a/d94/lbd 0 2024-04-24T23:13:56.999 INFO:tasks.workunit.client.0.smithi031.stdout:9/854: dwrite d2/dae/d14/d78/d84/d4a/f94 [0,4194304] 0 2024-04-24T23:13:56.999 INFO:tasks.workunit.client.0.smithi031.stdout:2/659: readlink d5/d28/d4e/d5e/la0 0 2024-04-24T23:13:57.001 INFO:tasks.workunit.client.0.smithi031.stdout:0/833: getdents da/d18/d5a/d7d/d10d 0 2024-04-24T23:13:57.006 INFO:tasks.workunit.client.0.smithi031.stdout:4/883: creat d3/d20/d25/d60/dc8/f11a x:0 0 0 2024-04-24T23:13:57.007 INFO:tasks.workunit.client.1.smithi040.stdout:0/560: chown df/c1d 16 1 2024-04-24T23:13:57.008 INFO:tasks.workunit.client.1.smithi040.stdout:8/496: mkdir d0/d38/d4c/da3 0 2024-04-24T23:13:57.009 INFO:tasks.workunit.client.0.smithi031.stdout:1/688: fsync d1/d31/f39 0 2024-04-24T23:13:57.024 INFO:tasks.workunit.client.1.smithi040.stdout:2/548: mkdir da/d1c/d37/d39/d3f/d52/d78/dc7/dcc 0 2024-04-24T23:13:57.024 INFO:tasks.workunit.client.1.smithi040.stdout:2/549: chown da/d1c/d37/d39/d3f/f88 415 1 2024-04-24T23:13:57.025 INFO:tasks.workunit.client.1.smithi040.stdout:3/534: creat d1/d9/d15/d45/fbe x:0 0 0 2024-04-24T23:13:57.026 INFO:tasks.workunit.client.1.smithi040.stdout:3/535: dread - d1/d2a/d6a/f8c zero size 2024-04-24T23:13:57.030 INFO:tasks.workunit.client.1.smithi040.stdout:4/430: symlink da/d3f/d40/l9a 0 2024-04-24T23:13:57.032 INFO:tasks.workunit.client.1.smithi040.stdout:9/518: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f48 [0,4194304] 0 2024-04-24T23:13:57.033 INFO:tasks.workunit.client.1.smithi040.stdout:0/561: mkdir df/d12/d4b/db8 0 2024-04-24T23:13:57.037 INFO:tasks.workunit.client.1.smithi040.stdout:8/497: unlink d0/d16/d19/d4f/f4e 0 2024-04-24T23:13:57.039 INFO:tasks.workunit.client.1.smithi040.stdout:1/481: dwrite da/ff [4194304,4194304] 0 2024-04-24T23:13:57.045 INFO:tasks.workunit.client.1.smithi040.stdout:6/468: sync 2024-04-24T23:13:57.047 INFO:tasks.workunit.client.1.smithi040.stdout:5/468: truncate d2/d2f/d14/f76 246738 0 2024-04-24T23:13:57.049 INFO:tasks.workunit.client.1.smithi040.stdout:4/431: chown da/d3f/l54 289926074 1 2024-04-24T23:13:57.055 INFO:tasks.workunit.client.1.smithi040.stdout:4/432: dwrite da/d3f/d40/d41/d60/f24 [0,4194304] 0 2024-04-24T23:13:57.055 INFO:tasks.workunit.client.1.smithi040.stdout:8/498: unlink d0/d16/d19/d4f/d77/l8d 0 2024-04-24T23:13:57.056 INFO:tasks.workunit.client.1.smithi040.stdout:8/499: chown d0/d8/da0 40062 1 2024-04-24T23:13:57.058 INFO:tasks.workunit.client.1.smithi040.stdout:2/550: creat da/d1c/d7e/fcd x:0 0 0 2024-04-24T23:13:57.059 INFO:tasks.workunit.client.1.smithi040.stdout:3/536: mkdir d1/d80/dbf 0 2024-04-24T23:13:57.062 INFO:tasks.workunit.client.1.smithi040.stdout:8/500: write d0/d8/dc/f3a [123998,122496] 0 2024-04-24T23:13:57.064 INFO:tasks.workunit.client.1.smithi040.stdout:5/469: fdatasync d2/d7/d8/d64/f93 0 2024-04-24T23:13:57.067 INFO:tasks.workunit.client.1.smithi040.stdout:7/527: dwrite db/d12/d20/d52/f68 [0,4194304] 0 2024-04-24T23:13:57.073 INFO:tasks.workunit.client.1.smithi040.stdout:0/562: mknod df/d12/d4b/db8/cb9 0 2024-04-24T23:13:57.074 INFO:tasks.workunit.client.1.smithi040.stdout:7/528: write db/d12/d20/d52/d6c/fae [340429,121909] 0 2024-04-24T23:13:57.075 INFO:tasks.workunit.client.1.smithi040.stdout:1/482: dwrite da/d12/f42 [4194304,4194304] 0 2024-04-24T23:13:57.077 INFO:tasks.workunit.client.1.smithi040.stdout:7/529: chown db/d12 243 1 2024-04-24T23:13:57.079 INFO:tasks.workunit.client.1.smithi040.stdout:1/483: fdatasync da/d12/d14/f2d 0 2024-04-24T23:13:57.081 INFO:tasks.workunit.client.1.smithi040.stdout:8/501: dwrite d0/d52/f84 [4194304,4194304] 0 2024-04-24T23:13:57.096 INFO:tasks.workunit.client.1.smithi040.stdout:5/470: rename d2/d7/d8/d1f/d57/d27/d3d/l80 to d2/d7/d8/d64/d6b/l9a 0 2024-04-24T23:13:57.097 INFO:tasks.workunit.client.1.smithi040.stdout:3/537: dread d1/f16 [4194304,4194304] 0 2024-04-24T23:13:57.097 INFO:tasks.workunit.client.1.smithi040.stdout:4/433: mkdir da/d3f/d40/d9b 0 2024-04-24T23:13:57.099 INFO:tasks.workunit.client.1.smithi040.stdout:3/538: write d1/d2a/d8d/d7c/fae [5065818,126915] 0 2024-04-24T23:13:57.105 INFO:tasks.workunit.client.0.smithi031.stdout:7/785: mknod d1/d2/dd/d33/d5e/d64/c111 0 2024-04-24T23:13:57.107 INFO:tasks.workunit.client.1.smithi040.stdout:6/469: write d7/d1a/d20/d51/f26 [86003,110246] 0 2024-04-24T23:13:57.109 INFO:tasks.workunit.client.1.smithi040.stdout:1/484: read - da/d12/d18/d2a/d8b/d97/f98 zero size 2024-04-24T23:13:57.112 INFO:tasks.workunit.client.0.smithi031.stdout:8/911: fdatasync d8/d1a/d1d/d35/d58/d5b/f91 0 2024-04-24T23:13:57.113 INFO:tasks.workunit.client.0.smithi031.stdout:8/912: readlink d8/d1a/d1d/d35/l106 0 2024-04-24T23:13:57.114 INFO:tasks.workunit.client.0.smithi031.stdout:2/660: mkdir d5/d8/d45/d5b/d49/dc4/def 0 2024-04-24T23:13:57.116 INFO:tasks.workunit.client.0.smithi031.stdout:2/661: chown d5/d6/d29/d26/db0/dc2/fab 7522585 1 2024-04-24T23:13:57.119 INFO:tasks.workunit.client.1.smithi040.stdout:2/551: creat da/d10/d4d/da4/fce x:0 0 0 2024-04-24T23:13:57.120 INFO:tasks.workunit.client.0.smithi031.stdout:4/884: truncate d3/d20/d25/d60/d63/f8b 674778 0 2024-04-24T23:13:57.121 INFO:tasks.workunit.client.1.smithi040.stdout:9/519: getdents d7/da/d17/d40 0 2024-04-24T23:13:57.128 INFO:tasks.workunit.client.1.smithi040.stdout:5/471: unlink d2/d62/f98 0 2024-04-24T23:13:57.128 INFO:tasks.workunit.client.0.smithi031.stdout:5/723: dwrite d0/d7/d56/d7c/d7e/fb8 [0,4194304] 0 2024-04-24T23:13:57.129 INFO:tasks.workunit.client.1.smithi040.stdout:5/472: write d2/d7/d8/d1f/d57/d27/f54 [2799181,62786] 0 2024-04-24T23:13:57.130 INFO:tasks.workunit.client.1.smithi040.stdout:7/530: write db/f71 [958911,98169] 0 2024-04-24T23:13:57.130 INFO:tasks.workunit.client.0.smithi031.stdout:5/724: chown d0/d6/d3b/d4b/d1d/d2e/dc6/f6f 1353900 1 2024-04-24T23:13:57.130 INFO:tasks.workunit.client.1.smithi040.stdout:4/434: chown da/f1a 131 1 2024-04-24T23:13:57.131 INFO:tasks.workunit.client.0.smithi031.stdout:3/711: link d6/df/d16/d2a/d95/d9b/d1a/l4e d6/df/d16/d57/ld4 0 2024-04-24T23:13:57.133 INFO:tasks.workunit.client.0.smithi031.stdout:3/712: stat d6/d14/d1d/d59/f9a 0 2024-04-24T23:13:57.136 INFO:tasks.workunit.client.0.smithi031.stdout:6/815: getdents d13/d15/d1d/d23/d4d/d7b 0 2024-04-24T23:13:57.137 INFO:tasks.workunit.client.0.smithi031.stdout:9/855: write d2/dae/f17 [4472483,73151] 0 2024-04-24T23:13:57.138 INFO:tasks.workunit.client.1.smithi040.stdout:7/531: dwrite db/f71 [0,4194304] 0 2024-04-24T23:13:57.138 INFO:tasks.workunit.client.1.smithi040.stdout:7/532: chown db/d12/d20 2 1 2024-04-24T23:13:57.139 INFO:tasks.workunit.client.0.smithi031.stdout:5/725: dwrite d0/d6/d3b/d4f/fd6 [0,4194304] 0 2024-04-24T23:13:57.142 INFO:tasks.workunit.client.0.smithi031.stdout:5/726: readlink d0/d6/d3b/d4b/d1d/d2e/d28/l99 0 2024-04-24T23:13:57.143 INFO:tasks.workunit.client.1.smithi040.stdout:3/539: rename d1/d9/d18/d36/f82 to d1/d2d/fc0 0 2024-04-24T23:13:57.144 INFO:tasks.workunit.client.0.smithi031.stdout:5/727: write d0/d6/d3b/d4b/dc/f84 [2685908,5170] 0 2024-04-24T23:13:57.145 INFO:tasks.workunit.client.0.smithi031.stdout:0/834: write da/d18/d1a/d37/d41/f83 [5337951,58995] 0 2024-04-24T23:13:57.147 INFO:tasks.workunit.client.0.smithi031.stdout:0/835: chown da/d11/f25 2 1 2024-04-24T23:13:57.147 INFO:tasks.workunit.client.1.smithi040.stdout:7/533: dwrite db/d12/d3f/d7a/f9e [0,4194304] 0 2024-04-24T23:13:57.151 INFO:tasks.workunit.client.1.smithi040.stdout:6/470: rmdir d7 39 2024-04-24T23:13:57.154 INFO:tasks.workunit.client.1.smithi040.stdout:1/485: chown da/d12/d23/f3c 4 1 2024-04-24T23:13:57.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/913: creat d8/d15/d9a/f134 x:0 0 0 2024-04-24T23:13:57.155 INFO:tasks.workunit.client.1.smithi040.stdout:1/486: dread - da/d31/f7c zero size 2024-04-24T23:13:57.156 INFO:tasks.workunit.client.1.smithi040.stdout:1/487: fdatasync da/d12/d18/d2a/d41/f6a 0 2024-04-24T23:13:57.157 INFO:tasks.workunit.client.0.smithi031.stdout:2/662: rmdir d5/d8 39 2024-04-24T23:13:57.158 INFO:tasks.workunit.client.1.smithi040.stdout:8/502: mknod d0/d8/d22/d81/ca4 0 2024-04-24T23:13:57.163 INFO:tasks.workunit.client.0.smithi031.stdout:4/885: truncate d3/d20/d2d/d44/fe4 2878629 0 2024-04-24T23:13:57.164 INFO:tasks.workunit.client.1.smithi040.stdout:9/520: chown d7/da/l13 87 1 2024-04-24T23:13:57.164 INFO:tasks.workunit.client.0.smithi031.stdout:7/786: write d1/d2/dd/d33/d5e/d64/d6d/de1/fc3 [3996108,6566] 0 2024-04-24T23:13:57.172 INFO:tasks.workunit.client.1.smithi040.stdout:5/473: mkdir d2/d7/d8/d1f/d57/d27/d3c/d9b 0 2024-04-24T23:13:57.173 INFO:tasks.workunit.client.0.smithi031.stdout:3/713: creat d6/d14/d1d/d69/fd5 x:0 0 0 2024-04-24T23:13:57.175 INFO:tasks.workunit.client.0.smithi031.stdout:3/714: write d6/df/d16/d2a/d95/d9b/d1a/d25/d83/fb8 [139367,105005] 0 2024-04-24T23:13:57.184 INFO:tasks.workunit.client.1.smithi040.stdout:0/563: link df/d12/f1f df/d1e/d2b/d6e/fba 0 2024-04-24T23:13:57.185 INFO:tasks.workunit.client.1.smithi040.stdout:0/564: truncate df/d1e/d20/fb2 241695 0 2024-04-24T23:13:57.186 INFO:tasks.workunit.client.1.smithi040.stdout:0/565: truncate df/d1e/d20/d21/d41/d52/f87 406881 0 2024-04-24T23:13:57.188 INFO:tasks.workunit.client.1.smithi040.stdout:7/534: mkdir db/d43/db9 0 2024-04-24T23:13:57.191 INFO:tasks.workunit.client.0.smithi031.stdout:0/836: dread da/d20/d8e/fd4 [0,4194304] 0 2024-04-24T23:13:57.191 INFO:tasks.workunit.client.0.smithi031.stdout:5/728: dwrite d0/d6/d3b/d4b/fac [0,4194304] 0 2024-04-24T23:13:57.193 INFO:tasks.workunit.client.1.smithi040.stdout:3/540: dwrite d1/d9/f21 [4194304,4194304] 0 2024-04-24T23:13:57.196 INFO:tasks.workunit.client.0.smithi031.stdout:4/886: creat d3/d20/d25/d60/f11b x:0 0 0 2024-04-24T23:13:57.199 INFO:tasks.workunit.client.1.smithi040.stdout:8/503: creat d0/d16/d1b/d58/fa5 x:0 0 0 2024-04-24T23:13:57.199 INFO:tasks.workunit.client.0.smithi031.stdout:7/787: mknod d1/d2/dd/d33/d5e/d64/d6d/da1/c112 0 2024-04-24T23:13:57.200 INFO:tasks.workunit.client.0.smithi031.stdout:7/788: readlink d1/d2/dd/d33/d5e/l9a 0 2024-04-24T23:13:57.205 INFO:tasks.workunit.client.1.smithi040.stdout:9/521: mknod d7/d92/d97/cb6 0 2024-04-24T23:13:57.206 INFO:tasks.workunit.client.1.smithi040.stdout:8/504: dwrite d0/d16/d1b/f54 [0,4194304] 0 2024-04-24T23:13:57.207 INFO:tasks.workunit.client.0.smithi031.stdout:8/914: dwrite d8/f28 [4194304,4194304] 0 2024-04-24T23:13:57.209 INFO:tasks.workunit.client.0.smithi031.stdout:8/915: dread - d8/d15/d37/d43/daf/f133 zero size 2024-04-24T23:13:57.209 INFO:tasks.workunit.client.1.smithi040.stdout:0/566: unlink df/d1e/d20/d21/d41/c84 0 2024-04-24T23:13:57.210 INFO:tasks.workunit.client.0.smithi031.stdout:3/715: readlink d6/d14/la2 0 2024-04-24T23:13:57.211 INFO:tasks.workunit.client.1.smithi040.stdout:0/567: write df/d12/d4b/f6a [472713,53254] 0 2024-04-24T23:13:57.212 INFO:tasks.workunit.client.1.smithi040.stdout:7/535: fsync db/d12/f29 0 2024-04-24T23:13:57.214 INFO:tasks.workunit.client.1.smithi040.stdout:6/471: dread - d7/d1a/f42 zero size 2024-04-24T23:13:57.214 INFO:tasks.workunit.client.0.smithi031.stdout:6/816: creat d13/d15/d1d/d23/d44/d47/d69/dd5/f110 x:0 0 0 2024-04-24T23:13:57.215 INFO:tasks.workunit.client.1.smithi040.stdout:6/472: write d7/d1a/d20/f61 [684202,2131] 0 2024-04-24T23:13:57.217 INFO:tasks.workunit.client.0.smithi031.stdout:9/856: rename d2/dae/d85/f9b to d2/dae/d85/f106 0 2024-04-24T23:13:57.220 INFO:tasks.workunit.client.0.smithi031.stdout:2/663: symlink d5/d8/d15/d65/lf0 0 2024-04-24T23:13:57.222 INFO:tasks.workunit.client.1.smithi040.stdout:3/541: rename d1/d2a/d8d/f7b to d1/d2d/fc1 0 2024-04-24T23:13:57.225 INFO:tasks.workunit.client.1.smithi040.stdout:5/474: getdents d2/d7/d8/d1f/d90 0 2024-04-24T23:13:57.226 INFO:tasks.workunit.client.0.smithi031.stdout:0/837: creat da/d86/da3/dcd/ddc/f118 x:0 0 0 2024-04-24T23:13:57.227 INFO:tasks.workunit.client.1.smithi040.stdout:8/505: mknod d0/d16/d19/ca6 0 2024-04-24T23:13:57.228 INFO:tasks.workunit.client.0.smithi031.stdout:4/887: symlink d3/d20/d2d/l11c 0 2024-04-24T23:13:57.230 INFO:tasks.workunit.client.0.smithi031.stdout:1/689: getdents d1/d3e/da3/d69 0 2024-04-24T23:13:57.230 INFO:tasks.workunit.client.0.smithi031.stdout:4/888: write d3/d20/d25/fb0 [2859211,40801] 0 2024-04-24T23:13:57.234 INFO:tasks.workunit.client.0.smithi031.stdout:7/789: symlink d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/l113 0 2024-04-24T23:13:57.236 INFO:tasks.workunit.client.1.smithi040.stdout:2/552: truncate da/d29/d55/f58 3673032 0 2024-04-24T23:13:57.236 INFO:tasks.workunit.client.0.smithi031.stdout:1/690: dread d1/d33/d36/fa1 [0,4194304] 0 2024-04-24T23:13:57.237 INFO:tasks.workunit.client.1.smithi040.stdout:1/488: creat da/d12/d18/fa6 x:0 0 0 2024-04-24T23:13:57.237 INFO:tasks.workunit.client.1.smithi040.stdout:1/489: dread - da/d12/d18/fa6 zero size 2024-04-24T23:13:57.245 INFO:tasks.workunit.client.1.smithi040.stdout:9/522: rename d7/da/d17/d40/d55/d2f/d1f/d4e/ca7 to d7/da/d17/d40/d55/d2f/cb7 0 2024-04-24T23:13:57.251 INFO:tasks.workunit.client.0.smithi031.stdout:6/817: rmdir d13/d15/d1d 39 2024-04-24T23:13:57.253 INFO:tasks.workunit.client.1.smithi040.stdout:5/475: mknod d2/d7/d8/d1f/d57/d27/d3c/c9c 0 2024-04-24T23:13:57.254 INFO:tasks.workunit.client.0.smithi031.stdout:9/857: mkdir d2/dae/d14/d78/d84/d7e/dc7/d107 0 2024-04-24T23:13:57.255 INFO:tasks.workunit.client.1.smithi040.stdout:8/506: mkdir d0/d8/dc/d34/da7 0 2024-04-24T23:13:57.258 INFO:tasks.workunit.client.1.smithi040.stdout:8/507: dread d0/d8/d55/f64 [0,4194304] 0 2024-04-24T23:13:57.261 INFO:tasks.workunit.client.0.smithi031.stdout:5/729: creat d0/d6/d3b/d4b/d1d/d2e/d28/deb/fee x:0 0 0 2024-04-24T23:13:57.262 INFO:tasks.workunit.client.1.smithi040.stdout:6/473: write d7/f60 [804020,4591] 0 2024-04-24T23:13:57.268 INFO:tasks.workunit.client.1.smithi040.stdout:0/568: rename df/d1e/f74 to df/d1e/d20/d61/fbb 0 2024-04-24T23:13:57.272 INFO:tasks.workunit.client.1.smithi040.stdout:9/523: mkdir d7/da/d17/d40/d55/d2f/d1f/d34/db8 0 2024-04-24T23:13:57.273 INFO:tasks.workunit.client.1.smithi040.stdout:9/524: dread - d7/da/d17/d40/fb4 zero size 2024-04-24T23:13:57.274 INFO:tasks.workunit.client.0.smithi031.stdout:2/664: dread d5/d6/f16 [4194304,4194304] 0 2024-04-24T23:13:57.275 INFO:tasks.workunit.client.1.smithi040.stdout:9/525: write d7/da/d17/d40/d55/d61/f8f [4600584,71524] 0 2024-04-24T23:13:57.276 INFO:tasks.workunit.client.1.smithi040.stdout:2/553: dread da/d10/d15/f63 [0,4194304] 0 2024-04-24T23:13:57.277 INFO:tasks.workunit.client.0.smithi031.stdout:7/790: rename d1/d2/dd/d33/d5e/d64/d6d/la3 to d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/l114 0 2024-04-24T23:13:57.277 INFO:tasks.workunit.client.0.smithi031.stdout:3/716: dwrite d6/df/d16/d2a/d95/d9b/d1a/f97 [0,4194304] 0 2024-04-24T23:13:57.280 INFO:tasks.workunit.client.1.smithi040.stdout:3/542: link d1/d9/d15/d45/f4c d1/d2a/d6a/fc2 0 2024-04-24T23:13:57.281 INFO:tasks.workunit.client.0.smithi031.stdout:3/717: rename d6/df/d16/d2a/d95/d9b to d6/df/d16/d2a/d95/d9b/d60/dd6 22 2024-04-24T23:13:57.281 INFO:tasks.workunit.client.1.smithi040.stdout:3/543: write d1/d9/d18/da7/fad [148005,64096] 0 2024-04-24T23:13:57.283 INFO:tasks.workunit.client.0.smithi031.stdout:8/916: mkdir d8/d15/d37/d43/da5/d135 0 2024-04-24T23:13:57.284 INFO:tasks.workunit.client.0.smithi031.stdout:8/917: chown d8/d15/d37/d43/da5 1 1 2024-04-24T23:13:57.285 INFO:tasks.workunit.client.1.smithi040.stdout:5/476: unlink d2/l5 0 2024-04-24T23:13:57.286 INFO:tasks.workunit.client.0.smithi031.stdout:8/918: truncate d8/d15/d37/d43/d7b/fe9 318468 0 2024-04-24T23:13:57.290 INFO:tasks.workunit.client.1.smithi040.stdout:7/536: link db/d12/d20/d56/c96 db/d2f/d37/cba 0 2024-04-24T23:13:57.290 INFO:tasks.workunit.client.0.smithi031.stdout:6/818: dread - d13/d15/d1d/d23/d2d/dbf/f108 zero size 2024-04-24T23:13:57.292 INFO:tasks.workunit.client.0.smithi031.stdout:9/858: symlink d2/d43/d9d/dc0/l108 0 2024-04-24T23:13:57.294 INFO:tasks.workunit.client.1.smithi040.stdout:6/474: rmdir d7/de/d89 39 2024-04-24T23:13:57.295 INFO:tasks.workunit.client.1.smithi040.stdout:1/490: rmdir da/d12/d55 39 2024-04-24T23:13:57.302 INFO:tasks.workunit.client.0.smithi031.stdout:7/791: rmdir d1/d2/dd/d33/d5e/d64/d6d/da1 39 2024-04-24T23:13:57.304 INFO:tasks.workunit.client.1.smithi040.stdout:9/526: rename d7/da/d17/d40/l4f to d7/da/d17/d40/d55/d2f/d1f/d4e/lb9 0 2024-04-24T23:13:57.308 INFO:tasks.workunit.client.0.smithi031.stdout:3/718: symlink d6/df/d16/d2a/d95/d9b/d1a/d25/ld7 0 2024-04-24T23:13:57.309 INFO:tasks.workunit.client.0.smithi031.stdout:5/730: write d0/d6/d3b/d4b/dc/d8b/fa6 [4808742,115314] 0 2024-04-24T23:13:57.313 INFO:tasks.workunit.client.0.smithi031.stdout:7/792: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea/f103 [0,4194304] 0 2024-04-24T23:13:57.316 INFO:tasks.workunit.client.0.smithi031.stdout:6/819: mkdir d13/d15/d1d/d23/d44/da0/da6/d111 0 2024-04-24T23:13:57.318 INFO:tasks.workunit.client.0.smithi031.stdout:1/691: dwrite d1/f34 [0,4194304] 0 2024-04-24T23:13:57.318 INFO:tasks.workunit.client.1.smithi040.stdout:7/537: creat db/d12/d7d/fbb x:0 0 0 2024-04-24T23:13:57.321 INFO:tasks.workunit.client.0.smithi031.stdout:8/919: write d8/f38 [38019,105071] 0 2024-04-24T23:13:57.323 INFO:tasks.workunit.client.1.smithi040.stdout:6/475: fdatasync d7/f3f 0 2024-04-24T23:13:57.324 INFO:tasks.workunit.client.1.smithi040.stdout:6/476: chown f2 7922334 1 2024-04-24T23:13:57.326 INFO:tasks.workunit.client.0.smithi031.stdout:4/889: creat d3/d20/d25/f11d x:0 0 0 2024-04-24T23:13:57.327 INFO:tasks.workunit.client.1.smithi040.stdout:0/569: creat df/d1e/d2b/db6/fbc x:0 0 0 2024-04-24T23:13:57.327 INFO:tasks.workunit.client.0.smithi031.stdout:4/890: write d3/f11 [3630184,8914] 0 2024-04-24T23:13:57.330 INFO:tasks.workunit.client.1.smithi040.stdout:9/527: mknod d7/d5e/cba 0 2024-04-24T23:13:57.333 INFO:tasks.workunit.client.0.smithi031.stdout:2/665: mkdir d5/d8/d45/d5b/d49/dc4/de5/df1 0 2024-04-24T23:13:57.333 INFO:tasks.workunit.client.1.smithi040.stdout:3/544: link d1/d2d/de/d30/d37/f74 d1/d2d/fc3 0 2024-04-24T23:13:57.337 INFO:tasks.workunit.client.1.smithi040.stdout:7/538: mknod db/d12/d20/d52/d6c/d8f/cbc 0 2024-04-24T23:13:57.338 INFO:tasks.workunit.client.0.smithi031.stdout:5/731: rmdir d0/d6/d3b/d4b/dc/d8b/d9b/da8 39 2024-04-24T23:13:57.347 INFO:tasks.workunit.client.1.smithi040.stdout:0/570: dread df/d1e/d2b/d6e/fba [4194304,4194304] 0 2024-04-24T23:13:57.350 INFO:tasks.workunit.client.0.smithi031.stdout:7/793: dread d1/d2/dd/d33/d5e/d64/d6d/de1/f5b [0,4194304] 0 2024-04-24T23:13:57.351 INFO:tasks.workunit.client.1.smithi040.stdout:5/477: creat d2/d7/d8/d1f/f9d x:0 0 0 2024-04-24T23:13:57.352 INFO:tasks.workunit.client.0.smithi031.stdout:1/692: write d1/d3e/da3/f77 [185812,117009] 0 2024-04-24T23:13:57.353 INFO:tasks.workunit.client.0.smithi031.stdout:3/719: dwrite d6/d14/d1d/d59/f5e [0,4194304] 0 2024-04-24T23:13:57.353 INFO:tasks.workunit.client.1.smithi040.stdout:7/539: creat db/d12/d62/fbd x:0 0 0 2024-04-24T23:13:57.355 INFO:tasks.workunit.client.1.smithi040.stdout:7/540: read db/f71 [512461,91660] 0 2024-04-24T23:13:57.356 INFO:tasks.workunit.client.0.smithi031.stdout:9/859: write d2/dae/d14/d78/d84/d4a/f71 [2481276,66563] 0 2024-04-24T23:13:57.357 INFO:tasks.workunit.client.1.smithi040.stdout:8/508: getdents d0/d38/d4c/d5d 0 2024-04-24T23:13:57.361 INFO:tasks.workunit.client.0.smithi031.stdout:4/891: fsync d3/f75 0 2024-04-24T23:13:57.361 INFO:tasks.workunit.client.1.smithi040.stdout:6/477: symlink d7/d1a/d20/d51/d1e/d46/d85/l94 0 2024-04-24T23:13:57.362 INFO:tasks.workunit.client.1.smithi040.stdout:5/478: dwrite d2/d7/d8/d1f/f9d [0,4194304] 0 2024-04-24T23:13:57.363 INFO:tasks.workunit.client.1.smithi040.stdout:1/491: write da/d12/d18/d2a/f3d [4104319,65855] 0 2024-04-24T23:13:57.364 INFO:tasks.workunit.client.1.smithi040.stdout:3/545: dwrite d1/d2a/f47 [0,4194304] 0 2024-04-24T23:13:57.364 INFO:tasks.workunit.client.1.smithi040.stdout:5/479: readlink d2/d2f/l5b 0 2024-04-24T23:13:57.367 INFO:tasks.workunit.client.1.smithi040.stdout:6/478: chown d7/d1a/d20/d51/d1e/d5d/d52/c4c 23238849 1 2024-04-24T23:13:57.367 INFO:tasks.workunit.client.1.smithi040.stdout:5/480: readlink d2/d7/d8/d1f/l88 0 2024-04-24T23:13:57.369 INFO:tasks.workunit.client.0.smithi031.stdout:6/820: mknod d13/d15/d1d/d23/d44/d47/c112 0 2024-04-24T23:13:57.376 INFO:tasks.workunit.client.1.smithi040.stdout:8/509: creat d0/d16/d1b/d3f/fa8 x:0 0 0 2024-04-24T23:13:57.377 INFO:tasks.workunit.client.1.smithi040.stdout:8/510: readlink d0/d8/d22/l24 0 2024-04-24T23:13:57.377 INFO:tasks.workunit.client.0.smithi031.stdout:6/821: read d13/d15/d1d/d23/d24/d43/fb3 [1385211,118631] 0 2024-04-24T23:13:57.378 INFO:tasks.workunit.client.1.smithi040.stdout:1/492: chown da/d12/d23 651875 1 2024-04-24T23:13:57.381 INFO:tasks.workunit.client.0.smithi031.stdout:9/860: creat d2/dae/d14/d78/d84/d4a/dab/f109 x:0 0 0 2024-04-24T23:13:57.381 INFO:tasks.workunit.client.0.smithi031.stdout:8/920: dread d8/d15/d37/f85 [0,4194304] 0 2024-04-24T23:13:57.382 INFO:tasks.workunit.client.1.smithi040.stdout:0/571: unlink df/d1e/d20/d61/c9a 0 2024-04-24T23:13:57.383 INFO:tasks.workunit.client.0.smithi031.stdout:3/720: dread d6/df/d16/d2a/d95/d9b/d1a/f62 [0,4194304] 0 2024-04-24T23:13:57.385 INFO:tasks.workunit.client.1.smithi040.stdout:5/481: creat d2/f9e x:0 0 0 2024-04-24T23:13:57.385 INFO:tasks.workunit.client.0.smithi031.stdout:3/721: read - d6/d14/d1d/d59/f90 zero size 2024-04-24T23:13:57.389 INFO:tasks.workunit.client.1.smithi040.stdout:6/479: rename d7/d1a/d56/l80 to d7/d1a/d56/l95 0 2024-04-24T23:13:57.390 INFO:tasks.workunit.client.0.smithi031.stdout:4/892: mknod d3/d20/d2d/d44/d5b/d5d/c11e 0 2024-04-24T23:13:57.391 INFO:tasks.workunit.client.0.smithi031.stdout:1/693: write d1/da/d12/d7e/d7b/fb8 [1015416,50890] 0 2024-04-24T23:13:57.391 INFO:tasks.workunit.client.1.smithi040.stdout:7/541: getdents db/d43/db9 0 2024-04-24T23:13:57.393 INFO:tasks.workunit.client.1.smithi040.stdout:8/511: mknod d0/d16/d1b/d3f/d44/ca9 0 2024-04-24T23:13:57.396 INFO:tasks.workunit.client.1.smithi040.stdout:0/572: creat df/d1e/d2b/db6/fbd x:0 0 0 2024-04-24T23:13:57.396 INFO:tasks.workunit.client.0.smithi031.stdout:9/861: dread d2/f12 [4194304,4194304] 0 2024-04-24T23:13:57.397 INFO:tasks.workunit.client.0.smithi031.stdout:8/921: fsync d8/d15/d37/d43/dbd/dc4/fc7 0 2024-04-24T23:13:57.398 INFO:tasks.workunit.client.1.smithi040.stdout:0/573: write df/d1e/d20/d21/d51/f8c [423806,106800] 0 2024-04-24T23:13:57.398 INFO:tasks.workunit.client.1.smithi040.stdout:0/574: stat df/d1e/d70 0 2024-04-24T23:13:57.403 INFO:tasks.workunit.client.0.smithi031.stdout:8/922: dread d8/d15/d37/d43/d5c/de1/f72 [0,4194304] 0 2024-04-24T23:13:57.406 INFO:tasks.workunit.client.1.smithi040.stdout:8/512: creat d0/d38/faa x:0 0 0 2024-04-24T23:13:57.407 INFO:tasks.workunit.client.0.smithi031.stdout:3/722: rmdir d6/df/d16/d2a/d95/d9b/d1a/d25/dd1 39 2024-04-24T23:13:57.408 INFO:tasks.workunit.client.0.smithi031.stdout:6/822: write d13/d15/d1d/d23/d4d/f6a [5320333,77439] 0 2024-04-24T23:13:57.408 INFO:tasks.workunit.client.1.smithi040.stdout:1/493: symlink da/d12/d14/d58/la7 0 2024-04-24T23:13:57.410 INFO:tasks.workunit.client.0.smithi031.stdout:5/732: creat d0/d6/d3b/d4b/dc/fef x:0 0 0 2024-04-24T23:13:57.412 INFO:tasks.workunit.client.0.smithi031.stdout:4/893: unlink d3/d5e/dc2/f117 0 2024-04-24T23:13:57.413 INFO:tasks.workunit.client.1.smithi040.stdout:3/546: rename d1/d9/d18/d36/d55/c83 to d1/d9/d39/cc4 0 2024-04-24T23:13:57.414 INFO:tasks.workunit.client.1.smithi040.stdout:3/547: fsync d1/d9/f27 0 2024-04-24T23:13:57.418 INFO:tasks.workunit.client.1.smithi040.stdout:6/480: mknod d7/d1a/d20/d51/d1e/d46/c96 0 2024-04-24T23:13:57.420 INFO:tasks.workunit.client.1.smithi040.stdout:8/513: creat d0/d16/d1b/d58/fab x:0 0 0 2024-04-24T23:13:57.421 INFO:tasks.workunit.client.1.smithi040.stdout:8/514: write d0/d16/d1b/f95 [133771,122294] 0 2024-04-24T23:13:57.425 INFO:tasks.workunit.client.1.smithi040.stdout:7/542: rename db/d12/d3f/d7a/fa3 to db/d12/d73/fbe 0 2024-04-24T23:13:57.428 INFO:tasks.workunit.client.1.smithi040.stdout:4/435: sync 2024-04-24T23:13:57.429 INFO:tasks.workunit.client.1.smithi040.stdout:1/494: dwrite da/d12/d18/d2a/d41/d4f/f9b [0,4194304] 0 2024-04-24T23:13:57.434 INFO:tasks.workunit.client.0.smithi031.stdout:2/666: getdents d5/d8/d45/d5b 0 2024-04-24T23:13:57.438 INFO:tasks.workunit.client.0.smithi031.stdout:2/667: write d5/d28/f37 [3881825,112928] 0 2024-04-24T23:13:57.440 INFO:tasks.workunit.client.1.smithi040.stdout:3/548: dwrite d1/d9/d39/f78 [0,4194304] 0 2024-04-24T23:13:57.440 INFO:tasks.workunit.client.1.smithi040.stdout:8/515: chown d0/d16/c6d 2950776 1 2024-04-24T23:13:57.441 INFO:tasks.workunit.client.0.smithi031.stdout:3/723: creat d6/df/d16/d53/fd8 x:0 0 0 2024-04-24T23:13:57.443 INFO:tasks.workunit.client.0.smithi031.stdout:3/724: write d6/df/d16/f35 [4274839,3501] 0 2024-04-24T23:13:57.443 INFO:tasks.workunit.client.0.smithi031.stdout:6/823: unlink d13/d15/d1d/l7a 0 2024-04-24T23:13:57.446 INFO:tasks.workunit.client.0.smithi031.stdout:5/733: read - d0/d7/fa3 zero size 2024-04-24T23:13:57.451 INFO:tasks.workunit.client.1.smithi040.stdout:7/543: rmdir db/d12/d20/d52/d44 39 2024-04-24T23:13:57.452 INFO:tasks.workunit.client.0.smithi031.stdout:7/794: getdents d1/dbe 0 2024-04-24T23:13:57.453 INFO:tasks.workunit.client.1.smithi040.stdout:7/544: write db/d12/d20/d52/d4b/f4f [466048,114811] 0 2024-04-24T23:13:57.453 INFO:tasks.workunit.client.0.smithi031.stdout:5/734: truncate d0/d6/d3b/d4b/dc/d25/d55/fed 632088 0 2024-04-24T23:13:57.454 INFO:tasks.workunit.client.1.smithi040.stdout:4/436: creat da/d1f/f9c x:0 0 0 2024-04-24T23:13:57.455 INFO:tasks.workunit.client.0.smithi031.stdout:6/824: dwrite d13/d15/d1d/d23/d44/d47/d69/fd4 [0,4194304] 0 2024-04-24T23:13:57.458 INFO:tasks.workunit.client.1.smithi040.stdout:6/481: dwrite d7/d1a/d20/d51/d1e/f3d [0,4194304] 0 2024-04-24T23:13:57.465 INFO:tasks.workunit.client.0.smithi031.stdout:9/862: creat d2/dae/d14/d78/d84/d4a/dab/dd1/f10a x:0 0 0 2024-04-24T23:13:57.466 INFO:tasks.workunit.client.1.smithi040.stdout:1/495: write da/d12/d18/d2a/d41/f62 [519964,19236] 0 2024-04-24T23:13:57.470 INFO:tasks.workunit.client.0.smithi031.stdout:2/668: mknod d5/d8/d45/cf2 0 2024-04-24T23:13:57.476 INFO:tasks.workunit.client.0.smithi031.stdout:4/894: rename d3/d20/d2d/d44/d5b/l82 to d3/d20/d25/d60/d63/db9/l11f 0 2024-04-24T23:13:57.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/725: dread d6/df/d16/d2a/d95/d9b/d1a/d25/f54 [0,4194304] 0 2024-04-24T23:13:57.479 INFO:tasks.workunit.client.0.smithi031.stdout:7/795: fdatasync d1/d2/d83/dc1/f102 0 2024-04-24T23:13:57.479 INFO:tasks.workunit.client.0.smithi031.stdout:3/726: fsync d6/df/d16/d53/fc6 0 2024-04-24T23:13:57.484 INFO:tasks.workunit.client.0.smithi031.stdout:1/694: link d1/da/d12/d65/dcf/d11/f20 d1/da/d12/d65/dcf/d11/d21/fdb 0 2024-04-24T23:13:57.486 INFO:tasks.workunit.client.0.smithi031.stdout:3/727: dwrite d6/fbe [0,4194304] 0 2024-04-24T23:13:57.492 INFO:tasks.workunit.client.0.smithi031.stdout:2/669: fdatasync d5/d28/f54 0 2024-04-24T23:13:57.494 INFO:tasks.workunit.client.1.smithi040.stdout:2/554: sync 2024-04-24T23:13:57.494 INFO:tasks.workunit.client.1.smithi040.stdout:9/528: sync 2024-04-24T23:13:57.494 INFO:tasks.workunit.client.1.smithi040.stdout:8/516: creat d0/d16/d19/fac x:0 0 0 2024-04-24T23:13:57.496 INFO:tasks.workunit.client.0.smithi031.stdout:4/895: symlink d3/d20/da7/l120 0 2024-04-24T23:13:57.497 INFO:tasks.workunit.client.1.smithi040.stdout:3/549: creat d1/d9/d18/da7/fc5 x:0 0 0 2024-04-24T23:13:57.498 INFO:tasks.workunit.client.0.smithi031.stdout:5/735: truncate d0/faa 1014180 0 2024-04-24T23:13:57.501 INFO:tasks.workunit.client.1.smithi040.stdout:7/545: fsync db/f71 0 2024-04-24T23:13:57.505 INFO:tasks.workunit.client.1.smithi040.stdout:4/437: unlink da/d3f/d6b/d3c/f3e 0 2024-04-24T23:13:57.509 INFO:tasks.workunit.client.0.smithi031.stdout:1/695: creat d1/d33/fdc x:0 0 0 2024-04-24T23:13:57.510 INFO:tasks.workunit.client.1.smithi040.stdout:0/575: getdents df/d1e/d20/d21/d41/d52 0 2024-04-24T23:13:57.510 INFO:tasks.workunit.client.1.smithi040.stdout:0/576: readlink df/d12/d1a/l78 0 2024-04-24T23:13:57.511 INFO:tasks.workunit.client.0.smithi031.stdout:9/863: dwrite d2/dae/d14/f46 [0,4194304] 0 2024-04-24T23:13:57.513 INFO:tasks.workunit.client.1.smithi040.stdout:1/496: creat da/d12/d18/d2a/d41/fa8 x:0 0 0 2024-04-24T23:13:57.513 INFO:tasks.workunit.client.0.smithi031.stdout:9/864: fsync d2/dae/d14/f19 0 2024-04-24T23:13:57.514 INFO:tasks.workunit.client.1.smithi040.stdout:1/497: chown da/d12/d18/d2a/f79 33420984 1 2024-04-24T23:13:57.514 INFO:tasks.workunit.client.0.smithi031.stdout:9/865: stat d2/dae/d14/d1d/d45/l61 0 2024-04-24T23:13:57.516 INFO:tasks.workunit.client.0.smithi031.stdout:9/866: chown d2/dae/d14/d1d/d45/fd8 3332 1 2024-04-24T23:13:57.520 INFO:tasks.workunit.client.1.smithi040.stdout:1/498: dwrite da/d12/d18/d2a/f5a [0,4194304] 0 2024-04-24T23:13:57.520 INFO:tasks.workunit.client.0.smithi031.stdout:2/670: creat d5/d99/ff3 x:0 0 0 2024-04-24T23:13:57.521 INFO:tasks.workunit.client.0.smithi031.stdout:2/671: chown d5/d8/d15/d56/d84 6483377 1 2024-04-24T23:13:57.522 INFO:tasks.workunit.client.1.smithi040.stdout:8/517: rename d0/d8/dc/l75 to d0/d8/d66/lad 0 2024-04-24T23:13:57.522 INFO:tasks.workunit.client.1.smithi040.stdout:1/499: fsync da/d35/d38/f3e 0 2024-04-24T23:13:57.522 INFO:tasks.workunit.client.1.smithi040.stdout:3/550: unlink d1/d2d/d72/fb8 0 2024-04-24T23:13:57.523 INFO:tasks.workunit.client.0.smithi031.stdout:4/896: dread - d3/d23/fc5 zero size 2024-04-24T23:13:57.523 INFO:tasks.workunit.client.1.smithi040.stdout:1/500: readlink da/d35/d61/l94 0 2024-04-24T23:13:57.523 INFO:tasks.workunit.client.1.smithi040.stdout:8/518: write d0/d16/d19/d4f/f6c [43598,61694] 0 2024-04-24T23:13:57.525 INFO:tasks.workunit.client.0.smithi031.stdout:5/736: creat d0/d6/d3b/d4b/d1d/d2e/d28/d77/ff0 x:0 0 0 2024-04-24T23:13:57.527 INFO:tasks.workunit.client.1.smithi040.stdout:4/438: mknod da/d3f/d6b/d3c/d69/c9d 0 2024-04-24T23:13:57.529 INFO:tasks.workunit.client.1.smithi040.stdout:1/501: dwrite f2 [0,4194304] 0 2024-04-24T23:13:57.531 INFO:tasks.workunit.client.1.smithi040.stdout:9/529: mkdir d7/dbb 0 2024-04-24T23:13:57.533 INFO:tasks.workunit.client.0.smithi031.stdout:1/696: creat d1/d3e/d46/dbd/fdd x:0 0 0 2024-04-24T23:13:57.536 INFO:tasks.workunit.client.0.smithi031.stdout:3/728: creat d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/fd9 x:0 0 0 2024-04-24T23:13:57.536 INFO:tasks.workunit.client.1.smithi040.stdout:5/482: sync 2024-04-24T23:13:57.536 INFO:tasks.workunit.client.1.smithi040.stdout:5/483: chown d2/d7/d8/d64/f68 186981 1 2024-04-24T23:13:57.543 INFO:tasks.workunit.client.1.smithi040.stdout:7/546: mkdir db/d12/d62/d8a/db3/dbf 0 2024-04-24T23:13:57.543 INFO:tasks.workunit.client.0.smithi031.stdout:3/729: dwrite d6/df/d16/d53/fa8 [0,4194304] 0 2024-04-24T23:13:57.545 INFO:tasks.workunit.client.0.smithi031.stdout:8/923: getdents d8/d1a/d1d/d50 0 2024-04-24T23:13:57.546 INFO:tasks.workunit.client.1.smithi040.stdout:3/551: creat d1/d2d/de/d30/d37/fc6 x:0 0 0 2024-04-24T23:13:57.548 INFO:tasks.workunit.client.0.smithi031.stdout:7/796: truncate d1/d2/dd/f7d 5095778 0 2024-04-24T23:13:57.553 INFO:tasks.workunit.client.0.smithi031.stdout:2/672: truncate d5/d8/d15/f50 161172 0 2024-04-24T23:13:57.555 INFO:tasks.workunit.client.1.smithi040.stdout:6/482: creat d7/de/f97 x:0 0 0 2024-04-24T23:13:57.558 INFO:tasks.workunit.client.1.smithi040.stdout:0/577: mkdir df/d1e/d20/d21/d51/d79/daa/dbe 0 2024-04-24T23:13:57.558 INFO:tasks.workunit.client.1.smithi040.stdout:0/578: fsync df/d12/f22 0 2024-04-24T23:13:57.559 INFO:tasks.workunit.client.0.smithi031.stdout:5/737: creat d0/d6/d32/d6b/ff1 x:0 0 0 2024-04-24T23:13:57.561 INFO:tasks.workunit.client.0.smithi031.stdout:6/825: getdents d13/d15/d1d/d23/d2d 0 2024-04-24T23:13:57.562 INFO:tasks.workunit.client.1.smithi040.stdout:1/502: creat da/d12/d18/d2a/d4d/d7d/fa9 x:0 0 0 2024-04-24T23:13:57.563 INFO:tasks.workunit.client.0.smithi031.stdout:6/826: stat d13/d15/d1d/d23/d44/d47/d69/dd5/feb 0 2024-04-24T23:13:57.564 INFO:tasks.workunit.client.1.smithi040.stdout:1/503: chown da/d12/d18/d2a/d41/f70 219 1 2024-04-24T23:13:57.564 INFO:tasks.workunit.client.1.smithi040.stdout:1/504: chown da/d35/d38/d47/f84 13997419 1 2024-04-24T23:13:57.565 INFO:tasks.workunit.client.1.smithi040.stdout:5/484: dread d2/d7/d8/db/d1e/f28 [0,4194304] 0 2024-04-24T23:13:57.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/730: chown d6/d14/l21 498523894 1 2024-04-24T23:13:57.572 INFO:tasks.workunit.client.1.smithi040.stdout:8/519: dread d0/d8/d22/d81/f74 [0,4194304] 0 2024-04-24T23:13:57.573 INFO:tasks.workunit.client.0.smithi031.stdout:8/924: unlink d8/d15/d37/d43/d7b/dbf/l11f 0 2024-04-24T23:13:57.577 INFO:tasks.workunit.client.1.smithi040.stdout:7/547: creat db/d12/d3f/d7a/fc0 x:0 0 0 2024-04-24T23:13:57.580 INFO:tasks.workunit.client.1.smithi040.stdout:3/552: creat d1/d9/d18/d36/d55/d7e/fc7 x:0 0 0 2024-04-24T23:13:57.581 INFO:tasks.workunit.client.1.smithi040.stdout:3/553: write d1/d2d/d2b/f42 [3006356,102450] 0 2024-04-24T23:13:57.582 INFO:tasks.workunit.client.1.smithi040.stdout:3/554: read - d1/d2d/d72/f90 zero size 2024-04-24T23:13:57.583 INFO:tasks.workunit.client.1.smithi040.stdout:9/530: dread d7/da/d17/d40/d55/d2f/f46 [0,4194304] 0 2024-04-24T23:13:57.583 INFO:tasks.workunit.client.1.smithi040.stdout:3/555: chown d1/d2d/fc0 21269152 1 2024-04-24T23:13:57.584 INFO:tasks.workunit.client.0.smithi031.stdout:4/897: dwrite d3/d20/d25/d60/d63/f108 [0,4194304] 0 2024-04-24T23:13:57.587 INFO:tasks.workunit.client.0.smithi031.stdout:7/797: rename d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/fe9 to d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/f115 0 2024-04-24T23:13:57.588 INFO:tasks.workunit.client.0.smithi031.stdout:1/697: write d1/da/d12/d7e/d7b/f93 [832543,123319] 0 2024-04-24T23:13:57.589 INFO:tasks.workunit.client.1.smithi040.stdout:4/439: mknod da/d3f/d6b/c9e 0 2024-04-24T23:13:57.592 INFO:tasks.workunit.client.1.smithi040.stdout:2/555: getdents da 0 2024-04-24T23:13:57.594 INFO:tasks.workunit.client.1.smithi040.stdout:5/485: fsync d2/d7/d8/d1f/d57/f91 0 2024-04-24T23:13:57.597 INFO:tasks.workunit.client.0.smithi031.stdout:9/867: truncate d2/d16/f20 7225455 0 2024-04-24T23:13:57.598 INFO:tasks.workunit.client.0.smithi031.stdout:9/868: chown d2/dae/d14/d1d/d45/d56/fed 2443 1 2024-04-24T23:13:57.599 INFO:tasks.workunit.client.1.smithi040.stdout:4/440: dread da/d3f/d40/d41/d60/f42 [0,4194304] 0 2024-04-24T23:13:57.599 INFO:tasks.workunit.client.0.smithi031.stdout:9/869: chown d2/d43/f70 65649 1 2024-04-24T23:13:57.600 INFO:tasks.workunit.client.0.smithi031.stdout:3/731: mknod d6/df/d16/d2a/cda 0 2024-04-24T23:13:57.601 INFO:tasks.workunit.client.0.smithi031.stdout:9/870: stat d2/dae/d14/d78/d84/d4a/d4b/d96/c97 0 2024-04-24T23:13:57.601 INFO:tasks.workunit.client.1.smithi040.stdout:8/520: symlink d0/d16/d19/d4f/d9e/d90/lae 0 2024-04-24T23:13:57.602 INFO:tasks.workunit.client.0.smithi031.stdout:9/871: truncate d2/fdf 780884 0 2024-04-24T23:13:57.602 INFO:tasks.workunit.client.1.smithi040.stdout:8/521: truncate d0/d16/d1b/f95 391826 0 2024-04-24T23:13:57.606 INFO:tasks.workunit.client.1.smithi040.stdout:5/486: dread d2/d7/d8/d64/f93 [0,4194304] 0 2024-04-24T23:13:57.611 INFO:tasks.workunit.client.0.smithi031.stdout:1/698: mknod d1/d33/cde 0 2024-04-24T23:13:57.613 INFO:tasks.workunit.client.1.smithi040.stdout:9/531: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc 0 2024-04-24T23:13:57.617 INFO:tasks.workunit.client.0.smithi031.stdout:6/827: dwrite d13/d15/d1d/d23/d44/d47/d69/faa [0,4194304] 0 2024-04-24T23:13:57.619 INFO:tasks.workunit.client.0.smithi031.stdout:6/828: stat d13/d15/d1d/d23/d44/c81 0 2024-04-24T23:13:57.621 INFO:tasks.workunit.client.1.smithi040.stdout:7/548: dwrite db/d12/d20/d52/f79 [0,4194304] 0 2024-04-24T23:13:57.622 INFO:tasks.workunit.client.1.smithi040.stdout:1/505: link da/d12/d18/d2a/d41/f70 da/d35/da3/faa 0 2024-04-24T23:13:57.623 INFO:tasks.workunit.client.0.smithi031.stdout:8/925: getdents d8/d15/d37/d43/da5/d135 0 2024-04-24T23:13:57.627 INFO:tasks.workunit.client.0.smithi031.stdout:4/898: symlink d3/d23/d37/d86/d113/l121 0 2024-04-24T23:13:57.631 INFO:tasks.workunit.client.0.smithi031.stdout:5/738: dwrite d0/d6/d3b/d4b/d1d/fb2 [0,4194304] 0 2024-04-24T23:13:57.633 INFO:tasks.workunit.client.0.smithi031.stdout:2/673: link d5/d99/ff3 d5/d28/ff4 0 2024-04-24T23:13:57.633 INFO:tasks.workunit.client.1.smithi040.stdout:6/483: rename d7/d1a/d20/d51/f79 to d7/d1a/d20/d51/d1e/d46/f98 0 2024-04-24T23:13:57.634 INFO:tasks.workunit.client.0.smithi031.stdout:2/674: fdatasync d5/d6/d29/d4f/fc5 0 2024-04-24T23:13:57.635 INFO:tasks.workunit.client.1.smithi040.stdout:6/484: write d7/d1a/d20/d51/d2a/f7c [905285,59590] 0 2024-04-24T23:13:57.638 INFO:tasks.workunit.client.1.smithi040.stdout:9/532: write d7/f82 [1448091,125382] 0 2024-04-24T23:13:57.643 INFO:tasks.workunit.client.1.smithi040.stdout:9/533: read d7/da/d17/d40/d55/d61/f8f [4182520,30432] 0 2024-04-24T23:13:57.644 INFO:tasks.workunit.client.1.smithi040.stdout:7/549: unlink db/d12/c2d 0 2024-04-24T23:13:57.645 INFO:tasks.workunit.client.1.smithi040.stdout:9/534: read d7/da/d17/d40/d55/d2f/f46 [1111153,63515] 0 2024-04-24T23:13:57.646 INFO:tasks.workunit.client.0.smithi031.stdout:8/926: mkdir d8/d1a/d1d/d50/dc0/d9d/d136 0 2024-04-24T23:13:57.647 INFO:tasks.workunit.client.1.smithi040.stdout:8/522: fsync d0/d16/d19/f5f 0 2024-04-24T23:13:57.648 INFO:tasks.workunit.client.1.smithi040.stdout:8/523: dread - d0/d16/d1b/f94 zero size 2024-04-24T23:13:57.651 INFO:tasks.workunit.client.0.smithi031.stdout:4/899: rmdir d3/d20/d2d/d44/d5b/d5d 39 2024-04-24T23:13:57.651 INFO:tasks.workunit.client.1.smithi040.stdout:1/506: dread da/d12/d14/f2d [0,4194304] 0 2024-04-24T23:13:57.653 INFO:tasks.workunit.client.1.smithi040.stdout:0/579: rename df/d1e/d20/c66 to df/d12/cbf 0 2024-04-24T23:13:57.654 INFO:tasks.workunit.client.0.smithi031.stdout:5/739: creat d0/d6/d3b/d4f/ff2 x:0 0 0 2024-04-24T23:13:57.656 INFO:tasks.workunit.client.0.smithi031.stdout:5/740: stat d0/d6/d3b/d4b/d1d/fb2 0 2024-04-24T23:13:57.657 INFO:tasks.workunit.client.0.smithi031.stdout:9/872: write d2/d16/dbe/f31 [1509892,49396] 0 2024-04-24T23:13:57.657 INFO:tasks.workunit.client.0.smithi031.stdout:5/741: read d0/d7/d56/fa7 [3689574,15465] 0 2024-04-24T23:13:57.659 INFO:tasks.workunit.client.0.smithi031.stdout:9/873: dread d2/fdc [0,4194304] 0 2024-04-24T23:13:57.659 INFO:tasks.workunit.client.0.smithi031.stdout:5/742: write d0/d6/d3b/d4b/dc/d25/f69 [1998467,86734] 0 2024-04-24T23:13:57.660 INFO:tasks.workunit.client.1.smithi040.stdout:5/487: write d2/d2f/f33 [233869,8663] 0 2024-04-24T23:13:57.680 INFO:tasks.workunit.client.0.smithi031.stdout:2/675: dread d5/d6/d29/f72 [0,4194304] 0 2024-04-24T23:13:57.684 INFO:tasks.workunit.client.1.smithi040.stdout:8/524: symlink d0/d16/d19/d71/d76/laf 0 2024-04-24T23:13:57.684 INFO:tasks.workunit.client.1.smithi040.stdout:2/556: write da/d1c/d4f/f61 [11824,40728] 0 2024-04-24T23:13:57.686 INFO:tasks.workunit.client.0.smithi031.stdout:2/676: chown d5/d6/d29/d26/db0/dc2/f79 20 1 2024-04-24T23:13:57.687 INFO:tasks.workunit.client.0.smithi031.stdout:7/798: dwrite d1/d2/f7 [0,4194304] 0 2024-04-24T23:13:57.691 INFO:tasks.workunit.client.0.smithi031.stdout:1/699: dwrite d1/da/d12/d65/dcf/d11/d21/d5a/f76 [0,4194304] 0 2024-04-24T23:13:57.692 INFO:tasks.workunit.client.1.smithi040.stdout:1/507: rename da/d31 to da/d35/dab 0 2024-04-24T23:13:57.692 INFO:tasks.workunit.client.0.smithi031.stdout:2/677: chown d5/d6/d29/d26/db0/dc2/dad/cd4 89829 1 2024-04-24T23:13:57.692 INFO:tasks.workunit.client.0.smithi031.stdout:6/829: dwrite d13/d1b/d32/d9c/fc4 [0,4194304] 0 2024-04-24T23:13:57.694 INFO:tasks.workunit.client.1.smithi040.stdout:8/525: write d0/d8/d55/f83 [282022,108034] 0 2024-04-24T23:13:57.695 INFO:tasks.workunit.client.1.smithi040.stdout:0/580: creat df/d1e/d20/d21/fc0 x:0 0 0 2024-04-24T23:13:57.696 INFO:tasks.workunit.client.0.smithi031.stdout:2/678: stat d5/d8/d45/d5b/d49/d2f/faa 0 2024-04-24T23:13:57.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/743: mknod d0/d6/d32/cf3 0 2024-04-24T23:13:57.703 INFO:tasks.workunit.client.1.smithi040.stdout:3/556: getdents d1/d2a/d8d 0 2024-04-24T23:13:57.709 INFO:tasks.workunit.client.0.smithi031.stdout:4/900: fsync d3/d23/d69/d3f/f57 0 2024-04-24T23:13:57.711 INFO:tasks.workunit.client.0.smithi031.stdout:8/927: truncate d8/d15/d37/f2e 1357702 0 2024-04-24T23:13:57.711 INFO:tasks.workunit.client.1.smithi040.stdout:1/508: mkdir da/d12/d23/dac 0 2024-04-24T23:13:57.711 INFO:tasks.workunit.client.0.smithi031.stdout:8/928: fsync d8/d15/d37/d43/daf/fb9 0 2024-04-24T23:13:57.714 INFO:tasks.workunit.client.1.smithi040.stdout:0/581: symlink df/d1e/d89/lc1 0 2024-04-24T23:13:57.715 INFO:tasks.workunit.client.1.smithi040.stdout:0/582: write df/d14/f96 [539923,83344] 0 2024-04-24T23:13:57.717 INFO:tasks.workunit.client.0.smithi031.stdout:7/799: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/f116 x:0 0 0 2024-04-24T23:13:57.719 INFO:tasks.workunit.client.0.smithi031.stdout:2/679: creat d5/d8/d45/ff5 x:0 0 0 2024-04-24T23:13:57.719 INFO:tasks.workunit.client.1.smithi040.stdout:4/441: sync 2024-04-24T23:13:57.723 INFO:tasks.workunit.client.0.smithi031.stdout:5/744: truncate d0/d6/d3b/d4f/fce 317105 0 2024-04-24T23:13:57.725 INFO:tasks.workunit.client.0.smithi031.stdout:5/745: stat d0/d6/d3b/d4b/d1d/d2e/d28 0 2024-04-24T23:13:57.726 INFO:tasks.workunit.client.0.smithi031.stdout:4/901: dread - d3/d20/d2d/d44/d5b/d5d/f103 zero size 2024-04-24T23:13:57.727 INFO:tasks.workunit.client.1.smithi040.stdout:1/509: symlink da/d12/d14/d25/d5e/lad 0 2024-04-24T23:13:57.728 INFO:tasks.workunit.client.1.smithi040.stdout:6/485: write d7/d1a/d20/d51/d2f/f5b [1724005,17783] 0 2024-04-24T23:13:57.729 INFO:tasks.workunit.client.1.smithi040.stdout:6/486: chown d7/l18 67028294 1 2024-04-24T23:13:57.730 INFO:tasks.workunit.client.1.smithi040.stdout:6/487: dread - d7/d1a/d20/d51/d1e/f86 zero size 2024-04-24T23:13:57.731 INFO:tasks.workunit.client.0.smithi031.stdout:9/874: dwrite d2/f75 [0,4194304] 0 2024-04-24T23:13:57.732 INFO:tasks.workunit.client.1.smithi040.stdout:4/442: dread da/d45/d58/f73 [0,4194304] 0 2024-04-24T23:13:57.735 INFO:tasks.workunit.client.0.smithi031.stdout:6/830: mknod d13/d15/d1d/ddd/c113 0 2024-04-24T23:13:57.744 INFO:tasks.workunit.client.0.smithi031.stdout:2/680: mknod d5/d8/d15/d56/d84/d75/cf6 0 2024-04-24T23:13:57.744 INFO:tasks.workunit.client.1.smithi040.stdout:2/557: dwrite da/d10/d15/f30 [0,4194304] 0 2024-04-24T23:13:57.745 INFO:tasks.workunit.client.1.smithi040.stdout:2/558: chown da/d29/d55/fa9 1 1 2024-04-24T23:13:57.747 INFO:tasks.workunit.client.1.smithi040.stdout:5/488: rmdir d2/d7/d8/d1f/d57/d27/d66 0 2024-04-24T23:13:57.748 INFO:tasks.workunit.client.1.smithi040.stdout:5/489: chown d2/d7/d8/f34 8 1 2024-04-24T23:13:57.750 INFO:tasks.workunit.client.1.smithi040.stdout:7/550: getdents db/d43 0 2024-04-24T23:13:57.754 INFO:tasks.workunit.client.1.smithi040.stdout:1/510: symlink da/d12/d14/d25/d5e/lae 0 2024-04-24T23:13:57.755 INFO:tasks.workunit.client.0.smithi031.stdout:8/929: write d8/d15/d37/d43/d7b/d87/fa0 [639070,1096] 0 2024-04-24T23:13:57.758 INFO:tasks.workunit.client.1.smithi040.stdout:7/551: dread db/d12/d73/fbe [0,4194304] 0 2024-04-24T23:13:57.759 INFO:tasks.workunit.client.0.smithi031.stdout:9/875: rename d2/dae/d14/d78/d84/d4a/la2 to d2/dae/d14/d78/d84/d4a/dab/dad/db5/l10b 0 2024-04-24T23:13:57.760 INFO:tasks.workunit.client.1.smithi040.stdout:9/535: sync 2024-04-24T23:13:57.762 INFO:tasks.workunit.client.0.smithi031.stdout:6/831: fdatasync d13/fc5 0 2024-04-24T23:13:57.764 INFO:tasks.workunit.client.1.smithi040.stdout:0/583: symlink df/d1e/d70/lc2 0 2024-04-24T23:13:57.768 INFO:tasks.workunit.client.0.smithi031.stdout:7/800: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d84/f9e [0,4194304] 0 2024-04-24T23:13:57.769 INFO:tasks.workunit.client.0.smithi031.stdout:2/681: mknod d5/d6/d29/d26/db0/dc7/cf7 0 2024-04-24T23:13:57.774 INFO:tasks.workunit.client.0.smithi031.stdout:4/902: dread d3/d23/d3b/f41 [0,4194304] 0 2024-04-24T23:13:57.774 INFO:tasks.workunit.client.0.smithi031.stdout:2/682: read d5/d99/da2/fd2 [2266162,115057] 0 2024-04-24T23:13:57.775 INFO:tasks.workunit.client.1.smithi040.stdout:5/490: mkdir d2/d7/d8/d1f/d57/d3e/d48/d9f 0 2024-04-24T23:13:57.777 INFO:tasks.workunit.client.0.smithi031.stdout:5/746: dread d0/d6/d3b/f68 [0,4194304] 0 2024-04-24T23:13:57.780 INFO:tasks.workunit.client.1.smithi040.stdout:1/511: mknod da/d12/d23/caf 0 2024-04-24T23:13:57.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/876: rename d2/dae/d14/l29 to d2/d43/d9d/dc0/dd5/l10c 0 2024-04-24T23:13:57.786 INFO:tasks.workunit.client.0.smithi031.stdout:1/700: link d1/d33/d36/fa7 d1/da/d10/d52/fdf 0 2024-04-24T23:13:57.787 INFO:tasks.workunit.client.0.smithi031.stdout:1/701: dread - d1/da/d12/d7e/d7b/d4e/fd7 zero size 2024-04-24T23:13:57.788 INFO:tasks.workunit.client.1.smithi040.stdout:8/526: getdents d0/d16/d19/d4f/d9e/d90/d91 0 2024-04-24T23:13:57.789 INFO:tasks.workunit.client.1.smithi040.stdout:8/527: truncate d0/d8/d22/f62 4344350 0 2024-04-24T23:13:57.791 INFO:tasks.workunit.client.0.smithi031.stdout:6/832: creat d13/d15/d1d/d23/d24/d43/f114 x:0 0 0 2024-04-24T23:13:57.793 INFO:tasks.workunit.client.1.smithi040.stdout:2/559: write da/d1c/f27 [599467,1679] 0 2024-04-24T23:13:57.796 INFO:tasks.workunit.client.0.smithi031.stdout:8/930: unlink d8/fd 0 2024-04-24T23:13:57.799 INFO:tasks.workunit.client.0.smithi031.stdout:4/903: creat d3/d5e/dc4/f122 x:0 0 0 2024-04-24T23:13:57.799 INFO:tasks.workunit.client.1.smithi040.stdout:3/557: link d1/d2d/de/c4a d1/cc8 0 2024-04-24T23:13:57.800 INFO:tasks.workunit.client.1.smithi040.stdout:2/560: dwrite da/d10/f2b [0,4194304] 0 2024-04-24T23:13:57.800 INFO:tasks.workunit.client.1.smithi040.stdout:3/558: readlink d1/d2d/de/d30/d37/l7a 0 2024-04-24T23:13:57.802 INFO:tasks.workunit.client.1.smithi040.stdout:5/491: stat d2/d7/fe 0 2024-04-24T23:13:57.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/512: rename da/d12/d18/d2a/d32/c88 to da/d35/d61/cb0 0 2024-04-24T23:13:57.805 INFO:tasks.workunit.client.1.smithi040.stdout:4/443: dread da/d3f/d6b/d2b/f52 [4194304,4194304] 0 2024-04-24T23:13:57.811 INFO:tasks.workunit.client.0.smithi031.stdout:1/702: truncate d1/f18 3740719 0 2024-04-24T23:13:57.813 INFO:tasks.workunit.client.1.smithi040.stdout:8/528: mknod d0/d38/d4c/cb0 0 2024-04-24T23:13:57.813 INFO:tasks.workunit.client.0.smithi031.stdout:1/703: truncate d1/da/d12/d7e/d7b/d4e/fd7 604524 0 2024-04-24T23:13:57.816 INFO:tasks.workunit.client.1.smithi040.stdout:0/584: dwrite df/d1e/f3e [0,4194304] 0 2024-04-24T23:13:57.818 INFO:tasks.workunit.client.0.smithi031.stdout:2/683: dwrite d5/d6/d29/f12 [4194304,4194304] 0 2024-04-24T23:13:57.820 INFO:tasks.workunit.client.1.smithi040.stdout:3/559: creat d1/d2a/d6a/fc9 x:0 0 0 2024-04-24T23:13:57.821 INFO:tasks.workunit.client.1.smithi040.stdout:3/560: chown d1/d2d/de/d30/c57 33162374 1 2024-04-24T23:13:57.823 INFO:tasks.workunit.client.0.smithi031.stdout:5/747: link d0/d6/d3b/d4b/d1d/d2e/dc6/f21 d0/d7/d56/d7c/d7e/ff4 0 2024-04-24T23:13:57.826 INFO:tasks.workunit.client.0.smithi031.stdout:9/877: fdatasync d2/f5 0 2024-04-24T23:13:57.826 INFO:tasks.workunit.client.1.smithi040.stdout:4/444: creat da/d3f/d6b/d3c/f9f x:0 0 0 2024-04-24T23:13:57.828 INFO:tasks.workunit.client.1.smithi040.stdout:4/445: write da/d3f/d40/d41/d62/f6f [729610,47949] 0 2024-04-24T23:13:57.829 INFO:tasks.workunit.client.1.smithi040.stdout:6/488: getdents d7/d1a/d20/d51/d1e 0 2024-04-24T23:13:57.832 INFO:tasks.workunit.client.1.smithi040.stdout:6/489: dread d7/d1a/d20/d51/d1e/d46/d69/f8d [0,4194304] 0 2024-04-24T23:13:57.834 INFO:tasks.workunit.client.1.smithi040.stdout:6/490: write d7/d1a/d20/d51/d1e/f3d [1879805,76283] 0 2024-04-24T23:13:57.837 INFO:tasks.workunit.client.1.smithi040.stdout:9/536: link d7/da/d17/d40/c58 d7/d92/d9c/cbd 0 2024-04-24T23:13:57.839 INFO:tasks.workunit.client.1.smithi040.stdout:9/537: dread - d7/da/d17/d40/d55/d2f/f7c zero size 2024-04-24T23:13:57.840 INFO:tasks.workunit.client.1.smithi040.stdout:0/585: mkdir df/d12/d58/dc3 0 2024-04-24T23:13:57.841 INFO:tasks.workunit.client.1.smithi040.stdout:6/491: dwrite d7/de/f5e [0,4194304] 0 2024-04-24T23:13:57.841 INFO:tasks.workunit.client.0.smithi031.stdout:4/904: mkdir d3/d23/d69/d3f/d123 0 2024-04-24T23:13:57.842 INFO:tasks.workunit.client.1.smithi040.stdout:9/538: dread - d7/da/d17/d40/d55/d2f/d1f/d31/f98 zero size 2024-04-24T23:13:57.849 INFO:tasks.workunit.client.0.smithi031.stdout:5/748: dread d0/d6/d3b/d4b/dc/f47 [0,4194304] 0 2024-04-24T23:13:57.850 INFO:tasks.workunit.client.0.smithi031.stdout:5/749: readlink d0/d6/d3b/d4f/lcb 0 2024-04-24T23:13:57.855 INFO:tasks.workunit.client.1.smithi040.stdout:4/446: fsync da/d3f/d6b/f79 0 2024-04-24T23:13:57.856 INFO:tasks.workunit.client.0.smithi031.stdout:7/801: link d1/l55 d1/d2/dd/l117 0 2024-04-24T23:13:57.857 INFO:tasks.workunit.client.1.smithi040.stdout:8/529: mkdir d0/d16/d1b/d58/d8f/db1 0 2024-04-24T23:13:57.858 INFO:tasks.workunit.client.0.smithi031.stdout:6/833: link d13/f7c d13/d15/d1d/d23/d44/d47/d10c/f115 0 2024-04-24T23:13:57.858 INFO:tasks.workunit.client.1.smithi040.stdout:8/530: fsync d0/d16/d1b/d58/fab 0 2024-04-24T23:13:57.861 INFO:tasks.workunit.client.0.smithi031.stdout:6/834: truncate d13/d15/d1d/d23/d44/d47/d69/dd5/f110 663330 0 2024-04-24T23:13:57.865 INFO:tasks.workunit.client.1.smithi040.stdout:0/586: mknod df/d1e/d20/d21/d41/d8a/cc4 0 2024-04-24T23:13:57.865 INFO:tasks.workunit.client.1.smithi040.stdout:8/531: dwrite d0/d16/d1b/d3f/fa8 [0,4194304] 0 2024-04-24T23:13:57.865 INFO:tasks.workunit.client.0.smithi031.stdout:6/835: dread d13/d15/d1d/d23/f2b [0,4194304] 0 2024-04-24T23:13:57.866 INFO:tasks.workunit.client.1.smithi040.stdout:3/561: getdents d1/d80/dbf 0 2024-04-24T23:13:57.870 INFO:tasks.workunit.client.1.smithi040.stdout:3/562: read d1/d2d/f60 [114292,86773] 0 2024-04-24T23:13:57.870 INFO:tasks.workunit.client.1.smithi040.stdout:6/492: creat d7/de/d84/f99 x:0 0 0 2024-04-24T23:13:57.872 INFO:tasks.workunit.client.0.smithi031.stdout:9/878: rename d2/d43/d6e/c6f to d2/dae/d14/d78/d84/d4a/d4b/d96/c10d 0 2024-04-24T23:13:57.873 INFO:tasks.workunit.client.1.smithi040.stdout:7/552: sync 2024-04-24T23:13:57.874 INFO:tasks.workunit.client.0.smithi031.stdout:1/704: creat d1/da/fe0 x:0 0 0 2024-04-24T23:13:57.875 INFO:tasks.workunit.client.1.smithi040.stdout:5/492: creat d2/d2f/fa0 x:0 0 0 2024-04-24T23:13:57.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/705: truncate d1/d3e/d46/dbd/fdd 666250 0 2024-04-24T23:13:57.878 INFO:tasks.workunit.client.0.smithi031.stdout:8/931: getdents d8/d15/d37/d43/d7b/dd3 0 2024-04-24T23:13:57.880 INFO:tasks.workunit.client.1.smithi040.stdout:6/493: dread d7/d1a/d20/d51/d2f/f5c [0,4194304] 0 2024-04-24T23:13:57.881 INFO:tasks.workunit.client.1.smithi040.stdout:0/587: symlink df/d1e/d70/lc5 0 2024-04-24T23:13:57.882 INFO:tasks.workunit.client.0.smithi031.stdout:5/750: unlink d0/d6/d3b/d4b/l6a 0 2024-04-24T23:13:57.882 INFO:tasks.workunit.client.1.smithi040.stdout:0/588: chown df/d1e/d20/d21/d51/d79/daa 88685 1 2024-04-24T23:13:57.891 INFO:tasks.workunit.client.1.smithi040.stdout:3/563: creat d1/d2a/d8d/fca x:0 0 0 2024-04-24T23:13:57.892 INFO:tasks.workunit.client.1.smithi040.stdout:3/564: write d1/d9/d18/d36/d55/f89 [91790,123807] 0 2024-04-24T23:13:57.893 INFO:tasks.workunit.client.0.smithi031.stdout:1/706: read d1/da/d12/d65/dcf/f2b [1771236,123841] 0 2024-04-24T23:13:57.894 INFO:tasks.workunit.client.0.smithi031.stdout:4/905: creat d3/f124 x:0 0 0 2024-04-24T23:13:57.894 INFO:tasks.workunit.client.1.smithi040.stdout:3/565: fsync d1/d9/d39/fb0 0 2024-04-24T23:13:57.895 INFO:tasks.workunit.client.1.smithi040.stdout:6/494: dread d7/d1a/d20/d51/d1e/f49 [0,4194304] 0 2024-04-24T23:13:57.895 INFO:tasks.workunit.client.0.smithi031.stdout:8/932: rmdir d8/d15/d37/d43/d7b 39 2024-04-24T23:13:57.896 INFO:tasks.workunit.client.1.smithi040.stdout:3/566: readlink d1/d9/d18/d36/d61/lb6 0 2024-04-24T23:13:57.896 INFO:tasks.workunit.client.0.smithi031.stdout:8/933: readlink d8/d15/d37/d43/d5c/de1/l55 0 2024-04-24T23:13:57.897 INFO:tasks.workunit.client.1.smithi040.stdout:1/513: getdents da/d12/d14/d58 0 2024-04-24T23:13:57.897 INFO:tasks.workunit.client.0.smithi031.stdout:8/934: chown d8/d15/f123 28947 1 2024-04-24T23:13:57.898 INFO:tasks.workunit.client.0.smithi031.stdout:6/836: write d13/d15/d1d/d23/d2d/f4c [764782,119801] 0 2024-04-24T23:13:57.899 INFO:tasks.workunit.client.1.smithi040.stdout:3/567: dread d1/d9/d15/f1b [8388608,4194304] 0 2024-04-24T23:13:57.899 INFO:tasks.workunit.client.1.smithi040.stdout:3/568: stat d1/d2a 0 2024-04-24T23:13:57.900 INFO:tasks.workunit.client.1.smithi040.stdout:7/553: unlink db/d12/d20/d52/f79 0 2024-04-24T23:13:57.901 INFO:tasks.workunit.client.1.smithi040.stdout:8/532: write d0/d8/dc/d34/f3c [1360546,110922] 0 2024-04-24T23:13:57.905 INFO:tasks.workunit.client.0.smithi031.stdout:6/837: dread d13/d15/d1d/d23/d44/d47/d69/faa [0,4194304] 0 2024-04-24T23:13:57.905 INFO:tasks.workunit.client.1.smithi040.stdout:5/493: truncate d2/d7/d8/d1f/d57/d27/d3c/f95 632683 0 2024-04-24T23:13:57.906 INFO:tasks.workunit.client.1.smithi040.stdout:2/561: sync 2024-04-24T23:13:57.906 INFO:tasks.workunit.client.1.smithi040.stdout:5/494: read d2/d7/d8/d1f/f9d [3541939,119698] 0 2024-04-24T23:13:57.907 INFO:tasks.workunit.client.0.smithi031.stdout:7/802: dwrite d1/d2/f11 [0,4194304] 0 2024-04-24T23:13:57.907 INFO:tasks.workunit.client.1.smithi040.stdout:3/569: dwrite d1/d9/f27 [12582912,4194304] 0 2024-04-24T23:13:57.908 INFO:tasks.workunit.client.1.smithi040.stdout:2/562: dread - da/d1c/d37/d39/d79/da3/fb3 zero size 2024-04-24T23:13:57.909 INFO:tasks.workunit.client.0.smithi031.stdout:9/879: creat d2/dae/d14/d1d/d45/d99/d9c/d100/f10e x:0 0 0 2024-04-24T23:13:57.911 INFO:tasks.workunit.client.1.smithi040.stdout:0/589: creat df/d12/d4b/fc6 x:0 0 0 2024-04-24T23:13:57.912 INFO:tasks.workunit.client.1.smithi040.stdout:4/447: write da/d3f/d40/d47/d75/f97 [191170,120376] 0 2024-04-24T23:13:57.918 INFO:tasks.workunit.client.1.smithi040.stdout:9/539: creat d7/fbe x:0 0 0 2024-04-24T23:13:57.921 INFO:tasks.workunit.client.1.smithi040.stdout:9/540: read d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [7072637,88731] 0 2024-04-24T23:13:57.923 INFO:tasks.workunit.client.1.smithi040.stdout:9/541: write d7/da/d17/d40/d55/d2f/d1f/d34/f78 [783583,71683] 0 2024-04-24T23:13:57.924 INFO:tasks.workunit.client.1.smithi040.stdout:7/554: write db/d12/d20/d52/d44/fac [732821,26825] 0 2024-04-24T23:13:57.925 INFO:tasks.workunit.client.1.smithi040.stdout:9/542: dread - d7/da/d17/d40/d55/d2f/f7c zero size 2024-04-24T23:13:57.926 INFO:tasks.workunit.client.0.smithi031.stdout:9/880: dread d2/dae/d14/d78/d84/d4a/f91 [0,4194304] 0 2024-04-24T23:13:57.927 INFO:tasks.workunit.client.1.smithi040.stdout:8/533: fdatasync d0/d8/dc/f63 0 2024-04-24T23:13:57.927 INFO:tasks.workunit.client.1.smithi040.stdout:9/543: write d7/da/d17/d40/d55/d2f/d28/f6b [204663,70945] 0 2024-04-24T23:13:57.928 INFO:tasks.workunit.client.0.smithi031.stdout:4/906: mknod d3/d23/d3b/df2/c125 0 2024-04-24T23:13:57.932 INFO:tasks.workunit.client.0.smithi031.stdout:8/935: mknod d8/d15/d37/d43/d7b/d87/dad/c137 0 2024-04-24T23:13:57.933 INFO:tasks.workunit.client.0.smithi031.stdout:7/803: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/f35 [6315266,59093] 0 2024-04-24T23:13:57.936 INFO:tasks.workunit.client.1.smithi040.stdout:3/570: symlink d1/d2a/d6a/d94/lcb 0 2024-04-24T23:13:57.937 INFO:tasks.workunit.client.1.smithi040.stdout:3/571: chown d1/d9/d18/da7/cb1 712603291 1 2024-04-24T23:13:57.938 INFO:tasks.workunit.client.1.smithi040.stdout:3/572: fdatasync d1/d2a/f9a 0 2024-04-24T23:13:57.938 INFO:tasks.workunit.client.0.smithi031.stdout:6/838: fdatasync d13/d15/d1d/d23/d2d/f90 0 2024-04-24T23:13:57.940 INFO:tasks.workunit.client.0.smithi031.stdout:1/707: getdents d1/da/d12/d65/dcf/d11 0 2024-04-24T23:13:57.940 INFO:tasks.workunit.client.0.smithi031.stdout:6/839: write d13/d15/d1d/d23/d24/d43/f114 [730740,49654] 0 2024-04-24T23:13:57.942 INFO:tasks.workunit.client.0.smithi031.stdout:4/907: truncate d3/d23/d3b/fb4 7388421 0 2024-04-24T23:13:57.944 INFO:tasks.workunit.client.0.smithi031.stdout:4/908: chown d3/d20/d25/d95/ca6 51385950 1 2024-04-24T23:13:57.944 INFO:tasks.workunit.client.1.smithi040.stdout:3/573: dwrite d1/d2d/de/d30/d37/fc6 [0,4194304] 0 2024-04-24T23:13:57.944 INFO:tasks.workunit.client.1.smithi040.stdout:3/574: readlink d1/d9/l1e 0 2024-04-24T23:13:57.948 INFO:tasks.workunit.client.1.smithi040.stdout:1/514: mknod da/d35/da3/cb1 0 2024-04-24T23:13:57.949 INFO:tasks.workunit.client.1.smithi040.stdout:3/575: dwrite d1/d2d/f24 [4194304,4194304] 0 2024-04-24T23:13:57.957 INFO:tasks.workunit.client.0.smithi031.stdout:1/708: rmdir d1/da/d12/d65/dcf/d11/d21/d5a 39 2024-04-24T23:13:57.958 INFO:tasks.workunit.client.0.smithi031.stdout:1/709: fdatasync d1/da/f2c 0 2024-04-24T23:13:57.964 INFO:tasks.workunit.client.1.smithi040.stdout:9/544: rename d7/da/d17/d40/d55/d2f/d1f/d34 to d7/da/d17/d40/d55/d2f/d1f/d4e/dbf 0 2024-04-24T23:13:57.967 INFO:tasks.workunit.client.0.smithi031.stdout:5/751: getdents d0/d6/d3b/d4b/dc/d86 0 2024-04-24T23:13:57.970 INFO:tasks.workunit.client.0.smithi031.stdout:7/804: write d1/d2/dd/f7d [3365968,104917] 0 2024-04-24T23:13:57.970 INFO:tasks.workunit.client.1.smithi040.stdout:2/563: symlink da/d1c/d37/d39/lcf 0 2024-04-24T23:13:57.971 INFO:tasks.workunit.client.1.smithi040.stdout:5/495: write d2/d7/d8/d1f/d57/d27/d3c/f47 [2815016,3692] 0 2024-04-24T23:13:57.972 INFO:tasks.workunit.client.1.smithi040.stdout:0/590: mknod df/d12/d49/cc7 0 2024-04-24T23:13:57.972 INFO:tasks.workunit.client.0.smithi031.stdout:1/710: rmdir d1/da/d12/d7e 39 2024-04-24T23:13:57.974 INFO:tasks.workunit.client.1.smithi040.stdout:0/591: write df/d1e/d20/d21/d51/f8f [709934,91001] 0 2024-04-24T23:13:57.974 INFO:tasks.workunit.client.1.smithi040.stdout:0/592: fsync df/d12/d58/d5b/fa0 0 2024-04-24T23:13:57.975 INFO:tasks.workunit.client.1.smithi040.stdout:0/593: chown df/d12/d1a/cb4 2806 1 2024-04-24T23:13:57.975 INFO:tasks.workunit.client.0.smithi031.stdout:6/840: creat d13/d1b/d32/db4/d106/f116 x:0 0 0 2024-04-24T23:13:57.976 INFO:tasks.workunit.client.0.smithi031.stdout:6/841: fsync d13/d1b/d32/d9c/fed 0 2024-04-24T23:13:57.977 INFO:tasks.workunit.client.0.smithi031.stdout:6/842: stat d13/d1b/d32/db4/cc7 0 2024-04-24T23:13:57.977 INFO:tasks.workunit.client.0.smithi031.stdout:8/936: mkdir d8/d1a/d1d/d50/d138 0 2024-04-24T23:13:57.989 INFO:tasks.workunit.client.1.smithi040.stdout:0/594: dread df/d12/f1f [0,4194304] 0 2024-04-24T23:13:58.021 INFO:tasks.workunit.client.0.smithi031.stdout:0/838: sync 2024-04-24T23:13:58.026 INFO:tasks.workunit.client.0.smithi031.stdout:8/937: dread d8/d1a/f2f [0,4194304] 0 2024-04-24T23:13:58.026 INFO:tasks.workunit.client.1.smithi040.stdout:3/576: creat d1/d80/fcc x:0 0 0 2024-04-24T23:13:58.027 INFO:tasks.workunit.client.0.smithi031.stdout:7/805: mknod d1/d2/d83/dc1/c118 0 2024-04-24T23:13:58.027 INFO:tasks.workunit.client.1.smithi040.stdout:3/577: dread d1/d2a/d6a/d94/f98 [0,4194304] 0 2024-04-24T23:13:58.029 INFO:tasks.workunit.client.1.smithi040.stdout:3/578: read d1/d9/d2e/f5f [121409,55692] 0 2024-04-24T23:13:58.035 INFO:tasks.workunit.client.1.smithi040.stdout:8/534: symlink d0/d16/lb2 0 2024-04-24T23:13:58.035 INFO:tasks.workunit.client.1.smithi040.stdout:8/535: chown d0/l13 1489233 1 2024-04-24T23:13:58.039 INFO:tasks.workunit.client.1.smithi040.stdout:2/564: fdatasync da/f11 0 2024-04-24T23:13:58.040 INFO:tasks.workunit.client.1.smithi040.stdout:7/555: dwrite f2 [0,4194304] 0 2024-04-24T23:13:58.041 INFO:tasks.workunit.client.1.smithi040.stdout:2/565: truncate da/d81/fab 4474625 0 2024-04-24T23:13:58.124 INFO:tasks.workunit.client.1.smithi040.stdout:4/448: link da/d3f/d6b/d3c/c88 da/d45/d48/d6e/ca0 0 2024-04-24T23:13:58.126 INFO:tasks.workunit.client.1.smithi040.stdout:6/495: rmdir d7/de/d89/d8b 0 2024-04-24T23:13:58.127 INFO:tasks.workunit.client.1.smithi040.stdout:1/515: fdatasync da/d35/dab/f52 0 2024-04-24T23:13:58.134 INFO:tasks.workunit.client.1.smithi040.stdout:8/536: rename d0/d16/d1b/f54 to d0/d52/fb3 0 2024-04-24T23:13:58.135 INFO:tasks.workunit.client.1.smithi040.stdout:8/537: fdatasync d0/d16/d19/d4f/f72 0 2024-04-24T23:13:58.136 INFO:tasks.workunit.client.1.smithi040.stdout:8/538: write d0/d8/d22/f62 [640279,76150] 0 2024-04-24T23:13:58.138 INFO:tasks.workunit.client.1.smithi040.stdout:5/496: mkdir d2/d7/d8/d1f/da1 0 2024-04-24T23:13:58.139 INFO:tasks.workunit.client.1.smithi040.stdout:5/497: chown d2/d7/d8/d1f/d57/d27/d3c/d43/f6a 7024 1 2024-04-24T23:13:58.139 INFO:tasks.workunit.client.1.smithi040.stdout:2/566: mknod da/d29/d55/cd0 0 2024-04-24T23:13:58.140 INFO:tasks.workunit.client.1.smithi040.stdout:4/449: creat da/d3f/d6b/d3c/d69/fa1 x:0 0 0 2024-04-24T23:13:58.141 INFO:tasks.workunit.client.1.smithi040.stdout:4/450: read - da/d45/d48/d6e/f92 zero size 2024-04-24T23:13:58.142 INFO:tasks.workunit.client.1.smithi040.stdout:0/595: mknod df/d14/cc8 0 2024-04-24T23:13:58.144 INFO:tasks.workunit.client.1.smithi040.stdout:1/516: unlink da/d12/d14/d58/la7 0 2024-04-24T23:13:58.146 INFO:tasks.workunit.client.1.smithi040.stdout:5/498: dwrite d2/d7/d8/d1f/d57/d27/d3c/d43/f6a [0,4194304] 0 2024-04-24T23:13:58.148 INFO:tasks.workunit.client.1.smithi040.stdout:7/556: rename db/d2f/d37/d7e/f97 to db/d12/d3f/fc1 0 2024-04-24T23:13:58.152 INFO:tasks.workunit.client.1.smithi040.stdout:9/545: link d7/da/d17/ca0 d7/da/d17/d40/d55/d2f/d1f/d4e/d83/cc0 0 2024-04-24T23:13:58.157 INFO:tasks.workunit.client.1.smithi040.stdout:8/539: symlink d0/d16/d19/d71/lb4 0 2024-04-24T23:13:58.158 INFO:tasks.workunit.client.1.smithi040.stdout:8/540: truncate d0/d16/d1b/f95 1113277 0 2024-04-24T23:13:58.161 INFO:tasks.workunit.client.1.smithi040.stdout:4/451: fsync da/d3f/d6b/d2b/f39 0 2024-04-24T23:13:58.162 INFO:tasks.workunit.client.1.smithi040.stdout:4/452: chown da/d3f/d40/d9b 33871510 1 2024-04-24T23:13:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:57 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:57 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:57 smithi040 ceph-mon[33128]: Reconfiguring grafana.smithi031 (dependencies changed)... 2024-04-24T23:13:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:57 smithi040 ceph-mon[33128]: Reconfiguring daemon grafana.smithi031 on smithi031 2024-04-24T23:13:58.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:13:57 smithi040 ceph-mon[33128]: pgmap v32: 65 pgs: 65 active+clean; 2.6 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 75 MiB/s rd, 246 MiB/s wr, 508 op/s 2024-04-24T23:13:58.163 INFO:tasks.workunit.client.1.smithi040.stdout:4/453: write da/d3f/d6b/d2b/f6d [22307,9084] 0 2024-04-24T23:13:58.172 INFO:tasks.workunit.client.1.smithi040.stdout:6/496: rename d7/d1a/d20/d51/c39 to d7/d1a/d56/c9a 0 2024-04-24T23:13:58.173 INFO:tasks.workunit.client.1.smithi040.stdout:9/546: mknod d7/d92/cc1 0 2024-04-24T23:13:58.176 INFO:tasks.workunit.client.0.smithi031.stdout:3/732: sync 2024-04-24T23:13:58.180 INFO:tasks.workunit.client.1.smithi040.stdout:4/454: unlink da/d3f/d40/d41/d60/l66 0 2024-04-24T23:13:58.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:57 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:58.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:57 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:13:58.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:57 smithi031 ceph-mon[26411]: Reconfiguring grafana.smithi031 (dependencies changed)... 2024-04-24T23:13:58.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:57 smithi031 ceph-mon[26411]: Reconfiguring daemon grafana.smithi031 on smithi031 2024-04-24T23:13:58.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:13:57 smithi031 ceph-mon[26411]: pgmap v32: 65 pgs: 65 active+clean; 2.6 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 75 MiB/s rd, 246 MiB/s wr, 508 op/s 2024-04-24T23:13:58.251 INFO:tasks.workunit.client.0.smithi031.stdout:2/684: sync 2024-04-24T23:13:58.254 INFO:tasks.workunit.client.0.smithi031.stdout:0/839: unlink da/d20/d4e/l6f 0 2024-04-24T23:13:58.256 INFO:tasks.workunit.client.1.smithi040.stdout:1/517: mknod da/d12/cb2 0 2024-04-24T23:13:58.257 INFO:tasks.workunit.client.0.smithi031.stdout:8/938: fsync d8/d15/d37/d43/f4b 0 2024-04-24T23:13:58.258 INFO:tasks.workunit.client.1.smithi040.stdout:3/579: getdents d1/d9/d18/d36/d55/d7e 0 2024-04-24T23:13:58.260 INFO:tasks.workunit.client.1.smithi040.stdout:2/567: rename da/d1c/d37/d39/d3f/d52/d78/lc8 to da/d1c/d37/ld1 0 2024-04-24T23:13:58.261 INFO:tasks.workunit.client.0.smithi031.stdout:1/711: symlink d1/da/d12/d65/dcf/d11/d21/d5a/le1 0 2024-04-24T23:13:58.263 INFO:tasks.workunit.client.0.smithi031.stdout:4/909: link d3/f5c d3/d20/d25/d60/d63/f126 0 2024-04-24T23:13:58.264 INFO:tasks.workunit.client.1.smithi040.stdout:8/541: getdents d0/d8/dc/d34/da7 0 2024-04-24T23:13:58.267 INFO:tasks.workunit.client.1.smithi040.stdout:8/542: dread d0/d52/fb3 [0,4194304] 0 2024-04-24T23:13:58.267 INFO:tasks.workunit.client.1.smithi040.stdout:4/455: creat da/d45/d48/fa2 x:0 0 0 2024-04-24T23:13:58.269 INFO:tasks.workunit.client.0.smithi031.stdout:5/752: creat d0/d7/d56/d7c/ff5 x:0 0 0 2024-04-24T23:13:58.272 INFO:tasks.workunit.client.0.smithi031.stdout:2/685: unlink d5/d8/fcd 0 2024-04-24T23:13:58.272 INFO:tasks.workunit.client.1.smithi040.stdout:4/456: dwrite da/d3f/d40/d41/d60/f23 [0,4194304] 0 2024-04-24T23:13:58.273 INFO:tasks.workunit.client.1.smithi040.stdout:5/499: rename d2/d2f/d14/d7f to d2/d7/d8/db/da2 0 2024-04-24T23:13:58.280 INFO:tasks.workunit.client.1.smithi040.stdout:0/596: link df/d12/d1a/d71/c85 df/d1e/d20/d21/d51/d79/daa/dbe/cc9 0 2024-04-24T23:13:58.282 INFO:tasks.workunit.client.0.smithi031.stdout:7/806: mknod d1/d2/c119 0 2024-04-24T23:13:58.284 INFO:tasks.workunit.client.1.smithi040.stdout:8/543: creat d0/d16/d19/fb5 x:0 0 0 2024-04-24T23:13:58.286 INFO:tasks.workunit.client.1.smithi040.stdout:8/544: write d0/d8/d55/f83 [951791,116873] 0 2024-04-24T23:13:58.287 INFO:tasks.workunit.client.0.smithi031.stdout:4/910: symlink d3/d23/d69/d3f/d9e/ddb/l127 0 2024-04-24T23:13:58.288 INFO:tasks.workunit.client.0.smithi031.stdout:4/911: dread - d3/d23/d69/d3f/f10c zero size 2024-04-24T23:13:58.289 INFO:tasks.workunit.client.0.smithi031.stdout:6/843: rmdir d13/d15/d1d/d23/d44/d47/d69/dd5/de4 0 2024-04-24T23:13:58.290 INFO:tasks.workunit.client.1.smithi040.stdout:9/547: dwrite d7/da/d17/d40/d55/d2f/d1f/f5a [4194304,4194304] 0 2024-04-24T23:13:58.290 INFO:tasks.workunit.client.1.smithi040.stdout:9/548: readlink d7/da/d17/d40/d55/lb2 0 2024-04-24T23:13:58.294 INFO:tasks.workunit.client.1.smithi040.stdout:3/580: link d1/d9/d18/d36/d55/d62/f8b d1/d2d/fcd 0 2024-04-24T23:13:58.295 INFO:tasks.workunit.client.0.smithi031.stdout:8/939: write d8/d1a/d1d/d50/fb7 [2649499,94533] 0 2024-04-24T23:13:58.296 INFO:tasks.workunit.client.1.smithi040.stdout:4/457: symlink da/d78/la3 0 2024-04-24T23:13:58.297 INFO:tasks.workunit.client.1.smithi040.stdout:4/458: chown da/d3f/d40/d41/d60/c33 2922 1 2024-04-24T23:13:58.298 INFO:tasks.workunit.client.0.smithi031.stdout:1/712: write d1/d33/f47 [412494,41327] 0 2024-04-24T23:13:58.298 INFO:tasks.workunit.client.1.smithi040.stdout:4/459: chown da/d3f/d6b/c95 487 1 2024-04-24T23:13:58.299 INFO:tasks.workunit.client.1.smithi040.stdout:1/518: truncate da/ff 3800278 0 2024-04-24T23:13:58.302 INFO:tasks.workunit.client.0.smithi031.stdout:2/686: dread d5/d6/d29/d26/db0/dc2/f79 [0,4194304] 0 2024-04-24T23:13:58.307 INFO:tasks.workunit.client.0.smithi031.stdout:4/912: mkdir d3/d20/d25/d95/d128 0 2024-04-24T23:13:58.308 INFO:tasks.workunit.client.0.smithi031.stdout:4/913: write d3/d20/d2d/d44/d5b/d5d/f97 [3912229,44987] 0 2024-04-24T23:13:58.308 INFO:tasks.workunit.client.1.smithi040.stdout:3/581: mknod d1/d2a/d8d/d7c/cce 0 2024-04-24T23:13:58.309 INFO:tasks.workunit.client.1.smithi040.stdout:3/582: write d1/d2a/f9a [1812896,12197] 0 2024-04-24T23:13:58.311 INFO:tasks.workunit.client.1.smithi040.stdout:3/583: fsync d1/d9/d18/d36/d55/f67 0 2024-04-24T23:13:58.312 INFO:tasks.workunit.client.0.smithi031.stdout:0/840: write da/d22/f27 [3444663,14405] 0 2024-04-24T23:13:58.316 INFO:tasks.workunit.client.1.smithi040.stdout:9/549: mkdir d7/d92/d9c/dc2 0 2024-04-24T23:13:58.316 INFO:tasks.workunit.client.1.smithi040.stdout:1/519: dread da/d12/f42 [4194304,4194304] 0 2024-04-24T23:13:58.318 INFO:tasks.workunit.client.1.smithi040.stdout:7/557: rename db/d43/c91 to db/d12/cc2 0 2024-04-24T23:13:58.320 INFO:tasks.workunit.client.0.smithi031.stdout:7/807: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/c11a 0 2024-04-24T23:13:58.321 INFO:tasks.workunit.client.1.smithi040.stdout:5/500: write d2/d7/f36 [776288,108304] 0 2024-04-24T23:13:58.326 INFO:tasks.workunit.client.0.smithi031.stdout:6/844: mknod d13/d15/d1d/d23/d2d/dbf/d10b/c117 0 2024-04-24T23:13:58.326 INFO:tasks.workunit.client.0.smithi031.stdout:2/687: write d5/d28/d43/f97 [728491,85286] 0 2024-04-24T23:13:58.327 INFO:tasks.workunit.client.1.smithi040.stdout:2/568: getdents da/d10/d15 0 2024-04-24T23:13:58.327 INFO:tasks.workunit.client.0.smithi031.stdout:2/688: chown d5/d6/d29/d26/db0/dc2/dad/cd4 12 1 2024-04-24T23:13:58.329 INFO:tasks.workunit.client.0.smithi031.stdout:5/753: creat d0/d7/ff6 x:0 0 0 2024-04-24T23:13:58.330 INFO:tasks.workunit.client.0.smithi031.stdout:2/689: dread d5/d6/d29/f12 [8388608,4194304] 0 2024-04-24T23:13:58.330 INFO:tasks.workunit.client.1.smithi040.stdout:9/550: symlink d7/d92/d97/lc3 0 2024-04-24T23:13:58.331 INFO:tasks.workunit.client.0.smithi031.stdout:0/841: symlink da/d18/d5a/d7d/l119 0 2024-04-24T23:13:58.334 INFO:tasks.workunit.client.0.smithi031.stdout:8/940: truncate d8/d1a/d1d/d35/d58/f64 186623 0 2024-04-24T23:13:58.336 INFO:tasks.workunit.client.0.smithi031.stdout:4/914: dwrite d3/f4a [0,4194304] 0 2024-04-24T23:13:58.337 INFO:tasks.workunit.client.1.smithi040.stdout:7/558: creat db/d2f/d37/d7e/fc3 x:0 0 0 2024-04-24T23:13:58.338 INFO:tasks.workunit.client.1.smithi040.stdout:7/559: write db/d12/d7d/fbb [341576,42751] 0 2024-04-24T23:13:58.344 INFO:tasks.workunit.client.1.smithi040.stdout:7/560: dread db/d12/d3f/fa4 [0,4194304] 0 2024-04-24T23:13:58.345 INFO:tasks.workunit.client.1.smithi040.stdout:1/520: dwrite da/d12/d14/d25/d5e/fa4 [0,4194304] 0 2024-04-24T23:13:58.345 INFO:tasks.workunit.client.1.smithi040.stdout:9/551: symlink d7/da/d17/d40/d55/d2f/d1f/lc4 0 2024-04-24T23:13:58.347 INFO:tasks.workunit.client.0.smithi031.stdout:1/713: creat d1/d3e/da3/fe2 x:0 0 0 2024-04-24T23:13:58.348 INFO:tasks.workunit.client.0.smithi031.stdout:0/842: creat da/d86/da3/dcd/de7/f11a x:0 0 0 2024-04-24T23:13:58.352 INFO:tasks.workunit.client.1.smithi040.stdout:2/569: dread da/d29/f6b [0,4194304] 0 2024-04-24T23:13:58.353 INFO:tasks.workunit.client.1.smithi040.stdout:2/570: write da/d10/f34 [470523,52282] 0 2024-04-24T23:13:58.354 INFO:tasks.workunit.client.0.smithi031.stdout:6/845: getdents d13/d1b/d32/db4/d79/dd7/dde/d107 0 2024-04-24T23:13:58.354 INFO:tasks.workunit.client.1.smithi040.stdout:1/521: chown da/d12/f54 22 1 2024-04-24T23:13:58.357 INFO:tasks.workunit.client.1.smithi040.stdout:9/552: creat d7/da/d17/d40/d55/d2f/d1f/fc5 x:0 0 0 2024-04-24T23:13:58.360 INFO:tasks.workunit.client.1.smithi040.stdout:6/497: rename d7/d1a/d20/d51/d1e/d46/d85/l94 to d7/d1a/d20/l9b 0 2024-04-24T23:13:58.361 INFO:tasks.workunit.client.1.smithi040.stdout:6/498: truncate d7/d1a/f67 1263200 0 2024-04-24T23:13:58.363 INFO:tasks.workunit.client.1.smithi040.stdout:3/584: getdents d1/d2d/d2b 0 2024-04-24T23:13:58.363 INFO:tasks.workunit.client.1.smithi040.stdout:6/499: truncate d7/de/d84/f91 658997 0 2024-04-24T23:13:58.366 INFO:tasks.workunit.client.0.smithi031.stdout:1/714: fdatasync d1/d3e/fbe 0 2024-04-24T23:13:58.370 INFO:tasks.workunit.client.0.smithi031.stdout:1/715: dread d1/d33/d36/fa7 [0,4194304] 0 2024-04-24T23:13:58.370 INFO:tasks.workunit.client.0.smithi031.stdout:8/941: fsync d8/d1a/f7e 0 2024-04-24T23:13:58.371 INFO:tasks.workunit.client.1.smithi040.stdout:3/585: dwrite d1/d9/d2e/fa1 [0,4194304] 0 2024-04-24T23:13:58.372 INFO:tasks.workunit.client.1.smithi040.stdout:3/586: chown d1/d2d/de/d30 8075643 1 2024-04-24T23:13:58.373 INFO:tasks.workunit.client.1.smithi040.stdout:3/587: dread - d1/fb2 zero size 2024-04-24T23:13:58.373 INFO:tasks.workunit.client.0.smithi031.stdout:4/915: getdents d3/d23/d69/dce 0 2024-04-24T23:13:58.373 INFO:tasks.workunit.client.0.smithi031.stdout:9/881: sync 2024-04-24T23:13:58.376 INFO:tasks.workunit.client.0.smithi031.stdout:7/808: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d2c/f11b x:0 0 0 2024-04-24T23:13:58.377 INFO:tasks.workunit.client.1.smithi040.stdout:9/553: dread d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f78 [0,4194304] 0 2024-04-24T23:13:58.377 INFO:tasks.workunit.client.0.smithi031.stdout:1/716: dwrite d1/da/fad [0,4194304] 0 2024-04-24T23:13:58.378 INFO:tasks.workunit.client.1.smithi040.stdout:9/554: stat d7/da/d17/d40/d55/d2f/d1f/d31 0 2024-04-24T23:13:58.380 INFO:tasks.workunit.client.1.smithi040.stdout:8/545: rename d0/d8/d22/l2f to d0/d8/dc/d34/lb6 0 2024-04-24T23:13:58.381 INFO:tasks.workunit.client.1.smithi040.stdout:3/588: dread d1/d2a/f47 [0,4194304] 0 2024-04-24T23:13:58.385 INFO:tasks.workunit.client.1.smithi040.stdout:5/501: getdents d2/d7/d8/d1f/d57/d27/d65 0 2024-04-24T23:13:58.387 INFO:tasks.workunit.client.0.smithi031.stdout:2/690: rmdir d5/d8/d45/d5b/d49/dc4/de5/df1 0 2024-04-24T23:13:58.389 INFO:tasks.workunit.client.0.smithi031.stdout:2/691: write d5/d6/d68/feb [765414,79914] 0 2024-04-24T23:13:58.390 INFO:tasks.workunit.client.1.smithi040.stdout:2/571: mknod da/d1c/cd2 0 2024-04-24T23:13:58.390 INFO:tasks.workunit.client.0.smithi031.stdout:2/692: stat d5/d8/d45/d5b/d49/f7e 0 2024-04-24T23:13:58.392 INFO:tasks.workunit.client.0.smithi031.stdout:0/843: unlink da/f19 0 2024-04-24T23:13:58.397 INFO:tasks.workunit.client.1.smithi040.stdout:4/460: rename da/d3f/d6b/d2b/f5f to da/d3f/d40/d47/fa4 0 2024-04-24T23:13:58.402 INFO:tasks.workunit.client.0.smithi031.stdout:8/942: fdatasync d8/d15/d37/d62/fb3 0 2024-04-24T23:13:58.402 INFO:tasks.workunit.client.0.smithi031.stdout:8/943: dread - d8/d15/d37/d62/dfa/f131 zero size 2024-04-24T23:13:58.403 INFO:tasks.workunit.client.1.smithi040.stdout:4/461: dread - da/d45/d48/d6e/f92 zero size 2024-04-24T23:13:58.403 INFO:tasks.workunit.client.1.smithi040.stdout:8/546: chown d0/d8/f29 1 1 2024-04-24T23:13:58.403 INFO:tasks.workunit.client.1.smithi040.stdout:8/547: dread - d0/d16/d19/d4f/f70 zero size 2024-04-24T23:13:58.403 INFO:tasks.workunit.client.1.smithi040.stdout:4/462: dread da/d3f/d40/d47/d75/f97 [0,4194304] 0 2024-04-24T23:13:58.404 INFO:tasks.workunit.client.1.smithi040.stdout:1/522: dwrite da/d35/dab/f66 [0,4194304] 0 2024-04-24T23:13:58.405 INFO:tasks.workunit.client.1.smithi040.stdout:0/597: sync 2024-04-24T23:13:58.405 INFO:tasks.workunit.client.1.smithi040.stdout:4/463: chown da/d3f/d40/d47/d75 2863305 1 2024-04-24T23:13:58.407 INFO:tasks.workunit.client.1.smithi040.stdout:3/589: rmdir d1/d9/d18/d36/d55 39 2024-04-24T23:13:58.408 INFO:tasks.workunit.client.0.smithi031.stdout:3/733: sync 2024-04-24T23:13:58.408 INFO:tasks.workunit.client.1.smithi040.stdout:6/500: write d7/de/d4e/f64 [186552,48429] 0 2024-04-24T23:13:58.409 INFO:tasks.workunit.client.1.smithi040.stdout:3/590: chown d1/d2a/d6a/f8e 185596 1 2024-04-24T23:13:58.409 INFO:tasks.workunit.client.1.smithi040.stdout:6/501: readlink d7/d1a/d20/l29 0 2024-04-24T23:13:58.411 INFO:tasks.workunit.client.0.smithi031.stdout:7/809: creat d1/d2/dd/d33/d5e/d64/d6d/de1/da0/f11c x:0 0 0 2024-04-24T23:13:58.411 INFO:tasks.workunit.client.0.smithi031.stdout:6/846: dwrite d13/d15/d1d/d23/d24/d63/f96 [0,4194304] 0 2024-04-24T23:13:58.413 INFO:tasks.workunit.client.1.smithi040.stdout:6/502: dread d7/d1a/d20/d51/d1e/f3d [0,4194304] 0 2024-04-24T23:13:58.413 INFO:tasks.workunit.client.1.smithi040.stdout:5/502: symlink d2/d7/d8/d1f/d57/d27/d65/la3 0 2024-04-24T23:13:58.417 INFO:tasks.workunit.client.1.smithi040.stdout:6/503: dread d7/d1a/d20/d51/d2f/f5c [4194304,4194304] 0 2024-04-24T23:13:58.419 INFO:tasks.workunit.client.0.smithi031.stdout:3/734: dwrite d6/fc4 [0,4194304] 0 2024-04-24T23:13:58.428 INFO:tasks.workunit.client.0.smithi031.stdout:5/754: link d0/d6/d32/f39 d0/d6/d3b/d4b/d1d/d2e/d28/d77/ff7 0 2024-04-24T23:13:58.432 INFO:tasks.workunit.client.1.smithi040.stdout:7/561: rename db/d2f/f5b to db/d12/d20/d52/d6c/fc4 0 2024-04-24T23:13:58.433 INFO:tasks.workunit.client.1.smithi040.stdout:7/562: readlink db/d12/d62/d8a/lad 0 2024-04-24T23:13:58.434 INFO:tasks.workunit.client.0.smithi031.stdout:2/693: mkdir d5/d8/d45/d5b/d49/d2f/df8 0 2024-04-24T23:13:58.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/844: dread - da/d11/f7f zero size 2024-04-24T23:13:58.441 INFO:tasks.workunit.client.1.smithi040.stdout:1/523: unlink da/d35/dab/f66 0 2024-04-24T23:13:58.442 INFO:tasks.workunit.client.1.smithi040.stdout:1/524: read da/d12/d18/d2a/d41/f6a [730222,78361] 0 2024-04-24T23:13:58.443 INFO:tasks.workunit.client.1.smithi040.stdout:4/464: chown da/d3f/d40/d47/c5b 3233 1 2024-04-24T23:13:58.447 INFO:tasks.workunit.client.1.smithi040.stdout:0/598: dread df/d1e/da7/f69 [0,4194304] 0 2024-04-24T23:13:58.447 INFO:tasks.workunit.client.1.smithi040.stdout:3/591: dread d1/d9/d2e/f84 [0,4194304] 0 2024-04-24T23:13:58.447 INFO:tasks.workunit.client.0.smithi031.stdout:9/882: symlink d2/dae/d14/d78/d84/d4a/d4b/l10f 0 2024-04-24T23:13:58.450 INFO:tasks.workunit.client.1.smithi040.stdout:5/503: mkdir d2/d7/d8/d1f/d57/d27/d3c/da4 0 2024-04-24T23:13:58.451 INFO:tasks.workunit.client.1.smithi040.stdout:5/504: chown d2/d7/d8/f39 82861 1 2024-04-24T23:13:58.456 INFO:tasks.workunit.client.1.smithi040.stdout:2/572: link da/d10/d7a/f8d da/d1c/d7e/fd3 0 2024-04-24T23:13:58.459 INFO:tasks.workunit.client.1.smithi040.stdout:8/548: write d0/d16/d19/f35 [1378171,61358] 0 2024-04-24T23:13:58.459 INFO:tasks.workunit.client.0.smithi031.stdout:7/810: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/f11d x:0 0 0 2024-04-24T23:13:58.460 INFO:tasks.workunit.client.1.smithi040.stdout:8/549: chown d0 272274 1 2024-04-24T23:13:58.460 INFO:tasks.workunit.client.1.smithi040.stdout:9/555: creat d7/da/d17/d40/d55/fc6 x:0 0 0 2024-04-24T23:13:58.460 INFO:tasks.workunit.client.0.smithi031.stdout:1/717: read - d1/da/d12/d7e/d7b/d4e/f94 zero size 2024-04-24T23:13:58.462 INFO:tasks.workunit.client.0.smithi031.stdout:1/718: fsync d1/da/d12/d65/dcf/d11/d21/f28 0 2024-04-24T23:13:58.464 INFO:tasks.workunit.client.0.smithi031.stdout:3/735: rename d6/df/d16/d53/fa5 to d6/d55/fdb 0 2024-04-24T23:13:58.466 INFO:tasks.workunit.client.1.smithi040.stdout:7/563: creat db/d12/d7d/fc5 x:0 0 0 2024-04-24T23:13:58.466 INFO:tasks.workunit.client.1.smithi040.stdout:9/556: dwrite d7/da/d17/d40/d55/d2f/f7c [0,4194304] 0 2024-04-24T23:13:58.467 INFO:tasks.workunit.client.0.smithi031.stdout:2/694: mknod d5/d6/d68/cf9 0 2024-04-24T23:13:58.467 INFO:tasks.workunit.client.1.smithi040.stdout:7/564: chown db/d12/d73/f9c 1698950 1 2024-04-24T23:13:58.468 INFO:tasks.workunit.client.0.smithi031.stdout:4/916: dwrite d3/d20/d25/d60/d63/f8b [0,4194304] 0 2024-04-24T23:13:58.469 INFO:tasks.workunit.client.0.smithi031.stdout:2/695: chown d5/f58 1994 1 2024-04-24T23:13:58.470 INFO:tasks.workunit.client.1.smithi040.stdout:7/565: dread db/d12/d3f/d7a/f9e [0,4194304] 0 2024-04-24T23:13:58.471 INFO:tasks.workunit.client.0.smithi031.stdout:4/917: fsync d3/d20/d25/d60/ff8 0 2024-04-24T23:13:58.474 INFO:tasks.workunit.client.0.smithi031.stdout:0/845: mkdir da/d20/d4e/dc9/d11b 0 2024-04-24T23:13:58.478 INFO:tasks.workunit.client.0.smithi031.stdout:2/696: truncate d5/d8/d45/d5b/d49/d6f/fb6 1006942 0 2024-04-24T23:13:58.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/736: dwrite d6/df/d16/d2a/d95/d9b/d1a/f62 [0,4194304] 0 2024-04-24T23:13:58.478 INFO:tasks.workunit.client.0.smithi031.stdout:8/944: symlink d8/d15/d37/d43/l139 0 2024-04-24T23:13:58.484 INFO:tasks.workunit.client.0.smithi031.stdout:8/945: read d8/fa8 [4048753,17160] 0 2024-04-24T23:13:58.485 INFO:tasks.workunit.client.0.smithi031.stdout:4/918: dwrite d3/d20/d25/d60/ff8 [4194304,4194304] 0 2024-04-24T23:13:58.487 INFO:tasks.workunit.client.0.smithi031.stdout:6/847: mkdir d13/d1b/de3/d118 0 2024-04-24T23:13:58.489 INFO:tasks.workunit.client.0.smithi031.stdout:6/848: chown d13/d15/d1d/l20 46002679 1 2024-04-24T23:13:58.500 INFO:tasks.workunit.client.0.smithi031.stdout:0/846: dread - da/d18/f85 zero size 2024-04-24T23:13:58.500 INFO:tasks.workunit.client.0.smithi031.stdout:4/919: dwrite d3/d20/d25/f11d [0,4194304] 0 2024-04-24T23:13:58.503 INFO:tasks.workunit.client.0.smithi031.stdout:9/883: mknod d2/c110 0 2024-04-24T23:13:58.505 INFO:tasks.workunit.client.0.smithi031.stdout:8/946: dwrite d8/d15/d37/ffc [0,4194304] 0 2024-04-24T23:13:58.506 INFO:tasks.workunit.client.0.smithi031.stdout:7/811: dwrite d1/d2/f12 [0,4194304] 0 2024-04-24T23:13:58.507 INFO:tasks.workunit.client.0.smithi031.stdout:2/697: read d5/d6/d29/f61 [1849813,98956] 0 2024-04-24T23:13:58.508 INFO:tasks.workunit.client.0.smithi031.stdout:6/849: creat d13/d1b/d32/db4/d79/dd7/f119 x:0 0 0 2024-04-24T23:13:58.509 INFO:tasks.workunit.client.0.smithi031.stdout:3/737: fsync d6/df/d16/d2a/d95/d9b/d1a/d25/f9f 0 2024-04-24T23:13:58.511 INFO:tasks.workunit.client.0.smithi031.stdout:0/847: write da/d11/d6a/dba/fca [754811,123858] 0 2024-04-24T23:13:58.515 INFO:tasks.workunit.client.0.smithi031.stdout:6/850: chown d13/d15/d1d/d23/d44/da0/da6 14 1 2024-04-24T23:13:58.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/812: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/l113 0 2024-04-24T23:13:58.516 INFO:tasks.workunit.client.0.smithi031.stdout:1/719: creat d1/da/fe3 x:0 0 0 2024-04-24T23:13:58.518 INFO:tasks.workunit.client.1.smithi040.stdout:4/465: rmdir da/d78 39 2024-04-24T23:13:58.519 INFO:tasks.workunit.client.0.smithi031.stdout:9/884: unlink d2/d43/d9d/dbd/fda 0 2024-04-24T23:13:58.520 INFO:tasks.workunit.client.0.smithi031.stdout:9/885: stat d2/d16 0 2024-04-24T23:13:58.522 INFO:tasks.workunit.client.1.smithi040.stdout:1/525: rename da/d35/d61 to da/d12/d18/d2a/d32/db3 0 2024-04-24T23:13:58.527 INFO:tasks.workunit.client.1.smithi040.stdout:3/592: mkdir d1/d9/d2e/dcf 0 2024-04-24T23:13:58.527 INFO:tasks.workunit.client.1.smithi040.stdout:1/526: read da/d12/d18/d2a/d41/f6a [771402,62944] 0 2024-04-24T23:13:58.528 INFO:tasks.workunit.client.1.smithi040.stdout:3/593: fdatasync d1/d2a/d6a/fc9 0 2024-04-24T23:13:58.528 INFO:tasks.workunit.client.0.smithi031.stdout:2/698: read d5/f27 [3024656,107317] 0 2024-04-24T23:13:58.529 INFO:tasks.workunit.client.1.smithi040.stdout:0/599: truncate df/d1e/d2b/d6e/fba 11445242 0 2024-04-24T23:13:58.530 INFO:tasks.workunit.client.0.smithi031.stdout:8/947: dread d8/d15/d37/f10b [0,4194304] 0 2024-04-24T23:13:58.531 INFO:tasks.workunit.client.0.smithi031.stdout:0/848: mknod da/d18/d1a/d37/d41/d92/c11c 0 2024-04-24T23:13:58.532 INFO:tasks.workunit.client.0.smithi031.stdout:0/849: truncate da/d20/f44 4728004 0 2024-04-24T23:13:58.533 INFO:tasks.workunit.client.1.smithi040.stdout:5/505: creat d2/d7/d8/d1f/d57/d27/d65/fa5 x:0 0 0 2024-04-24T23:13:58.534 INFO:tasks.workunit.client.0.smithi031.stdout:6/851: truncate d13/d15/d1d/d23/d24/d63/f67 423039 0 2024-04-24T23:13:58.538 INFO:tasks.workunit.client.1.smithi040.stdout:8/550: rename d0/d28/d9c to d0/d28/db7 0 2024-04-24T23:13:58.539 INFO:tasks.workunit.client.0.smithi031.stdout:1/720: fdatasync d1/da/d12/d65/dcf/d11/f24 0 2024-04-24T23:13:58.542 INFO:tasks.workunit.client.0.smithi031.stdout:9/886: mknod d2/dae/d14/d1d/d45/d65/c111 0 2024-04-24T23:13:58.542 INFO:tasks.workunit.client.1.smithi040.stdout:8/551: dread d0/d8/d55/f83 [0,4194304] 0 2024-04-24T23:13:58.543 INFO:tasks.workunit.client.1.smithi040.stdout:9/557: symlink d7/d92/lc7 0 2024-04-24T23:13:58.544 INFO:tasks.workunit.client.0.smithi031.stdout:3/738: fsync d6/df/d16/f1c 0 2024-04-24T23:13:58.548 INFO:tasks.workunit.client.1.smithi040.stdout:4/466: unlink da/d3f/d6b/d3c/d69/c9d 0 2024-04-24T23:13:58.549 INFO:tasks.workunit.client.1.smithi040.stdout:4/467: chown da/d3f/d40/d41/d60 243919114 1 2024-04-24T23:13:58.549 INFO:tasks.workunit.client.0.smithi031.stdout:8/948: creat d8/d15/f13a x:0 0 0 2024-04-24T23:13:58.549 INFO:tasks.workunit.client.1.smithi040.stdout:9/558: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f49 [0,4194304] 0 2024-04-24T23:13:58.550 INFO:tasks.workunit.client.1.smithi040.stdout:3/594: mknod d1/d2d/cd0 0 2024-04-24T23:13:58.550 INFO:tasks.workunit.client.0.smithi031.stdout:2/699: dwrite d5/d99/da2/fd1 [0,4194304] 0 2024-04-24T23:13:58.552 INFO:tasks.workunit.client.1.smithi040.stdout:2/573: dread da/d10/d15/f2c [0,4194304] 0 2024-04-24T23:13:58.554 INFO:tasks.workunit.client.1.smithi040.stdout:1/527: mknod da/d12/d18/d2a/d8b/d97/cb4 0 2024-04-24T23:13:58.555 INFO:tasks.workunit.client.0.smithi031.stdout:6/852: mknod d13/d1b/d32/db4/c11a 0 2024-04-24T23:13:58.555 INFO:tasks.workunit.client.1.smithi040.stdout:1/528: truncate da/d12/d18/d2a/d4d/d7d/fa9 527654 0 2024-04-24T23:13:58.559 INFO:tasks.workunit.client.1.smithi040.stdout:0/600: symlink df/d1e/d89/lca 0 2024-04-24T23:13:58.561 INFO:tasks.workunit.client.1.smithi040.stdout:0/601: fsync df/d1e/d20/d21/d51/f8f 0 2024-04-24T23:13:58.561 INFO:tasks.workunit.client.1.smithi040.stdout:0/602: dread - df/d12/d58/f7d zero size 2024-04-24T23:13:58.568 INFO:tasks.workunit.client.0.smithi031.stdout:9/887: rmdir d2/dae/d14/d78 39 2024-04-24T23:13:58.570 INFO:tasks.workunit.client.0.smithi031.stdout:4/920: write d3/d20/d2d/fbe [933074,130757] 0 2024-04-24T23:13:58.573 INFO:tasks.workunit.client.1.smithi040.stdout:9/559: dread d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [0,4194304] 0 2024-04-24T23:13:58.575 INFO:tasks.workunit.client.1.smithi040.stdout:7/566: mknod db/d43/db7/cc6 0 2024-04-24T23:13:58.576 INFO:tasks.workunit.client.1.smithi040.stdout:7/567: dread - db/d12/d20/d52/d44/fab zero size 2024-04-24T23:13:58.577 INFO:tasks.workunit.client.1.smithi040.stdout:4/468: fdatasync da/d3f/d40/d47/d75/f97 0 2024-04-24T23:13:58.578 INFO:tasks.workunit.client.1.smithi040.stdout:6/504: truncate d7/d1a/d20/d51/d1e/d46/f98 3338636 0 2024-04-24T23:13:58.578 INFO:tasks.workunit.client.1.smithi040.stdout:6/505: chown d7/d1a/d56 25622537 1 2024-04-24T23:13:58.579 INFO:tasks.workunit.client.0.smithi031.stdout:0/850: unlink da/d18/d1a/f1e 0 2024-04-24T23:13:58.580 INFO:tasks.workunit.client.1.smithi040.stdout:3/595: rmdir d1/d2d/d2b 39 2024-04-24T23:13:58.581 INFO:tasks.workunit.client.1.smithi040.stdout:6/506: dread d7/d1a/f53 [0,4194304] 0 2024-04-24T23:13:58.581 INFO:tasks.workunit.client.0.smithi031.stdout:2/700: symlink d5/d8/d45/lfa 0 2024-04-24T23:13:58.583 INFO:tasks.workunit.client.0.smithi031.stdout:6/853: mkdir d13/d15/d1d/d23/d4d/d7b/d11b 0 2024-04-24T23:13:58.585 INFO:tasks.workunit.client.0.smithi031.stdout:7/813: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d2c/f11e x:0 0 0 2024-04-24T23:13:58.586 INFO:tasks.workunit.client.1.smithi040.stdout:6/507: dread d7/d1a/d20/d51/f26 [0,4194304] 0 2024-04-24T23:13:58.588 INFO:tasks.workunit.client.0.smithi031.stdout:7/814: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/fe7 [1250669,100876] 0 2024-04-24T23:13:58.588 INFO:tasks.workunit.client.1.smithi040.stdout:5/506: creat d2/d7/d8/d1f/d57/d27/d3d/d99/fa6 x:0 0 0 2024-04-24T23:13:58.589 INFO:tasks.workunit.client.0.smithi031.stdout:7/815: stat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/fef 0 2024-04-24T23:13:58.590 INFO:tasks.workunit.client.0.smithi031.stdout:9/888: dread - d2/d16/dbe/d7d/fac zero size 2024-04-24T23:13:58.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/816: write d1/d2/dd/d33/d5e/d64/d6d/de1/d84/f9e [3398455,122306] 0 2024-04-24T23:13:58.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/921: rename d3/d20/d2d/d44/c6a to d3/da3/d104/c129 0 2024-04-24T23:13:58.598 INFO:tasks.workunit.client.1.smithi040.stdout:1/529: dread da/ff [0,4194304] 0 2024-04-24T23:13:58.599 INFO:tasks.workunit.client.1.smithi040.stdout:7/568: rmdir db/d12/d20/d52 39 2024-04-24T23:13:58.599 INFO:tasks.workunit.client.1.smithi040.stdout:1/530: write da/d35/d38/f3e [2032565,6140] 0 2024-04-24T23:13:58.601 INFO:tasks.workunit.client.0.smithi031.stdout:2/701: unlink d5/d28/d4e/fac 0 2024-04-24T23:13:58.602 INFO:tasks.workunit.client.0.smithi031.stdout:6/854: truncate d13/d15/d1d/d23/d44/d47/f4a 680234 0 2024-04-24T23:13:58.607 INFO:tasks.workunit.client.0.smithi031.stdout:1/721: creat d1/da/fe4 x:0 0 0 2024-04-24T23:13:58.608 INFO:tasks.workunit.client.1.smithi040.stdout:3/596: creat d1/d2d/de/d30/d37/fd1 x:0 0 0 2024-04-24T23:13:58.614 INFO:tasks.workunit.client.1.smithi040.stdout:2/574: rename da/d1c/d37/d39/d3f/d52/d78/lb6 to da/d1c/d4f/ld4 0 2024-04-24T23:13:58.616 INFO:tasks.workunit.client.1.smithi040.stdout:6/508: creat d7/d1a/d20/d51/d1e/d5d/d52/f9c x:0 0 0 2024-04-24T23:13:58.617 INFO:tasks.workunit.client.0.smithi031.stdout:5/755: sync 2024-04-24T23:13:58.619 INFO:tasks.workunit.client.1.smithi040.stdout:5/507: symlink d2/d7/d8/d1f/d57/d3e/la7 0 2024-04-24T23:13:58.620 INFO:tasks.workunit.client.0.smithi031.stdout:9/889: dread d2/f5 [0,4194304] 0 2024-04-24T23:13:58.623 INFO:tasks.workunit.client.1.smithi040.stdout:0/603: unlink cd 0 2024-04-24T23:13:58.624 INFO:tasks.workunit.client.0.smithi031.stdout:3/739: creat d6/df/d16/d2a/d95/d9b/d1a/fdc x:0 0 0 2024-04-24T23:13:58.626 INFO:tasks.workunit.client.1.smithi040.stdout:9/560: getdents d7/d92/d9c/dc2 0 2024-04-24T23:13:58.627 INFO:tasks.workunit.client.0.smithi031.stdout:2/702: truncate d5/d28/f3a 202180 0 2024-04-24T23:13:58.630 INFO:tasks.workunit.client.0.smithi031.stdout:6/855: creat d13/d15/d1d/d23/d24/d63/f11c x:0 0 0 2024-04-24T23:13:58.631 INFO:tasks.workunit.client.1.smithi040.stdout:1/531: write da/d12/f21 [4534755,92570] 0 2024-04-24T23:13:58.633 INFO:tasks.workunit.client.0.smithi031.stdout:2/703: dwrite d5/d28/d4e/d5e/fb4 [0,4194304] 0 2024-04-24T23:13:58.635 INFO:tasks.workunit.client.0.smithi031.stdout:8/949: symlink d8/d15/d37/d43/d5c/df4/l13b 0 2024-04-24T23:13:58.638 INFO:tasks.workunit.client.0.smithi031.stdout:8/950: chown d8/d15/d37/d43/d7b/d87/dad/fda 8044 1 2024-04-24T23:13:58.640 INFO:tasks.workunit.client.0.smithi031.stdout:4/922: dwrite d3/fbc [0,4194304] 0 2024-04-24T23:13:58.644 INFO:tasks.workunit.client.1.smithi040.stdout:3/597: creat d1/d2a/fd2 x:0 0 0 2024-04-24T23:13:58.647 INFO:tasks.workunit.client.0.smithi031.stdout:9/890: unlink d2/d43/f5d 0 2024-04-24T23:13:58.649 INFO:tasks.workunit.client.1.smithi040.stdout:2/575: mknod da/d1c/d7e/cd5 0 2024-04-24T23:13:58.650 INFO:tasks.workunit.client.0.smithi031.stdout:1/722: dwrite d1/da/d12/d65/fa6 [0,4194304] 0 2024-04-24T23:13:58.650 INFO:tasks.workunit.client.0.smithi031.stdout:9/891: chown d2/d43/d9d/dc0/l108 182305 1 2024-04-24T23:13:58.650 INFO:tasks.workunit.client.0.smithi031.stdout:5/756: write d0/d6/d3b/d4b/d1d/d2e/d28/d77/ff7 [1747301,62553] 0 2024-04-24T23:13:58.651 INFO:tasks.workunit.client.1.smithi040.stdout:4/469: dwrite da/d3f/d40/d41/d60/f82 [0,4194304] 0 2024-04-24T23:13:58.653 INFO:tasks.workunit.client.0.smithi031.stdout:0/851: link da/d18/d1a/f29 da/d86/da3/dcd/de7/ddb/f11d 0 2024-04-24T23:13:58.654 INFO:tasks.workunit.client.0.smithi031.stdout:0/852: fdatasync da/d20/d4e/f51 0 2024-04-24T23:13:58.655 INFO:tasks.workunit.client.1.smithi040.stdout:4/470: write da/d3f/d6b/d2b/f46 [1003429,6484] 0 2024-04-24T23:13:58.655 INFO:tasks.workunit.client.1.smithi040.stdout:0/604: rmdir df/d1e/d2b 39 2024-04-24T23:13:58.657 INFO:tasks.workunit.client.1.smithi040.stdout:0/605: fsync df/d1e/f2f 0 2024-04-24T23:13:58.660 INFO:tasks.workunit.client.0.smithi031.stdout:6/856: truncate d13/d15/d1d/d23/d24/d43/f71 1228008 0 2024-04-24T23:13:58.661 INFO:tasks.workunit.client.1.smithi040.stdout:8/552: getdents d0/d16 0 2024-04-24T23:13:58.665 INFO:tasks.workunit.client.1.smithi040.stdout:1/532: creat da/d12/d18/d2a/d41/fb5 x:0 0 0 2024-04-24T23:13:58.666 INFO:tasks.workunit.client.1.smithi040.stdout:1/533: write da/d12/d18/f78 [4000113,40500] 0 2024-04-24T23:13:58.669 INFO:tasks.workunit.client.1.smithi040.stdout:3/598: mknod d1/d9/d15/d45/cd3 0 2024-04-24T23:13:58.670 INFO:tasks.workunit.client.1.smithi040.stdout:5/508: creat d2/d7/d8/d1f/da1/fa8 x:0 0 0 2024-04-24T23:13:58.671 INFO:tasks.workunit.client.0.smithi031.stdout:0/853: mkdir da/d11/d6a/dba/d11e 0 2024-04-24T23:13:58.673 INFO:tasks.workunit.client.1.smithi040.stdout:4/471: rmdir da/d45/d48/d6e 39 2024-04-24T23:13:58.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/854: dread da/d11/f100 [0,4194304] 0 2024-04-24T23:13:58.676 INFO:tasks.workunit.client.0.smithi031.stdout:7/817: getdents d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2 0 2024-04-24T23:13:58.677 INFO:tasks.workunit.client.1.smithi040.stdout:9/561: rename d7/da/d17/d40/d55/d2f/d1f/d31/l54 to d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/lc8 0 2024-04-24T23:13:58.678 INFO:tasks.workunit.client.0.smithi031.stdout:7/818: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/le3 4866787 1 2024-04-24T23:13:58.678 INFO:tasks.workunit.client.0.smithi031.stdout:8/951: mknod d8/d15/d37/d43/d7b/d87/c13c 0 2024-04-24T23:13:58.680 INFO:tasks.workunit.client.1.smithi040.stdout:8/553: chown d0/d8/dc/d34/c36 68543 1 2024-04-24T23:13:58.682 INFO:tasks.workunit.client.0.smithi031.stdout:5/757: mknod d0/d6/d3b/d4b/dc/d86/dc3/cf8 0 2024-04-24T23:13:58.683 INFO:tasks.workunit.client.1.smithi040.stdout:7/569: symlink db/d12/d20/d52/d6c/d8f/db8/lc7 0 2024-04-24T23:13:58.683 INFO:tasks.workunit.client.0.smithi031.stdout:5/758: truncate d0/d7/d56/d7c/d7e/fb8 5101402 0 2024-04-24T23:13:58.684 INFO:tasks.workunit.client.0.smithi031.stdout:5/759: chown d0/d6/d32/d6b/fa0 111462029 1 2024-04-24T23:13:58.685 INFO:tasks.workunit.client.0.smithi031.stdout:5/760: write d0/d6/d3b/d4f/fd6 [761167,1348] 0 2024-04-24T23:13:58.689 INFO:tasks.workunit.client.1.smithi040.stdout:3/599: dread d1/d9/d18/d36/d55/f89 [0,4194304] 0 2024-04-24T23:13:58.689 INFO:tasks.workunit.client.0.smithi031.stdout:3/740: dread d6/df/d16/d2a/d95/d9b/d1a/f22 [4194304,4194304] 0 2024-04-24T23:13:58.689 INFO:tasks.workunit.client.0.smithi031.stdout:2/704: rename d5/d8/d15/d56/d84/d75/lc8 to d5/d8/d15/d56/lfb 0 2024-04-24T23:13:58.690 INFO:tasks.workunit.client.1.smithi040.stdout:3/600: truncate d1/d2d/fcd 191755 0 2024-04-24T23:13:58.690 INFO:tasks.workunit.client.0.smithi031.stdout:3/741: fsync d6/df/d16/d2a/f31 0 2024-04-24T23:13:58.691 INFO:tasks.workunit.client.0.smithi031.stdout:0/855: creat da/d20/d8e/f11f x:0 0 0 2024-04-24T23:13:58.692 INFO:tasks.workunit.client.0.smithi031.stdout:3/742: readlink d6/df/d16/d2a/d95/d9b/d29/l85 0 2024-04-24T23:13:58.692 INFO:tasks.workunit.client.1.smithi040.stdout:5/509: mkdir d2/d7/d8/d1f/d57/d27/d3c/da9 0 2024-04-24T23:13:58.696 INFO:tasks.workunit.client.1.smithi040.stdout:4/472: symlink da/d3f/d40/d41/la5 0 2024-04-24T23:13:58.697 INFO:tasks.workunit.client.0.smithi031.stdout:2/705: dread d5/fd7 [0,4194304] 0 2024-04-24T23:13:58.697 INFO:tasks.workunit.client.1.smithi040.stdout:4/473: write da/d45/d48/f7d [1227928,112913] 0 2024-04-24T23:13:58.700 INFO:tasks.workunit.client.0.smithi031.stdout:6/857: creat d13/d1b/f11d x:0 0 0 2024-04-24T23:13:58.703 INFO:tasks.workunit.client.1.smithi040.stdout:9/562: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/dc9 0 2024-04-24T23:13:58.705 INFO:tasks.workunit.client.1.smithi040.stdout:8/554: mkdir d0/d8/d22/db8 0 2024-04-24T23:13:58.706 INFO:tasks.workunit.client.1.smithi040.stdout:8/555: fsync d0/d16/d1b/d58/fa5 0 2024-04-24T23:13:58.707 INFO:tasks.workunit.client.1.smithi040.stdout:7/570: creat db/d12/d20/d52/d6c/d8f/db8/fc8 x:0 0 0 2024-04-24T23:13:58.710 INFO:tasks.workunit.client.0.smithi031.stdout:3/743: rmdir d6/df/d16/d57 39 2024-04-24T23:13:58.713 INFO:tasks.workunit.client.1.smithi040.stdout:6/509: sync 2024-04-24T23:13:58.714 INFO:tasks.workunit.client.0.smithi031.stdout:2/706: creat d5/d8/d15/d56/d84/d75/ffc x:0 0 0 2024-04-24T23:13:58.714 INFO:tasks.workunit.client.1.smithi040.stdout:6/510: chown d7/d1a/f42 9107 1 2024-04-24T23:13:58.714 INFO:tasks.workunit.client.1.smithi040.stdout:5/510: mknod d2/d7/d8/db/caa 0 2024-04-24T23:13:58.715 INFO:tasks.workunit.client.1.smithi040.stdout:5/511: readlink d2/d7/d8/d1f/d57/d27/d3c/l5c 0 2024-04-24T23:13:58.715 INFO:tasks.workunit.client.0.smithi031.stdout:6/858: symlink d13/d15/d1d/d23/d2d/dbf/l11e 0 2024-04-24T23:13:58.716 INFO:tasks.workunit.client.0.smithi031.stdout:7/819: dwrite d1/f13 [0,4194304] 0 2024-04-24T23:13:58.718 INFO:tasks.workunit.client.0.smithi031.stdout:5/761: symlink d0/d7/lf9 0 2024-04-24T23:13:58.720 INFO:tasks.workunit.client.0.smithi031.stdout:5/762: write d0/fec [532012,116164] 0 2024-04-24T23:13:58.721 INFO:tasks.workunit.client.1.smithi040.stdout:0/606: dwrite df/d12/d1a/d71/fa2 [0,4194304] 0 2024-04-24T23:13:58.724 INFO:tasks.workunit.client.1.smithi040.stdout:4/474: creat da/d45/d58/fa6 x:0 0 0 2024-04-24T23:13:58.726 INFO:tasks.workunit.client.0.smithi031.stdout:3/744: chown d6/f8 0 1 2024-04-24T23:13:58.728 INFO:tasks.workunit.client.0.smithi031.stdout:6/859: creat d13/d15/d1d/d23/d2d/d9e/f11f x:0 0 0 2024-04-24T23:13:58.729 INFO:tasks.workunit.client.1.smithi040.stdout:8/556: symlink d0/d16/d19/d4f/lb9 0 2024-04-24T23:13:58.729 INFO:tasks.workunit.client.1.smithi040.stdout:8/557: chown d0/d52/l7f 3 1 2024-04-24T23:13:58.731 INFO:tasks.workunit.client.0.smithi031.stdout:2/707: truncate d5/d6/d29/d26/db0/dc2/f44 3371256 0 2024-04-24T23:13:58.733 INFO:tasks.workunit.client.0.smithi031.stdout:2/708: write d5/d8/d15/d65/f8e [430474,44125] 0 2024-04-24T23:13:58.734 INFO:tasks.workunit.client.0.smithi031.stdout:7/820: unlink d1/d2/dd/d33/d5e/d64/faa 0 2024-04-24T23:13:58.735 INFO:tasks.workunit.client.0.smithi031.stdout:2/709: read d5/d8/d45/d5b/d49/d2f/faa [56008,35868] 0 2024-04-24T23:13:58.737 INFO:tasks.workunit.client.0.smithi031.stdout:1/723: rename d1/da/d12/d65/dcf/d11/d21/d5a/le1 to d1/da/d61/le5 0 2024-04-24T23:13:58.739 INFO:tasks.workunit.client.1.smithi040.stdout:2/576: getdents da/d1c/d37/d39/d3f 0 2024-04-24T23:13:58.740 INFO:tasks.workunit.client.0.smithi031.stdout:3/745: dread d6/f7b [0,4194304] 0 2024-04-24T23:13:58.741 INFO:tasks.workunit.client.1.smithi040.stdout:3/601: mknod d1/d9/d15/d45/d96/cd4 0 2024-04-24T23:13:58.743 INFO:tasks.workunit.client.1.smithi040.stdout:6/511: readlink d7/d1a/d56/l95 0 2024-04-24T23:13:58.749 INFO:tasks.workunit.client.1.smithi040.stdout:4/475: dread da/f37 [0,4194304] 0 2024-04-24T23:13:58.749 INFO:tasks.workunit.client.1.smithi040.stdout:1/534: rename da/d12/d18/d2a/c46 to da/cb6 0 2024-04-24T23:13:58.750 INFO:tasks.workunit.client.0.smithi031.stdout:2/710: chown d5/d28/d4e/d5e/l8b 12 1 2024-04-24T23:13:58.751 INFO:tasks.workunit.client.1.smithi040.stdout:9/563: dwrite d7/f57 [0,4194304] 0 2024-04-24T23:13:58.752 INFO:tasks.workunit.client.1.smithi040.stdout:8/558: dread d0/d8/d22/f6a [0,4194304] 0 2024-04-24T23:13:58.758 INFO:tasks.workunit.client.0.smithi031.stdout:8/952: rename d8/d15/d37/d43/dbd to d8/d15/d37/d43/d7b/dbf/d13d 0 2024-04-24T23:13:58.760 INFO:tasks.workunit.client.0.smithi031.stdout:0/856: getdents da/d18/d1a/d37/d41 0 2024-04-24T23:13:58.764 INFO:tasks.workunit.client.1.smithi040.stdout:0/607: mknod df/d1e/d2b/ccb 0 2024-04-24T23:13:58.769 INFO:tasks.workunit.client.1.smithi040.stdout:9/564: stat d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/cab 0 2024-04-24T23:13:58.772 INFO:tasks.workunit.client.0.smithi031.stdout:3/746: link d6/d14/d1d/d59/f5e d6/d55/fdd 0 2024-04-24T23:13:58.773 INFO:tasks.workunit.client.1.smithi040.stdout:7/571: link db/c42 db/d12/d3f/d7a/cc9 0 2024-04-24T23:13:58.774 INFO:tasks.workunit.client.0.smithi031.stdout:3/747: stat d6/df/d16/d2a/d95/d9b/d1a/d25/ld7 0 2024-04-24T23:13:58.774 INFO:tasks.workunit.client.0.smithi031.stdout:7/821: rmdir d1/dbe/d10c 0 2024-04-24T23:13:58.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/711: mknod d5/d8/d45/d5b/d49/d2f/df8/cfd 0 2024-04-24T23:13:58.778 INFO:tasks.workunit.client.1.smithi040.stdout:3/602: link d1/d2a/d6a/fc9 d1/d80/dbf/fd5 0 2024-04-24T23:13:58.778 INFO:tasks.workunit.client.1.smithi040.stdout:3/603: truncate d1/d9/d18/f19 936120 0 2024-04-24T23:13:58.779 INFO:tasks.workunit.client.1.smithi040.stdout:0/608: mknod df/d1e/d70/ccc 0 2024-04-24T23:13:58.780 INFO:tasks.workunit.client.1.smithi040.stdout:4/476: creat da/d78/fa7 x:0 0 0 2024-04-24T23:13:58.783 INFO:tasks.workunit.client.0.smithi031.stdout:3/748: readlink d6/df/d16/d57/ld4 0 2024-04-24T23:13:58.784 INFO:tasks.workunit.client.1.smithi040.stdout:9/565: creat d7/da/d17/d40/d55/d2f/fca x:0 0 0 2024-04-24T23:13:58.785 INFO:tasks.workunit.client.1.smithi040.stdout:8/559: getdents d0/d38/d4c/d5d 0 2024-04-24T23:13:58.787 INFO:tasks.workunit.client.0.smithi031.stdout:2/712: creat d5/d8/d45/d5b/d49/dc4/ffe x:0 0 0 2024-04-24T23:13:58.789 INFO:tasks.workunit.client.1.smithi040.stdout:7/572: rename db/f71 to db/d12/d3f/d7a/fca 0 2024-04-24T23:13:58.790 INFO:tasks.workunit.client.0.smithi031.stdout:1/724: link d1/da/d12/c7f d1/d3e/d46/ce6 0 2024-04-24T23:13:58.791 INFO:tasks.workunit.client.1.smithi040.stdout:3/604: write d1/d9/d2e/fa8 [3388753,51572] 0 2024-04-24T23:13:58.792 INFO:tasks.workunit.client.1.smithi040.stdout:4/477: stat da/l17 0 2024-04-24T23:13:58.793 INFO:tasks.workunit.client.1.smithi040.stdout:4/478: chown da/d1f/c80 7 1 2024-04-24T23:13:58.794 INFO:tasks.workunit.client.1.smithi040.stdout:9/566: creat d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/fcb x:0 0 0 2024-04-24T23:13:58.798 INFO:tasks.workunit.client.1.smithi040.stdout:8/560: unlink d0/d16/d1b/d58/f68 0 2024-04-24T23:13:58.799 INFO:tasks.workunit.client.0.smithi031.stdout:2/713: mknod d5/d6/d29/d26/db0/dc2/dad/cff 0 2024-04-24T23:13:58.799 INFO:tasks.workunit.client.0.smithi031.stdout:2/714: dread - d5/f87 zero size 2024-04-24T23:13:58.801 INFO:tasks.workunit.client.0.smithi031.stdout:5/763: rename d0/d6/d3b/d4b/dc/l64 to d0/d7/d56/lfa 0 2024-04-24T23:13:58.803 INFO:tasks.workunit.client.0.smithi031.stdout:5/764: read d0/d6/d3b/d4b/dc/f47 [894671,102774] 0 2024-04-24T23:13:58.804 INFO:tasks.workunit.client.1.smithi040.stdout:7/573: rename db/d2f/l54 to db/d12/d20/d52/d4b/lcb 0 2024-04-24T23:13:58.807 INFO:tasks.workunit.client.0.smithi031.stdout:2/715: mknod d5/d6/d29/c100 0 2024-04-24T23:13:58.808 INFO:tasks.workunit.client.1.smithi040.stdout:2/577: write da/d10/d7a/f99 [425407,22498] 0 2024-04-24T23:13:58.808 INFO:tasks.workunit.client.0.smithi031.stdout:2/716: fsync d5/d28/f37 0 2024-04-24T23:13:58.808 INFO:tasks.workunit.client.1.smithi040.stdout:9/567: mknod d7/da/d17/d40/d55/d2f/d28/ccc 0 2024-04-24T23:13:58.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/717: chown d5/d6/cea 0 1 2024-04-24T23:13:58.812 INFO:tasks.workunit.client.0.smithi031.stdout:6/860: rename d13/d15/d1d/d23/d4d/l52 to d13/d15/d1d/d23/d4d/d7b/d11b/l120 0 2024-04-24T23:13:58.814 INFO:tasks.workunit.client.1.smithi040.stdout:8/561: creat d0/d16/d19/d4f/d9e/d90/d91/fba x:0 0 0 2024-04-24T23:13:58.814 INFO:tasks.workunit.client.1.smithi040.stdout:1/535: write da/d12/d18/d2a/f79 [2225906,53330] 0 2024-04-24T23:13:58.814 INFO:tasks.workunit.client.1.smithi040.stdout:8/562: write d0/f82 [3823088,35638] 0 2024-04-24T23:13:58.816 INFO:tasks.workunit.client.1.smithi040.stdout:3/605: rename d1/f32 to d1/d9/fd6 0 2024-04-24T23:13:58.817 INFO:tasks.workunit.client.1.smithi040.stdout:7/574: mknod db/d12/d3f/d7a/ccc 0 2024-04-24T23:13:58.822 INFO:tasks.workunit.client.0.smithi031.stdout:8/953: rename d8/d15/d37/d62/fbb to d8/d15/d37/d43/d7b/d87/dad/f13e 0 2024-04-24T23:13:58.823 INFO:tasks.workunit.client.1.smithi040.stdout:4/479: rename da/d3f/f56 to da/d45/d48/d6e/d70/fa8 0 2024-04-24T23:13:58.824 INFO:tasks.workunit.client.0.smithi031.stdout:6/861: creat d13/d1b/de3/f121 x:0 0 0 2024-04-24T23:13:58.826 INFO:tasks.workunit.client.1.smithi040.stdout:3/606: unlink d1/d2d/de/c40 0 2024-04-24T23:13:58.826 INFO:tasks.workunit.client.1.smithi040.stdout:7/575: creat db/d12/d7d/fcd x:0 0 0 2024-04-24T23:13:58.827 INFO:tasks.workunit.client.1.smithi040.stdout:7/576: write db/d12/d62/fbd [755168,115814] 0 2024-04-24T23:13:58.827 INFO:tasks.workunit.client.0.smithi031.stdout:1/725: rename d1/da/d10/l9a to d1/da/d12/d7e/le7 0 2024-04-24T23:13:58.831 INFO:tasks.workunit.client.0.smithi031.stdout:1/726: mkdir d1/d33/d36/de8 0 2024-04-24T23:13:58.832 INFO:tasks.workunit.client.1.smithi040.stdout:3/607: dwrite d1/d9/d2e/f5f [4194304,4194304] 0 2024-04-24T23:13:58.833 INFO:tasks.workunit.client.0.smithi031.stdout:8/954: mknod d8/d15/d37/d43/d7b/d87/c13f 0 2024-04-24T23:13:58.834 INFO:tasks.workunit.client.1.smithi040.stdout:8/563: rename d0/d16/d19/d71/lb4 to d0/d16/lbb 0 2024-04-24T23:13:58.836 INFO:tasks.workunit.client.1.smithi040.stdout:2/578: link da/d1c/d37/d39/d3f/f67 da/d29/d55/da7/fd6 0 2024-04-24T23:13:58.836 INFO:tasks.workunit.client.1.smithi040.stdout:9/568: link d7/da/d17/d40/d55/d2f/l20 d7/d5e/da1/lcd 0 2024-04-24T23:13:58.837 INFO:tasks.workunit.client.0.smithi031.stdout:1/727: mknod d1/da/dbc/ce9 0 2024-04-24T23:13:58.837 INFO:tasks.workunit.client.1.smithi040.stdout:9/569: chown d7/d5e/da1/ca2 4697 1 2024-04-24T23:13:58.838 INFO:tasks.workunit.client.1.smithi040.stdout:1/536: link da/d12/d23/f45 da/d12/d18/d2a/d8b/d97/fb7 0 2024-04-24T23:13:58.838 INFO:tasks.workunit.client.1.smithi040.stdout:5/512: sync 2024-04-24T23:13:58.840 INFO:tasks.workunit.client.0.smithi031.stdout:1/728: mknod d1/da/d61/d90/cea 0 2024-04-24T23:13:58.890 INFO:tasks.workunit.client.1.smithi040.stdout:6/512: sync 2024-04-24T23:13:58.897 INFO:tasks.workunit.client.0.smithi031.stdout:8/955: creat d8/f140 x:0 0 0 2024-04-24T23:13:58.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/729: mkdir d1/da/d12/d65/dcf/d11/deb 0 2024-04-24T23:13:58.904 INFO:tasks.workunit.client.1.smithi040.stdout:7/577: rename c7 to db/d12/d3f/cce 0 2024-04-24T23:13:58.910 INFO:tasks.workunit.client.0.smithi031.stdout:0/857: write da/d18/d1a/d37/f5f [2908419,40799] 0 2024-04-24T23:13:58.913 INFO:tasks.workunit.client.1.smithi040.stdout:1/537: creat da/d12/d23/fb8 x:0 0 0 2024-04-24T23:13:58.914 INFO:tasks.workunit.client.1.smithi040.stdout:1/538: chown da/d12/d14/d25/c7a 168961126 1 2024-04-24T23:13:58.918 INFO:tasks.workunit.client.0.smithi031.stdout:0/858: rename da/d20/d4e/f57 to da/d86/da3/dcd/ddc/f120 0 2024-04-24T23:13:58.920 INFO:tasks.workunit.client.0.smithi031.stdout:3/749: write f0 [7290102,1245] 0 2024-04-24T23:13:58.922 INFO:tasks.workunit.client.1.smithi040.stdout:7/578: mknod db/d12/d20/d52/d6c/d8f/db8/ccf 0 2024-04-24T23:13:58.924 INFO:tasks.workunit.client.1.smithi040.stdout:2/579: mkdir da/d10/d4d/dd7 0 2024-04-24T23:13:58.925 INFO:tasks.workunit.client.0.smithi031.stdout:0/859: creat da/d11/d6a/dba/dc0/f121 x:0 0 0 2024-04-24T23:13:58.927 INFO:tasks.workunit.client.1.smithi040.stdout:5/513: dread d2/d7/d8/d1f/d57/f44 [4194304,4194304] 0 2024-04-24T23:13:58.930 INFO:tasks.workunit.client.1.smithi040.stdout:0/609: sync 2024-04-24T23:13:58.931 INFO:tasks.workunit.client.1.smithi040.stdout:6/513: rename d7/d1a/d20/c92 to d7/d1a/d20/d51/d1e/d46/d69/d87/c9d 0 2024-04-24T23:13:58.932 INFO:tasks.workunit.client.1.smithi040.stdout:7/579: unlink db/l1a 0 2024-04-24T23:13:58.934 INFO:tasks.workunit.client.1.smithi040.stdout:9/570: link d7/da/l69 d7/da/d17/d40/d55/d2f/d1f/d4e/lce 0 2024-04-24T23:13:58.941 INFO:tasks.workunit.client.1.smithi040.stdout:7/580: unlink db/d12/d3f/lb4 0 2024-04-24T23:13:58.942 INFO:tasks.workunit.client.1.smithi040.stdout:2/580: mknod da/d1c/d37/d39/cd8 0 2024-04-24T23:13:58.943 INFO:tasks.workunit.client.1.smithi040.stdout:2/581: read da/d10/d15/f30 [3520941,11546] 0 2024-04-24T23:13:58.946 INFO:tasks.workunit.client.1.smithi040.stdout:4/480: write da/d45/d48/d6e/d70/f72 [512642,128276] 0 2024-04-24T23:13:58.950 INFO:tasks.workunit.client.1.smithi040.stdout:5/514: rmdir d2/d7/d8/d1f/d57/d3e/d48/d9f 0 2024-04-24T23:13:58.950 INFO:tasks.workunit.client.1.smithi040.stdout:5/515: readlink d2/d7/d8/db/d1e/l60 0 2024-04-24T23:13:58.952 INFO:tasks.workunit.client.1.smithi040.stdout:5/516: readlink d2/d7/d8/d1f/d57/d27/l4c 0 2024-04-24T23:13:58.952 INFO:tasks.workunit.client.1.smithi040.stdout:8/564: chown d0/d16/d1b/d58/d8f 354 1 2024-04-24T23:13:58.952 INFO:tasks.workunit.client.1.smithi040.stdout:6/514: creat d7/de/f9e x:0 0 0 2024-04-24T23:13:58.953 INFO:tasks.workunit.client.1.smithi040.stdout:4/481: creat da/d3f/d40/d47/d71/fa9 x:0 0 0 2024-04-24T23:13:58.955 INFO:tasks.workunit.client.1.smithi040.stdout:7/581: rename db/f18 to db/d12/d20/d52/d6c/d8f/fd0 0 2024-04-24T23:13:58.955 INFO:tasks.workunit.client.1.smithi040.stdout:3/608: dwrite d1/d9/d2e/f84 [0,4194304] 0 2024-04-24T23:13:58.957 INFO:tasks.workunit.client.1.smithi040.stdout:2/582: link da/d10/d4d/d6c/f9c da/d10/d4d/da4/fd9 0 2024-04-24T23:13:58.960 INFO:tasks.workunit.client.1.smithi040.stdout:8/565: creat d0/d28/fbc x:0 0 0 2024-04-24T23:13:58.962 INFO:tasks.workunit.client.1.smithi040.stdout:6/515: mkdir d7/d1a/d9f 0 2024-04-24T23:13:58.965 INFO:tasks.workunit.client.1.smithi040.stdout:4/482: chown da/ce 0 1 2024-04-24T23:13:58.965 INFO:tasks.workunit.client.1.smithi040.stdout:7/582: mknod db/d12/d3f/cd1 0 2024-04-24T23:13:58.965 INFO:tasks.workunit.client.1.smithi040.stdout:2/583: dwrite da/d10/d15/daa/fc3 [0,4194304] 0 2024-04-24T23:13:58.966 INFO:tasks.workunit.client.1.smithi040.stdout:4/483: dread - da/d3f/d40/d41/f4e zero size 2024-04-24T23:13:58.967 INFO:tasks.workunit.client.1.smithi040.stdout:7/583: fsync db/d12/d20/d52/d6c/fae 0 2024-04-24T23:13:58.968 INFO:tasks.workunit.client.1.smithi040.stdout:5/517: mkdir d2/d7/dab 0 2024-04-24T23:13:58.974 INFO:tasks.workunit.client.1.smithi040.stdout:2/584: symlink da/d29/d55/lda 0 2024-04-24T23:13:58.975 INFO:tasks.workunit.client.1.smithi040.stdout:2/585: chown da/d10 49823 1 2024-04-24T23:13:58.978 INFO:tasks.workunit.client.1.smithi040.stdout:1/539: dwrite da/d12/d23/f30 [0,4194304] 0 2024-04-24T23:13:58.980 INFO:tasks.workunit.client.1.smithi040.stdout:2/586: dread da/d1c/d37/d39/d3f/f7b [0,4194304] 0 2024-04-24T23:13:58.981 INFO:tasks.workunit.client.1.smithi040.stdout:9/571: write f2 [2388728,87320] 0 2024-04-24T23:13:58.983 INFO:tasks.workunit.client.1.smithi040.stdout:0/610: dwrite df/d1e/da7/f65 [0,4194304] 0 2024-04-24T23:13:58.984 INFO:tasks.workunit.client.1.smithi040.stdout:9/572: write d7/f37 [369892,114986] 0 2024-04-24T23:13:58.986 INFO:tasks.workunit.client.1.smithi040.stdout:0/611: fdatasync df/d1e/d2b/db6/fbd 0 2024-04-24T23:13:58.986 INFO:tasks.workunit.client.1.smithi040.stdout:7/584: mkdir db/d12/d7d/dd2 0 2024-04-24T23:13:58.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/765: dwrite d0/d6/d32/d6b/f7b [0,4194304] 0 2024-04-24T23:13:58.986 INFO:tasks.workunit.client.0.smithi031.stdout:6/862: dwrite d13/d15/d1d/d23/d4d/ff4 [0,4194304] 0 2024-04-24T23:13:58.988 INFO:tasks.workunit.client.0.smithi031.stdout:7/822: dwrite d1/d2/dd/d33/f4e [0,4194304] 0 2024-04-24T23:13:58.991 INFO:tasks.workunit.client.0.smithi031.stdout:1/730: dwrite d1/da/d61/d80/f84 [0,4194304] 0 2024-04-24T23:13:58.994 INFO:tasks.workunit.client.0.smithi031.stdout:1/731: chown d1/d3e/da2/fb0 191 1 2024-04-24T23:13:58.995 INFO:tasks.workunit.client.0.smithi031.stdout:3/750: dwrite d6/f8 [0,4194304] 0 2024-04-24T23:13:59.000 INFO:tasks.workunit.client.1.smithi040.stdout:5/518: creat d2/d7/d8/db/da2/fac x:0 0 0 2024-04-24T23:13:59.002 INFO:tasks.workunit.client.0.smithi031.stdout:6/863: mkdir d13/d1b/d32/db4/d122 0 2024-04-24T23:13:59.004 INFO:tasks.workunit.client.0.smithi031.stdout:6/864: dread - d13/d15/d1d/d23/d24/d8b/ff3 zero size 2024-04-24T23:13:59.006 INFO:tasks.workunit.client.0.smithi031.stdout:6/865: write d13/d15/d1d/d23/d2d/dbf/fec [1362603,87839] 0 2024-04-24T23:13:59.008 INFO:tasks.workunit.client.1.smithi040.stdout:1/540: mkdir da/d12/d14/d25/db9 0 2024-04-24T23:13:59.014 INFO:tasks.workunit.client.1.smithi040.stdout:4/484: truncate da/f59 313098 0 2024-04-24T23:13:59.015 INFO:tasks.workunit.client.1.smithi040.stdout:4/485: write da/d3f/d6b/d2b/f55 [2384247,6906] 0 2024-04-24T23:13:59.016 INFO:tasks.workunit.client.1.smithi040.stdout:8/566: write d0/d8/d22/d81/f74 [1682933,119665] 0 2024-04-24T23:13:59.022 INFO:tasks.workunit.client.0.smithi031.stdout:7/823: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/c11f 0 2024-04-24T23:13:59.022 INFO:tasks.workunit.client.1.smithi040.stdout:4/486: dwrite da/d3f/d6b/d3c/f9f [0,4194304] 0 2024-04-24T23:13:59.023 INFO:tasks.workunit.client.1.smithi040.stdout:6/516: creat d7/d1a/d20/d51/d1e/d46/fa0 x:0 0 0 2024-04-24T23:13:59.023 INFO:tasks.workunit.client.0.smithi031.stdout:7/824: stat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/c11a 0 2024-04-24T23:13:59.024 INFO:tasks.workunit.client.1.smithi040.stdout:6/517: readlink d7/d1a/d20/d51/d1e/l41 0 2024-04-24T23:13:59.024 INFO:tasks.workunit.client.1.smithi040.stdout:5/519: write d2/d7/d8/f34 [5644919,35030] 0 2024-04-24T23:13:59.026 INFO:tasks.workunit.client.1.smithi040.stdout:1/541: write da/d12/d55/f6b [5042539,116000] 0 2024-04-24T23:13:59.027 INFO:tasks.workunit.client.1.smithi040.stdout:2/587: creat da/d10/d4d/dd7/fdb x:0 0 0 2024-04-24T23:13:59.029 INFO:tasks.workunit.client.1.smithi040.stdout:1/542: fdatasync da/d12/d18/d2a/d41/f62 0 2024-04-24T23:13:59.032 INFO:tasks.workunit.client.0.smithi031.stdout:0/860: truncate da/d11/f90 4572111 0 2024-04-24T23:13:59.033 INFO:tasks.workunit.client.0.smithi031.stdout:0/861: chown da/d86/c95 4747106 1 2024-04-24T23:13:59.035 INFO:tasks.workunit.client.0.smithi031.stdout:0/862: read da/d20/f52 [24332,128702] 0 2024-04-24T23:13:59.041 INFO:tasks.workunit.client.1.smithi040.stdout:4/487: mkdir da/d3f/d40/d47/d75/daa 0 2024-04-24T23:13:59.043 INFO:tasks.workunit.client.0.smithi031.stdout:1/732: dwrite d1/d3e/da2/fb2 [0,4194304] 0 2024-04-24T23:13:59.045 INFO:tasks.workunit.client.0.smithi031.stdout:5/766: write d0/d6/d32/f17 [549552,66629] 0 2024-04-24T23:13:59.046 INFO:tasks.workunit.client.1.smithi040.stdout:6/518: mknod d7/d1a/d20/d51/d1e/d5d/ca1 0 2024-04-24T23:13:59.047 INFO:tasks.workunit.client.1.smithi040.stdout:6/519: chown d7/d1a/d20/d51/d2a/c74 921 1 2024-04-24T23:13:59.048 INFO:tasks.workunit.client.1.smithi040.stdout:3/609: sync 2024-04-24T23:13:59.052 INFO:tasks.workunit.client.1.smithi040.stdout:5/520: truncate d2/d7/d8/db/d1e/f2e 862504 0 2024-04-24T23:13:59.053 INFO:tasks.workunit.client.1.smithi040.stdout:1/543: readlink da/d12/d18/d2a/d41/d4f/l82 0 2024-04-24T23:13:59.056 INFO:tasks.workunit.client.1.smithi040.stdout:7/585: dwrite db/d2f/d37/fa6 [0,4194304] 0 2024-04-24T23:13:59.057 INFO:tasks.workunit.client.1.smithi040.stdout:5/521: truncate d2/d7/d8/d1f/d57/d27/d3d/d99/fa6 234133 0 2024-04-24T23:13:59.059 INFO:tasks.workunit.client.1.smithi040.stdout:9/573: rename d7/da/d17/d40/d55/d2f/c9a to d7/da/d17/d40/d55/ccf 0 2024-04-24T23:13:59.060 INFO:tasks.workunit.client.1.smithi040.stdout:9/574: fdatasync d7/da/d17/d40/d55/d2f/d28/f88 0 2024-04-24T23:13:59.062 INFO:tasks.workunit.client.0.smithi031.stdout:4/923: sync 2024-04-24T23:13:59.064 INFO:tasks.workunit.client.1.smithi040.stdout:3/610: rmdir d1/d2d 39 2024-04-24T23:13:59.067 INFO:tasks.workunit.client.1.smithi040.stdout:1/544: truncate da/ff 1964093 0 2024-04-24T23:13:59.069 INFO:tasks.workunit.client.1.smithi040.stdout:0/612: getdents df/d1e/da7 0 2024-04-24T23:13:59.070 INFO:tasks.workunit.client.1.smithi040.stdout:7/586: chown db/d12/d20/d52/f23 55 1 2024-04-24T23:13:59.071 INFO:tasks.workunit.client.1.smithi040.stdout:5/522: symlink d2/d7/d8/d1f/da1/lad 0 2024-04-24T23:13:59.074 INFO:tasks.workunit.client.1.smithi040.stdout:8/567: rename d0/l2 to d0/d16/d19/lbd 0 2024-04-24T23:13:59.076 INFO:tasks.workunit.client.1.smithi040.stdout:8/568: write d0/f82 [3391673,44762] 0 2024-04-24T23:13:59.078 INFO:tasks.workunit.client.1.smithi040.stdout:9/575: creat d7/d5e/da1/fd0 x:0 0 0 2024-04-24T23:13:59.078 INFO:tasks.workunit.client.1.smithi040.stdout:7/587: dwrite db/d2f/f6d [0,4194304] 0 2024-04-24T23:13:59.082 INFO:tasks.workunit.client.1.smithi040.stdout:2/588: dread da/d29/f7f [0,4194304] 0 2024-04-24T23:13:59.085 INFO:tasks.workunit.client.1.smithi040.stdout:4/488: dwrite da/f18 [4194304,4194304] 0 2024-04-24T23:13:59.086 INFO:tasks.workunit.client.1.smithi040.stdout:1/545: symlink da/d35/d38/d47/lba 0 2024-04-24T23:13:59.094 INFO:tasks.workunit.client.1.smithi040.stdout:6/520: link d7/d1a/d56/l59 d7/de/d4e/la2 0 2024-04-24T23:13:59.099 INFO:tasks.workunit.client.1.smithi040.stdout:8/569: unlink d0/d38/d4c/l6b 0 2024-04-24T23:13:59.099 INFO:tasks.workunit.client.1.smithi040.stdout:9/576: chown d7/d5e/da1/lcd 895509 1 2024-04-24T23:13:59.100 INFO:tasks.workunit.client.1.smithi040.stdout:9/577: dread - d7/da/d17/d40/d55/d61/d6c/f9b zero size 2024-04-24T23:13:59.103 INFO:tasks.workunit.client.1.smithi040.stdout:3/611: write d1/d9/f17 [1130588,6683] 0 2024-04-24T23:13:59.104 INFO:tasks.workunit.client.1.smithi040.stdout:3/612: fdatasync d1/d9/d18/d36/d55/d7e/fc7 0 2024-04-24T23:13:59.107 INFO:tasks.workunit.client.1.smithi040.stdout:4/489: mknod da/d3f/d6b/d3c/d69/cab 0 2024-04-24T23:13:59.108 INFO:tasks.workunit.client.1.smithi040.stdout:5/523: write d2/d2f/d14/f61 [1382892,34373] 0 2024-04-24T23:13:59.112 INFO:tasks.workunit.client.1.smithi040.stdout:1/546: dread da/d12/d14/f3f [0,4194304] 0 2024-04-24T23:13:59.114 INFO:tasks.workunit.client.1.smithi040.stdout:4/490: dwrite da/d45/d48/d6e/d70/f72 [0,4194304] 0 2024-04-24T23:13:59.121 INFO:tasks.workunit.client.1.smithi040.stdout:9/578: mknod d7/da/d17/d40/d55/d2f/d28/cd1 0 2024-04-24T23:13:59.124 INFO:tasks.workunit.client.1.smithi040.stdout:3/613: mknod d1/d2a/d6a/d94/cd7 0 2024-04-24T23:13:59.127 INFO:tasks.workunit.client.0.smithi031.stdout:3/751: rmdir d6/d14/d71/dc9 0 2024-04-24T23:13:59.128 INFO:tasks.workunit.client.1.smithi040.stdout:1/547: dread da/d12/d18/d2a/f5a [0,4194304] 0 2024-04-24T23:13:59.130 INFO:tasks.workunit.client.0.smithi031.stdout:6/866: creat d13/d15/d1d/f123 x:0 0 0 2024-04-24T23:13:59.132 INFO:tasks.workunit.client.0.smithi031.stdout:7/825: rename d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/lb0 to d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/l120 0 2024-04-24T23:13:59.134 INFO:tasks.workunit.client.1.smithi040.stdout:5/524: truncate d2/d7/d8/d1f/d57/d27/d3d/f7a 1256432 0 2024-04-24T23:13:59.137 INFO:tasks.workunit.client.1.smithi040.stdout:4/491: creat da/d45/fac x:0 0 0 2024-04-24T23:13:59.138 INFO:tasks.workunit.client.0.smithi031.stdout:1/733: creat d1/d33/d36/d81/fec x:0 0 0 2024-04-24T23:13:59.139 INFO:tasks.workunit.client.1.smithi040.stdout:2/589: truncate da/d1c/d37/d8a/f9a 1327098 0 2024-04-24T23:13:59.143 INFO:tasks.workunit.client.1.smithi040.stdout:9/579: unlink d7/da/d17/d40/d55/d2f/d1f/d31/c38 0 2024-04-24T23:13:59.145 INFO:tasks.workunit.client.0.smithi031.stdout:4/924: rename d3/da3/d104 to d3/d23/d37/d86/d113/d12a 0 2024-04-24T23:13:59.148 INFO:tasks.workunit.client.1.smithi040.stdout:1/548: mknod da/d12/d18/d2a/d41/d8c/cbb 0 2024-04-24T23:13:59.148 INFO:tasks.workunit.client.0.smithi031.stdout:6/867: chown d13/d15/d1d/d23/d44/d47/c56 4914 1 2024-04-24T23:13:59.149 INFO:tasks.workunit.client.1.smithi040.stdout:9/580: dwrite d7/da/d17/d40/d55/d2f/fca [0,4194304] 0 2024-04-24T23:13:59.150 INFO:tasks.workunit.client.1.smithi040.stdout:1/549: chown da/d12/d18/d2a/d8b/d97/cb4 355 1 2024-04-24T23:13:59.150 INFO:tasks.workunit.client.1.smithi040.stdout:5/525: mkdir d2/dae 0 2024-04-24T23:13:59.151 INFO:tasks.workunit.client.0.smithi031.stdout:9/892: sync 2024-04-24T23:13:59.152 INFO:tasks.workunit.client.0.smithi031.stdout:1/734: unlink d1/da/d10/f4d 0 2024-04-24T23:13:59.156 INFO:tasks.workunit.client.1.smithi040.stdout:0/613: sync 2024-04-24T23:13:59.157 INFO:tasks.workunit.client.1.smithi040.stdout:0/614: write df/d1e/d2b/db6/fbd [763147,88493] 0 2024-04-24T23:13:59.159 INFO:tasks.workunit.client.1.smithi040.stdout:2/590: mkdir da/d1c/d37/d39/d3f/d52/d78/dc7/ddc 0 2024-04-24T23:13:59.160 INFO:tasks.workunit.client.0.smithi031.stdout:1/735: dwrite d1/da/d12/d65/fa6 [4194304,4194304] 0 2024-04-24T23:13:59.161 INFO:tasks.workunit.client.0.smithi031.stdout:7/826: mkdir d1/d121 0 2024-04-24T23:13:59.163 INFO:tasks.workunit.client.0.smithi031.stdout:1/736: dread d1/da/d10/d52/fdf [0,4194304] 0 2024-04-24T23:13:59.164 INFO:tasks.workunit.client.1.smithi040.stdout:7/588: link db/d12/d3f/cce db/d2f/d37/cd3 0 2024-04-24T23:13:59.164 INFO:tasks.workunit.client.1.smithi040.stdout:7/589: fsync db/f2e 0 2024-04-24T23:13:59.171 INFO:tasks.workunit.client.0.smithi031.stdout:0/863: creat da/d22/d62/f122 x:0 0 0 2024-04-24T23:13:59.173 INFO:tasks.workunit.client.0.smithi031.stdout:1/737: dwrite d1/d33/d41/d9b/da8/fd1 [0,4194304] 0 2024-04-24T23:13:59.175 INFO:tasks.workunit.client.1.smithi040.stdout:3/614: mknod d1/d2d/de/cd8 0 2024-04-24T23:13:59.176 INFO:tasks.workunit.client.1.smithi040.stdout:9/581: dread d7/da/d17/d40/d55/d2f/d28/f88 [0,4194304] 0 2024-04-24T23:13:59.178 INFO:tasks.workunit.client.1.smithi040.stdout:8/570: truncate d0/d16/f57 153632 0 2024-04-24T23:13:59.179 INFO:tasks.workunit.client.1.smithi040.stdout:6/521: getdents d7 0 2024-04-24T23:13:59.180 INFO:tasks.workunit.client.1.smithi040.stdout:6/522: stat d7/d1a/l83 0 2024-04-24T23:13:59.180 INFO:tasks.workunit.client.0.smithi031.stdout:4/925: write d3/d20/d25/fc3 [1820871,32730] 0 2024-04-24T23:13:59.182 INFO:tasks.workunit.client.1.smithi040.stdout:1/550: write da/d35/dab/f7c [22376,58945] 0 2024-04-24T23:13:59.186 INFO:tasks.workunit.client.1.smithi040.stdout:8/571: dwrite d0/d8/dc/f86 [0,4194304] 0 2024-04-24T23:13:59.186 INFO:tasks.workunit.client.0.smithi031.stdout:4/926: dwrite d3/f7 [0,4194304] 0 2024-04-24T23:13:59.188 INFO:tasks.workunit.client.0.smithi031.stdout:7/827: read d1/d2/f12 [1835825,12163] 0 2024-04-24T23:13:59.190 INFO:tasks.workunit.client.1.smithi040.stdout:8/572: read d0/d8/d22/d81/f74 [258397,88062] 0 2024-04-24T23:13:59.192 INFO:tasks.workunit.client.1.smithi040.stdout:8/573: stat d0/d16/d19/d4f/d9e/d90 0 2024-04-24T23:13:59.194 INFO:tasks.workunit.client.1.smithi040.stdout:3/615: symlink d1/d80/ld9 0 2024-04-24T23:13:59.196 INFO:tasks.workunit.client.0.smithi031.stdout:9/893: mkdir d2/d112 0 2024-04-24T23:13:59.198 INFO:tasks.workunit.client.0.smithi031.stdout:1/738: unlink d1/da/d12/d7e/d7b/d4e/l5f 0 2024-04-24T23:13:59.200 INFO:tasks.workunit.client.0.smithi031.stdout:5/767: getdents d0/d6/d3b/d4b/d1d/d2e 0 2024-04-24T23:13:59.200 INFO:tasks.workunit.client.0.smithi031.stdout:2/718: sync 2024-04-24T23:13:59.200 INFO:tasks.workunit.client.0.smithi031.stdout:8/956: sync 2024-04-24T23:13:59.202 INFO:tasks.workunit.client.1.smithi040.stdout:6/523: dwrite d7/d1a/d20/f23 [0,4194304] 0 2024-04-24T23:13:59.203 INFO:tasks.workunit.client.0.smithi031.stdout:3/752: link d6/df/d16/d2a/d95/d9b/d1a/lb0 d6/d14/lde 0 2024-04-24T23:13:59.204 INFO:tasks.workunit.client.1.smithi040.stdout:4/492: link da/d3f/d6b/d2b/l44 da/d3f/d40/d47/d75/d7c/lad 0 2024-04-24T23:13:59.206 INFO:tasks.workunit.client.0.smithi031.stdout:4/927: mkdir d3/d23/d69/d3f/d9e/d12b 0 2024-04-24T23:13:59.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/719: read d5/d8/d15/d65/f8e [207705,38894] 0 2024-04-24T23:13:59.209 INFO:tasks.workunit.client.0.smithi031.stdout:1/739: dwrite d1/d33/d41/d9b/fcd [0,4194304] 0 2024-04-24T23:13:59.211 INFO:tasks.workunit.client.1.smithi040.stdout:0/615: mkdir df/d12/d58/dc3/dcd 0 2024-04-24T23:13:59.211 INFO:tasks.workunit.client.0.smithi031.stdout:2/720: truncate d5/d6/d29/d4f/fc5 940529 0 2024-04-24T23:13:59.212 INFO:tasks.workunit.client.1.smithi040.stdout:6/524: dwrite d7/d1a/d20/d51/d1e/d46/d69/f6b [0,4194304] 0 2024-04-24T23:13:59.214 INFO:tasks.workunit.client.0.smithi031.stdout:6/868: creat d13/d1b/f124 x:0 0 0 2024-04-24T23:13:59.216 INFO:tasks.workunit.client.1.smithi040.stdout:7/590: mkdir db/d43/db9/dd4 0 2024-04-24T23:13:59.220 INFO:tasks.workunit.client.0.smithi031.stdout:8/957: rmdir d8/d15/d37/d43/d5c/de1 39 2024-04-24T23:13:59.222 INFO:tasks.workunit.client.1.smithi040.stdout:8/574: dwrite d0/d16/f7c [0,4194304] 0 2024-04-24T23:13:59.223 INFO:tasks.workunit.client.0.smithi031.stdout:5/768: creat d0/d6/d3b/d4b/d1d/d2e/ffb x:0 0 0 2024-04-24T23:13:59.227 INFO:tasks.workunit.client.1.smithi040.stdout:5/526: dread d2/d7/d8/d1f/d57/d27/f32 [0,4194304] 0 2024-04-24T23:13:59.230 INFO:tasks.workunit.client.1.smithi040.stdout:3/616: dwrite d1/f79 [0,4194304] 0 2024-04-24T23:13:59.232 INFO:tasks.workunit.client.1.smithi040.stdout:3/617: truncate d1/d9/d18/da7/fc5 879758 0 2024-04-24T23:13:59.234 INFO:tasks.workunit.client.1.smithi040.stdout:3/618: truncate d1/d2d/fcd 923668 0 2024-04-24T23:13:59.238 INFO:tasks.workunit.client.0.smithi031.stdout:7/828: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dd9/f122 x:0 0 0 2024-04-24T23:13:59.244 INFO:tasks.workunit.client.1.smithi040.stdout:7/591: creat db/d12/d62/d8a/fd5 x:0 0 0 2024-04-24T23:13:59.245 INFO:tasks.workunit.client.0.smithi031.stdout:9/894: mknod d2/dae/d14/d78/d84/d4a/d4b/daf/c113 0 2024-04-24T23:13:59.246 INFO:tasks.workunit.client.1.smithi040.stdout:9/582: rmdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/dc9 0 2024-04-24T23:13:59.249 INFO:tasks.workunit.client.0.smithi031.stdout:8/958: chown d8/d15/d37/d43/d7b/d87/dad/cc9 828 1 2024-04-24T23:13:59.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/864: write da/fb [422089,102479] 0 2024-04-24T23:13:59.251 INFO:tasks.workunit.client.0.smithi031.stdout:5/769: creat d0/d7/d74/de2/ffc x:0 0 0 2024-04-24T23:13:59.252 INFO:tasks.workunit.client.0.smithi031.stdout:5/770: write d0/d6/d3b/d4f/ff2 [316702,57551] 0 2024-04-24T23:13:59.254 INFO:tasks.workunit.client.0.smithi031.stdout:5/771: readlink d0/d6/d3b/d4b/dc/d25/l5c 0 2024-04-24T23:13:59.254 INFO:tasks.workunit.client.0.smithi031.stdout:4/928: mkdir d3/d12c 0 2024-04-24T23:13:59.259 INFO:tasks.workunit.client.1.smithi040.stdout:2/591: getdents da/d1c/d37 0 2024-04-24T23:13:59.260 INFO:tasks.workunit.client.0.smithi031.stdout:2/721: mknod d5/d8/d45/d5b/d49/c101 0 2024-04-24T23:13:59.261 INFO:tasks.workunit.client.1.smithi040.stdout:2/592: write da/d1c/d37/d39/f60 [2523396,95618] 0 2024-04-24T23:13:59.262 INFO:tasks.workunit.client.0.smithi031.stdout:6/869: mkdir d13/d1b/de3/d118/d125 0 2024-04-24T23:13:59.263 INFO:tasks.workunit.client.1.smithi040.stdout:6/525: dwrite d7/d1a/d20/f3a [0,4194304] 0 2024-04-24T23:13:59.265 INFO:tasks.workunit.client.0.smithi031.stdout:7/829: unlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/c11a 0 2024-04-24T23:13:59.267 INFO:tasks.workunit.client.1.smithi040.stdout:3/619: write d1/d2d/fc0 [5179907,44244] 0 2024-04-24T23:13:59.269 INFO:tasks.workunit.client.1.smithi040.stdout:6/526: dread d7/d1a/f67 [0,4194304] 0 2024-04-24T23:13:59.271 INFO:tasks.workunit.client.1.smithi040.stdout:4/493: truncate da/d3f/d6b/d3c/f9f 2687919 0 2024-04-24T23:13:59.272 INFO:tasks.workunit.client.0.smithi031.stdout:1/740: truncate d1/d33/d41/d9b/fcd 3086966 0 2024-04-24T23:13:59.272 INFO:tasks.workunit.client.1.smithi040.stdout:4/494: read da/f2f [895467,24434] 0 2024-04-24T23:13:59.273 INFO:tasks.workunit.client.0.smithi031.stdout:8/959: readlink d8/d1a/d1d/l26 0 2024-04-24T23:13:59.276 INFO:tasks.workunit.client.1.smithi040.stdout:9/583: unlink f2 0 2024-04-24T23:13:59.277 INFO:tasks.workunit.client.1.smithi040.stdout:8/575: fsync d0/d16/f3b 0 2024-04-24T23:13:59.280 INFO:tasks.workunit.client.1.smithi040.stdout:1/551: getdents da/d35 0 2024-04-24T23:13:59.281 INFO:tasks.workunit.client.1.smithi040.stdout:1/552: readlink da/d12/d18/d2a/d4d/l8f 0 2024-04-24T23:13:59.283 INFO:tasks.workunit.client.0.smithi031.stdout:2/722: unlink d5/d6/d29/d26/db0/dc2/dad/fe4 0 2024-04-24T23:13:59.283 INFO:tasks.workunit.client.1.smithi040.stdout:0/616: truncate df/d1e/d20/f43 3728689 0 2024-04-24T23:13:59.284 INFO:tasks.workunit.client.0.smithi031.stdout:2/723: readlink d5/d8/d45/lfa 0 2024-04-24T23:13:59.289 INFO:tasks.workunit.client.0.smithi031.stdout:7/830: symlink d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/df2/l123 0 2024-04-24T23:13:59.290 INFO:tasks.workunit.client.0.smithi031.stdout:7/831: stat d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f5d 0 2024-04-24T23:13:59.290 INFO:tasks.workunit.client.1.smithi040.stdout:2/593: creat da/d81/da8/fdd x:0 0 0 2024-04-24T23:13:59.291 INFO:tasks.workunit.client.1.smithi040.stdout:2/594: stat da/d29/d55/lba 0 2024-04-24T23:13:59.294 INFO:tasks.workunit.client.0.smithi031.stdout:1/741: creat d1/da/dbc/fed x:0 0 0 2024-04-24T23:13:59.296 INFO:tasks.workunit.client.1.smithi040.stdout:3/620: mknod d1/d9/d2e/cda 0 2024-04-24T23:13:59.296 INFO:tasks.workunit.client.0.smithi031.stdout:6/870: write d13/d15/d1d/f8c [3570836,22622] 0 2024-04-24T23:13:59.296 INFO:tasks.workunit.client.0.smithi031.stdout:8/960: fdatasync d8/d15/f123 0 2024-04-24T23:13:59.297 INFO:tasks.workunit.client.1.smithi040.stdout:2/595: dwrite da/d81/fab [0,4194304] 0 2024-04-24T23:13:59.299 INFO:tasks.workunit.client.1.smithi040.stdout:9/584: dread d7/da/d17/d40/d55/d2f/f6f [4194304,4194304] 0 2024-04-24T23:13:59.299 INFO:tasks.workunit.client.1.smithi040.stdout:2/596: readlink da/d1c/d37/d39/d3f/l49 0 2024-04-24T23:13:59.300 INFO:tasks.workunit.client.1.smithi040.stdout:2/597: dread - da/d10/d4d/da4/fd9 zero size 2024-04-24T23:13:59.303 INFO:tasks.workunit.client.0.smithi031.stdout:3/753: link d6/df/d16/d2a/d95/d9b/d29/l85 d6/df/d16/d2a/d95/d9b/d1a/d25/ldf 0 2024-04-24T23:13:59.307 INFO:tasks.workunit.client.0.smithi031.stdout:2/724: fdatasync d5/d6/fc1 0 2024-04-24T23:13:59.308 INFO:tasks.workunit.client.1.smithi040.stdout:8/576: mknod d0/d16/d1b/d58/cbe 0 2024-04-24T23:13:59.308 INFO:tasks.workunit.client.1.smithi040.stdout:8/577: fdatasync d0/d8/dc/f96 0 2024-04-24T23:13:59.309 INFO:tasks.workunit.client.0.smithi031.stdout:9/895: creat d2/dae/d14/d1d/d45/f114 x:0 0 0 2024-04-24T23:13:59.310 INFO:tasks.workunit.client.1.smithi040.stdout:1/553: mkdir da/d35/d38/d47/dbc 0 2024-04-24T23:13:59.312 INFO:tasks.workunit.client.1.smithi040.stdout:5/527: rename d2/d7/d8/d64/d6b/c73 to d2/d7/d8/d64/caf 0 2024-04-24T23:13:59.313 INFO:tasks.workunit.client.1.smithi040.stdout:0/617: rmdir df/d1e/d20/d21/d51/da3 39 2024-04-24T23:13:59.315 INFO:tasks.workunit.client.0.smithi031.stdout:0/865: link da/d22/d62/fb4 da/f123 0 2024-04-24T23:13:59.315 INFO:tasks.workunit.client.1.smithi040.stdout:7/592: write db/d12/d20/f22 [1458864,93327] 0 2024-04-24T23:13:59.316 INFO:tasks.workunit.client.0.smithi031.stdout:0/866: write da/d18/d5a/d7d/f116 [792962,110858] 0 2024-04-24T23:13:59.318 INFO:tasks.workunit.client.0.smithi031.stdout:6/871: truncate d13/d15/d1d/d23/d24/d8b/f9d 968566 0 2024-04-24T23:13:59.320 INFO:tasks.workunit.client.1.smithi040.stdout:9/585: dread d7/da/f39 [0,4194304] 0 2024-04-24T23:13:59.321 INFO:tasks.workunit.client.0.smithi031.stdout:4/929: rename d3/d20/d2d/d44/d5b/d5d/c93 to d3/d20/d25/c12d 0 2024-04-24T23:13:59.325 INFO:tasks.workunit.client.0.smithi031.stdout:4/930: dread d3/f4a [0,4194304] 0 2024-04-24T23:13:59.326 INFO:tasks.workunit.client.1.smithi040.stdout:4/495: fsync da/d3f/d40/d47/fa4 0 2024-04-24T23:13:59.328 INFO:tasks.workunit.client.1.smithi040.stdout:4/496: truncate da/d45/d58/f98 68414 0 2024-04-24T23:13:59.329 INFO:tasks.workunit.client.0.smithi031.stdout:7/832: symlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d2c/l124 0 2024-04-24T23:13:59.333 INFO:tasks.workunit.client.0.smithi031.stdout:1/742: link d1/da/d12/d65/fa6 d1/da/d12/d65/dcf/d11/fee 0 2024-04-24T23:13:59.337 INFO:tasks.workunit.client.0.smithi031.stdout:6/872: creat d13/d15/d1d/d23/d24/d8b/f126 x:0 0 0 2024-04-24T23:13:59.338 INFO:tasks.workunit.client.0.smithi031.stdout:0/867: dread da/d18/d1a/d37/d41/f66 [0,4194304] 0 2024-04-24T23:13:59.338 INFO:tasks.workunit.client.1.smithi040.stdout:2/598: creat da/d10/d4d/dd7/fde x:0 0 0 2024-04-24T23:13:59.341 INFO:tasks.workunit.client.0.smithi031.stdout:3/754: write d6/fa1 [3298025,97890] 0 2024-04-24T23:13:59.343 INFO:tasks.workunit.client.0.smithi031.stdout:2/725: write d5/d8/d15/f2b [3142245,56793] 0 2024-04-24T23:13:59.343 INFO:tasks.workunit.client.1.smithi040.stdout:3/621: dread d1/d9/d15/f1d [0,4194304] 0 2024-04-24T23:13:59.346 INFO:tasks.workunit.client.0.smithi031.stdout:9/896: dread d2/dae/f13 [0,4194304] 0 2024-04-24T23:13:59.348 INFO:tasks.workunit.client.0.smithi031.stdout:7/833: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/f125 x:0 0 0 2024-04-24T23:13:59.349 INFO:tasks.workunit.client.1.smithi040.stdout:8/578: creat d0/d28/db7/fbf x:0 0 0 2024-04-24T23:13:59.351 INFO:tasks.workunit.client.1.smithi040.stdout:5/528: mknod d2/d7/d8/d1f/d57/d27/d65/cb0 0 2024-04-24T23:13:59.353 INFO:tasks.workunit.client.0.smithi031.stdout:6/873: mknod d13/d15/d1d/d23/d44/da0/da6/c127 0 2024-04-24T23:13:59.354 INFO:tasks.workunit.client.1.smithi040.stdout:7/593: creat db/d12/d20/d56/fd6 x:0 0 0 2024-04-24T23:13:59.356 INFO:tasks.workunit.client.0.smithi031.stdout:2/726: mkdir d5/d99/d102 0 2024-04-24T23:13:59.358 INFO:tasks.workunit.client.1.smithi040.stdout:2/599: symlink da/d1c/d37/d39/d3f/d52/d78/dc7/ldf 0 2024-04-24T23:13:59.358 INFO:tasks.workunit.client.1.smithi040.stdout:0/618: dwrite df/d1e/d20/d61/fbb [0,4194304] 0 2024-04-24T23:13:59.358 INFO:tasks.workunit.client.0.smithi031.stdout:9/897: truncate d2/d43/fe6 239083 0 2024-04-24T23:13:59.360 INFO:tasks.workunit.client.1.smithi040.stdout:3/622: mknod d1/d9/d18/d36/d55/d7e/cdb 0 2024-04-24T23:13:59.361 INFO:tasks.workunit.client.0.smithi031.stdout:4/931: mknod d3/d20/d25/d60/d63/c12e 0 2024-04-24T23:13:59.363 INFO:tasks.workunit.client.0.smithi031.stdout:1/743: symlink d1/d3e/da3/lef 0 2024-04-24T23:13:59.365 INFO:tasks.workunit.client.1.smithi040.stdout:8/579: creat d0/d8/d66/fc0 x:0 0 0 2024-04-24T23:13:59.365 INFO:tasks.workunit.client.0.smithi031.stdout:9/898: dwrite d2/dae/d14/d78/d84/d4a/d4b/d96/fa8 [0,4194304] 0 2024-04-24T23:13:59.366 INFO:tasks.workunit.client.1.smithi040.stdout:2/600: dwrite da/d10/d4d/dd7/fdb [0,4194304] 0 2024-04-24T23:13:59.366 INFO:tasks.workunit.client.1.smithi040.stdout:8/580: chown d0/d8/d55/f83 5370 1 2024-04-24T23:13:59.369 INFO:tasks.workunit.client.0.smithi031.stdout:9/899: truncate d2/dae/d14/d78/d84/da6/ff4 460110 0 2024-04-24T23:13:59.370 INFO:tasks.workunit.client.0.smithi031.stdout:9/900: chown d2/d43/d9d/dc0 0 1 2024-04-24T23:13:59.372 INFO:tasks.workunit.client.0.smithi031.stdout:8/961: rename d8/d15/d37/d43/c8a to d8/d1a/d1d/d50/dc0/c141 0 2024-04-24T23:13:59.373 INFO:tasks.workunit.client.1.smithi040.stdout:6/527: getdents d7/d1a/d20/d51/d1e 0 2024-04-24T23:13:59.375 INFO:tasks.workunit.client.0.smithi031.stdout:6/874: symlink d13/d15/d1d/d23/d4d/d7b/d11b/l128 0 2024-04-24T23:13:59.377 INFO:tasks.workunit.client.0.smithi031.stdout:3/755: write d6/d55/fdd [4195048,45009] 0 2024-04-24T23:13:59.379 INFO:tasks.workunit.client.1.smithi040.stdout:7/594: mkdir db/d12/d20/d52/d44/dd7 0 2024-04-24T23:13:59.381 INFO:tasks.workunit.client.1.smithi040.stdout:0/619: mknod df/d12/d4b/db8/cce 0 2024-04-24T23:13:59.384 INFO:tasks.workunit.client.1.smithi040.stdout:3/623: read - d1/d2d/de/d30/f58 zero size 2024-04-24T23:13:59.384 INFO:tasks.workunit.client.1.smithi040.stdout:1/554: truncate da/d12/d18/f6c 2765612 0 2024-04-24T23:13:59.385 INFO:tasks.workunit.client.0.smithi031.stdout:9/901: fsync d2/d43/d9d/dc0/fe8 0 2024-04-24T23:13:59.386 INFO:tasks.workunit.client.1.smithi040.stdout:3/624: fdatasync d1/d9/d18/d36/fa2 0 2024-04-24T23:13:59.387 INFO:tasks.workunit.client.1.smithi040.stdout:1/555: dread - da/d12/d23/fb8 zero size 2024-04-24T23:13:59.388 INFO:tasks.workunit.client.0.smithi031.stdout:6/875: fdatasync d13/d15/d1d/d23/d24/d8b/f9b 0 2024-04-24T23:13:59.388 INFO:tasks.workunit.client.1.smithi040.stdout:7/595: dwrite db/d12/d20/d52/d6c/d8f/db8/fc8 [0,4194304] 0 2024-04-24T23:13:59.389 INFO:tasks.workunit.client.0.smithi031.stdout:6/876: chown d13/d15/d1d/d23/d2d/dbf/fec 631 1 2024-04-24T23:13:59.390 INFO:tasks.workunit.client.0.smithi031.stdout:3/756: symlink d6/df/d16/d2a/le0 0 2024-04-24T23:13:59.390 INFO:tasks.workunit.client.1.smithi040.stdout:1/556: chown da/d12/d14/l8d 22062 1 2024-04-24T23:13:59.391 INFO:tasks.workunit.client.0.smithi031.stdout:3/757: fsync d6/df/d16/d53/fc6 0 2024-04-24T23:13:59.392 INFO:tasks.workunit.client.1.smithi040.stdout:8/581: dread d0/d8/d22/f6a [0,4194304] 0 2024-04-24T23:13:59.393 INFO:tasks.workunit.client.1.smithi040.stdout:8/582: write d0/d16/d1b/f95 [974473,64165] 0 2024-04-24T23:13:59.397 INFO:tasks.workunit.client.0.smithi031.stdout:0/868: getdents da/d11/d6a 0 2024-04-24T23:13:59.400 INFO:tasks.workunit.client.1.smithi040.stdout:2/601: truncate da/d1c/d7e/fd3 3753275 0 2024-04-24T23:13:59.400 INFO:tasks.workunit.client.1.smithi040.stdout:2/602: chown da/d81/f97 14665279 1 2024-04-24T23:13:59.401 INFO:tasks.workunit.client.0.smithi031.stdout:9/902: dread d2/fdc [0,4194304] 0 2024-04-24T23:13:59.405 INFO:tasks.workunit.client.1.smithi040.stdout:5/529: link d2/d7/d8/d1f/d31/c40 d2/d7/d8/db/d1e/cb1 0 2024-04-24T23:13:59.409 INFO:tasks.workunit.client.1.smithi040.stdout:7/596: write db/f10 [6074845,97227] 0 2024-04-24T23:13:59.409 INFO:tasks.workunit.client.0.smithi031.stdout:6/877: truncate d13/d1b/dd0/fd9 87888 0 2024-04-24T23:13:59.412 INFO:tasks.workunit.client.1.smithi040.stdout:5/530: dwrite d2/d2f/d14/f17 [0,4194304] 0 2024-04-24T23:13:59.412 INFO:tasks.workunit.client.1.smithi040.stdout:0/620: write df/d1e/f40 [1312586,23990] 0 2024-04-24T23:13:59.412 INFO:tasks.workunit.client.0.smithi031.stdout:3/758: mkdir d6/df/d16/d57/de1 0 2024-04-24T23:13:59.414 INFO:tasks.workunit.client.1.smithi040.stdout:0/621: chown df/d14/f96 751026 1 2024-04-24T23:13:59.415 INFO:tasks.workunit.client.0.smithi031.stdout:7/834: getdents d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa 0 2024-04-24T23:13:59.419 INFO:tasks.workunit.client.1.smithi040.stdout:0/622: truncate df/d1e/d20/d21/d41/d52/f87 776841 0 2024-04-24T23:13:59.419 INFO:tasks.workunit.client.1.smithi040.stdout:7/597: dwrite db/d2f/d37/fa5 [0,4194304] 0 2024-04-24T23:13:59.421 INFO:tasks.workunit.client.0.smithi031.stdout:9/903: dread - d2/d43/d6e/fd9 zero size 2024-04-24T23:13:59.424 INFO:tasks.workunit.client.1.smithi040.stdout:8/583: rename d0/d16/d19/d4f to d0/d7e/dc1 0 2024-04-24T23:13:59.427 INFO:tasks.workunit.client.1.smithi040.stdout:1/557: dwrite da/d12/d18/d2a/f5a [0,4194304] 0 2024-04-24T23:13:59.428 INFO:tasks.workunit.client.0.smithi031.stdout:6/878: creat d13/d15/d1d/ddd/f129 x:0 0 0 2024-04-24T23:13:59.430 INFO:tasks.workunit.client.1.smithi040.stdout:6/528: truncate d7/de/ff 2112384 0 2024-04-24T23:13:59.431 INFO:tasks.workunit.client.1.smithi040.stdout:3/625: dwrite d1/d9/d18/d36/d55/f89 [0,4194304] 0 2024-04-24T23:13:59.431 INFO:tasks.workunit.client.0.smithi031.stdout:2/727: getdents d5/d8/d45 0 2024-04-24T23:13:59.433 INFO:tasks.workunit.client.0.smithi031.stdout:2/728: write d5/d8/d15/d56/d84/d75/ffc [148926,58356] 0 2024-04-24T23:13:59.435 INFO:tasks.workunit.client.0.smithi031.stdout:9/904: truncate d2/d16/dbe/d7d/fac 1000331 0 2024-04-24T23:13:59.437 INFO:tasks.workunit.client.1.smithi040.stdout:6/529: dread d7/d1a/d20/f3a [0,4194304] 0 2024-04-24T23:13:59.439 INFO:tasks.workunit.client.0.smithi031.stdout:1/744: rename d1/da/d10/d52/d62 to d1/da/df0 0 2024-04-24T23:13:59.441 INFO:tasks.workunit.client.1.smithi040.stdout:9/586: sync 2024-04-24T23:13:59.444 INFO:tasks.workunit.client.1.smithi040.stdout:0/623: unlink df/d12/d1a/f67 0 2024-04-24T23:13:59.445 INFO:tasks.workunit.client.0.smithi031.stdout:3/759: symlink d6/df/d16/le2 0 2024-04-24T23:13:59.447 INFO:tasks.workunit.client.1.smithi040.stdout:5/531: rename d2/d2f/f33 to d2/d62/fb2 0 2024-04-24T23:13:59.448 INFO:tasks.workunit.client.0.smithi031.stdout:7/835: fdatasync d1/d2/d60/fd4 0 2024-04-24T23:13:59.449 INFO:tasks.workunit.client.0.smithi031.stdout:2/729: stat d5/d8/d45/d5b/d49/d6f/c7b 0 2024-04-24T23:13:59.453 INFO:tasks.workunit.client.0.smithi031.stdout:9/905: creat d2/dae/d14/d78/d84/d4a/d4b/daf/f115 x:0 0 0 2024-04-24T23:13:59.454 INFO:tasks.workunit.client.1.smithi040.stdout:3/626: read - d1/f88 zero size 2024-04-24T23:13:59.455 INFO:tasks.workunit.client.1.smithi040.stdout:2/603: write da/d10/d15/f2c [1729496,35937] 0 2024-04-24T23:13:59.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/836: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/ff0 [0,4194304] 0 2024-04-24T23:13:59.458 INFO:tasks.workunit.client.0.smithi031.stdout:8/962: rename d8/d15/d37/d62/dfa to d8/d1a/d1d/d50/d142 0 2024-04-24T23:13:59.460 INFO:tasks.workunit.client.0.smithi031.stdout:1/745: dread - d1/da/d12/d65/dcf/d11/d21/d5a/dbb/fc0 zero size 2024-04-24T23:13:59.461 INFO:tasks.workunit.client.0.smithi031.stdout:1/746: dread - d1/d3e/da3/fe2 zero size 2024-04-24T23:13:59.470 INFO:tasks.workunit.client.0.smithi031.stdout:9/906: truncate d2/dae/d14/d1d/f7b 2886892 0 2024-04-24T23:13:59.476 INFO:tasks.workunit.client.0.smithi031.stdout:1/747: symlink d1/d3e/d46/d74/lf1 0 2024-04-24T23:13:59.476 INFO:tasks.workunit.client.1.smithi040.stdout:8/584: dwrite d0/d8/d22/d81/f74 [0,4194304] 0 2024-04-24T23:13:59.477 INFO:tasks.workunit.client.0.smithi031.stdout:6/879: write fe [212091,100976] 0 2024-04-24T23:13:59.477 INFO:tasks.workunit.client.1.smithi040.stdout:8/585: chown d0/d7e/dc1/d9e/d90/d91/fba 4711431 1 2024-04-24T23:13:59.480 INFO:tasks.workunit.client.0.smithi031.stdout:2/730: write d5/d6/d29/f5a [2391964,50273] 0 2024-04-24T23:13:59.481 INFO:tasks.workunit.client.1.smithi040.stdout:4/497: sync 2024-04-24T23:13:59.482 INFO:tasks.workunit.client.0.smithi031.stdout:2/731: dread d5/fd7 [0,4194304] 0 2024-04-24T23:13:59.485 INFO:tasks.workunit.client.0.smithi031.stdout:8/963: creat d8/d15/dcf/f143 x:0 0 0 2024-04-24T23:13:59.486 INFO:tasks.workunit.client.0.smithi031.stdout:1/748: chown d1/da/d12/c14 137 1 2024-04-24T23:13:59.487 INFO:tasks.workunit.client.0.smithi031.stdout:8/964: chown d8/d15/d37/d43/d7b/dbf/d13d/dc4/fc7 1874 1 2024-04-24T23:13:59.487 INFO:tasks.workunit.client.0.smithi031.stdout:1/749: chown d1/da/d12/c7f 0 1 2024-04-24T23:13:59.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/880: mknod d13/d1b/d32/db4/d79/c12a 0 2024-04-24T23:13:59.493 INFO:tasks.workunit.client.0.smithi031.stdout:3/760: rename d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/cd2 to d6/df/ce3 0 2024-04-24T23:13:59.496 INFO:tasks.workunit.client.1.smithi040.stdout:7/598: rename db/f10 to db/d2f/d37/fd8 0 2024-04-24T23:13:59.497 INFO:tasks.workunit.client.1.smithi040.stdout:7/599: truncate db/d12/d73/fbe 3451572 0 2024-04-24T23:13:59.497 INFO:tasks.workunit.client.0.smithi031.stdout:2/732: unlink d5/d8/d45/ff5 0 2024-04-24T23:13:59.499 INFO:tasks.workunit.client.0.smithi031.stdout:8/965: fdatasync d8/d15/d37/d43/daf/f115 0 2024-04-24T23:13:59.501 INFO:tasks.workunit.client.1.smithi040.stdout:2/604: creat da/d1c/d37/d39/d79/fe0 x:0 0 0 2024-04-24T23:13:59.502 INFO:tasks.workunit.client.0.smithi031.stdout:6/881: symlink d13/d1b/d32/db4/d106/l12b 0 2024-04-24T23:13:59.503 INFO:tasks.workunit.client.1.smithi040.stdout:3/627: fdatasync d1/d9/d18/d36/d55/d62/f8b 0 2024-04-24T23:13:59.505 INFO:tasks.workunit.client.0.smithi031.stdout:7/837: rename d1/d2/d60 to d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/d126 0 2024-04-24T23:13:59.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/907: write d2/fdc [827759,72795] 0 2024-04-24T23:13:59.514 INFO:tasks.workunit.client.1.smithi040.stdout:4/498: fsync da/d3f/d40/d47/fa4 0 2024-04-24T23:13:59.514 INFO:tasks.workunit.client.1.smithi040.stdout:9/587: write d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f5b [1990408,25784] 0 2024-04-24T23:13:59.521 INFO:tasks.workunit.client.1.smithi040.stdout:6/530: rename d7/de/c73 to d7/d1a/d20/d8c/ca3 0 2024-04-24T23:13:59.522 INFO:tasks.workunit.client.0.smithi031.stdout:9/908: mknod d2/dae/d14/d1d/c116 0 2024-04-24T23:13:59.523 INFO:tasks.workunit.client.0.smithi031.stdout:7/838: dread d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/fa5 [0,4194304] 0 2024-04-24T23:13:59.524 INFO:tasks.workunit.client.1.smithi040.stdout:2/605: rmdir da/d10/d15/daa 39 2024-04-24T23:13:59.527 INFO:tasks.workunit.client.1.smithi040.stdout:6/531: dwrite d7/d1a/d20/d51/d2f/f5b [0,4194304] 0 2024-04-24T23:13:59.528 INFO:tasks.workunit.client.1.smithi040.stdout:6/532: read - d7/d1a/d20/d51/d1e/d46/fa0 zero size 2024-04-24T23:13:59.529 INFO:tasks.workunit.client.1.smithi040.stdout:6/533: chown d7/d1a/d20/c48 6675488 1 2024-04-24T23:13:59.531 INFO:tasks.workunit.client.0.smithi031.stdout:1/750: link d1/d33/d36/f8d d1/d33/d41/d9b/da8/ff2 0 2024-04-24T23:13:59.531 INFO:tasks.workunit.client.0.smithi031.stdout:1/751: fsync d1/d1d/f7a 0 2024-04-24T23:13:59.532 INFO:tasks.workunit.client.0.smithi031.stdout:1/752: fsync d1/da/f2c 0 2024-04-24T23:13:59.533 INFO:tasks.workunit.client.1.smithi040.stdout:1/558: link da/d12/d18/d2a/d4d/f75 da/d12/d14/fbd 0 2024-04-24T23:13:59.534 INFO:tasks.workunit.client.1.smithi040.stdout:1/559: chown da/d12/d18/d2a/d32/db3/c76 146860 1 2024-04-24T23:13:59.535 INFO:tasks.workunit.client.1.smithi040.stdout:8/586: mkdir d0/d8/d22/db8/dc2 0 2024-04-24T23:13:59.544 INFO:tasks.workunit.client.0.smithi031.stdout:3/761: creat d6/df/d16/d2a/d95/d9b/fe4 x:0 0 0 2024-04-24T23:13:59.547 INFO:tasks.workunit.client.1.smithi040.stdout:0/624: rename df/d1e/d70 to df/d1e/d20/d21/d51/d79/daa/dcf 0 2024-04-24T23:13:59.548 INFO:tasks.workunit.client.0.smithi031.stdout:2/733: dwrite d5/d8/d45/d5b/d49/f9f [0,4194304] 0 2024-04-24T23:13:59.550 INFO:tasks.workunit.client.1.smithi040.stdout:2/606: creat da/d1c/d37/fe1 x:0 0 0 2024-04-24T23:13:59.550 INFO:tasks.workunit.client.0.smithi031.stdout:6/882: link l0 d13/d15/d1d/d23/d44/d47/d69/l12c 0 2024-04-24T23:13:59.553 INFO:tasks.workunit.client.0.smithi031.stdout:6/883: truncate d13/d15/d1d/d23/d44/d47/d69/dd5/f110 1020782 0 2024-04-24T23:13:59.554 INFO:tasks.workunit.client.0.smithi031.stdout:8/966: rename d8/d15/d37/d62/fee to d8/d15/d37/f144 0 2024-04-24T23:13:59.555 INFO:tasks.workunit.client.1.smithi040.stdout:3/628: mknod d1/d2d/d2b/cdc 0 2024-04-24T23:13:59.556 INFO:tasks.workunit.client.0.smithi031.stdout:9/909: truncate d2/dae/d14/d1d/f35 1268624 0 2024-04-24T23:13:59.557 INFO:tasks.workunit.client.1.smithi040.stdout:6/534: creat d7/d1a/d20/d51/d1e/d5d/d52/fa4 x:0 0 0 2024-04-24T23:13:59.559 INFO:tasks.workunit.client.1.smithi040.stdout:1/560: mknod da/d12/d18/d2a/d32/cbe 0 2024-04-24T23:13:59.561 INFO:tasks.workunit.client.0.smithi031.stdout:1/753: mknod d1/da/d12/d65/dcf/d11/d21/cf3 0 2024-04-24T23:13:59.562 INFO:tasks.workunit.client.0.smithi031.stdout:1/754: write d1/d3e/da3/fe2 [548060,75216] 0 2024-04-24T23:13:59.563 INFO:tasks.workunit.client.0.smithi031.stdout:6/884: rmdir d13/d1b/d32/db4 39 2024-04-24T23:13:59.570 INFO:tasks.workunit.client.1.smithi040.stdout:0/625: mkdir df/d1e/da7/dd0 0 2024-04-24T23:13:59.571 INFO:tasks.workunit.client.1.smithi040.stdout:0/626: fsync df/d14/f88 0 2024-04-24T23:13:59.571 INFO:tasks.workunit.client.1.smithi040.stdout:0/627: stat f6 0 2024-04-24T23:13:59.572 INFO:tasks.workunit.client.0.smithi031.stdout:3/762: creat d6/df/d16/d57/de1/fe5 x:0 0 0 2024-04-24T23:13:59.575 INFO:tasks.workunit.client.1.smithi040.stdout:3/629: creat d1/d9/d15/fdd x:0 0 0 2024-04-24T23:13:59.577 INFO:tasks.workunit.client.0.smithi031.stdout:1/755: mkdir d1/d3e/d46/df4 0 2024-04-24T23:13:59.577 INFO:tasks.workunit.client.1.smithi040.stdout:2/607: dread da/d29/d55/da7/fd6 [0,4194304] 0 2024-04-24T23:13:59.577 INFO:tasks.workunit.client.1.smithi040.stdout:6/535: mknod d7/d1a/d56/ca5 0 2024-04-24T23:13:59.578 INFO:tasks.workunit.client.0.smithi031.stdout:9/910: dread d2/dae/d14/d1d/f47 [0,4194304] 0 2024-04-24T23:13:59.578 INFO:tasks.workunit.client.0.smithi031.stdout:9/911: chown d2/dae/d14/d1d/d45/f93 80 1 2024-04-24T23:13:59.580 INFO:tasks.workunit.client.0.smithi031.stdout:8/967: mkdir d8/d1a/d1d/d50/dc0/d145 0 2024-04-24T23:13:59.585 INFO:tasks.workunit.client.1.smithi040.stdout:8/587: dwrite d0/d16/f3b [0,4194304] 0 2024-04-24T23:13:59.726 INFO:tasks.workunit.client.0.smithi031.stdout:2/734: creat d5/d8/d45/d5b/d49/f103 x:0 0 0 2024-04-24T23:13:59.728 INFO:tasks.workunit.client.0.smithi031.stdout:2/735: truncate d5/d6/d29/d26/db0/dc2/dad/dbe/fe1 697360 0 2024-04-24T23:13:59.729 INFO:tasks.workunit.client.1.smithi040.stdout:5/532: rename d2/d7/d8/d1f/f67 to d2/d7/d8/d6d/fb3 0 2024-04-24T23:13:59.730 INFO:tasks.workunit.client.0.smithi031.stdout:1/756: unlink d1/da/f2c 0 2024-04-24T23:13:59.732 INFO:tasks.workunit.client.0.smithi031.stdout:6/885: write d13/d15/d1d/d23/d24/d43/d68/f72 [3137124,47786] 0 2024-04-24T23:13:59.733 INFO:tasks.workunit.client.0.smithi031.stdout:6/886: write f11 [4595696,79263] 0 2024-04-24T23:13:59.735 INFO:tasks.workunit.client.0.smithi031.stdout:9/912: mkdir d2/dae/d14/d1d/d45/d65/d117 0 2024-04-24T23:13:59.738 INFO:tasks.workunit.client.0.smithi031.stdout:3/763: unlink d6/df/d16/d2a/d95/d9b/c6f 0 2024-04-24T23:13:59.739 INFO:tasks.workunit.client.1.smithi040.stdout:2/608: mknod da/d10/d4d/dd7/ce2 0 2024-04-24T23:13:59.741 INFO:tasks.workunit.client.0.smithi031.stdout:2/736: truncate d5/d6/d29/f61 4022391 0 2024-04-24T23:13:59.743 INFO:tasks.workunit.client.1.smithi040.stdout:6/536: creat d7/de/d4e/fa6 x:0 0 0 2024-04-24T23:13:59.743 INFO:tasks.workunit.client.1.smithi040.stdout:5/533: dread d2/d7/d8/d1f/d57/d27/f54 [0,4194304] 0 2024-04-24T23:13:59.744 INFO:tasks.workunit.client.0.smithi031.stdout:8/968: write d8/d15/d37/f85 [2613856,7596] 0 2024-04-24T23:13:59.744 INFO:tasks.workunit.client.1.smithi040.stdout:5/534: truncate d2/d2f/fa0 147154 0 2024-04-24T23:13:59.748 INFO:tasks.workunit.client.1.smithi040.stdout:4/499: link da/d3f/d6b/d3c/d69/l99 da/d3f/lae 0 2024-04-24T23:13:59.749 INFO:tasks.workunit.client.0.smithi031.stdout:9/913: fdatasync d2/dae/d14/d78/d84/da6/fa7 0 2024-04-24T23:13:59.750 INFO:tasks.workunit.client.1.smithi040.stdout:5/535: dread d2/d2f/d14/f17 [0,4194304] 0 2024-04-24T23:13:59.750 INFO:tasks.workunit.client.1.smithi040.stdout:9/588: getdents d7/da/d17/d40/d55/d2f/d1f/d31 0 2024-04-24T23:13:59.751 INFO:tasks.workunit.client.1.smithi040.stdout:5/536: chown d2/d2f/l29 0 1 2024-04-24T23:13:59.752 INFO:tasks.workunit.client.1.smithi040.stdout:5/537: fsync d2/d7/d8/db/da2/fac 0 2024-04-24T23:13:59.753 INFO:tasks.workunit.client.1.smithi040.stdout:7/600: rename db/d12/d20/c6a to db/d12/d7d/cd9 0 2024-04-24T23:13:59.754 INFO:tasks.workunit.client.0.smithi031.stdout:3/764: creat d6/d14/d1d/d59/fe6 x:0 0 0 2024-04-24T23:13:59.755 INFO:tasks.workunit.client.0.smithi031.stdout:3/765: chown d6/df/d16/d57/de1 131523311 1 2024-04-24T23:13:59.756 INFO:tasks.workunit.client.0.smithi031.stdout:2/737: mkdir d5/d8/d45/d5b/d49/dc4/d104 0 2024-04-24T23:13:59.757 INFO:tasks.workunit.client.1.smithi040.stdout:7/601: dread db/d12/d20/d52/d6c/d8f/fd0 [0,4194304] 0 2024-04-24T23:13:59.758 INFO:tasks.workunit.client.1.smithi040.stdout:3/630: getdents d1/d9/d2e/dcf 0 2024-04-24T23:13:59.759 INFO:tasks.workunit.client.0.smithi031.stdout:1/757: dwrite d1/da/d12/d7e/d7b/d4e/f72 [0,4194304] 0 2024-04-24T23:13:59.759 INFO:tasks.workunit.client.1.smithi040.stdout:2/609: truncate da/d29/f4a 1811541 0 2024-04-24T23:13:59.760 INFO:tasks.workunit.client.1.smithi040.stdout:3/631: write d1/d9/d15/fdd [820972,1849] 0 2024-04-24T23:13:59.761 INFO:tasks.workunit.client.0.smithi031.stdout:6/887: symlink d13/d1b/d32/db4/d106/l12d 0 2024-04-24T23:13:59.762 INFO:tasks.workunit.client.1.smithi040.stdout:3/632: write d1/d9/f17 [1909898,85795] 0 2024-04-24T23:13:59.763 INFO:tasks.workunit.client.1.smithi040.stdout:8/588: dwrite d0/d7e/dc1/f80 [0,4194304] 0 2024-04-24T23:13:59.765 INFO:tasks.workunit.client.1.smithi040.stdout:4/500: creat da/d3f/d6b/d2b/faf x:0 0 0 2024-04-24T23:13:59.766 INFO:tasks.workunit.client.0.smithi031.stdout:8/969: creat d8/d1a/d1d/d50/dc0/d9d/d136/f146 x:0 0 0 2024-04-24T23:13:59.767 INFO:tasks.workunit.client.1.smithi040.stdout:9/589: creat d7/da/d17/d40/d55/d2f/fd2 x:0 0 0 2024-04-24T23:13:59.767 INFO:tasks.workunit.client.0.smithi031.stdout:9/914: read d2/dae/d14/d78/d84/d4a/fde [332937,107197] 0 2024-04-24T23:13:59.768 INFO:tasks.workunit.client.1.smithi040.stdout:4/501: read da/d3f/d40/d41/d60/f42 [259044,58992] 0 2024-04-24T23:13:59.769 INFO:tasks.workunit.client.1.smithi040.stdout:6/537: rename d7/d1a/d20/d51/d2f/c88 to d7/de/d84/ca7 0 2024-04-24T23:13:59.771 INFO:tasks.workunit.client.0.smithi031.stdout:3/766: rename d6/d14/la2 to d6/d14/d1d/d69/le7 0 2024-04-24T23:13:59.777 INFO:tasks.workunit.client.0.smithi031.stdout:1/758: mknod d1/da/d12/d7e/d7b/cf5 0 2024-04-24T23:13:59.780 INFO:tasks.workunit.client.1.smithi040.stdout:0/628: fdatasync df/d12/d1a/d71/f36 0 2024-04-24T23:13:59.783 INFO:tasks.workunit.client.1.smithi040.stdout:1/561: getdents da/d12/d23 0 2024-04-24T23:13:59.783 INFO:tasks.workunit.client.0.smithi031.stdout:2/738: fsync d5/d8/d15/f50 0 2024-04-24T23:13:59.785 INFO:tasks.workunit.client.1.smithi040.stdout:2/610: read da/d1c/d37/d8a/f91 [410540,20070] 0 2024-04-24T23:13:59.785 INFO:tasks.workunit.client.0.smithi031.stdout:8/970: creat d8/d15/d37/d43/d7b/dbf/f147 x:0 0 0 2024-04-24T23:13:59.786 INFO:tasks.workunit.client.1.smithi040.stdout:0/629: dwrite df/d1e/f2f [4194304,4194304] 0 2024-04-24T23:13:59.786 INFO:tasks.workunit.client.0.smithi031.stdout:8/971: chown d8/d1a/d1d/l26 6197504 1 2024-04-24T23:13:59.790 INFO:tasks.workunit.client.1.smithi040.stdout:3/633: write d1/d9/d18/d36/d61/fa6 [916568,32300] 0 2024-04-24T23:13:59.792 INFO:tasks.workunit.client.1.smithi040.stdout:3/634: chown d1/d9/d18/d36/l9b 52 1 2024-04-24T23:13:59.794 INFO:tasks.workunit.client.1.smithi040.stdout:0/630: dwrite df/d1e/f60 [0,4194304] 0 2024-04-24T23:13:59.794 INFO:tasks.workunit.client.1.smithi040.stdout:9/590: write d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f78 [3958181,101780] 0 2024-04-24T23:13:59.794 INFO:tasks.workunit.client.0.smithi031.stdout:3/767: read d6/d14/d1d/d69/f9c [1798100,35596] 0 2024-04-24T23:13:59.796 INFO:tasks.workunit.client.0.smithi031.stdout:3/768: read - d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/fd9 zero size 2024-04-24T23:13:59.796 INFO:tasks.workunit.client.1.smithi040.stdout:4/502: truncate f5 1861407 0 2024-04-24T23:13:59.800 INFO:tasks.workunit.client.0.smithi031.stdout:8/972: creat d8/d1a/d1d/d50/dc0/d9d/f148 x:0 0 0 2024-04-24T23:13:59.801 INFO:tasks.workunit.client.0.smithi031.stdout:9/915: write d2/d43/d9d/dc0/fcc [55684,17314] 0 2024-04-24T23:13:59.802 INFO:tasks.workunit.client.0.smithi031.stdout:1/759: symlink d1/da/d10/d52/lf6 0 2024-04-24T23:13:59.804 INFO:tasks.workunit.client.0.smithi031.stdout:6/888: creat d13/d1b/d32/f12e x:0 0 0 2024-04-24T23:13:59.806 INFO:tasks.workunit.client.0.smithi031.stdout:2/739: symlink d5/d8/l105 0 2024-04-24T23:13:59.808 INFO:tasks.workunit.client.0.smithi031.stdout:1/760: symlink d1/da/d12/d7e/d7b/lf7 0 2024-04-24T23:13:59.809 INFO:tasks.workunit.client.1.smithi040.stdout:9/591: mknod d7/d92/d9c/cd3 0 2024-04-24T23:13:59.812 INFO:tasks.workunit.client.1.smithi040.stdout:4/503: fsync da/d3f/d40/d47/d75/f97 0 2024-04-24T23:13:59.813 INFO:tasks.workunit.client.1.smithi040.stdout:6/538: creat d7/d1a/d9f/fa8 x:0 0 0 2024-04-24T23:13:59.813 INFO:tasks.workunit.client.1.smithi040.stdout:6/539: dread - d7/d1a/d9f/fa8 zero size 2024-04-24T23:13:59.816 INFO:tasks.workunit.client.0.smithi031.stdout:1/761: dread d1/da/d12/d65/fa0 [0,4194304] 0 2024-04-24T23:13:59.816 INFO:tasks.workunit.client.1.smithi040.stdout:2/611: creat da/d29/d55/da7/fe3 x:0 0 0 2024-04-24T23:13:59.817 INFO:tasks.workunit.client.0.smithi031.stdout:1/762: chown d1/d3e/da3/lef 393628830 1 2024-04-24T23:13:59.818 INFO:tasks.workunit.client.1.smithi040.stdout:2/612: read - da/d1c/d37/d39/d79/da3/fb3 zero size 2024-04-24T23:13:59.818 INFO:tasks.workunit.client.1.smithi040.stdout:8/589: link d0/d16/d19/f9a d0/d8/fc3 0 2024-04-24T23:13:59.825 INFO:tasks.workunit.client.0.smithi031.stdout:9/916: getdents d2/d43/d9d/dc0 0 2024-04-24T23:13:59.826 INFO:tasks.workunit.client.1.smithi040.stdout:6/540: creat d7/d1a/d9f/fa9 x:0 0 0 2024-04-24T23:13:59.827 INFO:tasks.workunit.client.0.smithi031.stdout:1/763: mkdir d1/da/d12/d65/dcf/d11/df8 0 2024-04-24T23:13:59.829 INFO:tasks.workunit.client.0.smithi031.stdout:6/889: link d13/d1b/d32/db4/c53 d13/c12f 0 2024-04-24T23:13:59.831 INFO:tasks.workunit.client.1.smithi040.stdout:2/613: creat da/d1c/d37/d8a/fe4 x:0 0 0 2024-04-24T23:13:59.831 INFO:tasks.workunit.client.0.smithi031.stdout:3/769: dwrite d6/d14/d71/dad/fb9 [0,4194304] 0 2024-04-24T23:13:59.834 INFO:tasks.workunit.client.1.smithi040.stdout:1/562: dwrite da/d12/f1c [0,4194304] 0 2024-04-24T23:13:59.834 INFO:tasks.workunit.client.0.smithi031.stdout:1/764: symlink d1/d3e/lf9 0 2024-04-24T23:13:59.836 INFO:tasks.workunit.client.1.smithi040.stdout:0/631: truncate df/d1e/da7/f7f 305175 0 2024-04-24T23:13:59.836 INFO:tasks.workunit.client.1.smithi040.stdout:0/632: symlink df/d1e/d20/d21/d41/d52/ld1 2 2024-04-24T23:13:59.837 INFO:tasks.workunit.client.1.smithi040.stdout:1/563: write da/d12/d23/f30 [4275060,21817] 0 2024-04-24T23:13:59.838 INFO:tasks.workunit.client.1.smithi040.stdout:0/633: chown df/d1e/d2b/l37 5479312 1 2024-04-24T23:13:59.840 INFO:tasks.workunit.client.1.smithi040.stdout:2/614: dwrite da/d1c/d37/d39/f76 [0,4194304] 0 2024-04-24T23:13:59.840 INFO:tasks.workunit.client.0.smithi031.stdout:6/890: fsync d13/d15/d1d/d23/d44/f80 0 2024-04-24T23:13:59.844 INFO:tasks.workunit.client.1.smithi040.stdout:6/541: mkdir d7/d1a/d20/d51/daa 0 2024-04-24T23:13:59.844 INFO:tasks.workunit.client.0.smithi031.stdout:2/740: dwrite d5/d6/d29/d26/db0/dc2/f44 [0,4194304] 0 2024-04-24T23:13:59.845 INFO:tasks.workunit.client.1.smithi040.stdout:6/542: write d7/de/f5e [1957050,61335] 0 2024-04-24T23:13:59.849 INFO:tasks.workunit.client.0.smithi031.stdout:1/765: dwrite d1/da/d12/d65/dcf/d11/f13 [8388608,4194304] 0 2024-04-24T23:13:59.852 INFO:tasks.workunit.client.0.smithi031.stdout:9/917: link d2/dae/d14/d78/d84/d4a/f8d d2/dae/d14/d78/d84/d4a/d4b/d96/f118 0 2024-04-24T23:13:59.856 INFO:tasks.workunit.client.0.smithi031.stdout:6/891: chown d13/d15/d1d/d23/d44/da0/da6/de6/c109 389 1 2024-04-24T23:13:59.861 INFO:tasks.workunit.client.1.smithi040.stdout:9/592: truncate d7/f57 4064839 0 2024-04-24T23:13:59.864 INFO:tasks.workunit.client.0.smithi031.stdout:2/741: creat d5/d8/d15/d56/d84/f106 x:0 0 0 2024-04-24T23:13:59.865 INFO:tasks.workunit.client.1.smithi040.stdout:4/504: creat da/d3f/fb0 x:0 0 0 2024-04-24T23:13:59.866 INFO:tasks.workunit.client.0.smithi031.stdout:6/892: dwrite d13/d15/d1d/d23/d2d/fff [0,4194304] 0 2024-04-24T23:13:59.869 INFO:tasks.workunit.client.0.smithi031.stdout:3/770: dread d6/df/d16/d2a/d95/d9b/d1a/f97 [0,4194304] 0 2024-04-24T23:13:59.874 INFO:tasks.workunit.client.1.smithi040.stdout:8/590: truncate d0/d8/dc/f3a 2334017 0 2024-04-24T23:13:59.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/766: symlink d1/da/d10/d52/lfa 0 2024-04-24T23:13:59.877 INFO:tasks.workunit.client.1.smithi040.stdout:0/634: write df/d12/d1a/d71/f36 [340329,107243] 0 2024-04-24T23:13:59.878 INFO:tasks.workunit.client.0.smithi031.stdout:6/893: read d13/d1b/f46 [2787162,62554] 0 2024-04-24T23:13:59.879 INFO:tasks.workunit.client.1.smithi040.stdout:0/635: write df/d1e/f2f [8310848,73041] 0 2024-04-24T23:13:59.879 INFO:tasks.workunit.client.1.smithi040.stdout:1/564: rename da/d35/d38/d47/l86 to da/d12/d18/d2a/d32/db3/lbf 0 2024-04-24T23:13:59.884 INFO:tasks.workunit.client.0.smithi031.stdout:1/767: creat d1/da/d12/d7e/d7b/d4e/d98/ffb x:0 0 0 2024-04-24T23:13:59.885 INFO:tasks.workunit.client.1.smithi040.stdout:9/593: read d7/f3a [2528719,55693] 0 2024-04-24T23:13:59.888 INFO:tasks.workunit.client.1.smithi040.stdout:9/594: stat d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/c95 0 2024-04-24T23:13:59.889 INFO:tasks.workunit.client.1.smithi040.stdout:2/615: dwrite da/d10/f31 [4194304,4194304] 0 2024-04-24T23:13:59.889 INFO:tasks.workunit.client.1.smithi040.stdout:6/543: symlink d7/d1a/d20/d51/d1e/d46/lab 0 2024-04-24T23:13:59.892 INFO:tasks.workunit.client.1.smithi040.stdout:5/538: sync 2024-04-24T23:13:59.892 INFO:tasks.workunit.client.0.smithi031.stdout:2/742: link d5/d8/d45/cf2 d5/d6/d29/c107 0 2024-04-24T23:13:59.895 INFO:tasks.workunit.client.0.smithi031.stdout:1/768: mkdir d1/da/d12/d7e/dfc 0 2024-04-24T23:13:59.896 INFO:tasks.workunit.client.1.smithi040.stdout:6/544: dwrite d7/f60 [0,4194304] 0 2024-04-24T23:13:59.896 INFO:tasks.workunit.client.1.smithi040.stdout:6/545: readlink d7/de/l19 0 2024-04-24T23:13:59.899 INFO:tasks.workunit.client.1.smithi040.stdout:0/636: symlink df/d12/d58/dc3/ld2 0 2024-04-24T23:13:59.900 INFO:tasks.workunit.client.0.smithi031.stdout:6/894: write d13/d1b/dd0/ff9 [420775,58278] 0 2024-04-24T23:13:59.903 INFO:tasks.workunit.client.1.smithi040.stdout:1/565: creat da/d12/d18/d2a/d41/fc0 x:0 0 0 2024-04-24T23:13:59.905 INFO:tasks.workunit.client.0.smithi031.stdout:1/769: symlink d1/da/d12/d7e/d7b/lfd 0 2024-04-24T23:13:59.906 INFO:tasks.workunit.client.1.smithi040.stdout:6/546: dread d7/d1a/d20/d51/d2f/f6f [0,4194304] 0 2024-04-24T23:13:59.907 INFO:tasks.workunit.client.0.smithi031.stdout:3/771: getdents d6/df/d16/d57/de1 0 2024-04-24T23:13:59.907 INFO:tasks.workunit.client.1.smithi040.stdout:1/566: dread da/d35/dab/f7c [0,4194304] 0 2024-04-24T23:13:59.908 INFO:tasks.workunit.client.1.smithi040.stdout:2/616: mknod da/ce5 0 2024-04-24T23:13:59.909 INFO:tasks.workunit.client.1.smithi040.stdout:9/595: dread d7/da/d17/d40/d55/d2f/d28/d67/faf [0,4194304] 0 2024-04-24T23:13:59.911 INFO:tasks.workunit.client.1.smithi040.stdout:0/637: dwrite df/d12/d1a/d71/f36 [0,4194304] 0 2024-04-24T23:13:59.912 INFO:tasks.workunit.client.1.smithi040.stdout:9/596: stat d7/da/d17/d40/d55/d2f/d28/f88 0 2024-04-24T23:13:59.913 INFO:tasks.workunit.client.1.smithi040.stdout:0/638: write df/d14/f88 [487630,52148] 0 2024-04-24T23:13:59.914 INFO:tasks.workunit.client.1.smithi040.stdout:0/639: stat df/d14/c4a 0 2024-04-24T23:13:59.914 INFO:tasks.workunit.client.0.smithi031.stdout:3/772: mknod d6/df/d16/d2a/d95/d9b/d29/dcb/ce8 0 2024-04-24T23:13:59.916 INFO:tasks.workunit.client.1.smithi040.stdout:0/640: chown df/d1e/f3e 222729 1 2024-04-24T23:13:59.916 INFO:tasks.workunit.client.0.smithi031.stdout:3/773: read - d6/df/d16/d53/fc6 zero size 2024-04-24T23:13:59.916 INFO:tasks.workunit.client.1.smithi040.stdout:0/641: readlink df/d14/la1 0 2024-04-24T23:13:59.918 INFO:tasks.workunit.client.1.smithi040.stdout:5/539: rmdir d2/d7/d8/d1f/d57/d27/d65 39 2024-04-24T23:13:59.918 INFO:tasks.workunit.client.0.smithi031.stdout:2/743: link d5/d8/d15/d65/lf0 d5/d28/d43/l108 0 2024-04-24T23:13:59.920 INFO:tasks.workunit.client.0.smithi031.stdout:1/770: mknod d1/da/d12/d65/dcf/cfe 0 2024-04-24T23:13:59.920 INFO:tasks.workunit.client.0.smithi031.stdout:1/771: chown d1/da 1050692 1 2024-04-24T23:13:59.922 INFO:tasks.workunit.client.0.smithi031.stdout:6/895: creat d13/d15/d1d/f130 x:0 0 0 2024-04-24T23:13:59.923 INFO:tasks.workunit.client.1.smithi040.stdout:9/597: dwrite d7/d92/fb5 [0,4194304] 0 2024-04-24T23:13:59.923 INFO:tasks.workunit.client.0.smithi031.stdout:6/896: chown d13/d15/d1d/d23/d44/d47/d69/f7f 3740560 1 2024-04-24T23:13:59.924 INFO:tasks.workunit.client.1.smithi040.stdout:4/505: creat da/d3f/d6b/fb1 x:0 0 0 2024-04-24T23:13:59.927 INFO:tasks.workunit.client.1.smithi040.stdout:0/642: dwrite df/d1e/d2b/db6/fbd [0,4194304] 0 2024-04-24T23:13:59.933 INFO:tasks.workunit.client.1.smithi040.stdout:8/591: truncate d0/d8/f4d 1012727 0 2024-04-24T23:13:59.934 INFO:tasks.workunit.client.0.smithi031.stdout:6/897: readlink d13/d15/d1d/d23/d2d/d9e/ld3 0 2024-04-24T23:13:59.938 INFO:tasks.workunit.client.1.smithi040.stdout:1/567: rename da/d12/d18/d2a/f5a to da/d35/d38/d47/fc1 0 2024-04-24T23:13:59.941 INFO:tasks.workunit.client.0.smithi031.stdout:3/774: write d6/df/d16/d2a/d95/d9b/d29/fa3 [117370,32804] 0 2024-04-24T23:13:59.942 INFO:tasks.workunit.client.0.smithi031.stdout:6/898: creat d13/d1b/de3/d118/d125/f131 x:0 0 0 2024-04-24T23:13:59.944 INFO:tasks.workunit.client.1.smithi040.stdout:4/506: chown da/d3f/d6b/l81 3837 1 2024-04-24T23:13:59.947 INFO:tasks.workunit.client.0.smithi031.stdout:1/772: dread d1/da/d12/d65/dcf/d11/f38 [0,4194304] 0 2024-04-24T23:13:59.949 INFO:tasks.workunit.client.0.smithi031.stdout:2/744: dwrite d5/d6/d29/d26/fd5 [0,4194304] 0 2024-04-24T23:13:59.950 INFO:tasks.workunit.client.0.smithi031.stdout:1/773: truncate d1/d3e/d46/f59 4504305 0 2024-04-24T23:13:59.956 INFO:tasks.workunit.client.1.smithi040.stdout:8/592: symlink d0/d16/d1b/lc4 0 2024-04-24T23:13:59.957 INFO:tasks.workunit.client.0.smithi031.stdout:2/745: read d5/d6/d29/d4f/f77 [2687619,57530] 0 2024-04-24T23:13:59.959 INFO:tasks.workunit.client.0.smithi031.stdout:2/746: read d5/d6/d29/f5a [4484753,18064] 0 2024-04-24T23:13:59.959 INFO:tasks.workunit.client.1.smithi040.stdout:1/568: stat da/cb6 0 2024-04-24T23:13:59.960 INFO:tasks.workunit.client.1.smithi040.stdout:5/540: mknod d2/d7/d8/d64/cb4 0 2024-04-24T23:13:59.961 INFO:tasks.workunit.client.1.smithi040.stdout:9/598: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4 0 2024-04-24T23:13:59.962 INFO:tasks.workunit.client.0.smithi031.stdout:2/747: symlink d5/d8/d45/l109 0 2024-04-24T23:13:59.962 INFO:tasks.workunit.client.0.smithi031.stdout:6/899: dwrite d13/d15/d1d/d23/d24/d43/d68/db8/fcc [0,4194304] 0 2024-04-24T23:13:59.963 INFO:tasks.workunit.client.1.smithi040.stdout:9/599: fdatasync d7/da/d17/d40/d55/d2f/fd2 0 2024-04-24T23:13:59.964 INFO:tasks.workunit.client.1.smithi040.stdout:2/617: truncate da/d1c/d37/d39/d3f/f67 623999 0 2024-04-24T23:13:59.967 INFO:tasks.workunit.client.0.smithi031.stdout:2/748: symlink d5/d8/d15/d56/d84/d75/l10a 0 2024-04-24T23:13:59.969 INFO:tasks.workunit.client.1.smithi040.stdout:6/547: creat d7/d1a/d20/d51/d1e/d46/fac x:0 0 0 2024-04-24T23:13:59.969 INFO:tasks.workunit.client.0.smithi031.stdout:6/900: write d13/d15/d1d/d23/d24/d43/f5a [679312,75347] 0 2024-04-24T23:13:59.972 INFO:tasks.workunit.client.1.smithi040.stdout:9/600: dwrite d7/da/d17/d40/d55/d2f/fd2 [0,4194304] 0 2024-04-24T23:13:59.972 INFO:tasks.workunit.client.1.smithi040.stdout:6/548: stat d7/d1a/d20/d51/d1e/d46/d69/f6b 0 2024-04-24T23:13:59.974 INFO:tasks.workunit.client.1.smithi040.stdout:4/507: creat da/d3f/d40/fb2 x:0 0 0 2024-04-24T23:13:59.975 INFO:tasks.workunit.client.1.smithi040.stdout:9/601: write d7/da/d17/d40/d55/d2f/d1f/d31/f98 [797331,74456] 0 2024-04-24T23:13:59.975 INFO:tasks.workunit.client.0.smithi031.stdout:2/749: dwrite d5/d28/d4e/d5e/fd9 [0,4194304] 0 2024-04-24T23:13:59.981 INFO:tasks.workunit.client.1.smithi040.stdout:2/618: rename da/d1c/d7e to da/d81/de6 0 2024-04-24T23:13:59.984 INFO:tasks.workunit.client.0.smithi031.stdout:2/750: rename d5/d99/da2/fd2 to d5/d6/d29/d26/db0/dc2/f10b 0 2024-04-24T23:13:59.986 INFO:tasks.workunit.client.1.smithi040.stdout:9/602: creat d7/d5e/fd5 x:0 0 0 2024-04-24T23:13:59.988 INFO:tasks.workunit.client.1.smithi040.stdout:5/541: rmdir d2/d7/d8/d1f/d57/d27/d3c/da9 0 2024-04-24T23:13:59.991 INFO:tasks.workunit.client.1.smithi040.stdout:8/593: getdents d0/d16/d1b 0 2024-04-24T23:13:59.992 INFO:tasks.workunit.client.1.smithi040.stdout:8/594: write d0/d8/d22/f62 [1923935,72501] 0 2024-04-24T23:13:59.993 INFO:tasks.workunit.client.1.smithi040.stdout:4/508: rename da/f1c to da/d3f/d40/d47/d71/fb3 0 2024-04-24T23:14:00.003 INFO:tasks.workunit.client.0.smithi031.stdout:6/901: dwrite d13/d15/d1d/d23/d24/d43/d68/db8/fcc [4194304,4194304] 0 2024-04-24T23:14:00.004 INFO:tasks.workunit.client.0.smithi031.stdout:6/902: stat d13/d15/d1d/d23/d2d/c100 0 2024-04-24T23:14:00.005 INFO:tasks.workunit.client.1.smithi040.stdout:6/549: write d7/d1a/f42 [852047,53260] 0 2024-04-24T23:14:00.006 INFO:tasks.workunit.client.1.smithi040.stdout:8/595: mknod d0/d16/d19/da2/cc5 0 2024-04-24T23:14:00.007 INFO:tasks.workunit.client.1.smithi040.stdout:7/602: sync 2024-04-24T23:14:00.007 INFO:tasks.workunit.client.1.smithi040.stdout:3/635: sync 2024-04-24T23:14:00.008 INFO:tasks.workunit.client.1.smithi040.stdout:8/596: fsync d0/d7e/dc1/d9e/d90/d91/f97 0 2024-04-24T23:14:00.010 INFO:tasks.workunit.client.0.smithi031.stdout:6/903: dread d13/d15/d1d/d23/d2d/fff [0,4194304] 0 2024-04-24T23:14:00.011 INFO:tasks.workunit.client.1.smithi040.stdout:9/603: dread d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f49 [0,4194304] 0 2024-04-24T23:14:00.013 INFO:tasks.workunit.client.1.smithi040.stdout:6/550: read d7/d1a/d20/d51/d1e/d46/f7d [102134,33928] 0 2024-04-24T23:14:00.015 INFO:tasks.workunit.client.0.smithi031.stdout:2/751: dwrite d5/d28/f86 [0,4194304] 0 2024-04-24T23:14:00.016 INFO:tasks.workunit.client.0.smithi031.stdout:2/752: readlink d5/d8/l105 0 2024-04-24T23:14:00.018 INFO:tasks.workunit.client.1.smithi040.stdout:2/619: write da/d10/d15/f32 [3364768,53835] 0 2024-04-24T23:14:00.018 INFO:tasks.workunit.client.0.smithi031.stdout:6/904: rename d13/d15/l42 to d13/d15/d1d/d23/d2d/dbf/d10b/l132 0 2024-04-24T23:14:00.019 INFO:tasks.workunit.client.1.smithi040.stdout:3/636: fsync d1/d9/f20 0 2024-04-24T23:14:00.021 INFO:tasks.workunit.client.1.smithi040.stdout:4/509: creat da/d45/d48/fb4 x:0 0 0 2024-04-24T23:14:00.021 INFO:tasks.workunit.client.0.smithi031.stdout:2/753: mknod d5/d6/d29/d4f/c10c 0 2024-04-24T23:14:00.022 INFO:tasks.workunit.client.1.smithi040.stdout:4/510: dread - da/d3f/d6b/d2b/faf zero size 2024-04-24T23:14:00.025 INFO:tasks.workunit.client.0.smithi031.stdout:6/905: creat d13/d15/d1d/d23/d4d/d7b/d11b/f133 x:0 0 0 2024-04-24T23:14:00.026 INFO:tasks.workunit.client.1.smithi040.stdout:8/597: creat d0/d16/d1b/d58/fc6 x:0 0 0 2024-04-24T23:14:00.026 INFO:tasks.workunit.client.1.smithi040.stdout:3/637: dwrite d1/d2d/d2b/f42 [4194304,4194304] 0 2024-04-24T23:14:00.027 INFO:tasks.workunit.client.1.smithi040.stdout:8/598: chown d0/d16/d19/da2 146 1 2024-04-24T23:14:00.028 INFO:tasks.workunit.client.1.smithi040.stdout:5/542: getdents d2/d7/d8 0 2024-04-24T23:14:00.029 INFO:tasks.workunit.client.1.smithi040.stdout:5/543: write d2/d7/d8/d1f/d57/d27/d3c/d43/f6a [3053669,9074] 0 2024-04-24T23:14:00.030 INFO:tasks.workunit.client.1.smithi040.stdout:9/604: unlink d7/da/c74 0 2024-04-24T23:14:00.042 INFO:tasks.workunit.client.1.smithi040.stdout:8/599: creat d0/d16/d19/fc7 x:0 0 0 2024-04-24T23:14:00.043 INFO:tasks.workunit.client.1.smithi040.stdout:2/620: write da/d1c/f57 [2393988,3751] 0 2024-04-24T23:14:00.043 INFO:tasks.workunit.client.1.smithi040.stdout:7/603: dwrite db/d12/d3f/d7a/f9e [4194304,4194304] 0 2024-04-24T23:14:00.043 INFO:tasks.workunit.client.1.smithi040.stdout:3/638: truncate d1/d9/d15/f1d 406281 0 2024-04-24T23:14:00.045 INFO:tasks.workunit.client.1.smithi040.stdout:7/604: stat db/d12/d20/d52/d6c 0 2024-04-24T23:14:00.045 INFO:tasks.workunit.client.1.smithi040.stdout:8/600: write d0/d7e/dc1/f72 [593419,56089] 0 2024-04-24T23:14:00.051 INFO:tasks.workunit.client.1.smithi040.stdout:4/511: mkdir da/d3f/d8d/db5 0 2024-04-24T23:14:00.053 INFO:tasks.workunit.client.1.smithi040.stdout:7/605: dwrite db/d12/f29 [0,4194304] 0 2024-04-24T23:14:00.058 INFO:tasks.workunit.client.1.smithi040.stdout:0/643: sync 2024-04-24T23:14:00.061 INFO:tasks.workunit.client.1.smithi040.stdout:2/621: rename da/d81/fab to da/d81/fe7 0 2024-04-24T23:14:00.064 INFO:tasks.workunit.client.1.smithi040.stdout:8/601: mknod d0/d16/d1b/d3f/cc8 0 2024-04-24T23:14:00.065 INFO:tasks.workunit.client.1.smithi040.stdout:9/605: write d7/f3a [789069,10607] 0 2024-04-24T23:14:00.067 INFO:tasks.workunit.client.1.smithi040.stdout:5/544: write d2/d7/d8/d1f/d57/f44 [3494637,69890] 0 2024-04-24T23:14:00.067 INFO:tasks.workunit.client.1.smithi040.stdout:9/606: chown d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc 6859 1 2024-04-24T23:14:00.069 INFO:tasks.workunit.client.1.smithi040.stdout:4/512: write da/d3f/d6b/f32 [1524408,112935] 0 2024-04-24T23:14:00.073 INFO:tasks.workunit.client.1.smithi040.stdout:6/551: dread d7/d1a/d20/d51/d1e/d46/f98 [0,4194304] 0 2024-04-24T23:14:00.074 INFO:tasks.workunit.client.1.smithi040.stdout:3/639: write d1/d2a/f6d [117493,16577] 0 2024-04-24T23:14:00.078 INFO:tasks.workunit.client.1.smithi040.stdout:8/602: mknod d0/d38/d4c/d5d/cc9 0 2024-04-24T23:14:00.078 INFO:tasks.workunit.client.1.smithi040.stdout:8/603: chown d0/d8 492500 1 2024-04-24T23:14:00.082 INFO:tasks.workunit.client.1.smithi040.stdout:7/606: mknod db/d2f/da8/cda 0 2024-04-24T23:14:00.085 INFO:tasks.workunit.client.1.smithi040.stdout:4/513: dread da/f59 [0,4194304] 0 2024-04-24T23:14:00.085 INFO:tasks.workunit.client.1.smithi040.stdout:0/644: dread df/d12/d58/d5b/fa0 [0,4194304] 0 2024-04-24T23:14:00.089 INFO:tasks.workunit.client.1.smithi040.stdout:5/545: dread d2/d7/d8/d64/d6b/f8b [0,4194304] 0 2024-04-24T23:14:00.089 INFO:tasks.workunit.client.1.smithi040.stdout:3/640: unlink d1/d2d/la3 0 2024-04-24T23:14:00.092 INFO:tasks.workunit.client.1.smithi040.stdout:2/622: creat da/d1c/d37/d39/d3f/d52/d78/dc7/dcc/fe8 x:0 0 0 2024-04-24T23:14:00.094 INFO:tasks.workunit.client.1.smithi040.stdout:5/546: dwrite d2/d2f/fa0 [0,4194304] 0 2024-04-24T23:14:00.096 INFO:tasks.workunit.client.0.smithi031.stdout:4/932: sync 2024-04-24T23:14:00.096 INFO:tasks.workunit.client.0.smithi031.stdout:0/869: sync 2024-04-24T23:14:00.096 INFO:tasks.workunit.client.0.smithi031.stdout:8/973: sync 2024-04-24T23:14:00.096 INFO:tasks.workunit.client.0.smithi031.stdout:5/772: sync 2024-04-24T23:14:00.096 INFO:tasks.workunit.client.0.smithi031.stdout:7/839: sync 2024-04-24T23:14:00.098 INFO:tasks.workunit.client.1.smithi040.stdout:7/607: creat db/d12/d62/fdb x:0 0 0 2024-04-24T23:14:00.098 INFO:tasks.workunit.client.0.smithi031.stdout:5/773: readlink d0/d6/d3b/d4b/d1d/l37 0 2024-04-24T23:14:00.099 INFO:tasks.workunit.client.1.smithi040.stdout:7/608: fsync db/d12/d20/d52/d44/fab 0 2024-04-24T23:14:00.099 INFO:tasks.workunit.client.1.smithi040.stdout:1/569: sync 2024-04-24T23:14:00.100 INFO:tasks.workunit.client.1.smithi040.stdout:7/609: write db/d12/f29 [214253,12250] 0 2024-04-24T23:14:00.103 INFO:tasks.workunit.client.1.smithi040.stdout:9/607: link d7/da/d17/d40/d55/d2f/fca d7/da/d17/d40/d55/d61/d6c/d8c/fd6 0 2024-04-24T23:14:00.105 INFO:tasks.workunit.client.1.smithi040.stdout:1/570: dwrite da/d12/d55/f8e [0,4194304] 0 2024-04-24T23:14:00.106 INFO:tasks.workunit.client.0.smithi031.stdout:5/774: dread d0/d6/d3b/d4b/d1d/f9d [0,4194304] 0 2024-04-24T23:14:00.108 INFO:tasks.workunit.client.1.smithi040.stdout:0/645: mkdir df/d1e/d20/d21/d41/d8a/dd3 0 2024-04-24T23:14:00.109 INFO:tasks.workunit.client.1.smithi040.stdout:6/552: write d7/d1a/d20/f3a [869621,52896] 0 2024-04-24T23:14:00.112 INFO:tasks.workunit.client.1.smithi040.stdout:3/641: mkdir d1/d2d/de/d30/dde 0 2024-04-24T23:14:00.114 INFO:tasks.workunit.client.1.smithi040.stdout:2/623: chown da/d10/d15/daa 78229342 1 2024-04-24T23:14:00.118 INFO:tasks.workunit.client.1.smithi040.stdout:0/646: dwrite df/d1e/d20/d21/d51/d79/fae [0,4194304] 0 2024-04-24T23:14:00.119 INFO:tasks.workunit.client.1.smithi040.stdout:7/610: mknod db/d43/db9/cdc 0 2024-04-24T23:14:00.121 INFO:tasks.workunit.client.1.smithi040.stdout:9/608: mknod d7/da/d17/d40/d55/d61/cd7 0 2024-04-24T23:14:00.122 INFO:tasks.workunit.client.1.smithi040.stdout:9/609: chown d7/da/d17/d40/d55/d2f/d28/f6b 264 1 2024-04-24T23:14:00.123 INFO:tasks.workunit.client.1.smithi040.stdout:4/514: truncate da/d3f/d40/d47/d71/fb3 1382188 0 2024-04-24T23:14:00.124 INFO:tasks.workunit.client.1.smithi040.stdout:4/515: fsync da/d3f/d40/d41/d62/f6f 0 2024-04-24T23:14:00.124 INFO:tasks.workunit.client.1.smithi040.stdout:4/516: fdatasync f9 0 2024-04-24T23:14:00.126 INFO:tasks.workunit.client.1.smithi040.stdout:4/517: write da/d3f/d6b/d2b/f35 [1360509,89176] 0 2024-04-24T23:14:00.127 INFO:tasks.workunit.client.1.smithi040.stdout:6/553: unlink d7/d1a/d20/d51/d1e/l41 0 2024-04-24T23:14:00.128 INFO:tasks.workunit.client.0.smithi031.stdout:2/754: link d5/d28/f54 d5/d99/f10d 0 2024-04-24T23:14:00.128 INFO:tasks.workunit.client.0.smithi031.stdout:2/755: chown d5/d3d 16595 1 2024-04-24T23:14:00.129 INFO:tasks.workunit.client.0.smithi031.stdout:2/756: fdatasync d5/d99/da2/fd1 0 2024-04-24T23:14:00.130 INFO:tasks.workunit.client.0.smithi031.stdout:6/906: rmdir d13/d15/d1d/d23/d44/da0/da6/d111 0 2024-04-24T23:14:00.131 INFO:tasks.workunit.client.0.smithi031.stdout:6/907: dread - d13/d15/d1d/ddd/f129 zero size 2024-04-24T23:14:00.135 INFO:tasks.workunit.client.1.smithi040.stdout:5/547: link d2/d7/d8/d1f/da1/fa8 d2/d62/fb5 0 2024-04-24T23:14:00.137 INFO:tasks.workunit.client.1.smithi040.stdout:2/624: dread - da/d1c/d37/d39/d3f/f88 zero size 2024-04-24T23:14:00.137 INFO:tasks.workunit.client.0.smithi031.stdout:9/918: sync 2024-04-24T23:14:00.140 INFO:tasks.workunit.client.0.smithi031.stdout:8/974: creat d8/d15/d37/d43/d7b/dc8/f149 x:0 0 0 2024-04-24T23:14:00.141 INFO:tasks.workunit.client.1.smithi040.stdout:9/610: dread d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f50 [0,4194304] 0 2024-04-24T23:14:00.144 INFO:tasks.workunit.client.0.smithi031.stdout:5/775: unlink d0/d6/d32/f5f 0 2024-04-24T23:14:00.148 INFO:tasks.workunit.client.0.smithi031.stdout:2/757: creat d5/d6/d29/d26/f10e x:0 0 0 2024-04-24T23:14:00.151 INFO:tasks.workunit.client.1.smithi040.stdout:8/604: truncate d0/d8/dc/f86 3854227 0 2024-04-24T23:14:00.152 INFO:tasks.workunit.client.1.smithi040.stdout:4/518: symlink da/d3f/d40/d41/lb6 0 2024-04-24T23:14:00.153 INFO:tasks.workunit.client.1.smithi040.stdout:4/519: stat da/d3f/d40/d41/d60/f34 0 2024-04-24T23:14:00.154 INFO:tasks.workunit.client.1.smithi040.stdout:3/642: creat d1/d9/d15/d45/daa/fdf x:0 0 0 2024-04-24T23:14:00.155 INFO:tasks.workunit.client.1.smithi040.stdout:4/520: dread da/d3f/d6b/d2b/f6d [0,4194304] 0 2024-04-24T23:14:00.155 INFO:tasks.workunit.client.1.smithi040.stdout:3/643: readlink d1/d9/d15/d45/l77 0 2024-04-24T23:14:00.156 INFO:tasks.workunit.client.1.smithi040.stdout:6/554: creat d7/d1a/d20/d51/d1e/d46/d69/d87/fad x:0 0 0 2024-04-24T23:14:00.158 INFO:tasks.workunit.client.0.smithi031.stdout:9/919: mkdir d2/d43/d9d/dc0/dd5/d119 0 2024-04-24T23:14:00.159 INFO:tasks.workunit.client.1.smithi040.stdout:5/548: creat d2/fb6 x:0 0 0 2024-04-24T23:14:00.161 INFO:tasks.workunit.client.0.smithi031.stdout:8/975: fsync d8/d15/d37/d43/daf/f120 0 2024-04-24T23:14:00.161 INFO:tasks.workunit.client.1.smithi040.stdout:6/555: dwrite d7/d1a/d20/d51/d1e/d46/d69/f6b [0,4194304] 0 2024-04-24T23:14:00.163 INFO:tasks.workunit.client.1.smithi040.stdout:2/625: mknod da/d1c/d37/d8a/ce9 0 2024-04-24T23:14:00.163 INFO:tasks.workunit.client.1.smithi040.stdout:6/556: chown d7/d1a/d20/d51/d1e/d46/d69/f8d 1859 1 2024-04-24T23:14:00.164 INFO:tasks.workunit.client.1.smithi040.stdout:2/626: truncate da/d1c/d37/fc4 731384 0 2024-04-24T23:14:00.165 INFO:tasks.workunit.client.0.smithi031.stdout:5/776: fsync d0/d6/d3b/d4b/dc/f47 0 2024-04-24T23:14:00.171 INFO:tasks.workunit.client.1.smithi040.stdout:9/611: fsync d7/da/d17/d40/d55/d61/d6c/d8c/fd6 0 2024-04-24T23:14:00.172 INFO:tasks.workunit.client.1.smithi040.stdout:9/612: chown d7/da/d17/c5f 42529 1 2024-04-24T23:14:00.172 INFO:tasks.workunit.client.0.smithi031.stdout:7/840: mkdir d1/d2/deb/d127 0 2024-04-24T23:14:00.173 INFO:tasks.workunit.client.1.smithi040.stdout:1/571: link f2 da/d12/d14/d58/fc2 0 2024-04-24T23:14:00.174 INFO:tasks.workunit.client.1.smithi040.stdout:9/613: write d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f89 [1375726,51089] 0 2024-04-24T23:14:00.177 INFO:tasks.workunit.client.0.smithi031.stdout:0/870: write da/d18/d5a/f5b [266890,117969] 0 2024-04-24T23:14:00.183 INFO:tasks.workunit.client.0.smithi031.stdout:4/933: dwrite d3/d23/d69/f13 [0,4194304] 0 2024-04-24T23:14:00.184 INFO:tasks.workunit.client.0.smithi031.stdout:8/976: rename d8/d1a/d73/l99 to d8/d15/d37/d43/d7b/dbf/dcc/l14a 0 2024-04-24T23:14:00.190 INFO:tasks.workunit.client.0.smithi031.stdout:2/758: link d5/d3d/f90 d5/d3d/f10f 0 2024-04-24T23:14:00.194 INFO:tasks.workunit.client.0.smithi031.stdout:6/908: link d13/d1b/d32/c66 d13/d1b/d32/d9c/c134 0 2024-04-24T23:14:00.196 INFO:tasks.workunit.client.1.smithi040.stdout:6/557: creat d7/fae x:0 0 0 2024-04-24T23:14:00.200 INFO:tasks.workunit.client.0.smithi031.stdout:0/871: rmdir da/d22/db0 39 2024-04-24T23:14:00.201 INFO:tasks.workunit.client.0.smithi031.stdout:0/872: chown da/d11/ffc 877361548 1 2024-04-24T23:14:00.201 INFO:tasks.workunit.client.1.smithi040.stdout:2/627: readlink da/d29/l85 0 2024-04-24T23:14:00.202 INFO:tasks.workunit.client.0.smithi031.stdout:0/873: readlink da/d86/da3/dcd/de7/ddb/le2 0 2024-04-24T23:14:00.207 INFO:tasks.workunit.client.0.smithi031.stdout:5/777: rename d0/d6/d3b/d4b/dc/d8b/d9b to d0/d6/d3b/d4b/dc/d86/dfd 0 2024-04-24T23:14:00.210 INFO:tasks.workunit.client.1.smithi040.stdout:1/572: creat da/d12/d14/fc3 x:0 0 0 2024-04-24T23:14:00.213 INFO:tasks.workunit.client.0.smithi031.stdout:7/841: creat d1/d2/deb/d127/f128 x:0 0 0 2024-04-24T23:14:00.215 INFO:tasks.workunit.client.1.smithi040.stdout:8/605: unlink d0/d16/lbb 0 2024-04-24T23:14:00.216 INFO:tasks.workunit.client.0.smithi031.stdout:6/909: symlink d13/d1b/d32/db4/d79/dd7/l135 0 2024-04-24T23:14:00.219 INFO:tasks.workunit.client.0.smithi031.stdout:0/874: mknod da/d18/d1a/d37/d41/c124 0 2024-04-24T23:14:00.220 INFO:tasks.workunit.client.0.smithi031.stdout:4/934: mkdir d3/d23/d69/d3f/d123/d12f 0 2024-04-24T23:14:00.221 INFO:tasks.workunit.client.0.smithi031.stdout:5/778: truncate d0/d6/d3b/d4b/d1d/fa4 81760 0 2024-04-24T23:14:00.224 INFO:tasks.workunit.client.0.smithi031.stdout:8/977: symlink d8/d1a/d1d/d35/l14b 0 2024-04-24T23:14:00.225 INFO:tasks.workunit.client.0.smithi031.stdout:8/978: fsync d8/d1a/d1d/d50/d142/f131 0 2024-04-24T23:14:00.225 INFO:tasks.workunit.client.1.smithi040.stdout:5/549: dwrite d2/d7/d8/d1f/d57/d27/f54 [0,4194304] 0 2024-04-24T23:14:00.226 INFO:tasks.workunit.client.0.smithi031.stdout:2/759: mknod d5/d99/d102/c110 0 2024-04-24T23:14:00.227 INFO:tasks.workunit.client.1.smithi040.stdout:0/647: getdents df/d12/d58/d5b 0 2024-04-24T23:14:00.227 INFO:tasks.workunit.client.1.smithi040.stdout:0/648: fsync df/d1e/f3e 0 2024-04-24T23:14:00.228 INFO:tasks.workunit.client.0.smithi031.stdout:3/775: sync 2024-04-24T23:14:00.228 INFO:tasks.workunit.client.0.smithi031.stdout:1/774: sync 2024-04-24T23:14:00.229 INFO:tasks.workunit.client.0.smithi031.stdout:7/842: chown d1/d2/d83 16458736 1 2024-04-24T23:14:00.231 INFO:tasks.workunit.client.1.smithi040.stdout:7/611: getdents db/d43 0 2024-04-24T23:14:00.233 INFO:tasks.workunit.client.0.smithi031.stdout:9/920: dwrite d2/d43/fe6 [0,4194304] 0 2024-04-24T23:14:00.233 INFO:tasks.workunit.client.0.smithi031.stdout:6/910: creat d13/d1b/d32/db4/d106/f136 x:0 0 0 2024-04-24T23:14:00.235 INFO:tasks.workunit.client.0.smithi031.stdout:0/875: stat da/d11/f90 0 2024-04-24T23:14:00.237 INFO:tasks.workunit.client.1.smithi040.stdout:1/573: creat da/d12/d55/fc4 x:0 0 0 2024-04-24T23:14:00.237 INFO:tasks.workunit.client.1.smithi040.stdout:2/628: dread da/d1c/f5c [0,4194304] 0 2024-04-24T23:14:00.238 INFO:tasks.workunit.client.1.smithi040.stdout:2/629: dread - da/d1c/d37/d39/d77/f68 zero size 2024-04-24T23:14:00.240 INFO:tasks.workunit.client.1.smithi040.stdout:2/630: readlink da/d1c/d37/d39/d3f/d52/d78/dc7/ldf 0 2024-04-24T23:14:00.240 INFO:tasks.workunit.client.1.smithi040.stdout:9/614: symlink d7/ld8 0 2024-04-24T23:14:00.241 INFO:tasks.workunit.client.1.smithi040.stdout:9/615: stat d7/da/c4a 0 2024-04-24T23:14:00.243 INFO:tasks.workunit.client.1.smithi040.stdout:8/606: creat d0/d7e/dc1/d9e/d90/fca x:0 0 0 2024-04-24T23:14:00.250 INFO:tasks.workunit.client.1.smithi040.stdout:0/649: dread df/d12/d1a/fad [0,4194304] 0 2024-04-24T23:14:00.251 INFO:tasks.workunit.client.0.smithi031.stdout:1/775: mkdir d1/da/d12/d65/dcf/d11/d21/d5a/dbb/dff 0 2024-04-24T23:14:00.253 INFO:tasks.workunit.client.0.smithi031.stdout:6/911: dread - d13/d15/fcf zero size 2024-04-24T23:14:00.261 INFO:tasks.workunit.client.1.smithi040.stdout:7/612: unlink db/d12/d20/d52/d4b/c82 0 2024-04-24T23:14:00.263 INFO:tasks.workunit.client.0.smithi031.stdout:4/935: dread d3/d20/d25/d60/fc0 [0,4194304] 0 2024-04-24T23:14:00.266 INFO:tasks.workunit.client.1.smithi040.stdout:9/616: read d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f78 [945090,39408] 0 2024-04-24T23:14:00.267 INFO:tasks.workunit.client.1.smithi040.stdout:9/617: chown d7/da/l13 86870466 1 2024-04-24T23:14:00.267 INFO:tasks.workunit.client.0.smithi031.stdout:7/843: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/f69 [1932216,91572] 0 2024-04-24T23:14:00.268 INFO:tasks.workunit.client.1.smithi040.stdout:9/618: write d7/da/d17/d40/d55/d61/d6c/f91 [328792,17062] 0 2024-04-24T23:14:00.269 INFO:tasks.workunit.client.0.smithi031.stdout:2/760: dwrite d5/d99/fdb [0,4194304] 0 2024-04-24T23:14:00.271 INFO:tasks.workunit.client.1.smithi040.stdout:4/521: rename da/d1f/c4c to da/cb7 0 2024-04-24T23:14:00.272 INFO:tasks.workunit.client.1.smithi040.stdout:4/522: write da/f2f [1297214,52051] 0 2024-04-24T23:14:00.272 INFO:tasks.workunit.client.1.smithi040.stdout:5/550: write d2/d7/d8/f13 [1571636,27922] 0 2024-04-24T23:14:00.272 INFO:tasks.workunit.client.1.smithi040.stdout:4/523: readlink da/d3f/l54 0 2024-04-24T23:14:00.273 INFO:tasks.workunit.client.0.smithi031.stdout:8/979: fsync d8/d1a/d1d/f78 0 2024-04-24T23:14:00.274 INFO:tasks.workunit.client.1.smithi040.stdout:3/644: getdents d1/d2a/d8d/d7c 0 2024-04-24T23:14:00.274 INFO:tasks.workunit.client.1.smithi040.stdout:4/524: stat da/d3f/d6b/d2b/l89 0 2024-04-24T23:14:00.274 INFO:tasks.workunit.client.0.smithi031.stdout:2/761: write d5/d6/d29/d26/db0/dc2/fd0 [1568909,124959] 0 2024-04-24T23:14:00.275 INFO:tasks.workunit.client.0.smithi031.stdout:7/844: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/c71 815 1 2024-04-24T23:14:00.276 INFO:tasks.workunit.client.1.smithi040.stdout:6/558: read d7/d1a/f53 [342346,95040] 0 2024-04-24T23:14:00.276 INFO:tasks.workunit.client.0.smithi031.stdout:9/921: dread d2/dae/d14/d1d/f7b [0,4194304] 0 2024-04-24T23:14:00.277 INFO:tasks.workunit.client.1.smithi040.stdout:1/574: write da/d35/f9f [434538,99729] 0 2024-04-24T23:14:00.277 INFO:tasks.workunit.client.0.smithi031.stdout:8/980: write d8/d15/d37/d43/d7b/fe9 [208938,43910] 0 2024-04-24T23:14:00.278 INFO:tasks.workunit.client.0.smithi031.stdout:9/922: chown d2/dae/d14/d78/d84/d4a/dab 165615 1 2024-04-24T23:14:00.279 INFO:tasks.workunit.client.1.smithi040.stdout:1/575: fdatasync da/d12/d23/f30 0 2024-04-24T23:14:00.280 INFO:tasks.workunit.client.0.smithi031.stdout:6/912: mknod d13/d15/d1d/ddd/c137 0 2024-04-24T23:14:00.281 INFO:tasks.workunit.client.0.smithi031.stdout:9/923: stat d2/dae/d14/d78/d84/d4a/dab/db7/df3 0 2024-04-24T23:14:00.282 INFO:tasks.workunit.client.0.smithi031.stdout:3/776: mkdir d6/df/d16/d2a/d95/d9b/d29/db7/de9 0 2024-04-24T23:14:00.284 INFO:tasks.workunit.client.1.smithi040.stdout:6/559: dwrite d7/d1a/d20/d51/d1e/f72 [0,4194304] 0 2024-04-24T23:14:00.284 INFO:tasks.workunit.client.0.smithi031.stdout:3/777: stat d6/d14/d1d/f9d 0 2024-04-24T23:14:00.284 INFO:tasks.workunit.client.0.smithi031.stdout:9/924: chown d2/dae/f13 1086 1 2024-04-24T23:14:00.285 INFO:tasks.workunit.client.0.smithi031.stdout:7/845: dwrite d1/f13 [4194304,4194304] 0 2024-04-24T23:14:00.290 INFO:tasks.workunit.client.1.smithi040.stdout:2/631: rename da/d1c/d37/d8a/f9a to da/d1c/d37/d39/d79/da3/fea 0 2024-04-24T23:14:00.291 INFO:tasks.workunit.client.0.smithi031.stdout:5/779: mknod d0/d6/d3b/d4b/d1d/d2e/dc6/cfe 0 2024-04-24T23:14:00.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/846: dread d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/fa5 [0,4194304] 0 2024-04-24T23:14:00.302 INFO:tasks.workunit.client.0.smithi031.stdout:5/780: dwrite d0/d6/d3b/d4b/dc/fea [0,4194304] 0 2024-04-24T23:14:00.343 INFO:tasks.workunit.client.1.smithi040.stdout:8/607: sync 2024-04-24T23:14:00.436 INFO:tasks.workunit.client.1.smithi040.stdout:4/525: mkdir da/d3f/d40/d41/d8e/db8 0 2024-04-24T23:14:00.439 INFO:tasks.workunit.client.0.smithi031.stdout:4/936: write d3/d20/d2d/d44/d5b/d5d/f65 [4820831,76708] 0 2024-04-24T23:14:00.441 INFO:tasks.workunit.client.0.smithi031.stdout:4/937: write d3/d20/d2d/fbe [1274839,42068] 0 2024-04-24T23:14:00.448 INFO:tasks.workunit.client.1.smithi040.stdout:1/576: symlink da/d35/da3/lc5 0 2024-04-24T23:14:00.448 INFO:tasks.workunit.client.1.smithi040.stdout:0/650: truncate df/d1e/d20/f43 4222018 0 2024-04-24T23:14:00.451 INFO:tasks.workunit.client.1.smithi040.stdout:0/651: write df/d1e/d20/d21/d51/f8c [1493666,74114] 0 2024-04-24T23:14:00.453 INFO:tasks.workunit.client.1.smithi040.stdout:5/551: dwrite d2/d7/d8/d64/f93 [0,4194304] 0 2024-04-24T23:14:00.462 INFO:tasks.workunit.client.0.smithi031.stdout:4/938: dread d3/d23/d69/f30 [0,4194304] 0 2024-04-24T23:14:00.462 INFO:tasks.workunit.client.1.smithi040.stdout:7/613: symlink db/d12/d62/d8a/db3/dbf/ldd 0 2024-04-24T23:14:00.466 INFO:tasks.workunit.client.1.smithi040.stdout:6/560: creat d7/d1a/d56/faf x:0 0 0 2024-04-24T23:14:00.469 INFO:tasks.workunit.client.1.smithi040.stdout:9/619: symlink d7/d92/d9c/dc2/ld9 0 2024-04-24T23:14:00.471 INFO:tasks.workunit.client.1.smithi040.stdout:0/652: dread df/d1e/f3e [0,4194304] 0 2024-04-24T23:14:00.473 INFO:tasks.workunit.client.1.smithi040.stdout:2/632: chown da/f19 182566679 1 2024-04-24T23:14:00.481 INFO:tasks.workunit.client.1.smithi040.stdout:8/608: mknod d0/d8/dc/d34/ccb 0 2024-04-24T23:14:00.482 INFO:tasks.workunit.client.0.smithi031.stdout:7/847: dread d1/d2/f12 [0,4194304] 0 2024-04-24T23:14:00.485 INFO:tasks.workunit.client.1.smithi040.stdout:4/526: rmdir da/d3f/d40/d47 39 2024-04-24T23:14:00.486 INFO:tasks.workunit.client.1.smithi040.stdout:4/527: read - da/d3f/d6b/d3c/d69/fa1 zero size 2024-04-24T23:14:00.487 INFO:tasks.workunit.client.1.smithi040.stdout:4/528: stat da/d3f/d40/d9b 0 2024-04-24T23:14:00.491 INFO:tasks.workunit.client.0.smithi031.stdout:7/848: dread d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/d126/fd4 [0,4194304] 0 2024-04-24T23:14:00.492 INFO:tasks.workunit.client.0.smithi031.stdout:7/849: fdatasync d1/d2/dd/d33/d5e/d64/d6d/dae/db1/f109 0 2024-04-24T23:14:00.511 INFO:tasks.workunit.client.0.smithi031.stdout:7/850: dread d1/d2/dd/d33/d5e/d64/d6d/de1/f5b [0,4194304] 0 2024-04-24T23:14:00.512 INFO:tasks.workunit.client.0.smithi031.stdout:0/876: getdents da/d11/d6a/de8 0 2024-04-24T23:14:00.514 INFO:tasks.workunit.client.0.smithi031.stdout:7/851: chown d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea/f103 20092 1 2024-04-24T23:14:00.515 INFO:tasks.workunit.client.0.smithi031.stdout:4/939: creat d3/d20/d25/d60/dc8/f130 x:0 0 0 2024-04-24T23:14:00.526 INFO:tasks.workunit.client.0.smithi031.stdout:9/925: truncate d2/dae/d14/d1d/d45/f5e 205225 0 2024-04-24T23:14:00.526 INFO:tasks.workunit.client.0.smithi031.stdout:0/877: dread da/d22/f27 [0,4194304] 0 2024-04-24T23:14:00.550 INFO:tasks.workunit.client.0.smithi031.stdout:4/940: dread d3/d20/d25/d7e/fc9 [0,4194304] 0 2024-04-24T23:14:00.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/778: getdents d6/df/d16 0 2024-04-24T23:14:00.570 INFO:tasks.workunit.client.0.smithi031.stdout:0/878: dread da/d18/d1a/f1f [0,4194304] 0 2024-04-24T23:14:00.571 INFO:tasks.workunit.client.0.smithi031.stdout:0/879: dread - da/d20/d8e/f11f zero size 2024-04-24T23:14:00.575 INFO:tasks.workunit.client.0.smithi031.stdout:3/779: rename d6/df/d16/d2a/d95/d9b/d29/f79 to d6/df/d16/d2a/d95/d9b/d29/fea 0 2024-04-24T23:14:00.578 INFO:tasks.workunit.client.0.smithi031.stdout:4/941: creat d3/d20/d25/d60/d63/f131 x:0 0 0 2024-04-24T23:14:00.580 INFO:tasks.workunit.client.0.smithi031.stdout:0/880: dread - da/d11/d6a/dba/ff7 zero size 2024-04-24T23:14:00.582 INFO:tasks.workunit.client.1.smithi040.stdout:1/577: creat da/d12/d14/d58/fc6 x:0 0 0 2024-04-24T23:14:00.586 INFO:tasks.workunit.client.0.smithi031.stdout:4/942: creat d3/d23/d69/d3f/d9e/ddb/f132 x:0 0 0 2024-04-24T23:14:00.597 INFO:tasks.workunit.client.1.smithi040.stdout:6/561: read - d7/d1a/d56/f7f zero size 2024-04-24T23:14:00.606 INFO:tasks.workunit.client.0.smithi031.stdout:3/780: getdents d6/df/d16/d2a/d95 0 2024-04-24T23:14:00.609 INFO:tasks.workunit.client.0.smithi031.stdout:4/943: dread d3/d5e/dc4/ff7 [0,4194304] 0 2024-04-24T23:14:00.614 INFO:tasks.workunit.client.0.smithi031.stdout:3/781: unlink d6/d14/d1d/d69/fa7 0 2024-04-24T23:14:00.614 INFO:tasks.workunit.client.1.smithi040.stdout:4/529: dread f5 [0,4194304] 0 2024-04-24T23:14:00.615 INFO:tasks.workunit.client.1.smithi040.stdout:4/530: fsync da/d3f/d40/fb2 0 2024-04-24T23:14:00.619 INFO:tasks.workunit.client.1.smithi040.stdout:6/562: dread d7/d1a/d20/d51/d1e/d5d/f7b [0,4194304] 0 2024-04-24T23:14:00.625 INFO:tasks.workunit.client.0.smithi031.stdout:3/782: rename d6/df/d16/d2a/d95/d9b/d29/l85 to d6/df/d16/leb 0 2024-04-24T23:14:00.631 INFO:tasks.workunit.client.0.smithi031.stdout:3/783: dwrite d6/d14/d1d/fcd [0,4194304] 0 2024-04-24T23:14:00.632 INFO:tasks.workunit.client.0.smithi031.stdout:8/981: sync 2024-04-24T23:14:00.634 INFO:tasks.workunit.client.0.smithi031.stdout:4/944: creat d3/d23/d69/d3f/dea/f133 x:0 0 0 2024-04-24T23:14:00.640 INFO:tasks.workunit.client.1.smithi040.stdout:4/531: creat da/d3f/d8d/fb9 x:0 0 0 2024-04-24T23:14:00.645 INFO:tasks.workunit.client.1.smithi040.stdout:6/563: mkdir d7/d1a/d20/d8c/db0 0 2024-04-24T23:14:00.650 INFO:tasks.workunit.client.1.smithi040.stdout:6/564: dwrite d7/d1a/d20/f3a [0,4194304] 0 2024-04-24T23:14:00.651 INFO:tasks.workunit.client.0.smithi031.stdout:2/762: creat d5/d28/d4e/f111 x:0 0 0 2024-04-24T23:14:00.651 INFO:tasks.workunit.client.0.smithi031.stdout:4/945: rename d3/d5e/dc2/cfe to d3/d20/d25/d60/c134 0 2024-04-24T23:14:00.652 INFO:tasks.workunit.client.1.smithi040.stdout:6/565: write d7/d1a/d20/d51/d1e/f86 [660774,14438] 0 2024-04-24T23:14:00.654 INFO:tasks.workunit.client.0.smithi031.stdout:8/982: creat d8/d15/d37/d43/d5c/f14c x:0 0 0 2024-04-24T23:14:00.657 INFO:tasks.workunit.client.0.smithi031.stdout:2/763: truncate d5/d6/d68/f6b 690851 0 2024-04-24T23:14:00.662 INFO:tasks.workunit.client.1.smithi040.stdout:8/609: getdents d0/d8/d22 0 2024-04-24T23:14:00.663 INFO:tasks.workunit.client.1.smithi040.stdout:8/610: chown d0/d38/d4c/cb0 248 1 2024-04-24T23:14:00.663 INFO:tasks.workunit.client.1.smithi040.stdout:1/578: getdents da/d12/d14/d25/d5e 0 2024-04-24T23:14:00.664 INFO:tasks.workunit.client.1.smithi040.stdout:8/611: write d0/d7e/dc1/d9e/d90/fca [218283,21737] 0 2024-04-24T23:14:00.669 INFO:tasks.workunit.client.1.smithi040.stdout:1/579: dwrite da/d12/d14/d25/d5e/fa4 [0,4194304] 0 2024-04-24T23:14:00.670 INFO:tasks.workunit.client.0.smithi031.stdout:8/983: symlink d8/d1a/d1d/d35/l14d 0 2024-04-24T23:14:00.671 INFO:tasks.workunit.client.1.smithi040.stdout:8/612: dread - d0/d16/d19/fac zero size 2024-04-24T23:14:00.673 INFO:tasks.workunit.client.1.smithi040.stdout:8/613: write d0/d7e/dc1/d9e/d90/fca [120777,100913] 0 2024-04-24T23:14:00.680 INFO:tasks.workunit.client.0.smithi031.stdout:2/764: dread d5/d8/d15/d56/d84/d75/f91 [0,4194304] 0 2024-04-24T23:14:00.683 INFO:tasks.workunit.client.1.smithi040.stdout:8/614: dwrite d0/d16/d1b/d58/fc6 [0,4194304] 0 2024-04-24T23:14:00.691 INFO:tasks.workunit.client.0.smithi031.stdout:1/776: write d1/da/d12/d65/dcf/fe [72661,41132] 0 2024-04-24T23:14:00.713 INFO:tasks.workunit.client.1.smithi040.stdout:8/615: unlink d0/l1c 0 2024-04-24T23:14:00.715 INFO:tasks.workunit.client.1.smithi040.stdout:8/616: symlink d0/d16/d1b/d3f/d44/lcc 0 2024-04-24T23:14:00.718 INFO:tasks.workunit.client.1.smithi040.stdout:8/617: creat d0/d7e/dc1/d77/fcd x:0 0 0 2024-04-24T23:14:00.723 INFO:tasks.workunit.client.0.smithi031.stdout:1/777: rmdir d1/d3e 39 2024-04-24T23:14:00.730 INFO:tasks.workunit.client.0.smithi031.stdout:1/778: link d1/d33/d41/d9b/da8/fa9 d1/da/d12/d65/dcf/f100 0 2024-04-24T23:14:00.730 INFO:tasks.workunit.client.0.smithi031.stdout:6/913: dwrite d13/d15/d1d/d23/d44/d47/f4a [0,4194304] 0 2024-04-24T23:14:00.735 INFO:tasks.workunit.client.1.smithi040.stdout:8/618: dread d0/d16/d1b/d3f/fa8 [0,4194304] 0 2024-04-24T23:14:00.736 INFO:tasks.workunit.client.1.smithi040.stdout:8/619: dread - d0/d8/d66/fc0 zero size 2024-04-24T23:14:00.736 INFO:tasks.workunit.client.0.smithi031.stdout:6/914: fdatasync d13/d15/d1d/d23/d24/d8b/ff3 0 2024-04-24T23:14:00.737 INFO:tasks.workunit.client.1.smithi040.stdout:8/620: chown d0/d16/d19/d71/d76 17615747 1 2024-04-24T23:14:00.737 INFO:tasks.workunit.client.0.smithi031.stdout:6/915: chown d13/d15/d1d/f130 881484851 1 2024-04-24T23:14:00.738 INFO:tasks.workunit.client.0.smithi031.stdout:6/916: chown d13/d1b/f6b 50595 1 2024-04-24T23:14:00.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:00 smithi031 ceph-mon[26411]: pgmap v33: 65 pgs: 65 active+clean; 2.6 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 62 MiB/s rd, 184 MiB/s wr, 384 op/s 2024-04-24T23:14:00.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:00 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:00.753 INFO:tasks.workunit.client.0.smithi031.stdout:7/852: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d84/f129 x:0 0 0 2024-04-24T23:14:00.756 INFO:tasks.workunit.client.0.smithi031.stdout:7/853: truncate d1/d2/dd/d33/d5e/d64/d6d/de1/d84/f129 259067 0 2024-04-24T23:14:00.757 INFO:tasks.workunit.client.0.smithi031.stdout:9/926: creat d2/dae/f11a x:0 0 0 2024-04-24T23:14:00.761 INFO:tasks.workunit.client.0.smithi031.stdout:9/927: truncate d2/dae/d14/d1d/d45/fd8 101024 0 2024-04-24T23:14:00.762 INFO:tasks.workunit.client.0.smithi031.stdout:9/928: chown d2/d43/d9d/dc0/ce5 6219232 1 2024-04-24T23:14:00.763 INFO:tasks.workunit.client.0.smithi031.stdout:0/881: write da/d18/f85 [85588,8165] 0 2024-04-24T23:14:00.764 INFO:tasks.workunit.client.0.smithi031.stdout:9/929: symlink d2/dae/d14/d1d/d45/d65/l11b 0 2024-04-24T23:14:00.767 INFO:tasks.workunit.client.1.smithi040.stdout:9/620: mknod d7/da/d17/d40/d55/cda 0 2024-04-24T23:14:00.768 INFO:tasks.workunit.client.0.smithi031.stdout:0/882: dread da/d22/f30 [0,4194304] 0 2024-04-24T23:14:00.769 INFO:tasks.workunit.client.0.smithi031.stdout:9/930: creat d2/dae/d14/d78/d84/d7e/dc7/d107/f11c x:0 0 0 2024-04-24T23:14:00.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/883: chown da/d18/d1a/d37/d111 102240 1 2024-04-24T23:14:00.769 INFO:tasks.workunit.client.1.smithi040.stdout:1/580: rmdir da/d12 39 2024-04-24T23:14:00.771 INFO:tasks.workunit.client.1.smithi040.stdout:0/653: mkdir df/dd4 0 2024-04-24T23:14:00.772 INFO:tasks.workunit.client.0.smithi031.stdout:9/931: truncate d2/dae/f10 2872161 0 2024-04-24T23:14:00.773 INFO:tasks.workunit.client.1.smithi040.stdout:9/621: dwrite d7/da/d17/d40/d55/d2f/d1f/fc5 [0,4194304] 0 2024-04-24T23:14:00.774 INFO:tasks.workunit.client.1.smithi040.stdout:9/622: fsync d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f5b 0 2024-04-24T23:14:00.775 INFO:tasks.workunit.client.0.smithi031.stdout:2/765: creat d5/d8/f112 x:0 0 0 2024-04-24T23:14:00.776 INFO:tasks.workunit.client.0.smithi031.stdout:2/766: chown d5/d8/d45/d5b/d49/d2f/faa 0 1 2024-04-24T23:14:00.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/767: write d5/d99/da2/fd1 [4129652,122316] 0 2024-04-24T23:14:00.777 INFO:tasks.workunit.client.0.smithi031.stdout:9/932: readlink d2/d16/l1e 0 2024-04-24T23:14:00.781 INFO:tasks.workunit.client.0.smithi031.stdout:0/884: creat da/d86/f125 x:0 0 0 2024-04-24T23:14:00.784 INFO:tasks.workunit.client.1.smithi040.stdout:9/623: symlink d7/da/d17/d40/d55/d61/d6c/d8c/ldb 0 2024-04-24T23:14:00.785 INFO:tasks.workunit.client.0.smithi031.stdout:5/781: mknod d0/d6/d3b/d4b/dc/cff 0 2024-04-24T23:14:00.787 INFO:tasks.workunit.client.1.smithi040.stdout:0/654: symlink df/ld5 0 2024-04-24T23:14:00.787 INFO:tasks.workunit.client.1.smithi040.stdout:9/624: dread d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f9d [0,4194304] 0 2024-04-24T23:14:00.790 INFO:tasks.workunit.client.1.smithi040.stdout:7/614: unlink db/d12/c3b 0 2024-04-24T23:14:00.790 INFO:tasks.workunit.client.0.smithi031.stdout:0/885: symlink da/d20/d4d/df8/l126 0 2024-04-24T23:14:00.791 INFO:tasks.workunit.client.1.smithi040.stdout:7/615: chown db/d12/d20/d52/d4b/c89 220343406 1 2024-04-24T23:14:00.792 INFO:tasks.workunit.client.0.smithi031.stdout:0/886: dread - da/d20/d4e/f110 zero size 2024-04-24T23:14:00.792 INFO:tasks.workunit.client.1.smithi040.stdout:9/625: dread d7/da/d17/d40/d55/d2f/d1f/d31/f98 [0,4194304] 0 2024-04-24T23:14:00.792 INFO:tasks.workunit.client.1.smithi040.stdout:7/616: chown db/d2f/d37/d7e/f7f 899567 1 2024-04-24T23:14:00.794 INFO:tasks.workunit.client.0.smithi031.stdout:9/933: mkdir d2/d16/de0/d11d 0 2024-04-24T23:14:00.796 INFO:tasks.workunit.client.0.smithi031.stdout:9/934: fdatasync d2/dae/d14/d78/d84/d4a/dab/db7/f104 0 2024-04-24T23:14:00.800 INFO:tasks.workunit.client.0.smithi031.stdout:3/784: stat d6/df/d16/d2a/f5a 0 2024-04-24T23:14:00.801 INFO:tasks.workunit.client.0.smithi031.stdout:2/768: dread d5/d6/d29/f61 [0,4194304] 0 2024-04-24T23:14:00.801 INFO:tasks.workunit.client.1.smithi040.stdout:0/655: dread f6 [0,4194304] 0 2024-04-24T23:14:00.802 INFO:tasks.workunit.client.1.smithi040.stdout:1/581: creat da/d12/d18/d2a/fc7 x:0 0 0 2024-04-24T23:14:00.803 INFO:tasks.workunit.client.1.smithi040.stdout:2/633: dwrite da/d1c/d37/d39/d3f/d52/f87 [0,4194304] 0 2024-04-24T23:14:00.803 INFO:tasks.workunit.client.0.smithi031.stdout:0/887: truncate da/d11/d6a/dba/dc0/fe3 419631 0 2024-04-24T23:14:00.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/582: stat da/d12/d18/d2a/d8b/d97/f98 0 2024-04-24T23:14:00.805 INFO:tasks.workunit.client.1.smithi040.stdout:9/626: mknod d7/d92/d97/cdc 0 2024-04-24T23:14:00.807 INFO:tasks.workunit.client.1.smithi040.stdout:7/617: read db/f24 [527903,110583] 0 2024-04-24T23:14:00.810 INFO:tasks.workunit.client.1.smithi040.stdout:1/583: dwrite da/d12/d14/fc3 [0,4194304] 0 2024-04-24T23:14:00.814 INFO:tasks.workunit.client.1.smithi040.stdout:0/656: symlink df/d1e/d20/d21/d51/d79/daa/ld6 0 2024-04-24T23:14:00.816 INFO:tasks.workunit.client.0.smithi031.stdout:9/935: mkdir d2/dae/d14/d78/d84/d4a/d4b/daf/df5/d11e 0 2024-04-24T23:14:00.818 INFO:tasks.workunit.client.1.smithi040.stdout:0/657: dread df/d1e/d2b/db6/fbd [0,4194304] 0 2024-04-24T23:14:00.819 INFO:tasks.workunit.client.1.smithi040.stdout:2/634: rename da/d1c/d37/d8a/f91 to da/d10/d15/daa/feb 0 2024-04-24T23:14:00.819 INFO:tasks.workunit.client.1.smithi040.stdout:1/584: dread da/d12/d18/d2a/d41/f62 [0,4194304] 0 2024-04-24T23:14:00.822 INFO:tasks.workunit.client.1.smithi040.stdout:9/627: symlink d7/da/d17/d40/d55/d2f/d28/d67/ldd 0 2024-04-24T23:14:00.822 INFO:tasks.workunit.client.1.smithi040.stdout:9/628: chown d7/da/f39 140 1 2024-04-24T23:14:00.826 INFO:tasks.workunit.client.1.smithi040.stdout:4/532: mknod da/d45/d48/cba 0 2024-04-24T23:14:00.829 INFO:tasks.workunit.client.0.smithi031.stdout:4/946: dwrite d3/d5e/fda [0,4194304] 0 2024-04-24T23:14:00.832 INFO:tasks.workunit.client.1.smithi040.stdout:6/566: dwrite d7/d1a/d20/f28 [0,4194304] 0 2024-04-24T23:14:00.833 INFO:tasks.workunit.client.1.smithi040.stdout:0/658: rename df/d1e/f2f to df/d1e/d89/fd7 0 2024-04-24T23:14:00.834 INFO:tasks.workunit.client.1.smithi040.stdout:3/645: creat d1/d2a/d6a/fe0 x:0 0 0 2024-04-24T23:14:00.837 INFO:tasks.workunit.client.1.smithi040.stdout:2/635: truncate da/d1c/d37/d39/f4e 981107 0 2024-04-24T23:14:00.838 INFO:tasks.workunit.client.1.smithi040.stdout:2/636: chown da/d10 17152392 1 2024-04-24T23:14:00.840 INFO:tasks.workunit.client.1.smithi040.stdout:7/618: symlink db/d12/lde 0 2024-04-24T23:14:00.840 INFO:tasks.workunit.client.0.smithi031.stdout:0/888: creat da/d22/d62/f127 x:0 0 0 2024-04-24T23:14:00.841 INFO:tasks.workunit.client.0.smithi031.stdout:0/889: fdatasync da/ffd 0 2024-04-24T23:14:00.842 INFO:tasks.workunit.client.1.smithi040.stdout:7/619: write db/d12/d62/fbd [265804,17124] 0 2024-04-24T23:14:00.842 INFO:tasks.workunit.client.0.smithi031.stdout:0/890: chown da/d20/d4d/df8/c108 36299 1 2024-04-24T23:14:00.843 INFO:tasks.workunit.client.1.smithi040.stdout:4/533: mkdir da/d78/dbb 0 2024-04-24T23:14:00.843 INFO:tasks.workunit.client.0.smithi031.stdout:4/947: mknod d3/d20/da7/c135 0 2024-04-24T23:14:00.844 INFO:tasks.workunit.client.1.smithi040.stdout:2/637: dread da/d1c/d37/d39/d3f/d52/f87 [0,4194304] 0 2024-04-24T23:14:00.845 INFO:tasks.workunit.client.0.smithi031.stdout:9/936: symlink d2/d43/d9d/dc0/dd5/d119/l11f 0 2024-04-24T23:14:00.847 INFO:tasks.workunit.client.1.smithi040.stdout:9/629: dread d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [0,4194304] 0 2024-04-24T23:14:00.848 INFO:tasks.workunit.client.1.smithi040.stdout:5/552: rmdir d2/d7/d8/d1f/d57/d27/d3c/d43 39 2024-04-24T23:14:00.850 INFO:tasks.workunit.client.1.smithi040.stdout:0/659: read df/d1e/da7/f69 [695877,41495] 0 2024-04-24T23:14:00.851 INFO:tasks.workunit.client.0.smithi031.stdout:6/917: creat d13/d15/d1d/f138 x:0 0 0 2024-04-24T23:14:00.851 INFO:tasks.workunit.client.0.smithi031.stdout:0/891: rmdir da/d18/d1a/dd6 39 2024-04-24T23:14:00.853 INFO:tasks.workunit.client.1.smithi040.stdout:6/567: mknod d7/d1a/d20/d51/d1e/d46/d85/cb1 0 2024-04-24T23:14:00.855 INFO:tasks.workunit.client.0.smithi031.stdout:8/984: unlink d8/l10 0 2024-04-24T23:14:00.858 INFO:tasks.workunit.client.0.smithi031.stdout:9/937: dread d2/d16/dbe/d7d/fac [0,4194304] 0 2024-04-24T23:14:00.859 INFO:tasks.workunit.client.0.smithi031.stdout:6/918: creat d13/d15/d1d/d23/d24/d43/d68/f139 x:0 0 0 2024-04-24T23:14:00.863 INFO:tasks.workunit.client.1.smithi040.stdout:1/585: dread f2 [0,4194304] 0 2024-04-24T23:14:00.864 INFO:tasks.workunit.client.0.smithi031.stdout:4/948: mkdir d3/d20/d25/d60/d63/d136 0 2024-04-24T23:14:00.869 INFO:tasks.workunit.client.1.smithi040.stdout:2/638: unlink da/d10/d15/f2c 0 2024-04-24T23:14:00.873 INFO:tasks.workunit.client.0.smithi031.stdout:6/919: creat d13/d1b/d32/db4/d79/dd7/f13a x:0 0 0 2024-04-24T23:14:00.873 INFO:tasks.workunit.client.1.smithi040.stdout:9/630: chown d7/da/l69 3863 1 2024-04-24T23:14:00.874 INFO:tasks.workunit.client.0.smithi031.stdout:6/920: chown d13/d15/d1d/d23/d2d/c7e 1562032038 1 2024-04-24T23:14:00.874 INFO:tasks.workunit.client.1.smithi040.stdout:5/553: fdatasync d2/d62/fb5 0 2024-04-24T23:14:00.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/779: write d1/da/d10/d52/fdf [1356062,76564] 0 2024-04-24T23:14:00.877 INFO:tasks.workunit.client.1.smithi040.stdout:7/620: dread db/d12/d20/d52/d4b/fa1 [0,4194304] 0 2024-04-24T23:14:00.878 INFO:tasks.workunit.client.1.smithi040.stdout:8/621: write d0/d8/f29 [483372,53088] 0 2024-04-24T23:14:00.878 INFO:tasks.workunit.client.1.smithi040.stdout:0/660: dread df/d12/d1a/f4e [0,4194304] 0 2024-04-24T23:14:00.879 INFO:tasks.workunit.client.1.smithi040.stdout:8/622: chown d0/d8/d55/f83 6543 1 2024-04-24T23:14:00.879 INFO:tasks.workunit.client.1.smithi040.stdout:0/661: write df/d1e/d20/d61/fb0 [564794,105381] 0 2024-04-24T23:14:00.880 INFO:tasks.workunit.client.1.smithi040.stdout:8/623: dread - d0/d8/d66/fc0 zero size 2024-04-24T23:14:00.880 INFO:tasks.workunit.client.1.smithi040.stdout:6/568: chown d7/d1a/d20/d51/d1e/d5d/d52/c4d 24 1 2024-04-24T23:14:00.881 INFO:tasks.workunit.client.1.smithi040.stdout:8/624: fdatasync d0/d7e/dc1/f6c 0 2024-04-24T23:14:00.882 INFO:tasks.workunit.client.0.smithi031.stdout:9/938: mkdir d2/dae/d14/d78/d84/d4a/d4b/daf/df5/d11e/d120 0 2024-04-24T23:14:00.884 INFO:tasks.workunit.client.1.smithi040.stdout:3/646: link d1/d2a/d8d/fca d1/d2a/d8d/d7c/fe1 0 2024-04-24T23:14:00.886 INFO:tasks.workunit.client.0.smithi031.stdout:7/854: write d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f5d [664904,129193] 0 2024-04-24T23:14:00.886 INFO:tasks.workunit.client.0.smithi031.stdout:0/892: creat da/d22/d62/f128 x:0 0 0 2024-04-24T23:14:00.888 INFO:tasks.workunit.client.0.smithi031.stdout:7/855: dread - d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/f11d zero size 2024-04-24T23:14:00.889 INFO:tasks.workunit.client.0.smithi031.stdout:6/921: symlink d13/d1b/de3/d118/l13b 0 2024-04-24T23:14:00.893 INFO:tasks.workunit.client.1.smithi040.stdout:1/586: rename da/d12/d18/d2a/f3d to da/d12/d14/d25/db9/fc8 0 2024-04-24T23:14:00.894 INFO:tasks.workunit.client.1.smithi040.stdout:1/587: truncate da/d35/d38/d47/f84 970564 0 2024-04-24T23:14:00.895 INFO:tasks.workunit.client.0.smithi031.stdout:1/780: write d1/d33/d41/d9b/fcd [2070687,88508] 0 2024-04-24T23:14:00.896 INFO:tasks.workunit.client.1.smithi040.stdout:1/588: write da/d12/d18/d2a/fc7 [275593,126949] 0 2024-04-24T23:14:00.896 INFO:tasks.workunit.client.0.smithi031.stdout:8/985: link d8/l30 d8/d15/d37/d43/d7b/dbf/l14e 0 2024-04-24T23:14:00.903 INFO:tasks.workunit.client.0.smithi031.stdout:8/986: dwrite d8/d15/f123 [0,4194304] 0 2024-04-24T23:14:00.904 INFO:tasks.workunit.client.1.smithi040.stdout:9/631: creat d7/da/d17/d40/d55/d61/d6c/d8c/fde x:0 0 0 2024-04-24T23:14:00.907 INFO:tasks.workunit.client.0.smithi031.stdout:0/893: creat da/d18/d5a/d7d/dbd/f129 x:0 0 0 2024-04-24T23:14:00.909 INFO:tasks.workunit.client.1.smithi040.stdout:5/554: chown d2/d7/d8/d1f/d57/d27/d3c/d43/c85 50239 1 2024-04-24T23:14:00.911 INFO:tasks.workunit.client.1.smithi040.stdout:0/662: symlink df/d1e/d20/d21/d51/d79/ld8 0 2024-04-24T23:14:00.911 INFO:tasks.workunit.client.0.smithi031.stdout:5/782: write d0/d6/d3b/d4b/dc/f84 [4357711,27970] 0 2024-04-24T23:14:00.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:00 smithi040 ceph-mon[33128]: pgmap v33: 65 pgs: 65 active+clean; 2.6 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 62 MiB/s rd, 184 MiB/s wr, 384 op/s 2024-04-24T23:14:00.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:00 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:00.913 INFO:tasks.workunit.client.0.smithi031.stdout:1/781: truncate d1/da/d12/d65/dcf/f70 231574 0 2024-04-24T23:14:00.915 INFO:tasks.workunit.client.1.smithi040.stdout:6/569: chown d7/d1a/d20/d51/d1e/d5d/d52/c82 13025988 1 2024-04-24T23:14:00.916 INFO:tasks.workunit.client.1.smithi040.stdout:8/625: symlink d0/d38/d4c/lce 0 2024-04-24T23:14:00.918 INFO:tasks.workunit.client.1.smithi040.stdout:3/647: read - d1/d9/d18/d36/d55/fa9 zero size 2024-04-24T23:14:00.922 INFO:tasks.workunit.client.0.smithi031.stdout:0/894: readlink da/d86/da3/d104/lcc 0 2024-04-24T23:14:00.923 INFO:tasks.workunit.client.0.smithi031.stdout:3/785: truncate d6/fc4 404597 0 2024-04-24T23:14:00.925 INFO:tasks.workunit.client.0.smithi031.stdout:6/922: mknod d13/d1b/d32/db4/d122/c13c 0 2024-04-24T23:14:00.925 INFO:tasks.workunit.client.0.smithi031.stdout:2/769: truncate d5/d28/d43/f97 1528382 0 2024-04-24T23:14:00.928 INFO:tasks.workunit.client.1.smithi040.stdout:1/589: rename da/d12/d18/d2a/d8b/d97/fb7 to da/d12/d23/fc9 0 2024-04-24T23:14:00.929 INFO:tasks.workunit.client.1.smithi040.stdout:1/590: rename da/d35/dab to da/d35/dab/dca 22 2024-04-24T23:14:00.934 INFO:tasks.workunit.client.0.smithi031.stdout:7/856: rename d1/d2/dd/l43 to d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d93/l12a 0 2024-04-24T23:14:00.936 INFO:tasks.workunit.client.1.smithi040.stdout:9/632: read d7/da/d17/d40/d55/d61/d6c/fa8 [2280437,26430] 0 2024-04-24T23:14:00.936 INFO:tasks.workunit.client.1.smithi040.stdout:5/555: symlink d2/d7/d8/db/d1e/lb7 0 2024-04-24T23:14:00.939 INFO:tasks.workunit.client.1.smithi040.stdout:5/556: dread d2/d7/d8/d1f/d57/d27/d3d/d99/fa6 [0,4194304] 0 2024-04-24T23:14:00.940 INFO:tasks.workunit.client.1.smithi040.stdout:6/570: rmdir d7/d1a/d20/d51/d1e/d46/d69/d87 39 2024-04-24T23:14:00.942 INFO:tasks.workunit.client.1.smithi040.stdout:9/633: dwrite d7/da/d17/d40/d55/d61/d6c/f7a [4194304,4194304] 0 2024-04-24T23:14:00.943 INFO:tasks.workunit.client.0.smithi031.stdout:0/895: unlink da/d11/d6a/fc5 0 2024-04-24T23:14:00.943 INFO:tasks.workunit.client.0.smithi031.stdout:0/896: dread - da/d18/d1a/d37/d41/fae zero size 2024-04-24T23:14:00.944 INFO:tasks.workunit.client.1.smithi040.stdout:9/634: write d7/da/d17/d40/d55/d61/d6c/d8c/fde [61571,118443] 0 2024-04-24T23:14:00.945 INFO:tasks.workunit.client.1.smithi040.stdout:8/626: symlink d0/d16/d1b/d58/lcf 0 2024-04-24T23:14:00.946 INFO:tasks.workunit.client.0.smithi031.stdout:3/786: mkdir d6/d14/d1d/d59/dec 0 2024-04-24T23:14:00.948 INFO:tasks.workunit.client.1.smithi040.stdout:3/648: unlink d1/d2d/fc0 0 2024-04-24T23:14:00.954 INFO:tasks.workunit.client.0.smithi031.stdout:7/857: fsync d1/d2/f11 0 2024-04-24T23:14:00.955 INFO:tasks.workunit.client.0.smithi031.stdout:5/783: symlink d0/d6/d3b/d4b/d1d/l100 0 2024-04-24T23:14:00.956 INFO:tasks.workunit.client.0.smithi031.stdout:7/858: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/f3c 10504 1 2024-04-24T23:14:00.957 INFO:tasks.workunit.client.0.smithi031.stdout:5/784: chown d0/d6/d3b/d4b/d1d/f9e 671199278 1 2024-04-24T23:14:00.957 INFO:tasks.workunit.client.0.smithi031.stdout:2/770: dread d5/d8/d45/d5b/d49/d2f/faa [0,4194304] 0 2024-04-24T23:14:00.958 INFO:tasks.workunit.client.0.smithi031.stdout:1/782: creat d1/d3e/da2/f101 x:0 0 0 2024-04-24T23:14:00.960 INFO:tasks.workunit.client.0.smithi031.stdout:8/987: link d8/d15/dcf/cdd d8/d15/d37/d43/d7b/dbf/dcc/c14f 0 2024-04-24T23:14:00.960 INFO:tasks.workunit.client.1.smithi040.stdout:4/534: dwrite da/d3f/f5a [0,4194304] 0 2024-04-24T23:14:00.961 INFO:tasks.workunit.client.1.smithi040.stdout:5/557: dread - d2/d7/d8/d1f/d57/f91 zero size 2024-04-24T23:14:00.962 INFO:tasks.workunit.client.0.smithi031.stdout:0/897: chown da/d86/dbf/ce6 2013 1 2024-04-24T23:14:00.964 INFO:tasks.workunit.client.0.smithi031.stdout:4/949: write d3/d20/d2d/d44/fb2 [2212672,96527] 0 2024-04-24T23:14:00.966 INFO:tasks.workunit.client.0.smithi031.stdout:3/787: mkdir d6/d14/d71/dad/ded 0 2024-04-24T23:14:00.968 INFO:tasks.workunit.client.1.smithi040.stdout:8/627: creat d0/d16/d1b/d58/d8f/fd0 x:0 0 0 2024-04-24T23:14:00.973 INFO:tasks.workunit.client.0.smithi031.stdout:1/783: dread d1/d3e/d46/dbd/fce [0,4194304] 0 2024-04-24T23:14:00.974 INFO:tasks.workunit.client.0.smithi031.stdout:7/859: write d1/d2/f2a [3490928,23023] 0 2024-04-24T23:14:00.977 INFO:tasks.workunit.client.1.smithi040.stdout:3/649: dread d1/d2d/de/f34 [0,4194304] 0 2024-04-24T23:14:00.977 INFO:tasks.workunit.client.1.smithi040.stdout:1/591: symlink da/d12/d18/d2a/lcb 0 2024-04-24T23:14:00.979 INFO:tasks.workunit.client.0.smithi031.stdout:5/785: creat d0/d6/d3b/d4b/dc/d25/f101 x:0 0 0 2024-04-24T23:14:00.979 INFO:tasks.workunit.client.1.smithi040.stdout:1/592: chown da/d12/d55/l65 1188 1 2024-04-24T23:14:00.980 INFO:tasks.workunit.client.1.smithi040.stdout:1/593: write da/d12/d23/fb8 [948423,68161] 0 2024-04-24T23:14:00.984 INFO:tasks.workunit.client.0.smithi031.stdout:8/988: fsync d8/d15/d37/d43/d7b/dbf/d13d/f124 0 2024-04-24T23:14:00.985 INFO:tasks.workunit.client.1.smithi040.stdout:3/650: dwrite d1/d2d/de/f2f [4194304,4194304] 0 2024-04-24T23:14:00.986 INFO:tasks.workunit.client.1.smithi040.stdout:4/535: symlink da/d3f/d40/d41/d8e/lbc 0 2024-04-24T23:14:00.987 INFO:tasks.workunit.client.0.smithi031.stdout:9/939: dwrite d2/dae/d14/d78/d84/d4a/dab/dad/db5/fe2 [0,4194304] 0 2024-04-24T23:14:00.987 INFO:tasks.workunit.client.1.smithi040.stdout:3/651: chown d1/d2d/de/d30/d37/f63 472868797 1 2024-04-24T23:14:00.988 INFO:tasks.workunit.client.1.smithi040.stdout:1/594: dwrite da/d12/f21 [0,4194304] 0 2024-04-24T23:14:00.989 INFO:tasks.workunit.client.0.smithi031.stdout:0/898: symlink da/d20/d8e/l12a 0 2024-04-24T23:14:00.989 INFO:tasks.workunit.client.1.smithi040.stdout:1/595: stat da/d12/d18/d2a/d41 0 2024-04-24T23:14:00.991 INFO:tasks.workunit.client.1.smithi040.stdout:1/596: write da/d12/f21 [5439934,120070] 0 2024-04-24T23:14:00.993 INFO:tasks.workunit.client.1.smithi040.stdout:7/621: sync 2024-04-24T23:14:00.994 INFO:tasks.workunit.client.1.smithi040.stdout:5/558: rename d2/d2f/d14/c97 to d2/d7/d8/d1f/d57/d27/d3c/d9b/cb8 0 2024-04-24T23:14:00.996 INFO:tasks.workunit.client.1.smithi040.stdout:5/559: readlink d2/lf 0 2024-04-24T23:14:00.998 INFO:tasks.workunit.client.1.smithi040.stdout:6/571: symlink d7/d1a/d20/d51/d1e/d46/lb2 0 2024-04-24T23:14:00.998 INFO:tasks.workunit.client.1.smithi040.stdout:5/560: fdatasync d2/d2f/fa0 0 2024-04-24T23:14:01.005 INFO:tasks.workunit.client.1.smithi040.stdout:8/628: mknod d0/d16/d19/d71/d76/cd1 0 2024-04-24T23:14:01.008 INFO:tasks.workunit.client.1.smithi040.stdout:4/536: dread da/fb [0,4194304] 0 2024-04-24T23:14:01.010 INFO:tasks.workunit.client.1.smithi040.stdout:9/635: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/ddf 0 2024-04-24T23:14:01.012 INFO:tasks.workunit.client.1.smithi040.stdout:2/639: write da/f9b [563512,123370] 0 2024-04-24T23:14:01.020 INFO:tasks.workunit.client.0.smithi031.stdout:5/786: dread d0/d6/d3b/d4b/d1d/f9d [0,4194304] 0 2024-04-24T23:14:01.022 INFO:tasks.workunit.client.0.smithi031.stdout:2/771: rename d5/d8/d45/d5b/d49/d2f/lb1 to d5/d6/d29/d26/db0/dc7/da7/l113 0 2024-04-24T23:14:01.023 INFO:tasks.workunit.client.0.smithi031.stdout:2/772: stat d5/d6/d68/fe7 0 2024-04-24T23:14:01.023 INFO:tasks.workunit.client.1.smithi040.stdout:0/663: link df/d12/d1a/d71/f28 df/d1e/d20/d21/fd9 0 2024-04-24T23:14:01.028 INFO:tasks.workunit.client.0.smithi031.stdout:8/989: truncate d8/d15/d37/d62/fb3 3393796 0 2024-04-24T23:14:01.029 INFO:tasks.workunit.client.1.smithi040.stdout:0/664: dwrite df/d1e/d20/d61/f86 [0,4194304] 0 2024-04-24T23:14:01.032 INFO:tasks.workunit.client.0.smithi031.stdout:9/940: truncate d2/dae/d85/fc4 1140840 0 2024-04-24T23:14:01.043 INFO:tasks.workunit.client.0.smithi031.stdout:6/923: getdents d13/d15/d1d/d23/d4d/d7b 0 2024-04-24T23:14:01.044 INFO:tasks.workunit.client.0.smithi031.stdout:4/950: write d3/d20/d2d/f79 [330558,5455] 0 2024-04-24T23:14:01.044 INFO:tasks.workunit.client.1.smithi040.stdout:5/561: write d2/d7/d8/db/d1e/f82 [1028405,47482] 0 2024-04-24T23:14:01.045 INFO:tasks.workunit.client.0.smithi031.stdout:7/860: mkdir d1/df8/d12b 0 2024-04-24T23:14:01.046 INFO:tasks.workunit.client.0.smithi031.stdout:6/924: chown d13/d15/d1d/d23/d24/d43/cee 92676715 1 2024-04-24T23:14:01.047 INFO:tasks.workunit.client.1.smithi040.stdout:6/572: dwrite d7/f71 [0,4194304] 0 2024-04-24T23:14:01.047 INFO:tasks.workunit.client.0.smithi031.stdout:4/951: chown d3/d20/d2d/d44/d5b/l6c 16652868 1 2024-04-24T23:14:01.047 INFO:tasks.workunit.client.0.smithi031.stdout:4/952: fsync d3/d20/d2d/fbe 0 2024-04-24T23:14:01.048 INFO:tasks.workunit.client.0.smithi031.stdout:6/925: write d13/d15/d1d/d23/d4d/ff4 [4879893,63497] 0 2024-04-24T23:14:01.048 INFO:tasks.workunit.client.1.smithi040.stdout:8/629: symlink d0/d8/d22/d81/ld2 0 2024-04-24T23:14:01.050 INFO:tasks.workunit.client.0.smithi031.stdout:6/926: stat d13/d15/d1d/d23/d2d/f3d 0 2024-04-24T23:14:01.052 INFO:tasks.workunit.client.1.smithi040.stdout:4/537: creat da/d45/d48/d6e/fbd x:0 0 0 2024-04-24T23:14:01.057 INFO:tasks.workunit.client.1.smithi040.stdout:9/636: creat d7/da/d17/d40/d55/d2f/d1f/d31/fe0 x:0 0 0 2024-04-24T23:14:01.066 INFO:tasks.workunit.client.1.smithi040.stdout:3/652: link d1/d9/f27 d1/d9/fe2 0 2024-04-24T23:14:01.066 INFO:tasks.workunit.client.1.smithi040.stdout:3/653: read - d1/d2a/d8d/d7c/fe1 zero size 2024-04-24T23:14:01.148 INFO:tasks.workunit.client.1.smithi040.stdout:7/622: creat db/d43/db9/dd4/fdf x:0 0 0 2024-04-24T23:14:01.152 INFO:tasks.workunit.client.0.smithi031.stdout:1/784: write d1/da/d12/d65/dcf/d11/fc2 [345861,12231] 0 2024-04-24T23:14:01.153 INFO:tasks.workunit.client.0.smithi031.stdout:8/990: creat d8/d1a/d1d/d35/d58/d107/f150 x:0 0 0 2024-04-24T23:14:01.155 INFO:tasks.workunit.client.0.smithi031.stdout:4/953: dwrite d3/d20/d25/d60/d63/f131 [0,4194304] 0 2024-04-24T23:14:01.156 INFO:tasks.workunit.client.0.smithi031.stdout:3/788: write d6/df/d16/f1c [11813,68089] 0 2024-04-24T23:14:01.160 INFO:tasks.workunit.client.1.smithi040.stdout:6/573: creat d7/d1a/d9f/fb3 x:0 0 0 2024-04-24T23:14:01.162 INFO:tasks.workunit.client.1.smithi040.stdout:8/630: fdatasync d0/d8/fc3 0 2024-04-24T23:14:01.163 INFO:tasks.workunit.client.0.smithi031.stdout:0/899: dwrite da/d86/dbf/ff2 [0,4194304] 0 2024-04-24T23:14:01.166 INFO:tasks.workunit.client.1.smithi040.stdout:4/538: creat da/d45/d48/d6e/d70/fbe x:0 0 0 2024-04-24T23:14:01.167 INFO:tasks.workunit.client.0.smithi031.stdout:0/900: write da/d11/d6a/dba/fca [1044516,75968] 0 2024-04-24T23:14:01.171 INFO:tasks.workunit.client.0.smithi031.stdout:2/773: dwrite d5/f58 [4194304,4194304] 0 2024-04-24T23:14:01.173 INFO:tasks.workunit.client.0.smithi031.stdout:2/774: chown d5/d8/d45/d5b/fbd 1295939204 1 2024-04-24T23:14:01.178 INFO:tasks.workunit.client.0.smithi031.stdout:9/941: dread d2/d16/dbe/f21 [0,4194304] 0 2024-04-24T23:14:01.188 INFO:tasks.workunit.client.1.smithi040.stdout:7/623: creat db/d2f/fe0 x:0 0 0 2024-04-24T23:14:01.189 INFO:tasks.workunit.client.1.smithi040.stdout:7/624: fdatasync db/d12/d20/d56/fd6 0 2024-04-24T23:14:01.189 INFO:tasks.workunit.client.1.smithi040.stdout:1/597: link da/d12/d55/l9e da/d12/d23/lcc 0 2024-04-24T23:14:01.192 INFO:tasks.workunit.client.1.smithi040.stdout:6/574: chown d7/c11 186590528 1 2024-04-24T23:14:01.198 INFO:tasks.workunit.client.1.smithi040.stdout:9/637: dwrite d7/da/d17/d40/d55/d2f/d1f/f8a [0,4194304] 0 2024-04-24T23:14:01.198 INFO:tasks.workunit.client.1.smithi040.stdout:0/665: write df/d12/d1a/d71/f28 [4367080,108078] 0 2024-04-24T23:14:01.203 INFO:tasks.workunit.client.1.smithi040.stdout:1/598: symlink da/d12/d14/d25/db9/lcd 0 2024-04-24T23:14:01.205 INFO:tasks.workunit.client.1.smithi040.stdout:8/631: dwrite d0/d16/f3b [0,4194304] 0 2024-04-24T23:14:01.207 INFO:tasks.workunit.client.1.smithi040.stdout:7/625: rename db/d12/d20/f21 to db/d12/d3f/d7a/fe1 0 2024-04-24T23:14:01.207 INFO:tasks.workunit.client.1.smithi040.stdout:5/562: creat d2/d7/d8/d1f/d57/d27/d3c/fb9 x:0 0 0 2024-04-24T23:14:01.208 INFO:tasks.workunit.client.1.smithi040.stdout:5/563: chown d2/d7/d8/d1f/f9d 15 1 2024-04-24T23:14:01.210 INFO:tasks.workunit.client.1.smithi040.stdout:4/539: dread da/d3f/d6b/d3c/f9f [0,4194304] 0 2024-04-24T23:14:01.215 INFO:tasks.workunit.client.1.smithi040.stdout:3/654: link d1/d2d/de/d30/d37/f63 d1/d2d/de/d30/d37/fe3 0 2024-04-24T23:14:01.215 INFO:tasks.workunit.client.1.smithi040.stdout:0/666: truncate df/d1e/da7/d6b/f6f 1067084 0 2024-04-24T23:14:01.217 INFO:tasks.workunit.client.1.smithi040.stdout:4/540: dwrite da/d45/d48/f7d [0,4194304] 0 2024-04-24T23:14:01.219 INFO:tasks.workunit.client.1.smithi040.stdout:8/632: creat d0/d7e/dc1/d9e/d90/d91/fd3 x:0 0 0 2024-04-24T23:14:01.222 INFO:tasks.workunit.client.1.smithi040.stdout:7/626: creat db/d12/d20/d52/d6c/d8f/fe2 x:0 0 0 2024-04-24T23:14:01.226 INFO:tasks.workunit.client.1.smithi040.stdout:3/655: dwrite d1/d9/d2e/fa8 [0,4194304] 0 2024-04-24T23:14:01.227 INFO:tasks.workunit.client.1.smithi040.stdout:4/541: dwrite da/d3f/fb0 [0,4194304] 0 2024-04-24T23:14:01.229 INFO:tasks.workunit.client.1.smithi040.stdout:5/564: read d2/d7/d8/f39 [79378,87669] 0 2024-04-24T23:14:01.229 INFO:tasks.workunit.client.1.smithi040.stdout:9/638: dread d7/da/d17/d40/d55/d2f/fca [0,4194304] 0 2024-04-24T23:14:01.229 INFO:tasks.workunit.client.1.smithi040.stdout:3/656: write d1/d9/f17 [726952,128423] 0 2024-04-24T23:14:01.236 INFO:tasks.workunit.client.1.smithi040.stdout:1/599: symlink da/d12/lce 0 2024-04-24T23:14:01.237 INFO:tasks.workunit.client.1.smithi040.stdout:3/657: dwrite d1/d2a/d8d/f99 [0,4194304] 0 2024-04-24T23:14:01.239 INFO:tasks.workunit.client.1.smithi040.stdout:8/633: unlink d0/d16/d19/d71/d76/l8a 0 2024-04-24T23:14:01.247 INFO:tasks.workunit.client.1.smithi040.stdout:2/640: link da/d1c/d37/d39/d79/caf da/d1c/d37/d39/d3f/db2/cec 0 2024-04-24T23:14:01.249 INFO:tasks.workunit.client.1.smithi040.stdout:0/667: mknod df/d1e/da7/cda 0 2024-04-24T23:14:01.250 INFO:tasks.workunit.client.1.smithi040.stdout:9/639: creat d7/d5e/da1/fe1 x:0 0 0 2024-04-24T23:14:01.250 INFO:tasks.workunit.client.1.smithi040.stdout:7/627: dread db/d12/d20/d52/d44/f53 [0,4194304] 0 2024-04-24T23:14:01.254 INFO:tasks.workunit.client.1.smithi040.stdout:1/600: rename da/d12/d55/l9e to da/d12/d18/d2a/d8b/d97/lcf 0 2024-04-24T23:14:01.255 INFO:tasks.workunit.client.1.smithi040.stdout:1/601: truncate da/d12/d14/f96 260415 0 2024-04-24T23:14:01.262 INFO:tasks.workunit.client.0.smithi031.stdout:6/927: dread d13/d15/d1d/d23/d24/d8b/fcd [0,4194304] 0 2024-04-24T23:14:01.269 INFO:tasks.workunit.client.1.smithi040.stdout:9/640: creat d7/da/d17/d40/d55/d2f/d28/fe2 x:0 0 0 2024-04-24T23:14:01.269 INFO:tasks.workunit.client.0.smithi031.stdout:6/928: dwrite d13/d1b/d32/d9c/fc4 [4194304,4194304] 0 2024-04-24T23:14:01.269 INFO:tasks.workunit.client.0.smithi031.stdout:1/785: symlink d1/da/d12/d7e/d7b/l102 0 2024-04-24T23:14:01.272 INFO:tasks.workunit.client.0.smithi031.stdout:8/991: fsync d8/d15/d37/d43/d7b/dd3/f119 0 2024-04-24T23:14:01.276 INFO:tasks.workunit.client.1.smithi040.stdout:8/634: truncate d0/d16/f57 638060 0 2024-04-24T23:14:01.278 INFO:tasks.workunit.client.0.smithi031.stdout:6/929: dwrite d13/d15/d1d/d23/d24/d8b/ff3 [0,4194304] 0 2024-04-24T23:14:01.280 INFO:tasks.workunit.client.0.smithi031.stdout:6/930: write d13/d15/d1d/d23/d44/f5b [3572482,23170] 0 2024-04-24T23:14:01.280 INFO:tasks.workunit.client.1.smithi040.stdout:2/641: mkdir da/d1c/d37/d39/d3f/d52/d78/dc7/ded 0 2024-04-24T23:14:01.281 INFO:tasks.workunit.client.1.smithi040.stdout:2/642: readlink da/d1c/d37/d39/d3f/d52/d78/dc7/ldf 0 2024-04-24T23:14:01.282 INFO:tasks.workunit.client.1.smithi040.stdout:2/643: stat da/d29/l53 0 2024-04-24T23:14:01.283 INFO:tasks.workunit.client.1.smithi040.stdout:0/668: mkdir df/d1e/d20/d21/d51/ddb 0 2024-04-24T23:14:01.284 INFO:tasks.workunit.client.0.smithi031.stdout:0/901: rmdir da/d86/da3/dcd 39 2024-04-24T23:14:01.291 INFO:tasks.workunit.client.0.smithi031.stdout:2/775: fdatasync d5/d6/d29/f61 0 2024-04-24T23:14:01.299 INFO:tasks.workunit.client.1.smithi040.stdout:2/644: creat da/d1c/d37/d39/d3f/d52/d78/dc7/fee x:0 0 0 2024-04-24T23:14:01.300 INFO:tasks.workunit.client.0.smithi031.stdout:4/954: dread d3/da3/fae [0,4194304] 0 2024-04-24T23:14:01.301 INFO:tasks.workunit.client.1.smithi040.stdout:0/669: creat df/d1e/d2b/db6/fdc x:0 0 0 2024-04-24T23:14:01.304 INFO:tasks.workunit.client.1.smithi040.stdout:0/670: read df/d1e/d20/d21/d51/f8f [477745,127113] 0 2024-04-24T23:14:01.305 INFO:tasks.workunit.client.0.smithi031.stdout:5/787: link d0/d6/d3b/d4f/fd6 d0/d6/d3b/d4b/dc/d86/dfd/f102 0 2024-04-24T23:14:01.305 INFO:tasks.workunit.client.1.smithi040.stdout:1/602: link da/d35/da3/lc5 da/d12/d18/d2a/d4d/d7d/ld0 0 2024-04-24T23:14:01.306 INFO:tasks.workunit.client.1.smithi040.stdout:3/658: dwrite d1/d2a/d6a/fc9 [0,4194304] 0 2024-04-24T23:14:01.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/788: chown d0/d7/lf9 2 1 2024-04-24T23:14:01.306 INFO:tasks.workunit.client.1.smithi040.stdout:4/542: truncate da/d3f/d40/d41/d60/f23 185250 0 2024-04-24T23:14:01.307 INFO:tasks.workunit.client.0.smithi031.stdout:1/786: mkdir d1/da/d12/d65/dcf/d11/d21/d5a/d103 0 2024-04-24T23:14:01.308 INFO:tasks.workunit.client.1.smithi040.stdout:8/635: truncate d0/d8/f4d 1945700 0 2024-04-24T23:14:01.310 INFO:tasks.workunit.client.1.smithi040.stdout:6/575: sync 2024-04-24T23:14:01.310 INFO:tasks.workunit.client.1.smithi040.stdout:8/636: truncate d0/d16/d1b/f95 1137632 0 2024-04-24T23:14:01.311 INFO:tasks.workunit.client.1.smithi040.stdout:2/645: truncate da/d10/d4d/da4/fd9 476549 0 2024-04-24T23:14:01.313 INFO:tasks.workunit.client.1.smithi040.stdout:9/641: creat d7/da/d17/d40/d55/d2f/d1f/d31/d3b/fe3 x:0 0 0 2024-04-24T23:14:01.316 INFO:tasks.workunit.client.1.smithi040.stdout:7/628: fsync db/d12/d73/fbe 0 2024-04-24T23:14:01.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/902: creat da/d11/d6a/dba/f12b x:0 0 0 2024-04-24T23:14:01.320 INFO:tasks.workunit.client.1.smithi040.stdout:1/603: symlink da/d12/d23/ld1 0 2024-04-24T23:14:01.321 INFO:tasks.workunit.client.1.smithi040.stdout:1/604: chown da/d12/d18/d2a/d4d/f75 667 1 2024-04-24T23:14:01.330 INFO:tasks.workunit.client.0.smithi031.stdout:5/789: read - d0/d6/d3b/d4b/d1d/d2e/fa1 zero size 2024-04-24T23:14:01.334 INFO:tasks.workunit.client.1.smithi040.stdout:9/642: mknod d7/d92/ce4 0 2024-04-24T23:14:01.334 INFO:tasks.workunit.client.0.smithi031.stdout:1/787: mkdir d1/da/d12/d7e/d104 0 2024-04-24T23:14:01.335 INFO:tasks.workunit.client.1.smithi040.stdout:2/646: read da/d10/d15/f30 [572025,128409] 0 2024-04-24T23:14:01.337 INFO:tasks.workunit.client.1.smithi040.stdout:0/671: mknod df/d1e/da7/dd0/cdd 0 2024-04-24T23:14:01.337 INFO:tasks.workunit.client.1.smithi040.stdout:4/543: dread da/d3f/d40/d41/d60/f4d [0,4194304] 0 2024-04-24T23:14:01.338 INFO:tasks.workunit.client.1.smithi040.stdout:0/672: write df/d12/f4c [5140843,51063] 0 2024-04-24T23:14:01.341 INFO:tasks.workunit.client.0.smithi031.stdout:5/790: dread d0/d6/d3b/f8d [0,4194304] 0 2024-04-24T23:14:01.342 INFO:tasks.workunit.client.1.smithi040.stdout:7/629: rename db/d12/d3f/d7a/fc0 to db/d43/db9/fe3 0 2024-04-24T23:14:01.342 INFO:tasks.workunit.client.1.smithi040.stdout:1/605: unlink da/d12/d14/d25/d5e/lae 0 2024-04-24T23:14:01.343 INFO:tasks.workunit.client.0.smithi031.stdout:2/776: write d5/d6/d29/fc9 [723803,10340] 0 2024-04-24T23:14:01.344 INFO:tasks.workunit.client.1.smithi040.stdout:6/576: rmdir d7/de/d89 39 2024-04-24T23:14:01.345 INFO:tasks.workunit.client.0.smithi031.stdout:9/942: creat d2/dae/d14/d1d/d45/f121 x:0 0 0 2024-04-24T23:14:01.346 INFO:tasks.workunit.client.1.smithi040.stdout:8/637: creat d0/d8/dc/d34/da7/fd4 x:0 0 0 2024-04-24T23:14:01.347 INFO:tasks.workunit.client.1.smithi040.stdout:7/630: dwrite db/d12/d20/d56/fd6 [0,4194304] 0 2024-04-24T23:14:01.348 INFO:tasks.workunit.client.0.smithi031.stdout:4/955: symlink d3/d20/d25/d60/d63/d136/l137 0 2024-04-24T23:14:01.350 INFO:tasks.workunit.client.0.smithi031.stdout:1/788: creat d1/d1d/f105 x:0 0 0 2024-04-24T23:14:01.351 INFO:tasks.workunit.client.1.smithi040.stdout:2/647: fsync da/f19 0 2024-04-24T23:14:01.354 INFO:tasks.workunit.client.0.smithi031.stdout:8/992: dwrite d8/d15/d9a/fc2 [0,4194304] 0 2024-04-24T23:14:01.355 INFO:tasks.workunit.client.1.smithi040.stdout:2/648: dwrite da/d1c/f57 [0,4194304] 0 2024-04-24T23:14:01.356 INFO:tasks.workunit.client.0.smithi031.stdout:8/993: write d8/d15/d37/d43/f6f [1320580,43972] 0 2024-04-24T23:14:01.362 INFO:tasks.workunit.client.1.smithi040.stdout:4/544: rename da/d3f/d6b/d3c/f9f to da/d3f/d8d/db5/fbf 0 2024-04-24T23:14:01.375 INFO:tasks.workunit.client.0.smithi031.stdout:0/903: rename da/d18/cc2 to da/d86/da3/dcd/c12c 0 2024-04-24T23:14:01.376 INFO:tasks.workunit.client.1.smithi040.stdout:8/638: rmdir d0/d8/d55 39 2024-04-24T23:14:01.386 INFO:tasks.workunit.client.0.smithi031.stdout:2/777: dwrite d5/d28/ff4 [0,4194304] 0 2024-04-24T23:14:01.386 INFO:tasks.workunit.client.0.smithi031.stdout:1/789: mkdir d1/d106 0 2024-04-24T23:14:01.387 INFO:tasks.workunit.client.0.smithi031.stdout:2/778: dread - d5/f87 zero size 2024-04-24T23:14:01.388 INFO:tasks.workunit.client.0.smithi031.stdout:1/790: chown d1/d3e/d46/dbd/fce 296876 1 2024-04-24T23:14:01.388 INFO:tasks.workunit.client.1.smithi040.stdout:6/577: write d7/d1a/d20/d51/d1e/f3d [1928880,9524] 0 2024-04-24T23:14:01.389 INFO:tasks.workunit.client.1.smithi040.stdout:2/649: truncate da/d29/f42 2575311 0 2024-04-24T23:14:01.391 INFO:tasks.workunit.client.0.smithi031.stdout:4/956: write d3/d20/d25/d95/ff3 [91436,99150] 0 2024-04-24T23:14:01.393 INFO:tasks.workunit.client.0.smithi031.stdout:9/943: dread d2/dae/d14/d78/d84/d4a/d4b/d96/f118 [0,4194304] 0 2024-04-24T23:14:01.393 INFO:tasks.workunit.client.0.smithi031.stdout:0/904: mkdir da/d20/d8e/d12d 0 2024-04-24T23:14:01.395 INFO:tasks.workunit.client.0.smithi031.stdout:9/944: write d2/dae/d14/f46 [721722,78034] 0 2024-04-24T23:14:01.397 INFO:tasks.workunit.client.1.smithi040.stdout:1/606: link da/d12/d18/d2a/d4d/d7d/fa9 da/d12/d18/d2a/d4d/fd2 0 2024-04-24T23:14:01.398 INFO:tasks.workunit.client.1.smithi040.stdout:1/607: write da/d12/d14/f24 [870826,38715] 0 2024-04-24T23:14:01.400 INFO:tasks.workunit.client.1.smithi040.stdout:8/639: mkdir d0/d38/d4c/dd5 0 2024-04-24T23:14:01.400 INFO:tasks.workunit.client.0.smithi031.stdout:8/994: dwrite d8/f22 [0,4194304] 0 2024-04-24T23:14:01.402 INFO:tasks.workunit.client.1.smithi040.stdout:0/673: dwrite df/d1e/d2b/d6e/fba [8388608,4194304] 0 2024-04-24T23:14:01.402 INFO:tasks.workunit.client.0.smithi031.stdout:2/779: fdatasync d5/d8/d15/f93 0 2024-04-24T23:14:01.402 INFO:tasks.workunit.client.1.smithi040.stdout:7/631: mknod db/d12/d20/ce4 0 2024-04-24T23:14:01.403 INFO:tasks.workunit.client.1.smithi040.stdout:7/632: readlink db/d43/l45 0 2024-04-24T23:14:01.404 INFO:tasks.workunit.client.1.smithi040.stdout:7/633: chown db/d12/d20/d52/f70 9 1 2024-04-24T23:14:01.404 INFO:tasks.workunit.client.0.smithi031.stdout:1/791: readlink d1/da/d12/d65/dcf/l88 0 2024-04-24T23:14:01.412 INFO:tasks.workunit.client.0.smithi031.stdout:5/791: rename d0/l65 to d0/d6/d3b/d4b/d1d/d2e/d28/d77/l103 0 2024-04-24T23:14:01.414 INFO:tasks.workunit.client.0.smithi031.stdout:4/957: fsync d3/d23/d69/d3f/f6d 0 2024-04-24T23:14:01.416 INFO:tasks.workunit.client.1.smithi040.stdout:6/578: creat d7/d1a/d9f/fb4 x:0 0 0 2024-04-24T23:14:01.420 INFO:tasks.workunit.client.1.smithi040.stdout:5/565: sync 2024-04-24T23:14:01.420 INFO:tasks.workunit.client.1.smithi040.stdout:1/608: rmdir da 39 2024-04-24T23:14:01.423 INFO:tasks.workunit.client.1.smithi040.stdout:8/640: symlink d0/d16/d1b/d3f/d44/ld6 0 2024-04-24T23:14:01.424 INFO:tasks.workunit.client.0.smithi031.stdout:8/995: mknod d8/d15/d37/d43/d7b/dbf/c151 0 2024-04-24T23:14:01.426 INFO:tasks.workunit.client.1.smithi040.stdout:7/634: fdatasync db/d12/d20/d52/d6c/d8f/fd0 0 2024-04-24T23:14:01.428 INFO:tasks.workunit.client.0.smithi031.stdout:2/780: chown d5/d8/d15/d56/d84/fd8 218 1 2024-04-24T23:14:01.430 INFO:tasks.workunit.client.1.smithi040.stdout:6/579: mkdir d7/d1a/db5 0 2024-04-24T23:14:01.430 INFO:tasks.workunit.client.1.smithi040.stdout:0/674: write df/d1e/d20/f3f [1008368,123706] 0 2024-04-24T23:14:01.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/905: mkdir da/d86/dbf/dd5/d12e 0 2024-04-24T23:14:01.436 INFO:tasks.workunit.client.0.smithi031.stdout:0/906: chown da/d18/d5a/d7d/f116 31098642 1 2024-04-24T23:14:01.437 INFO:tasks.workunit.client.1.smithi040.stdout:8/641: creat d0/d7e/dc1/d87/fd7 x:0 0 0 2024-04-24T23:14:01.438 INFO:tasks.workunit.client.1.smithi040.stdout:8/642: write d0/d8/dc/d34/da7/fd4 [836038,45767] 0 2024-04-24T23:14:01.440 INFO:tasks.workunit.client.1.smithi040.stdout:6/580: rmdir d7/d1a/d20/d51/d2a 39 2024-04-24T23:14:01.444 INFO:tasks.workunit.client.1.smithi040.stdout:2/650: creat da/d10/fef x:0 0 0 2024-04-24T23:14:01.444 INFO:tasks.workunit.client.1.smithi040.stdout:7/635: dwrite db/d12/d20/d52/d6c/fc4 [0,4194304] 0 2024-04-24T23:14:01.447 INFO:tasks.workunit.client.1.smithi040.stdout:1/609: mkdir da/d12/d18/d2a/d8b/dd3 0 2024-04-24T23:14:01.451 INFO:tasks.workunit.client.0.smithi031.stdout:8/996: mkdir d8/d1a/d1d/d50/dc0/d152 0 2024-04-24T23:14:01.454 INFO:tasks.workunit.client.0.smithi031.stdout:1/792: link d1/da/d12/d7e/d7b/d4e/lae d1/da/d12/d65/dcf/d11/l107 0 2024-04-24T23:14:01.456 INFO:tasks.workunit.client.1.smithi040.stdout:8/643: stat d0/d16/l18 0 2024-04-24T23:14:01.457 INFO:tasks.workunit.client.1.smithi040.stdout:7/636: rename db/d43/c95 to db/d12/d3f/ce5 0 2024-04-24T23:14:01.458 INFO:tasks.workunit.client.0.smithi031.stdout:2/781: mkdir d5/d8/d45/d5b/d49/dc4/def/d114 0 2024-04-24T23:14:01.459 INFO:tasks.workunit.client.1.smithi040.stdout:0/675: dwrite df/d1e/d89/fd7 [0,4194304] 0 2024-04-24T23:14:01.459 INFO:tasks.workunit.client.0.smithi031.stdout:2/782: fdatasync d5/d8/f112 0 2024-04-24T23:14:01.460 INFO:tasks.workunit.client.0.smithi031.stdout:2/783: fdatasync d5/d99/fdb 0 2024-04-24T23:14:01.461 INFO:tasks.workunit.client.0.smithi031.stdout:9/945: link d2/d16/dbe/l2c d2/dae/d14/l122 0 2024-04-24T23:14:01.462 INFO:tasks.workunit.client.1.smithi040.stdout:6/581: dwrite d7/de/d4e/fa6 [0,4194304] 0 2024-04-24T23:14:01.464 INFO:tasks.workunit.client.1.smithi040.stdout:6/582: chown d7/d1a/d20/d51/d1e/d46/fac 466 1 2024-04-24T23:14:01.465 INFO:tasks.workunit.client.0.smithi031.stdout:8/997: creat d8/d1a/d1d/d50/d142/f153 x:0 0 0 2024-04-24T23:14:01.469 INFO:tasks.workunit.client.0.smithi031.stdout:5/792: creat d0/d6/d3b/d4b/dc/de3/f104 x:0 0 0 2024-04-24T23:14:01.473 INFO:tasks.workunit.client.0.smithi031.stdout:0/907: dread da/d18/d1a/d37/f26 [0,4194304] 0 2024-04-24T23:14:01.475 INFO:tasks.workunit.client.1.smithi040.stdout:3/659: sync 2024-04-24T23:14:01.475 INFO:tasks.workunit.client.1.smithi040.stdout:9/643: sync 2024-04-24T23:14:01.475 INFO:tasks.workunit.client.1.smithi040.stdout:4/545: sync 2024-04-24T23:14:01.477 INFO:tasks.workunit.client.0.smithi031.stdout:9/946: chown d2/dae/d14/f5f 0 1 2024-04-24T23:14:01.483 INFO:tasks.workunit.client.0.smithi031.stdout:5/793: readlink d0/d7/d74/de2/le5 0 2024-04-24T23:14:01.485 INFO:tasks.workunit.client.0.smithi031.stdout:0/908: mkdir da/d20/d8e/d12f 0 2024-04-24T23:14:01.485 INFO:tasks.workunit.client.0.smithi031.stdout:0/909: readlink da/d18/d1a/l6b 0 2024-04-24T23:14:01.489 INFO:tasks.workunit.client.1.smithi040.stdout:2/651: write da/d29/d55/fa9 [4788256,16386] 0 2024-04-24T23:14:01.489 INFO:tasks.workunit.client.1.smithi040.stdout:2/652: chown da/d1c/l2a 2907 1 2024-04-24T23:14:01.492 INFO:tasks.workunit.client.0.smithi031.stdout:1/793: link d1/da/d12/d65/dcf/cfe d1/d3e/d46/d74/c108 0 2024-04-24T23:14:01.495 INFO:tasks.workunit.client.1.smithi040.stdout:5/566: sync 2024-04-24T23:14:01.495 INFO:tasks.workunit.client.1.smithi040.stdout:5/567: fdatasync d2/d7/d8/f13 0 2024-04-24T23:14:01.496 INFO:tasks.workunit.client.1.smithi040.stdout:5/568: chown d2/d7/d8/d1f/d31/f4e 3 1 2024-04-24T23:14:01.497 INFO:tasks.workunit.client.0.smithi031.stdout:0/910: mkdir da/d86/da3/d104/d130 0 2024-04-24T23:14:01.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/998: write d8/d15/d37/d43/d7b/f8c [2571686,23636] 0 2024-04-24T23:14:01.503 INFO:tasks.workunit.client.0.smithi031.stdout:0/911: symlink da/d86/da3/dcd/de7/ddb/l131 0 2024-04-24T23:14:01.504 INFO:tasks.workunit.client.1.smithi040.stdout:3/660: dread d1/d9/d18/d36/d55/f89 [0,4194304] 0 2024-04-24T23:14:01.509 INFO:tasks.workunit.client.0.smithi031.stdout:2/784: dread d5/d8/d45/d5b/d49/f7e [0,4194304] 0 2024-04-24T23:14:01.509 INFO:tasks.workunit.client.0.smithi031.stdout:8/999: mknod d8/d15/d37/d43/daf/dca/c154 0 2024-04-24T23:14:01.510 INFO:tasks.workunit.client.1.smithi040.stdout:3/661: dread d1/d2d/fc3 [0,4194304] 0 2024-04-24T23:14:01.511 INFO:tasks.workunit.client.0.smithi031.stdout:5/794: write d0/d6/d3b/d4b/dc/f47 [1455952,74643] 0 2024-04-24T23:14:01.513 INFO:tasks.workunit.client.0.smithi031.stdout:9/947: getdents d2/dae 0 2024-04-24T23:14:01.514 INFO:tasks.workunit.client.0.smithi031.stdout:9/948: write d2/dae/d14/f19 [2368846,107755] 0 2024-04-24T23:14:01.515 INFO:tasks.workunit.client.0.smithi031.stdout:1/794: creat d1/da/d61/f109 x:0 0 0 2024-04-24T23:14:01.520 INFO:tasks.workunit.client.0.smithi031.stdout:0/912: link da/d86/dbf/ff2 da/d20/d4e/dc9/f132 0 2024-04-24T23:14:01.522 INFO:tasks.workunit.client.0.smithi031.stdout:5/795: symlink d0/d6/d32/l105 0 2024-04-24T23:14:01.523 INFO:tasks.workunit.client.0.smithi031.stdout:1/795: dread - d1/da/d12/d65/fcb zero size 2024-04-24T23:14:01.524 INFO:tasks.workunit.client.0.smithi031.stdout:1/796: dread - d1/da/d12/d7e/d7b/d4e/d98/ffb zero size 2024-04-24T23:14:01.524 INFO:tasks.workunit.client.0.smithi031.stdout:1/797: chown d1/da/l1a 1858744 1 2024-04-24T23:14:01.525 INFO:tasks.workunit.client.0.smithi031.stdout:9/949: rename d2/dae/d14/d1d/d45/d56/ff2 to d2/dae/d85/f123 0 2024-04-24T23:14:01.530 INFO:tasks.workunit.client.0.smithi031.stdout:0/913: rename da/d18/d1a/f106 to da/d18/d5a/d7d/dbd/f133 0 2024-04-24T23:14:01.531 INFO:tasks.workunit.client.1.smithi040.stdout:8/644: mkdir d0/d16/d19/da2/dd8 0 2024-04-24T23:14:01.536 INFO:tasks.workunit.client.1.smithi040.stdout:0/676: mkdir df/d12/d58/dc3/dde 0 2024-04-24T23:14:01.538 INFO:tasks.workunit.client.1.smithi040.stdout:9/644: mkdir d7/d5e/de5 0 2024-04-24T23:14:01.546 INFO:tasks.workunit.client.1.smithi040.stdout:2/653: creat da/d1c/d37/d39/d3f/d52/d78/dc7/dcc/ff0 x:0 0 0 2024-04-24T23:14:01.547 INFO:tasks.workunit.client.0.smithi031.stdout:2/785: write d5/d8/f98 [650572,102121] 0 2024-04-24T23:14:01.547 INFO:tasks.workunit.client.0.smithi031.stdout:5/796: creat d0/d6/d3b/d4b/dc/f106 x:0 0 0 2024-04-24T23:14:01.548 INFO:tasks.workunit.client.0.smithi031.stdout:2/786: fsync d5/d8/d45/d5b/d49/dc4/fe9 0 2024-04-24T23:14:01.552 INFO:tasks.workunit.client.0.smithi031.stdout:9/950: rmdir d2/dae/d105 0 2024-04-24T23:14:01.555 INFO:tasks.workunit.client.0.smithi031.stdout:5/797: symlink d0/d6/d32/d6b/l107 0 2024-04-24T23:14:01.555 INFO:tasks.workunit.client.1.smithi040.stdout:7/637: dwrite db/d12/d62/f64 [0,4194304] 0 2024-04-24T23:14:01.556 INFO:tasks.workunit.client.1.smithi040.stdout:1/610: mkdir da/d12/d18/d2a/dd4 0 2024-04-24T23:14:01.559 INFO:tasks.workunit.client.0.smithi031.stdout:1/798: dwrite d1/da/d12/d7e/f9e [0,4194304] 0 2024-04-24T23:14:01.564 INFO:tasks.workunit.client.0.smithi031.stdout:0/914: dwrite da/d18/d5a/fb2 [0,4194304] 0 2024-04-24T23:14:01.567 INFO:tasks.workunit.client.0.smithi031.stdout:5/798: mknod d0/d6/d3b/d4b/dc/d86/c108 0 2024-04-24T23:14:01.568 INFO:tasks.workunit.client.0.smithi031.stdout:5/799: fdatasync d0/d6/d3b/d4b/dc/f84 0 2024-04-24T23:14:01.569 INFO:tasks.workunit.client.1.smithi040.stdout:6/583: creat d7/d1a/d20/d51/daa/fb6 x:0 0 0 2024-04-24T23:14:01.572 INFO:tasks.workunit.client.1.smithi040.stdout:9/645: write d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d [6755496,27117] 0 2024-04-24T23:14:01.573 INFO:tasks.workunit.client.1.smithi040.stdout:9/646: fsync d7/da/d17/d40/d55/d61/d6c/d8c/fde 0 2024-04-24T23:14:01.576 INFO:tasks.workunit.client.0.smithi031.stdout:1/799: rename d1/d3e/da3/fe2 to d1/da/d12/d65/dcf/d11/f10a 0 2024-04-24T23:14:01.578 INFO:tasks.workunit.client.1.smithi040.stdout:4/546: symlink da/d3f/d40/d47/d71/lc0 0 2024-04-24T23:14:01.579 INFO:tasks.workunit.client.0.smithi031.stdout:0/915: creat da/d18/d5a/d7d/dbd/f134 x:0 0 0 2024-04-24T23:14:01.579 INFO:tasks.workunit.client.0.smithi031.stdout:0/916: readlink da/d11/l28 0 2024-04-24T23:14:01.580 INFO:tasks.workunit.client.0.smithi031.stdout:2/787: write d5/d99/f10d [2332758,64353] 0 2024-04-24T23:14:01.581 INFO:tasks.workunit.client.1.smithi040.stdout:5/569: symlink d2/d2f/d14/lba 0 2024-04-24T23:14:01.582 INFO:tasks.workunit.client.0.smithi031.stdout:5/800: creat d0/d6/d3b/d4b/dc/d25/d55/f109 x:0 0 0 2024-04-24T23:14:01.583 INFO:tasks.workunit.client.0.smithi031.stdout:5/801: stat d0/d6/d3b/d4b/dc/d25/fe4 0 2024-04-24T23:14:01.586 INFO:tasks.workunit.client.1.smithi040.stdout:0/677: dwrite df/d1e/d2b/f9e [0,4194304] 0 2024-04-24T23:14:01.587 INFO:tasks.workunit.client.0.smithi031.stdout:1/800: fdatasync d1/d3e/da3/fc7 0 2024-04-24T23:14:01.588 INFO:tasks.workunit.client.1.smithi040.stdout:7/638: dread db/d12/f3e [0,4194304] 0 2024-04-24T23:14:01.590 INFO:tasks.workunit.client.0.smithi031.stdout:0/917: mkdir da/d18/d1a/d37/d41/d135 0 2024-04-24T23:14:01.601 INFO:tasks.workunit.client.0.smithi031.stdout:0/918: truncate da/d18/ff6 342554 0 2024-04-24T23:14:01.605 INFO:tasks.workunit.client.0.smithi031.stdout:9/951: link d2/dae/d14/d78/d84/d4a/f102 d2/d16/de0/d11d/f124 0 2024-04-24T23:14:01.605 INFO:tasks.workunit.client.0.smithi031.stdout:0/919: dread da/d22/f96 [0,4194304] 0 2024-04-24T23:14:01.606 INFO:tasks.workunit.client.1.smithi040.stdout:8/645: dwrite d0/d8/dc/f17 [0,4194304] 0 2024-04-24T23:14:01.610 INFO:tasks.workunit.client.1.smithi040.stdout:0/678: write df/d1e/da7/f65 [1416232,113356] 0 2024-04-24T23:14:01.612 INFO:tasks.workunit.client.0.smithi031.stdout:0/920: chown da/d18/ca7 1 1 2024-04-24T23:14:01.613 INFO:tasks.workunit.client.0.smithi031.stdout:5/802: dwrite d0/d6/d32/d6b/fa0 [0,4194304] 0 2024-04-24T23:14:01.615 INFO:tasks.workunit.client.0.smithi031.stdout:0/921: write da/d86/da3/dcd/de7/fc7 [7144538,54666] 0 2024-04-24T23:14:01.617 INFO:tasks.workunit.client.0.smithi031.stdout:9/952: symlink d2/dae/d14/d1d/l125 0 2024-04-24T23:14:01.619 INFO:tasks.workunit.client.1.smithi040.stdout:7/639: creat db/d12/d20/d56/fe6 x:0 0 0 2024-04-24T23:14:01.621 INFO:tasks.workunit.client.0.smithi031.stdout:1/801: creat d1/da/d10/d52/f10b x:0 0 0 2024-04-24T23:14:01.623 INFO:tasks.workunit.client.1.smithi040.stdout:1/611: truncate da/f7e 3617759 0 2024-04-24T23:14:01.625 INFO:tasks.workunit.client.1.smithi040.stdout:6/584: creat d7/de/d89/fb7 x:0 0 0 2024-04-24T23:14:01.626 INFO:tasks.workunit.client.1.smithi040.stdout:9/647: truncate d7/da/d17/d40/d55/d2f/d1f/d31/f8d 795683 0 2024-04-24T23:14:01.627 INFO:tasks.workunit.client.0.smithi031.stdout:5/803: symlink d0/d6/d3b/d4b/dc/d86/d91/l10a 0 2024-04-24T23:14:01.628 INFO:tasks.workunit.client.1.smithi040.stdout:4/547: mknod da/d3f/d40/d47/d75/daa/cc1 0 2024-04-24T23:14:01.629 INFO:tasks.workunit.client.0.smithi031.stdout:0/922: mkdir da/d86/da3/dcd/de7/ddb/d136 0 2024-04-24T23:14:01.630 INFO:tasks.workunit.client.1.smithi040.stdout:8/646: mknod d0/d8/d22/db8/cd9 0 2024-04-24T23:14:01.631 INFO:tasks.workunit.client.1.smithi040.stdout:8/647: read - d0/d7e/dc1/f70 zero size 2024-04-24T23:14:01.631 INFO:tasks.workunit.client.1.smithi040.stdout:9/648: dwrite d7/da/d17/fae [0,4194304] 0 2024-04-24T23:14:01.632 INFO:tasks.workunit.client.1.smithi040.stdout:8/648: stat d0/d7e/dc1/d87/fd7 0 2024-04-24T23:14:01.633 INFO:tasks.workunit.client.1.smithi040.stdout:9/649: write d7/d5e/da1/fd0 [767334,7303] 0 2024-04-24T23:14:01.634 INFO:tasks.workunit.client.1.smithi040.stdout:0/679: mkdir df/d1e/d89/ddf 0 2024-04-24T23:14:01.635 INFO:tasks.workunit.client.0.smithi031.stdout:5/804: creat d0/d6/d32/d6b/f10b x:0 0 0 2024-04-24T23:14:01.636 INFO:tasks.workunit.client.1.smithi040.stdout:7/640: truncate db/d12/d20/d52/f23 363363 0 2024-04-24T23:14:01.643 INFO:tasks.workunit.client.0.smithi031.stdout:9/953: unlink d2/cce 0 2024-04-24T23:14:01.644 INFO:tasks.workunit.client.1.smithi040.stdout:3/662: sync 2024-04-24T23:14:01.644 INFO:tasks.workunit.client.1.smithi040.stdout:8/649: dread d0/d8/dc/d34/f3c [0,4194304] 0 2024-04-24T23:14:01.645 INFO:tasks.workunit.client.1.smithi040.stdout:3/663: truncate d1/fb2 984053 0 2024-04-24T23:14:01.645 INFO:tasks.workunit.client.1.smithi040.stdout:8/650: write d0/d16/d19/fac [231779,108004] 0 2024-04-24T23:14:01.646 INFO:tasks.workunit.client.1.smithi040.stdout:4/548: rmdir da/d78 39 2024-04-24T23:14:01.648 INFO:tasks.workunit.client.1.smithi040.stdout:0/680: dread df/d12/d4b/f92 [0,4194304] 0 2024-04-24T23:14:01.650 INFO:tasks.workunit.client.1.smithi040.stdout:5/570: creat d2/d2f/fbb x:0 0 0 2024-04-24T23:14:01.653 INFO:tasks.workunit.client.0.smithi031.stdout:9/954: mknod d2/dae/d14/d78/d84/d4a/dab/db8/c126 0 2024-04-24T23:14:01.656 INFO:tasks.workunit.client.0.smithi031.stdout:5/805: rename d0/d6/d3b/d4b/dc/d86/dfd/da8/cb7 to d0/d7/d74/de2/c10c 0 2024-04-24T23:14:01.657 INFO:tasks.workunit.client.1.smithi040.stdout:9/650: rmdir d7/d5e/da1 39 2024-04-24T23:14:01.658 INFO:tasks.workunit.client.1.smithi040.stdout:9/651: fsync d7/da/d17/d40/d55/d61/d6c/f7a 0 2024-04-24T23:14:01.660 INFO:tasks.workunit.client.0.smithi031.stdout:1/802: dwrite d1/da/d12/d65/fcb [0,4194304] 0 2024-04-24T23:14:01.661 INFO:tasks.workunit.client.1.smithi040.stdout:3/664: dread d1/d9/fe2 [8388608,4194304] 0 2024-04-24T23:14:01.664 INFO:tasks.workunit.client.1.smithi040.stdout:7/641: mkdir db/d12/d62/de7 0 2024-04-24T23:14:01.671 INFO:tasks.workunit.client.0.smithi031.stdout:0/923: dwrite da/d18/d1a/d37/f3d [0,4194304] 0 2024-04-24T23:14:01.671 INFO:tasks.workunit.client.1.smithi040.stdout:1/612: dwrite da/d12/d18/d2a/d4d/f59 [0,4194304] 0 2024-04-24T23:14:01.674 INFO:tasks.workunit.client.1.smithi040.stdout:6/585: dwrite d7/d1a/d20/d51/f17 [0,4194304] 0 2024-04-24T23:14:01.683 INFO:tasks.workunit.client.0.smithi031.stdout:5/806: getdents d0/d7/d56/d7c 0 2024-04-24T23:14:01.688 INFO:tasks.workunit.client.1.smithi040.stdout:5/571: truncate d2/d7/d8/d1f/d57/d3e/f58 1843280 0 2024-04-24T23:14:01.689 INFO:tasks.workunit.client.0.smithi031.stdout:0/924: dread da/d20/d4e/f51 [0,4194304] 0 2024-04-24T23:14:01.690 INFO:tasks.workunit.client.0.smithi031.stdout:0/925: stat da/d20/d8e 0 2024-04-24T23:14:01.690 INFO:tasks.workunit.client.0.smithi031.stdout:1/803: dread d1/d33/d41/d9b/da8/fa9 [0,4194304] 0 2024-04-24T23:14:01.693 INFO:tasks.workunit.client.0.smithi031.stdout:9/955: write d2/dae/d14/d1d/d45/fd8 [687226,41991] 0 2024-04-24T23:14:01.695 INFO:tasks.workunit.client.0.smithi031.stdout:5/807: mknod d0/d7/d74/dbd/c10d 0 2024-04-24T23:14:01.696 INFO:tasks.workunit.client.0.smithi031.stdout:9/956: write d2/d43/fe6 [4190214,74043] 0 2024-04-24T23:14:01.698 INFO:tasks.workunit.client.1.smithi040.stdout:9/652: creat d7/d92/d9c/fe6 x:0 0 0 2024-04-24T23:14:01.700 INFO:tasks.workunit.client.0.smithi031.stdout:0/926: dwrite da/d86/da3/dcd/ddc/f118 [0,4194304] 0 2024-04-24T23:14:01.702 INFO:tasks.workunit.client.1.smithi040.stdout:3/665: unlink d1/d2d/c85 0 2024-04-24T23:14:01.703 INFO:tasks.workunit.client.1.smithi040.stdout:2/654: sync 2024-04-24T23:14:01.710 INFO:tasks.workunit.client.0.smithi031.stdout:1/804: link d1/d31/f39 d1/d3e/d46/d74/f10c 0 2024-04-24T23:14:01.711 INFO:tasks.workunit.client.1.smithi040.stdout:6/586: fsync d7/d1a/d20/d51/f26 0 2024-04-24T23:14:01.712 INFO:tasks.workunit.client.1.smithi040.stdout:6/587: chown d7/d1a/d20/d51/d2f/f35 1 1 2024-04-24T23:14:01.712 INFO:tasks.workunit.client.0.smithi031.stdout:5/808: rename d0/d6/d3b/d4b/dc/d25/d55/ce6 to d0/c10e 0 2024-04-24T23:14:01.717 INFO:tasks.workunit.client.0.smithi031.stdout:0/927: rename da/d18/d5a/d7d/fef to da/d20/d4e/dc9/d11b/f137 0 2024-04-24T23:14:01.718 INFO:tasks.workunit.client.0.smithi031.stdout:0/928: stat da/d86/da3/d104/d130 0 2024-04-24T23:14:01.720 INFO:tasks.workunit.client.0.smithi031.stdout:5/809: creat d0/d6/d3b/d4b/dc/d86/d91/f10f x:0 0 0 2024-04-24T23:14:01.725 INFO:tasks.workunit.client.0.smithi031.stdout:1/805: rename d1/da/d12/d65/dcf/f100 to d1/da/d12/d65/dcf/d11/d21/d5a/dbb/dff/f10d 0 2024-04-24T23:14:01.726 INFO:tasks.workunit.client.1.smithi040.stdout:2/655: mknod da/d81/cf1 0 2024-04-24T23:14:01.728 INFO:tasks.workunit.client.1.smithi040.stdout:7/642: write db/f61 [454431,21034] 0 2024-04-24T23:14:01.730 INFO:tasks.workunit.client.1.smithi040.stdout:1/613: getdents da/d35/d38/d47/dbc 0 2024-04-24T23:14:01.730 INFO:tasks.workunit.client.0.smithi031.stdout:9/957: write d2/dae/d14/d78/d84/d4a/f8d [1957990,119778] 0 2024-04-24T23:14:01.732 INFO:tasks.workunit.client.0.smithi031.stdout:9/958: chown d2/dae/d14/d78/fea 3 1 2024-04-24T23:14:01.732 INFO:tasks.workunit.client.1.smithi040.stdout:2/656: dwrite da/d1c/f57 [0,4194304] 0 2024-04-24T23:14:01.732 INFO:tasks.workunit.client.1.smithi040.stdout:4/549: creat da/d78/dbb/fc2 x:0 0 0 2024-04-24T23:14:01.733 INFO:tasks.workunit.client.0.smithi031.stdout:9/959: stat d2/dae/d14/d78/d84/d4a/dab/db7 0 2024-04-24T23:14:01.733 INFO:tasks.workunit.client.0.smithi031.stdout:0/929: mknod da/d18/d1a/d37/c138 0 2024-04-24T23:14:01.736 INFO:tasks.workunit.client.1.smithi040.stdout:8/651: truncate d0/d52/f84 6379458 0 2024-04-24T23:14:01.741 INFO:tasks.workunit.client.1.smithi040.stdout:3/666: mknod d1/d9/d39/ce4 0 2024-04-24T23:14:01.743 INFO:tasks.workunit.client.1.smithi040.stdout:0/681: sync 2024-04-24T23:14:01.744 INFO:tasks.workunit.client.1.smithi040.stdout:7/643: symlink db/d2f/da8/le8 0 2024-04-24T23:14:01.744 INFO:tasks.workunit.client.1.smithi040.stdout:0/682: readlink df/d12/d58/d5b/lac 0 2024-04-24T23:14:01.745 INFO:tasks.workunit.client.1.smithi040.stdout:1/614: unlink da/d12/d14/d25/c74 0 2024-04-24T23:14:01.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/810: dwrite d0/d6/d3b/d4b/dc/d86/dfd/f102 [0,4194304] 0 2024-04-24T23:14:01.751 INFO:tasks.workunit.client.0.smithi031.stdout:1/806: dread d1/da/d12/d65/dcf/d11/f24 [0,4194304] 0 2024-04-24T23:14:01.752 INFO:tasks.workunit.client.0.smithi031.stdout:1/807: stat d1/ff 0 2024-04-24T23:14:01.752 INFO:tasks.workunit.client.1.smithi040.stdout:6/588: mknod d7/de/cb8 0 2024-04-24T23:14:01.755 INFO:tasks.workunit.client.1.smithi040.stdout:5/572: creat d2/d2f/d14/fbc x:0 0 0 2024-04-24T23:14:01.758 INFO:tasks.workunit.client.1.smithi040.stdout:9/653: creat d7/da/d17/fe7 x:0 0 0 2024-04-24T23:14:01.759 INFO:tasks.workunit.client.0.smithi031.stdout:0/930: dwrite da/d22/f96 [0,4194304] 0 2024-04-24T23:14:01.761 INFO:tasks.workunit.client.0.smithi031.stdout:1/808: rmdir d1/da/d12/d65/dcf/d11/d21/d5a/dbb 39 2024-04-24T23:14:01.761 INFO:tasks.workunit.client.0.smithi031.stdout:0/931: write da/d22/d62/f122 [762999,119652] 0 2024-04-24T23:14:01.766 INFO:tasks.workunit.client.1.smithi040.stdout:7/644: symlink db/d12/d20/d52/d4b/le9 0 2024-04-24T23:14:01.767 INFO:tasks.workunit.client.0.smithi031.stdout:5/811: mknod d0/d6/d3b/d4b/d1d/d2e/c110 0 2024-04-24T23:14:01.769 INFO:tasks.workunit.client.0.smithi031.stdout:1/809: dread - d1/da/df0/fb7 zero size 2024-04-24T23:14:01.769 INFO:tasks.workunit.client.1.smithi040.stdout:5/573: dread d2/d7/d8/d1f/d57/d27/f32 [0,4194304] 0 2024-04-24T23:14:01.770 INFO:tasks.workunit.client.1.smithi040.stdout:4/550: symlink da/lc3 0 2024-04-24T23:14:01.771 INFO:tasks.workunit.client.0.smithi031.stdout:0/932: fdatasync da/d86/dbf/ff2 0 2024-04-24T23:14:01.772 INFO:tasks.workunit.client.1.smithi040.stdout:4/551: chown da/d3f/d6b/l81 227970888 1 2024-04-24T23:14:01.772 INFO:tasks.workunit.client.1.smithi040.stdout:4/552: chown l1 5741265 1 2024-04-24T23:14:01.779 INFO:tasks.workunit.client.1.smithi040.stdout:6/589: dread d7/de/f5e [0,4194304] 0 2024-04-24T23:14:01.782 INFO:tasks.workunit.client.0.smithi031.stdout:5/812: mknod d0/d6/d3b/d4b/d1d/d2e/d28/deb/c111 0 2024-04-24T23:14:01.783 INFO:tasks.workunit.client.1.smithi040.stdout:2/657: write da/d1c/d37/d39/f76 [4778121,61700] 0 2024-04-24T23:14:01.784 INFO:tasks.workunit.client.0.smithi031.stdout:1/810: mkdir d1/da/d12/d7e/dfc/d10e 0 2024-04-24T23:14:01.785 INFO:tasks.workunit.client.1.smithi040.stdout:6/590: dwrite d7/d1a/d20/d51/d1e/f86 [0,4194304] 0 2024-04-24T23:14:01.785 INFO:tasks.workunit.client.1.smithi040.stdout:7/645: symlink db/d43/db7/lea 0 2024-04-24T23:14:01.786 INFO:tasks.workunit.client.0.smithi031.stdout:1/811: write d1/d33/fdc [236619,122254] 0 2024-04-24T23:14:01.786 INFO:tasks.workunit.client.1.smithi040.stdout:0/683: getdents df/d12/d58/d5b 0 2024-04-24T23:14:01.786 INFO:tasks.workunit.client.1.smithi040.stdout:7/646: fsync db/d12/d7d/fc5 0 2024-04-24T23:14:01.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/813: mkdir d0/d7/d74/dbd/d112 0 2024-04-24T23:14:01.787 INFO:tasks.workunit.client.1.smithi040.stdout:0/684: fdatasync df/d12/f4c 0 2024-04-24T23:14:01.787 INFO:tasks.workunit.client.1.smithi040.stdout:8/652: sync 2024-04-24T23:14:01.788 INFO:tasks.workunit.client.0.smithi031.stdout:1/812: dread - d1/da/d12/d7e/d7b/d4e/f94 zero size 2024-04-24T23:14:01.788 INFO:tasks.workunit.client.1.smithi040.stdout:5/574: mknod d2/d7/d8/d1f/da1/cbd 0 2024-04-24T23:14:01.789 INFO:tasks.workunit.client.1.smithi040.stdout:7/647: chown db/d12/d20/d52/d4b/f57 81024 1 2024-04-24T23:14:01.789 INFO:tasks.workunit.client.1.smithi040.stdout:1/615: mknod da/d12/d18/d2a/d8b/dd3/cd5 0 2024-04-24T23:14:01.790 INFO:tasks.workunit.client.0.smithi031.stdout:1/813: dread - d1/da/fe0 zero size 2024-04-24T23:14:01.791 INFO:tasks.workunit.client.1.smithi040.stdout:1/616: chown da/d12/d23/dac 3 1 2024-04-24T23:14:01.791 INFO:tasks.workunit.client.1.smithi040.stdout:7/648: write db/d2f/d37/f94 [4083193,38543] 0 2024-04-24T23:14:01.791 INFO:tasks.workunit.client.0.smithi031.stdout:5/814: write d0/d6/d3b/d4b/d1d/d2e/d28/d77/ff7 [2904165,54025] 0 2024-04-24T23:14:01.792 INFO:tasks.workunit.client.1.smithi040.stdout:4/553: rename da/d3f/d6b/d3c/d69/f84 to da/d3f/d40/d41/d8e/db8/fc4 0 2024-04-24T23:14:01.792 INFO:tasks.workunit.client.0.smithi031.stdout:0/933: dwrite da/d11/f100 [0,4194304] 0 2024-04-24T23:14:01.798 INFO:tasks.workunit.client.1.smithi040.stdout:5/575: dread d2/d2f/d14/f61 [0,4194304] 0 2024-04-24T23:14:01.799 INFO:tasks.workunit.client.1.smithi040.stdout:5/576: chown d2/d7/d8/d1f/d57/d27/d3c/fb9 31630 1 2024-04-24T23:14:01.801 INFO:tasks.workunit.client.0.smithi031.stdout:1/814: truncate d1/da/d12/d65/fb6 758698 0 2024-04-24T23:14:01.802 INFO:tasks.workunit.client.1.smithi040.stdout:6/591: symlink d7/d1a/d20/d51/daa/lb9 0 2024-04-24T23:14:01.804 INFO:tasks.workunit.client.0.smithi031.stdout:0/934: mknod da/d22/d62/c139 0 2024-04-24T23:14:01.804 INFO:tasks.workunit.client.0.smithi031.stdout:5/815: dwrite d0/d6/d3b/d4b/dc/f9a [0,4194304] 0 2024-04-24T23:14:01.806 INFO:tasks.workunit.client.0.smithi031.stdout:1/815: mknod d1/d3e/c10f 0 2024-04-24T23:14:01.807 INFO:tasks.workunit.client.1.smithi040.stdout:8/653: mknod d0/d38/cda 0 2024-04-24T23:14:01.811 INFO:tasks.workunit.client.0.smithi031.stdout:1/816: rmdir d1/d33/d41/d9b 39 2024-04-24T23:14:01.812 INFO:tasks.workunit.client.1.smithi040.stdout:1/617: mkdir da/d35/dd6 0 2024-04-24T23:14:01.813 INFO:tasks.workunit.client.1.smithi040.stdout:7/649: mknod db/d12/d62/ceb 0 2024-04-24T23:14:01.814 INFO:tasks.workunit.client.0.smithi031.stdout:1/817: dread d1/da/d12/d7e/d7b/d4e/fd7 [0,4194304] 0 2024-04-24T23:14:01.815 INFO:tasks.workunit.client.1.smithi040.stdout:8/654: dwrite d0/d16/d19/fac [0,4194304] 0 2024-04-24T23:14:01.816 INFO:tasks.workunit.client.1.smithi040.stdout:0/685: rename df/d12/d4b/f8b to df/d12/d58/dc3/dcd/fe0 0 2024-04-24T23:14:01.818 INFO:tasks.workunit.client.1.smithi040.stdout:0/686: read df/d1e/d20/d21/d51/f8f [258783,30129] 0 2024-04-24T23:14:01.818 INFO:tasks.workunit.client.1.smithi040.stdout:4/554: creat da/d3f/d6b/d3c/fc5 x:0 0 0 2024-04-24T23:14:01.820 INFO:tasks.workunit.client.1.smithi040.stdout:9/654: creat d7/fe8 x:0 0 0 2024-04-24T23:14:01.821 INFO:tasks.workunit.client.1.smithi040.stdout:9/655: chown d7/da/d17/d40/d55/d2f 326248 1 2024-04-24T23:14:01.821 INFO:tasks.workunit.client.0.smithi031.stdout:1/818: creat d1/d33/f110 x:0 0 0 2024-04-24T23:14:01.822 INFO:tasks.workunit.client.0.smithi031.stdout:1/819: stat d1/da/d12/f83 0 2024-04-24T23:14:01.825 INFO:tasks.workunit.client.1.smithi040.stdout:3/667: getdents d1 0 2024-04-24T23:14:01.826 INFO:tasks.workunit.client.0.smithi031.stdout:5/816: getdents d0/d6/d3b/d4b/d1d/d2e 0 2024-04-24T23:14:01.828 INFO:tasks.workunit.client.0.smithi031.stdout:1/820: rename d1/da/d12/d7e/d7b/d4e/c67 to d1/d33/d36/c111 0 2024-04-24T23:14:01.832 INFO:tasks.workunit.client.0.smithi031.stdout:5/817: chown d0/d6/d3b/d4b/d1d/ld9 262596170 1 2024-04-24T23:14:01.835 INFO:tasks.workunit.client.0.smithi031.stdout:1/821: unlink d1/da/fe4 0 2024-04-24T23:14:01.835 INFO:tasks.workunit.client.0.smithi031.stdout:1/822: dread - d1/da/fe3 zero size 2024-04-24T23:14:01.836 INFO:tasks.workunit.client.1.smithi040.stdout:1/618: creat da/d12/d14/d25/d5e/fd7 x:0 0 0 2024-04-24T23:14:01.838 INFO:tasks.workunit.client.1.smithi040.stdout:1/619: write da/d12/d18/d2a/d41/fa8 [376989,48095] 0 2024-04-24T23:14:01.839 INFO:tasks.workunit.client.1.smithi040.stdout:8/655: mknod d0/d16/d1b/d58/d8f/cdb 0 2024-04-24T23:14:01.839 INFO:tasks.workunit.client.1.smithi040.stdout:2/658: dwrite da/f19 [0,4194304] 0 2024-04-24T23:14:01.840 INFO:tasks.workunit.client.0.smithi031.stdout:0/935: write da/d86/fb1 [252123,93701] 0 2024-04-24T23:14:01.844 INFO:tasks.workunit.client.1.smithi040.stdout:6/592: rename d7/d1a/d20/d51/d1e/d5d/d52 to d7/d1a/d20/d51/d2f/dba 0 2024-04-24T23:14:01.849 INFO:tasks.workunit.client.1.smithi040.stdout:4/555: fdatasync da/f37 0 2024-04-24T23:14:01.849 INFO:tasks.workunit.client.1.smithi040.stdout:4/556: readlink da/d3f/d40/d41/l76 0 2024-04-24T23:14:01.851 INFO:tasks.workunit.client.1.smithi040.stdout:5/577: mkdir d2/d2f/d14/dbe 0 2024-04-24T23:14:01.853 INFO:tasks.workunit.client.1.smithi040.stdout:4/557: dwrite da/d45/d58/f98 [0,4194304] 0 2024-04-24T23:14:01.854 INFO:tasks.workunit.client.1.smithi040.stdout:9/656: rmdir d7/da/d17/d40/d55/d2f/d1f/d4e/d83 39 2024-04-24T23:14:01.856 INFO:tasks.workunit.client.0.smithi031.stdout:5/818: link d0/d6/d3b/d4b/d1d/d2e/d28/d77/lcf d0/d6/d3b/d4b/dc/d8b/l113 0 2024-04-24T23:14:01.860 INFO:tasks.workunit.client.0.smithi031.stdout:1/823: dread d1/d33/d41/d9b/da8/fd1 [0,4194304] 0 2024-04-24T23:14:01.863 INFO:tasks.workunit.client.1.smithi040.stdout:1/620: creat da/d35/da3/fd8 x:0 0 0 2024-04-24T23:14:01.867 INFO:tasks.workunit.client.0.smithi031.stdout:1/824: mknod d1/d33/d36/d81/c112 0 2024-04-24T23:14:01.870 INFO:tasks.workunit.client.1.smithi040.stdout:8/656: creat d0/d16/d1b/d47/fdc x:0 0 0 2024-04-24T23:14:01.871 INFO:tasks.workunit.client.1.smithi040.stdout:8/657: dread - d0/d16/d1b/d58/fa5 zero size 2024-04-24T23:14:01.873 INFO:tasks.workunit.client.1.smithi040.stdout:7/650: rename db/d12/d62/d8a/l8e to db/d12/d20/d56/lec 0 2024-04-24T23:14:01.874 INFO:tasks.workunit.client.1.smithi040.stdout:8/658: dread d0/d16/f3b [0,4194304] 0 2024-04-24T23:14:01.875 INFO:tasks.workunit.client.1.smithi040.stdout:7/651: dread db/d12/d20/d52/d44/f53 [0,4194304] 0 2024-04-24T23:14:01.876 INFO:tasks.workunit.client.1.smithi040.stdout:6/593: mkdir d7/d1a/d20/d51/d1e/d5d/dbb 0 2024-04-24T23:14:01.880 INFO:tasks.workunit.client.1.smithi040.stdout:8/659: dwrite d0/d8/d66/f78 [0,4194304] 0 2024-04-24T23:14:01.881 INFO:tasks.workunit.client.1.smithi040.stdout:8/660: readlink d0/lb 0 2024-04-24T23:14:01.882 INFO:tasks.workunit.client.0.smithi031.stdout:0/936: write da/d20/d4d/fcf [451708,43105] 0 2024-04-24T23:14:01.884 INFO:tasks.workunit.client.1.smithi040.stdout:3/668: write d1/d9/d18/d36/d55/f56 [3517272,37044] 0 2024-04-24T23:14:01.886 INFO:tasks.workunit.client.0.smithi031.stdout:1/825: dread d1/da/fad [0,4194304] 0 2024-04-24T23:14:01.890 INFO:tasks.workunit.client.1.smithi040.stdout:4/558: rmdir da/d3f 39 2024-04-24T23:14:01.893 INFO:tasks.workunit.client.1.smithi040.stdout:9/657: creat d7/da/d17/d40/d55/d61/fe9 x:0 0 0 2024-04-24T23:14:01.895 INFO:tasks.workunit.client.0.smithi031.stdout:1/826: rmdir d1/d33/d41 39 2024-04-24T23:14:01.899 INFO:tasks.workunit.client.0.smithi031.stdout:5/819: dwrite d0/d6/d3b/d4b/dc/d25/f62 [4194304,4194304] 0 2024-04-24T23:14:01.901 INFO:tasks.workunit.client.0.smithi031.stdout:5/820: fdatasync d0/d7/d74/de2/ffc 0 2024-04-24T23:14:01.906 INFO:tasks.workunit.client.0.smithi031.stdout:5/821: symlink d0/d6/d3b/d4b/d1d/d2e/d28/d77/l114 0 2024-04-24T23:14:01.908 INFO:tasks.workunit.client.0.smithi031.stdout:5/822: chown d0/d6/d3b/d4b/d1d/d2e/c92 4205 1 2024-04-24T23:14:01.909 INFO:tasks.workunit.client.0.smithi031.stdout:1/827: write d1/da/d12/d7e/d7b/d4e/f94 [696998,80827] 0 2024-04-24T23:14:01.912 INFO:tasks.workunit.client.1.smithi040.stdout:8/661: mkdir d0/d38/d4c/ddd 0 2024-04-24T23:14:01.912 INFO:tasks.workunit.client.0.smithi031.stdout:5/823: read d0/d6/d3b/d4b/dc/f26 [4456506,77161] 0 2024-04-24T23:14:01.913 INFO:tasks.workunit.client.1.smithi040.stdout:2/659: dwrite da/f11 [4194304,4194304] 0 2024-04-24T23:14:01.913 INFO:tasks.workunit.client.0.smithi031.stdout:0/937: dwrite da/d11/f5d [0,4194304] 0 2024-04-24T23:14:01.914 INFO:tasks.workunit.client.1.smithi040.stdout:8/662: stat d0/d16/d1b/d3f/d44/ld6 0 2024-04-24T23:14:01.914 INFO:tasks.workunit.client.1.smithi040.stdout:5/578: unlink d2/d7/d8/d1f/d57/d3e/f58 0 2024-04-24T23:14:01.915 INFO:tasks.workunit.client.1.smithi040.stdout:8/663: dread - d0/d16/d1b/d47/fdc zero size 2024-04-24T23:14:01.916 INFO:tasks.workunit.client.1.smithi040.stdout:3/669: dread - d1/d9/d18/d36/d55/f5d zero size 2024-04-24T23:14:01.918 INFO:tasks.workunit.client.0.smithi031.stdout:1/828: creat d1/d3e/da3/d69/f113 x:0 0 0 2024-04-24T23:14:01.922 INFO:tasks.workunit.client.0.smithi031.stdout:5/824: creat d0/d6/f115 x:0 0 0 2024-04-24T23:14:01.923 INFO:tasks.workunit.client.1.smithi040.stdout:5/579: dread d2/d7/f36 [0,4194304] 0 2024-04-24T23:14:01.924 INFO:tasks.workunit.client.1.smithi040.stdout:5/580: chown d2/d7/d8/d6d/d89 16105686 1 2024-04-24T23:14:01.924 INFO:tasks.workunit.client.0.smithi031.stdout:0/938: mkdir da/d11/d6a/dba/d13a 0 2024-04-24T23:14:01.925 INFO:tasks.workunit.client.0.smithi031.stdout:0/939: stat da/d11/d6a/f94 0 2024-04-24T23:14:01.925 INFO:tasks.workunit.client.1.smithi040.stdout:5/581: readlink d2/d7/d8/db/l1a 0 2024-04-24T23:14:01.926 INFO:tasks.workunit.client.1.smithi040.stdout:4/559: dwrite da/d3f/d6b/d2b/f55 [0,4194304] 0 2024-04-24T23:14:01.930 INFO:tasks.workunit.client.0.smithi031.stdout:0/940: dread da/d18/d1a/d37/f26 [0,4194304] 0 2024-04-24T23:14:01.931 INFO:tasks.workunit.client.1.smithi040.stdout:7/652: link db/d12/d3f/fc1 db/d12/d62/d8a/db3/fed 0 2024-04-24T23:14:01.932 INFO:tasks.workunit.client.1.smithi040.stdout:6/594: rmdir d7/d1a/d20/d51/d1e/d46/d69/d87 39 2024-04-24T23:14:01.949 INFO:tasks.workunit.client.1.smithi040.stdout:6/595: dread d7/d1a/d20/d51/d1e/f72 [0,4194304] 0 2024-04-24T23:14:02.005 INFO:tasks.workunit.client.0.smithi031.stdout:0/941: rename da/d86/f125 to da/d18/d5a/f13b 0 2024-04-24T23:14:02.006 INFO:tasks.workunit.client.1.smithi040.stdout:2/660: creat da/d81/de6/ff2 x:0 0 0 2024-04-24T23:14:02.006 INFO:tasks.workunit.client.1.smithi040.stdout:2/661: chown da/d10/d15/daa 11 1 2024-04-24T23:14:02.013 INFO:tasks.workunit.client.0.smithi031.stdout:0/942: getdents da/d18/d1a/d37/d111 0 2024-04-24T23:14:02.014 INFO:tasks.workunit.client.0.smithi031.stdout:1/829: write d1/da/d12/d65/dcf/d11/fba [1005239,68804] 0 2024-04-24T23:14:02.015 INFO:tasks.workunit.client.0.smithi031.stdout:0/943: chown da/d18/d5a/d7d/f116 553167 1 2024-04-24T23:14:02.015 INFO:tasks.workunit.client.1.smithi040.stdout:9/658: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/cea 0 2024-04-24T23:14:02.016 INFO:tasks.workunit.client.0.smithi031.stdout:5/825: write d0/d6/fe0 [827532,71017] 0 2024-04-24T23:14:02.017 INFO:tasks.workunit.client.1.smithi040.stdout:1/621: creat da/fd9 x:0 0 0 2024-04-24T23:14:02.020 INFO:tasks.workunit.client.1.smithi040.stdout:6/596: sync 2024-04-24T23:14:02.022 INFO:tasks.workunit.client.0.smithi031.stdout:0/944: dwrite da/d18/d5a/d7d/f116 [0,4194304] 0 2024-04-24T23:14:02.023 INFO:tasks.workunit.client.0.smithi031.stdout:5/826: mknod d0/d6/d3b/d4b/dc/d25/c116 0 2024-04-24T23:14:02.026 INFO:tasks.workunit.client.1.smithi040.stdout:0/687: rename df/d1e/d2b to df/de1 0 2024-04-24T23:14:02.027 INFO:tasks.workunit.client.1.smithi040.stdout:0/688: fdatasync df/d12/d58/f7d 0 2024-04-24T23:14:02.032 INFO:tasks.workunit.client.0.smithi031.stdout:1/830: creat d1/da/d12/f114 x:0 0 0 2024-04-24T23:14:02.033 INFO:tasks.workunit.client.1.smithi040.stdout:0/689: dwrite df/d14/f88 [0,4194304] 0 2024-04-24T23:14:02.035 INFO:tasks.workunit.client.1.smithi040.stdout:0/690: truncate df/d1e/d20/d21/d51/f8f 1816079 0 2024-04-24T23:14:02.035 INFO:tasks.workunit.client.1.smithi040.stdout:2/662: symlink da/d1c/d37/d8a/lf3 0 2024-04-24T23:14:02.036 INFO:tasks.workunit.client.1.smithi040.stdout:2/663: chown da/d1c/d37/l51 12 1 2024-04-24T23:14:02.036 INFO:tasks.workunit.client.0.smithi031.stdout:5/827: mkdir d0/d117 0 2024-04-24T23:14:02.037 INFO:tasks.workunit.client.1.smithi040.stdout:8/664: truncate d0/d8/dc/f86 1504524 0 2024-04-24T23:14:02.038 INFO:tasks.workunit.client.1.smithi040.stdout:5/582: write d2/d7/d8/db/f75 [1170723,130482] 0 2024-04-24T23:14:02.045 INFO:tasks.workunit.client.1.smithi040.stdout:4/560: mknod da/cc6 0 2024-04-24T23:14:02.046 INFO:tasks.workunit.client.1.smithi040.stdout:4/561: write da/d1f/f9c [231783,75958] 0 2024-04-24T23:14:02.048 INFO:tasks.workunit.client.0.smithi031.stdout:0/945: creat da/d20/f13c x:0 0 0 2024-04-24T23:14:02.050 INFO:tasks.workunit.client.0.smithi031.stdout:0/946: chown da/d20/d4d/df8/c108 345 1 2024-04-24T23:14:02.050 INFO:tasks.workunit.client.1.smithi040.stdout:3/670: rename d1/d9/d18/c3e to d1/d9/ce5 0 2024-04-24T23:14:02.051 INFO:tasks.workunit.client.0.smithi031.stdout:0/947: fsync da/d86/da3/dcd/ddc/f118 0 2024-04-24T23:14:02.053 INFO:tasks.workunit.client.1.smithi040.stdout:6/597: dread d7/d1a/d20/d51/f26 [0,4194304] 0 2024-04-24T23:14:02.053 INFO:tasks.workunit.client.1.smithi040.stdout:6/598: fdatasync d7/d1a/d9f/fa8 0 2024-04-24T23:14:02.056 INFO:tasks.workunit.client.1.smithi040.stdout:6/599: dread d7/d1a/d20/f23 [4194304,4194304] 0 2024-04-24T23:14:02.056 INFO:tasks.workunit.client.1.smithi040.stdout:0/691: symlink df/d1e/d20/d21/d41/le2 0 2024-04-24T23:14:02.057 INFO:tasks.workunit.client.1.smithi040.stdout:0/692: fdatasync df/de1/d6e/fba 0 2024-04-24T23:14:02.058 INFO:tasks.workunit.client.1.smithi040.stdout:6/600: write d7/de/d89/fb7 [754435,57772] 0 2024-04-24T23:14:02.058 INFO:tasks.workunit.client.1.smithi040.stdout:8/665: dread d0/d8/f29 [0,4194304] 0 2024-04-24T23:14:02.059 INFO:tasks.workunit.client.0.smithi031.stdout:1/831: write d1/d33/d36/f91 [4552514,4795] 0 2024-04-24T23:14:02.061 INFO:tasks.workunit.client.1.smithi040.stdout:5/583: dread d2/d7/d8/d1f/f9d [0,4194304] 0 2024-04-24T23:14:02.061 INFO:tasks.workunit.client.0.smithi031.stdout:5/828: write d0/d6/d3b/d4b/dc/f6e [1482918,117410] 0 2024-04-24T23:14:02.062 INFO:tasks.workunit.client.1.smithi040.stdout:0/693: read f6 [1379336,121352] 0 2024-04-24T23:14:02.062 INFO:tasks.workunit.client.1.smithi040.stdout:5/584: stat d2/d2f/d14/dbe 0 2024-04-24T23:14:02.064 INFO:tasks.workunit.client.1.smithi040.stdout:5/585: write d2/d7/d8/db/f75 [1145270,53920] 0 2024-04-24T23:14:02.065 INFO:tasks.workunit.client.1.smithi040.stdout:9/659: creat d7/d5e/da1/feb x:0 0 0 2024-04-24T23:14:02.066 INFO:tasks.workunit.client.1.smithi040.stdout:6/601: dwrite d7/d1a/d9f/fb3 [0,4194304] 0 2024-04-24T23:14:02.067 INFO:tasks.workunit.client.0.smithi031.stdout:1/832: dwrite d1/da/d12/d65/dcf/d11/fc2 [0,4194304] 0 2024-04-24T23:14:02.073 INFO:tasks.workunit.client.1.smithi040.stdout:4/562: rename da/d3f/l7b to da/d78/lc7 0 2024-04-24T23:14:02.083 INFO:tasks.workunit.client.1.smithi040.stdout:3/671: dread d1/d2d/f60 [0,4194304] 0 2024-04-24T23:14:02.084 INFO:tasks.workunit.client.1.smithi040.stdout:2/664: symlink da/d1c/lf4 0 2024-04-24T23:14:02.085 INFO:tasks.workunit.client.1.smithi040.stdout:3/672: chown d1/d9/d18/d36/d55/d7e/fc7 0 1 2024-04-24T23:14:02.094 INFO:tasks.workunit.client.1.smithi040.stdout:9/660: symlink d7/da/d17/d40/lec 0 2024-04-24T23:14:02.096 INFO:tasks.workunit.client.1.smithi040.stdout:1/622: creat da/d35/d38/fda x:0 0 0 2024-04-24T23:14:02.099 INFO:tasks.workunit.client.1.smithi040.stdout:6/602: rename d7/de/l3b to d7/d1a/lbc 0 2024-04-24T23:14:02.103 INFO:tasks.workunit.client.1.smithi040.stdout:7/653: getdents db/d12/d20/d56 0 2024-04-24T23:14:02.108 INFO:tasks.workunit.client.1.smithi040.stdout:2/665: mkdir da/d1c/d37/d39/d3f/d52/d78/df5 0 2024-04-24T23:14:02.119 INFO:tasks.workunit.client.1.smithi040.stdout:9/661: mkdir d7/da/d17/d40/d55/d2f/ded 0 2024-04-24T23:14:02.122 INFO:tasks.workunit.client.1.smithi040.stdout:1/623: read da/f7e [3084415,9545] 0 2024-04-24T23:14:02.124 INFO:tasks.workunit.client.1.smithi040.stdout:1/624: dread - da/d12/d14/d25/d5e/fd7 zero size 2024-04-24T23:14:02.125 INFO:tasks.workunit.client.1.smithi040.stdout:1/625: chown da/d12/f21 3261608 1 2024-04-24T23:14:02.126 INFO:tasks.workunit.client.1.smithi040.stdout:4/563: dwrite da/d3f/d40/d41/f4e [0,4194304] 0 2024-04-24T23:14:02.126 INFO:tasks.workunit.client.1.smithi040.stdout:4/564: chown da/lc3 62765211 1 2024-04-24T23:14:02.127 INFO:tasks.workunit.client.1.smithi040.stdout:1/626: dread - da/d12/d18/d2a/d8b/d97/f98 zero size 2024-04-24T23:14:02.127 INFO:tasks.workunit.client.1.smithi040.stdout:4/565: chown da/d45/fac 401353 1 2024-04-24T23:14:02.128 INFO:tasks.workunit.client.1.smithi040.stdout:2/666: creat da/d81/de6/ff6 x:0 0 0 2024-04-24T23:14:02.129 INFO:tasks.workunit.client.1.smithi040.stdout:4/566: readlink da/d3f/d40/d41/d8e/lbc 0 2024-04-24T23:14:02.129 INFO:tasks.workunit.client.1.smithi040.stdout:8/666: write d0/d8/d55/f83 [372795,59556] 0 2024-04-24T23:14:02.130 INFO:tasks.workunit.client.1.smithi040.stdout:4/567: dread - da/d3f/d40/d47/d71/fa9 zero size 2024-04-24T23:14:02.132 INFO:tasks.workunit.client.1.smithi040.stdout:8/667: write d0/d7e/dc1/d77/fcd [22566,125495] 0 2024-04-24T23:14:02.133 INFO:tasks.workunit.client.1.smithi040.stdout:3/673: creat d1/d9/d2e/dcf/fe6 x:0 0 0 2024-04-24T23:14:02.138 INFO:tasks.workunit.client.1.smithi040.stdout:0/694: dwrite df/f34 [0,4194304] 0 2024-04-24T23:14:02.139 INFO:tasks.workunit.client.1.smithi040.stdout:6/603: write d7/d1a/d20/d51/f26 [686238,27486] 0 2024-04-24T23:14:02.144 INFO:tasks.workunit.client.1.smithi040.stdout:7/654: fdatasync db/d12/d20/d52/f28 0 2024-04-24T23:14:02.145 INFO:tasks.workunit.client.1.smithi040.stdout:7/655: fdatasync db/d12/d20/d52/d6c/d8f/db8/fc8 0 2024-04-24T23:14:02.148 INFO:tasks.workunit.client.1.smithi040.stdout:2/667: symlink da/d1c/d4f/lf7 0 2024-04-24T23:14:02.150 INFO:tasks.workunit.client.1.smithi040.stdout:4/568: creat da/d3f/d40/d47/d75/daa/fc8 x:0 0 0 2024-04-24T23:14:02.150 INFO:tasks.workunit.client.1.smithi040.stdout:2/668: write da/d29/d55/fa9 [2574581,88519] 0 2024-04-24T23:14:02.150 INFO:tasks.workunit.client.1.smithi040.stdout:7/656: dwrite db/d2f/d37/fa6 [0,4194304] 0 2024-04-24T23:14:02.152 INFO:tasks.workunit.client.1.smithi040.stdout:2/669: write da/d1c/d37/fc4 [659946,22152] 0 2024-04-24T23:14:02.154 INFO:tasks.workunit.client.0.smithi031.stdout:0/948: truncate da/ff 4015640 0 2024-04-24T23:14:02.155 INFO:tasks.workunit.client.0.smithi031.stdout:0/949: chown da/d11/d6a/dba/dc0/f121 130 1 2024-04-24T23:14:02.158 INFO:tasks.workunit.client.1.smithi040.stdout:4/569: dwrite da/d45/d48/d6e/fbd [0,4194304] 0 2024-04-24T23:14:02.162 INFO:tasks.workunit.client.0.smithi031.stdout:5/829: creat d0/d6/d3b/d4b/dc/d86/dc3/f118 x:0 0 0 2024-04-24T23:14:02.163 INFO:tasks.workunit.client.0.smithi031.stdout:5/830: stat d0/d7/d74/ldd 0 2024-04-24T23:14:02.164 INFO:tasks.workunit.client.1.smithi040.stdout:4/570: dwrite da/d3f/fb0 [0,4194304] 0 2024-04-24T23:14:02.169 INFO:tasks.workunit.client.0.smithi031.stdout:0/950: mknod da/d18/d5a/c13d 0 2024-04-24T23:14:02.171 INFO:tasks.workunit.client.1.smithi040.stdout:4/571: dwrite da/d3f/d6b/d2b/f35 [0,4194304] 0 2024-04-24T23:14:02.173 INFO:tasks.workunit.client.0.smithi031.stdout:5/831: mknod d0/d6/d3b/d4b/dc/d86/dc3/c119 0 2024-04-24T23:14:02.181 INFO:tasks.workunit.client.0.smithi031.stdout:5/832: dwrite d0/d6/d3b/d4b/dc/f84 [0,4194304] 0 2024-04-24T23:14:02.184 INFO:tasks.workunit.client.0.smithi031.stdout:5/833: stat d0/d6/d3b/d4b/d1d/d2e/d28/d77/ff7 0 2024-04-24T23:14:02.187 INFO:tasks.workunit.client.0.smithi031.stdout:1/833: getdents d1/da/d12/d7e/d7b 0 2024-04-24T23:14:02.189 INFO:tasks.workunit.client.1.smithi040.stdout:8/668: mkdir d0/d16/d1b/d58/d8f/db1/dde 0 2024-04-24T23:14:02.189 INFO:tasks.workunit.client.0.smithi031.stdout:5/834: mkdir d0/d6/d32/d6b/d11a 0 2024-04-24T23:14:02.190 INFO:tasks.workunit.client.0.smithi031.stdout:5/835: chown d0/d6/d32/d6b/f7b 24 1 2024-04-24T23:14:02.217 INFO:tasks.workunit.client.0.smithi031.stdout:1/834: rename d1/f54 to d1/d3e/d46/dbd/f115 0 2024-04-24T23:14:02.217 INFO:tasks.workunit.client.0.smithi031.stdout:1/835: chown d1/d3e/da3 560 1 2024-04-24T23:14:02.219 INFO:tasks.workunit.client.1.smithi040.stdout:2/670: chown da/d10/c1a 517906 1 2024-04-24T23:14:02.220 INFO:tasks.workunit.client.1.smithi040.stdout:2/671: truncate da/d1c/d37/d39/d3f/d52/d78/dc7/dcc/ff0 677565 0 2024-04-24T23:14:02.224 INFO:tasks.workunit.client.1.smithi040.stdout:5/586: getdents d2/d7/d8/d1f/d57/d27/d65 0 2024-04-24T23:14:02.227 INFO:tasks.workunit.client.0.smithi031.stdout:0/951: dwrite da/d20/d4e/f48 [4194304,4194304] 0 2024-04-24T23:14:02.228 INFO:tasks.workunit.client.1.smithi040.stdout:9/662: creat d7/da/fee x:0 0 0 2024-04-24T23:14:02.229 INFO:tasks.workunit.client.0.smithi031.stdout:1/836: dread d1/d33/d41/d9b/da8/fa9 [0,4194304] 0 2024-04-24T23:14:02.236 INFO:tasks.workunit.client.1.smithi040.stdout:6/604: mknod d7/d1a/d20/d51/d2a/cbd 0 2024-04-24T23:14:02.239 INFO:tasks.workunit.client.0.smithi031.stdout:0/952: chown da/d18/d1a/dd6/l81 6 1 2024-04-24T23:14:02.242 INFO:tasks.workunit.client.0.smithi031.stdout:1/837: mknod d1/da/d12/d65/dcf/d11/d21/d5a/c116 0 2024-04-24T23:14:02.245 INFO:tasks.workunit.client.0.smithi031.stdout:0/953: mknod da/d11/d6a/dba/d11e/c13e 0 2024-04-24T23:14:02.247 INFO:tasks.workunit.client.1.smithi040.stdout:5/587: symlink d2/d7/d8/d1f/d57/d27/d3c/d43/lbf 0 2024-04-24T23:14:02.249 INFO:tasks.workunit.client.0.smithi031.stdout:1/838: creat d1/d3e/d46/df4/f117 x:0 0 0 2024-04-24T23:14:02.251 INFO:tasks.workunit.client.1.smithi040.stdout:4/572: mknod da/d3f/d40/d47/d75/d7c/cc9 0 2024-04-24T23:14:02.253 INFO:tasks.workunit.client.1.smithi040.stdout:1/627: getdents da/d12/d18/d2a/d41 0 2024-04-24T23:14:02.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/839: fsync d1/da/d12/d65/dcf/d11/fee 0 2024-04-24T23:14:02.256 INFO:tasks.workunit.client.0.smithi031.stdout:0/954: creat da/d20/f13f x:0 0 0 2024-04-24T23:14:02.257 INFO:tasks.workunit.client.1.smithi040.stdout:2/672: symlink da/d1c/d37/d39/d3f/d52/d78/dc7/ded/lf8 0 2024-04-24T23:14:02.259 INFO:tasks.workunit.client.1.smithi040.stdout:3/674: getdents d1/d9 0 2024-04-24T23:14:02.260 INFO:tasks.workunit.client.1.smithi040.stdout:3/675: write d1/d9/d2e/fa1 [4172400,98697] 0 2024-04-24T23:14:02.263 INFO:tasks.workunit.client.1.smithi040.stdout:0/695: rename df/d12/d1a/d71/c85 to df/d1e/da7/ce3 0 2024-04-24T23:14:02.265 INFO:tasks.workunit.client.1.smithi040.stdout:0/696: fdatasync df/de1/d6e/fba 0 2024-04-24T23:14:02.267 INFO:tasks.workunit.client.1.smithi040.stdout:1/628: mknod da/d12/d18/d2a/d4d/cdb 0 2024-04-24T23:14:02.268 INFO:tasks.workunit.client.1.smithi040.stdout:1/629: readlink da/l53 0 2024-04-24T23:14:02.269 INFO:tasks.workunit.client.1.smithi040.stdout:7/657: getdents db/d12/d3f/d7a 0 2024-04-24T23:14:02.269 INFO:tasks.workunit.client.1.smithi040.stdout:3/676: dwrite d1/d9/f21 [0,4194304] 0 2024-04-24T23:14:02.270 INFO:tasks.workunit.client.1.smithi040.stdout:4/573: dread da/d3f/d40/d41/d60/f23 [0,4194304] 0 2024-04-24T23:14:02.270 INFO:tasks.workunit.client.0.smithi031.stdout:0/955: getdents da/d18/d5a/d7d 0 2024-04-24T23:14:02.270 INFO:tasks.workunit.client.0.smithi031.stdout:0/956: chown da/d20/d4e/dc9/f132 250905227 1 2024-04-24T23:14:02.274 INFO:tasks.workunit.client.1.smithi040.stdout:2/673: dread da/d1c/d37/d39/f76 [0,4194304] 0 2024-04-24T23:14:02.277 INFO:tasks.workunit.client.0.smithi031.stdout:0/957: symlink da/d86/da3/d10f/l140 0 2024-04-24T23:14:02.279 INFO:tasks.workunit.client.1.smithi040.stdout:5/588: rename d2/d7/d8/d1f/da1/cbd to d2/d7/dab/cc0 0 2024-04-24T23:14:02.280 INFO:tasks.workunit.client.0.smithi031.stdout:0/958: mknod da/d18/c141 0 2024-04-24T23:14:02.281 INFO:tasks.workunit.client.1.smithi040.stdout:0/697: symlink df/d1e/d20/d61/le4 0 2024-04-24T23:14:02.284 INFO:tasks.workunit.client.1.smithi040.stdout:4/574: symlink da/d1f/lca 0 2024-04-24T23:14:02.285 INFO:tasks.workunit.client.1.smithi040.stdout:3/677: truncate d1/d2a/d8d/d7c/fe1 35540 0 2024-04-24T23:14:02.287 INFO:tasks.workunit.client.1.smithi040.stdout:8/669: sync 2024-04-24T23:14:02.287 INFO:tasks.workunit.client.1.smithi040.stdout:2/674: unlink da/d1c/c74 0 2024-04-24T23:14:02.288 INFO:tasks.workunit.client.1.smithi040.stdout:8/670: chown d0/d8/dc/f96 9685810 1 2024-04-24T23:14:02.289 INFO:tasks.workunit.client.1.smithi040.stdout:9/663: getdents d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8 0 2024-04-24T23:14:02.289 INFO:tasks.workunit.client.1.smithi040.stdout:8/671: truncate d0/d8/dc/f96 29128 0 2024-04-24T23:14:02.292 INFO:tasks.workunit.client.1.smithi040.stdout:5/589: rename d2/d7/d8/d1f/d57/d27/l4c to d2/d7/d8/d6d/d89/lc1 0 2024-04-24T23:14:02.293 INFO:tasks.workunit.client.1.smithi040.stdout:7/658: symlink db/lee 0 2024-04-24T23:14:02.295 INFO:tasks.workunit.client.1.smithi040.stdout:4/575: creat da/d45/d48/d6e/d70/fcb x:0 0 0 2024-04-24T23:14:02.296 INFO:tasks.workunit.client.1.smithi040.stdout:3/678: symlink d1/d9/d18/d36/d61/le7 0 2024-04-24T23:14:02.296 INFO:tasks.workunit.client.0.smithi031.stdout:0/959: dwrite da/f99 [0,4194304] 0 2024-04-24T23:14:02.297 INFO:tasks.workunit.client.1.smithi040.stdout:2/675: mknod da/d1c/d37/d39/d3f/db2/cf9 0 2024-04-24T23:14:02.299 INFO:tasks.workunit.client.1.smithi040.stdout:0/698: rename df/d14/c4a to df/d1e/d20/d21/d41/d52/ce5 0 2024-04-24T23:14:02.301 INFO:tasks.workunit.client.0.smithi031.stdout:0/960: rmdir da/d20/d4e/dc9/d11b 39 2024-04-24T23:14:02.314 INFO:tasks.workunit.client.0.smithi031.stdout:0/961: write da/d22/f54 [2952491,43370] 0 2024-04-24T23:14:02.317 INFO:tasks.workunit.client.0.smithi031.stdout:0/962: symlink da/d20/d4e/dc9/d11b/l142 0 2024-04-24T23:14:02.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/963: unlink da/d22/c91 0 2024-04-24T23:14:02.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/964: readlink da/d18/d1a/db7/ldf 0 2024-04-24T23:14:02.325 INFO:tasks.workunit.client.0.smithi031.stdout:0/965: write da/d20/f52 [956724,40911] 0 2024-04-24T23:14:02.325 INFO:tasks.workunit.client.0.smithi031.stdout:0/966: stat da/d20/f13c 0 2024-04-24T23:14:02.330 INFO:tasks.workunit.client.0.smithi031.stdout:0/967: dread da/fd2 [0,4194304] 0 2024-04-24T23:14:02.331 INFO:tasks.workunit.client.0.smithi031.stdout:0/968: dread - da/d20/d4e/f110 zero size 2024-04-24T23:14:02.333 INFO:tasks.workunit.client.0.smithi031.stdout:0/969: fdatasync da/d18/d5a/d7d/dbd/f113 0 2024-04-24T23:14:02.336 INFO:tasks.workunit.client.0.smithi031.stdout:0/970: symlink da/d86/dbf/dd5/l143 0 2024-04-24T23:14:02.336 INFO:tasks.workunit.client.0.smithi031.stdout:0/971: stat da/d11/f59 0 2024-04-24T23:14:02.337 INFO:tasks.workunit.client.0.smithi031.stdout:0/972: read - da/d11/f115 zero size 2024-04-24T23:14:02.339 INFO:tasks.workunit.client.1.smithi040.stdout:5/590: chown d2/d7/d8/db/d1e/cb1 15873 1 2024-04-24T23:14:02.339 INFO:tasks.workunit.client.1.smithi040.stdout:3/679: mknod d1/d9/d15/d45/daa/ce8 0 2024-04-24T23:14:02.339 INFO:tasks.workunit.client.1.smithi040.stdout:2/676: mkdir da/d1c/d37/d8a/dfa 0 2024-04-24T23:14:02.340 INFO:tasks.workunit.client.0.smithi031.stdout:0/973: dread da/d18/d1a/dd6/f107 [0,4194304] 0 2024-04-24T23:14:02.340 INFO:tasks.workunit.client.1.smithi040.stdout:3/680: stat d1/d2a/d8d 0 2024-04-24T23:14:02.340 INFO:tasks.workunit.client.1.smithi040.stdout:8/672: dread d0/d52/fb3 [0,4194304] 0 2024-04-24T23:14:02.340 INFO:tasks.workunit.client.1.smithi040.stdout:1/630: getdents da/d12/d18/d2a/d8b/dd3 0 2024-04-24T23:14:02.341 INFO:tasks.workunit.client.1.smithi040.stdout:3/681: stat d1/d2d/de/d30/f58 0 2024-04-24T23:14:02.341 INFO:tasks.workunit.client.1.smithi040.stdout:4/576: mknod da/d3f/d6b/ccc 0 2024-04-24T23:14:02.342 INFO:tasks.workunit.client.1.smithi040.stdout:8/673: chown d0/d7e/dc1/d87/l9d 37 1 2024-04-24T23:14:02.342 INFO:tasks.workunit.client.0.smithi031.stdout:0/974: symlink da/d86/da3/dcd/ddc/d114/l144 0 2024-04-24T23:14:02.343 INFO:tasks.workunit.client.1.smithi040.stdout:1/631: write da/d12/d14/f2d [853407,16266] 0 2024-04-24T23:14:02.343 INFO:tasks.workunit.client.1.smithi040.stdout:8/674: dread - d0/d16/d1b/d47/fdc zero size 2024-04-24T23:14:02.344 INFO:tasks.workunit.client.1.smithi040.stdout:0/699: fsync df/d1e/d20/d21/d51/da3/d8e/f99 0 2024-04-24T23:14:02.345 INFO:tasks.workunit.client.1.smithi040.stdout:0/700: stat df/d12/f1f 0 2024-04-24T23:14:02.346 INFO:tasks.workunit.client.1.smithi040.stdout:3/682: mknod d1/d2a/d8d/ce9 0 2024-04-24T23:14:02.346 INFO:tasks.workunit.client.0.smithi031.stdout:0/975: fsync da/d18/d1a/f63 0 2024-04-24T23:14:02.348 INFO:tasks.workunit.client.1.smithi040.stdout:7/659: getdents db/d2f/da8 0 2024-04-24T23:14:02.348 INFO:tasks.workunit.client.1.smithi040.stdout:4/577: mknod da/d3f/d40/d41/d62/ccd 0 2024-04-24T23:14:02.350 INFO:tasks.workunit.client.1.smithi040.stdout:0/701: dwrite df/f34 [0,4194304] 0 2024-04-24T23:14:02.351 INFO:tasks.workunit.client.1.smithi040.stdout:1/632: readlink da/d12/d23/lcc 0 2024-04-24T23:14:02.356 INFO:tasks.workunit.client.1.smithi040.stdout:8/675: creat d0/d16/d1b/d58/d8f/fdf x:0 0 0 2024-04-24T23:14:02.362 INFO:tasks.workunit.client.1.smithi040.stdout:7/660: creat db/d12/d62/d8a/db3/fef x:0 0 0 2024-04-24T23:14:02.365 INFO:tasks.workunit.client.1.smithi040.stdout:3/683: write d1/d2a/d6a/d94/f98 [643931,126688] 0 2024-04-24T23:14:02.366 INFO:tasks.workunit.client.1.smithi040.stdout:3/684: write d1/d2a/f9a [2838342,24287] 0 2024-04-24T23:14:02.368 INFO:tasks.workunit.client.1.smithi040.stdout:7/661: dwrite db/d12/d62/fdb [0,4194304] 0 2024-04-24T23:14:02.368 INFO:tasks.workunit.client.0.smithi031.stdout:0/976: dwrite da/d86/da3/dcd/fec [0,4194304] 0 2024-04-24T23:14:02.368 INFO:tasks.workunit.client.1.smithi040.stdout:3/685: readlink d1/d2d/de/d30/d37/lab 0 2024-04-24T23:14:02.369 INFO:tasks.workunit.client.1.smithi040.stdout:7/662: fsync db/d12/d20/d52/f68 0 2024-04-24T23:14:02.370 INFO:tasks.workunit.client.1.smithi040.stdout:8/676: mknod d0/d28/db7/ce0 0 2024-04-24T23:14:02.370 INFO:tasks.workunit.client.1.smithi040.stdout:3/686: chown d1/d9/d18/da7/fad 2566162 1 2024-04-24T23:14:02.372 INFO:tasks.workunit.client.1.smithi040.stdout:3/687: chown d1/d2d/d2b/f4d 1646 1 2024-04-24T23:14:02.373 INFO:tasks.workunit.client.1.smithi040.stdout:2/677: rename da/f20 to da/d29/ffb 0 2024-04-24T23:14:02.374 INFO:tasks.workunit.client.1.smithi040.stdout:2/678: truncate da/d1c/d37/fe1 950593 0 2024-04-24T23:14:02.375 INFO:tasks.workunit.client.1.smithi040.stdout:2/679: chown da/d10/d4d/d6c 105487 1 2024-04-24T23:14:02.376 INFO:tasks.workunit.client.0.smithi031.stdout:0/977: dread da/f99 [0,4194304] 0 2024-04-24T23:14:02.378 INFO:tasks.workunit.client.1.smithi040.stdout:6/605: sync 2024-04-24T23:14:02.379 INFO:tasks.workunit.client.1.smithi040.stdout:6/606: chown d7/d1a/d20/d51/d1e/d46/l8f 95728 1 2024-04-24T23:14:02.383 INFO:tasks.workunit.client.1.smithi040.stdout:1/633: dread da/d12/d14/d25/db9/fc8 [4194304,4194304] 0 2024-04-24T23:14:02.421 INFO:tasks.workunit.client.1.smithi040.stdout:9/664: sync 2024-04-24T23:14:02.452 INFO:tasks.workunit.client.1.smithi040.stdout:0/702: write df/d12/d58/f72 [793515,50539] 0 2024-04-24T23:14:02.453 INFO:tasks.workunit.client.1.smithi040.stdout:7/663: mknod db/d12/d73/cf0 0 2024-04-24T23:14:02.453 INFO:tasks.workunit.client.1.smithi040.stdout:5/591: sync 2024-04-24T23:14:02.455 INFO:tasks.workunit.client.0.smithi031.stdout:0/978: creat da/d18/d5a/f145 x:0 0 0 2024-04-24T23:14:02.458 INFO:tasks.workunit.client.1.smithi040.stdout:8/677: rmdir d0/d8/dc/d34/da7 39 2024-04-24T23:14:02.459 INFO:tasks.workunit.client.1.smithi040.stdout:3/688: mknod d1/d9/d15/cea 0 2024-04-24T23:14:02.460 INFO:tasks.workunit.client.1.smithi040.stdout:4/578: truncate da/d45/d48/d6e/fbd 780326 0 2024-04-24T23:14:02.461 INFO:tasks.workunit.client.1.smithi040.stdout:4/579: fdatasync da/d3f/d6b/d2b/f39 0 2024-04-24T23:14:02.462 INFO:tasks.workunit.client.1.smithi040.stdout:2/680: symlink da/d81/da8/lfc 0 2024-04-24T23:14:02.466 INFO:tasks.workunit.client.1.smithi040.stdout:8/678: dwrite d0/d7e/dc1/d87/fd7 [0,4194304] 0 2024-04-24T23:14:02.467 INFO:tasks.workunit.client.1.smithi040.stdout:9/665: unlink d7/d92/cb0 0 2024-04-24T23:14:02.468 INFO:tasks.workunit.client.1.smithi040.stdout:3/689: dwrite d1/d9/f20 [0,4194304] 0 2024-04-24T23:14:02.473 INFO:tasks.workunit.client.1.smithi040.stdout:7/664: mknod db/d43/db9/cf1 0 2024-04-24T23:14:02.475 INFO:tasks.workunit.client.1.smithi040.stdout:7/665: chown db/d12/d20/d52/d4b/l69 84 1 2024-04-24T23:14:02.475 INFO:tasks.workunit.client.1.smithi040.stdout:5/592: truncate d2/d7/d8/db/f56 586237 0 2024-04-24T23:14:02.477 INFO:tasks.workunit.client.1.smithi040.stdout:5/593: chown d2/d62/fb2 15 1 2024-04-24T23:14:02.478 INFO:tasks.workunit.client.1.smithi040.stdout:8/679: dwrite d0/d7e/dc1/d9e/d90/d91/fd3 [0,4194304] 0 2024-04-24T23:14:02.478 INFO:tasks.workunit.client.1.smithi040.stdout:7/666: chown db/d12/d20/d52/d6c/d8f/db8/ccf 1044537137 1 2024-04-24T23:14:02.478 INFO:tasks.workunit.client.1.smithi040.stdout:5/594: stat d2/d7/d8/f34 0 2024-04-24T23:14:02.479 INFO:tasks.workunit.client.1.smithi040.stdout:7/667: chown db/d12/d20/d52/d6c/d8f/da7 33227459 1 2024-04-24T23:14:02.483 INFO:tasks.workunit.client.0.smithi031.stdout:0/979: getdents da/d18/d1a/db7 0 2024-04-24T23:14:02.485 INFO:tasks.workunit.client.1.smithi040.stdout:6/607: write d7/d1a/d20/d51/d2f/f35 [2600550,65924] 0 2024-04-24T23:14:02.519 INFO:tasks.workunit.client.1.smithi040.stdout:2/681: chown da/d1c/d37/d39/d3f/d52/d78/c8c 191 1 2024-04-24T23:14:02.520 INFO:tasks.workunit.client.1.smithi040.stdout:1/634: mknod da/d12/d18/d2a/cdc 0 2024-04-24T23:14:02.521 INFO:tasks.workunit.client.0.smithi031.stdout:0/980: truncate da/d11/fa0 1078144 0 2024-04-24T23:14:02.521 INFO:tasks.workunit.client.1.smithi040.stdout:3/690: read - d1/d2a/d6a/f8c zero size 2024-04-24T23:14:02.522 INFO:tasks.workunit.client.1.smithi040.stdout:3/691: dread - d1/d2d/de/d30/f58 zero size 2024-04-24T23:14:02.523 INFO:tasks.workunit.client.1.smithi040.stdout:9/666: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/def 0 2024-04-24T23:14:02.526 INFO:tasks.workunit.client.1.smithi040.stdout:5/595: rmdir d2/d7/d8/d1f/da1 39 2024-04-24T23:14:02.529 INFO:tasks.workunit.client.1.smithi040.stdout:4/580: mknod da/d3f/d6b/cce 0 2024-04-24T23:14:02.530 INFO:tasks.workunit.client.1.smithi040.stdout:2/682: symlink da/d1c/d37/d39/d79/lfd 0 2024-04-24T23:14:02.530 INFO:tasks.workunit.client.1.smithi040.stdout:2/683: readlink da/d29/l85 0 2024-04-24T23:14:02.534 INFO:tasks.workunit.client.1.smithi040.stdout:9/667: creat d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/ff0 x:0 0 0 2024-04-24T23:14:02.535 INFO:tasks.workunit.client.1.smithi040.stdout:8/680: creat d0/d52/d9b/fe1 x:0 0 0 2024-04-24T23:14:02.536 INFO:tasks.workunit.client.1.smithi040.stdout:9/668: readlink d7/da/d17/d40/d55/d2f/d1f/l7f 0 2024-04-24T23:14:02.537 INFO:tasks.workunit.client.1.smithi040.stdout:4/581: unlink da/d3f/d6b/d3c/d69/cab 0 2024-04-24T23:14:02.538 INFO:tasks.workunit.client.1.smithi040.stdout:5/596: dread d2/d7/d8/f13 [0,4194304] 0 2024-04-24T23:14:02.541 INFO:tasks.workunit.client.1.smithi040.stdout:7/668: creat db/d12/ff2 x:0 0 0 2024-04-24T23:14:02.543 INFO:tasks.workunit.client.1.smithi040.stdout:1/635: rmdir da/d12/d14/d90 0 2024-04-24T23:14:02.543 INFO:tasks.workunit.client.1.smithi040.stdout:9/669: dwrite d7/d5e/da1/feb [0,4194304] 0 2024-04-24T23:14:02.544 INFO:tasks.workunit.client.1.smithi040.stdout:1/636: chown da/d35/dab 170512181 1 2024-04-24T23:14:02.546 INFO:tasks.workunit.client.1.smithi040.stdout:4/582: chown da/d3f/d40/d47/d75/d7c/lad 167576 1 2024-04-24T23:14:02.546 INFO:tasks.workunit.client.1.smithi040.stdout:4/583: fdatasync da/d45/d58/f98 0 2024-04-24T23:14:02.548 INFO:tasks.workunit.client.1.smithi040.stdout:2/684: dread da/d10/f31 [4194304,4194304] 0 2024-04-24T23:14:02.548 INFO:tasks.workunit.client.1.smithi040.stdout:5/597: mknod d2/d7/d8/d6d/d89/cc2 0 2024-04-24T23:14:02.548 INFO:tasks.workunit.client.1.smithi040.stdout:4/584: truncate da/d78/dbb/fc2 169335 0 2024-04-24T23:14:02.555 INFO:tasks.workunit.client.1.smithi040.stdout:7/669: rename f5 to db/d12/d20/d56/ff3 0 2024-04-24T23:14:02.556 INFO:tasks.workunit.client.1.smithi040.stdout:7/670: fdatasync db/d12/f29 0 2024-04-24T23:14:02.557 INFO:tasks.workunit.client.1.smithi040.stdout:7/671: write db/d2f/d37/f94 [8924043,44190] 0 2024-04-24T23:14:02.558 INFO:tasks.workunit.client.1.smithi040.stdout:9/670: creat d7/d92/ff1 x:0 0 0 2024-04-24T23:14:02.561 INFO:tasks.workunit.client.1.smithi040.stdout:3/692: getdents d1/d9/d2e 0 2024-04-24T23:14:02.561 INFO:tasks.workunit.client.1.smithi040.stdout:3/693: stat d1/d9/d15/fdd 0 2024-04-24T23:14:02.562 INFO:tasks.workunit.client.1.smithi040.stdout:3/694: write d1/d2a/d6a/f35 [744691,8485] 0 2024-04-24T23:14:02.565 INFO:tasks.workunit.client.1.smithi040.stdout:3/695: dread d1/d2d/de/d30/fb9 [0,4194304] 0 2024-04-24T23:14:02.565 INFO:tasks.workunit.client.1.smithi040.stdout:4/585: symlink da/d45/d58/lcf 0 2024-04-24T23:14:02.567 INFO:tasks.workunit.client.1.smithi040.stdout:2/685: unlink da/d10/d4d/dd7/fde 0 2024-04-24T23:14:02.571 INFO:tasks.workunit.client.1.smithi040.stdout:2/686: truncate da/d1c/d37/d39/d79/da3/fb3 154904 0 2024-04-24T23:14:02.572 INFO:tasks.workunit.client.1.smithi040.stdout:0/703: sync 2024-04-24T23:14:02.572 INFO:tasks.workunit.client.1.smithi040.stdout:6/608: sync 2024-04-24T23:14:02.573 INFO:tasks.workunit.client.1.smithi040.stdout:7/672: mknod db/d2f/d37/cf4 0 2024-04-24T23:14:02.575 INFO:tasks.workunit.client.1.smithi040.stdout:9/671: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/df2 0 2024-04-24T23:14:02.578 INFO:tasks.workunit.client.1.smithi040.stdout:5/598: mkdir d2/d7/dc3 0 2024-04-24T23:14:02.580 INFO:tasks.workunit.client.1.smithi040.stdout:9/672: dwrite d7/da/d17/d40/d55/d61/fe9 [0,4194304] 0 2024-04-24T23:14:02.581 INFO:tasks.workunit.client.1.smithi040.stdout:9/673: write d7/fbe [725962,19115] 0 2024-04-24T23:14:02.582 INFO:tasks.workunit.client.1.smithi040.stdout:1/637: write da/d12/d18/d2a/d41/d4f/f9b [5032541,2680] 0 2024-04-24T23:14:02.582 INFO:tasks.workunit.client.1.smithi040.stdout:9/674: dread - d7/da/d17/d40/d55/d2f/d1f/d4e/f8b zero size 2024-04-24T23:14:02.584 INFO:tasks.workunit.client.1.smithi040.stdout:2/687: symlink da/d10/d4d/dd7/lfe 0 2024-04-24T23:14:02.585 INFO:tasks.workunit.client.1.smithi040.stdout:8/681: link d0/d7e/dc1/f80 d0/d16/d1b/d58/d8f/db1/fe2 0 2024-04-24T23:14:02.588 INFO:tasks.workunit.client.1.smithi040.stdout:6/609: mknod d7/d1a/d20/d51/d2f/cbe 0 2024-04-24T23:14:02.589 INFO:tasks.workunit.client.1.smithi040.stdout:8/682: fsync d0/d7e/dc1/d9e/d90/fca 0 2024-04-24T23:14:02.591 INFO:tasks.workunit.client.1.smithi040.stdout:8/683: write d0/d16/d1b/d58/fc6 [2043935,33994] 0 2024-04-24T23:14:02.593 INFO:tasks.workunit.client.1.smithi040.stdout:3/696: dwrite d1/d2a/d6a/f8c [0,4194304] 0 2024-04-24T23:14:02.600 INFO:tasks.workunit.client.1.smithi040.stdout:9/675: readlink d7/da/d17/d40/d55/d2f/d1f/d31/l7b 0 2024-04-24T23:14:02.602 INFO:tasks.workunit.client.1.smithi040.stdout:1/638: truncate da/d12/d18/d2a/d4d/fd2 383167 0 2024-04-24T23:14:02.606 INFO:tasks.workunit.client.1.smithi040.stdout:0/704: dwrite df/d1e/d20/d21/d51/da3/d8e/f99 [0,4194304] 0 2024-04-24T23:14:02.608 INFO:tasks.workunit.client.1.smithi040.stdout:5/599: symlink d2/d7/d8/d1f/d57/lc4 0 2024-04-24T23:14:02.609 INFO:tasks.workunit.client.1.smithi040.stdout:3/697: mkdir d1/d9/d15/d45/deb 0 2024-04-24T23:14:02.609 INFO:tasks.workunit.client.1.smithi040.stdout:4/586: link da/d1f/c5e da/d3f/d40/d41/d62/cd0 0 2024-04-24T23:14:02.611 INFO:tasks.workunit.client.1.smithi040.stdout:7/673: creat db/d12/d20/ff5 x:0 0 0 2024-04-24T23:14:02.612 INFO:tasks.workunit.client.1.smithi040.stdout:2/688: mknod da/d1c/d37/d39/d3f/cff 0 2024-04-24T23:14:02.617 INFO:tasks.workunit.client.1.smithi040.stdout:8/684: write d0/d38/faa [23375,51465] 0 2024-04-24T23:14:02.618 INFO:tasks.workunit.client.1.smithi040.stdout:0/705: read df/d12/f2a [4994021,2051] 0 2024-04-24T23:14:02.619 INFO:tasks.workunit.client.1.smithi040.stdout:4/587: readlink da/d78/lc7 0 2024-04-24T23:14:02.619 INFO:tasks.workunit.client.1.smithi040.stdout:0/706: chown df/d12/d4b/db8 127 1 2024-04-24T23:14:02.620 INFO:tasks.workunit.client.1.smithi040.stdout:3/698: mknod d1/d2d/d2b/cec 0 2024-04-24T23:14:02.621 INFO:tasks.workunit.client.1.smithi040.stdout:0/707: truncate df/d1e/f7b 2049837 0 2024-04-24T23:14:02.621 INFO:tasks.workunit.client.1.smithi040.stdout:7/674: creat db/d12/d7d/ff6 x:0 0 0 2024-04-24T23:14:02.623 INFO:tasks.workunit.client.1.smithi040.stdout:9/676: fsync d7/da/d17/d40/d55/d2f/d1f/d31/f98 0 2024-04-24T23:14:02.624 INFO:tasks.workunit.client.1.smithi040.stdout:7/675: write db/d12/d7d/fbb [1175683,43909] 0 2024-04-24T23:14:02.624 INFO:tasks.workunit.client.1.smithi040.stdout:2/689: write da/d81/fe7 [4802318,97836] 0 2024-04-24T23:14:02.624 INFO:tasks.workunit.client.1.smithi040.stdout:0/708: write df/d1e/d20/d61/fbb [3206123,60677] 0 2024-04-24T23:14:02.624 INFO:tasks.workunit.client.1.smithi040.stdout:6/610: dread d7/f9 [0,4194304] 0 2024-04-24T23:14:02.627 INFO:tasks.workunit.client.1.smithi040.stdout:0/709: chown df/d1e/da7 604538598 1 2024-04-24T23:14:02.627 INFO:tasks.workunit.client.1.smithi040.stdout:8/685: rmdir d0/d7e 39 2024-04-24T23:14:02.628 INFO:tasks.workunit.client.1.smithi040.stdout:5/600: mknod d2/d7/cc5 0 2024-04-24T23:14:02.629 INFO:tasks.workunit.client.1.smithi040.stdout:4/588: rmdir da/d3f/d8d 39 2024-04-24T23:14:02.631 INFO:tasks.workunit.client.1.smithi040.stdout:4/589: dread - da/d45/d58/fa6 zero size 2024-04-24T23:14:02.633 INFO:tasks.workunit.client.1.smithi040.stdout:7/676: dwrite db/d12/d7d/fbb [0,4194304] 0 2024-04-24T23:14:02.633 INFO:tasks.workunit.client.1.smithi040.stdout:4/590: fdatasync da/d3f/d40/d41/d60/f24 0 2024-04-24T23:14:02.633 INFO:tasks.workunit.client.1.smithi040.stdout:1/639: write da/ff [744313,86418] 0 2024-04-24T23:14:02.636 INFO:tasks.workunit.client.1.smithi040.stdout:1/640: dread - da/d12/d18/d2a/d41/fc0 zero size 2024-04-24T23:14:02.638 INFO:tasks.workunit.client.1.smithi040.stdout:6/611: write d7/d1a/d56/f7f [1040399,91625] 0 2024-04-24T23:14:02.638 INFO:tasks.workunit.client.1.smithi040.stdout:0/710: rmdir df/d12 39 2024-04-24T23:14:02.640 INFO:tasks.workunit.client.1.smithi040.stdout:3/699: mkdir d1/d80/dbf/ded 0 2024-04-24T23:14:02.663 INFO:tasks.workunit.client.0.smithi031.stdout:7/861: sync 2024-04-24T23:14:02.664 INFO:tasks.workunit.client.1.smithi040.stdout:4/591: dread - da/d3f/d40/d47/f93 zero size 2024-04-24T23:14:02.668 INFO:tasks.workunit.client.0.smithi031.stdout:7/862: symlink d1/d2/dd/d33/d5e/l12c 0 2024-04-24T23:14:02.668 INFO:tasks.workunit.client.0.smithi031.stdout:7/863: chown d1/d2/f12 6005 1 2024-04-24T23:14:02.670 INFO:tasks.workunit.client.1.smithi040.stdout:9/677: link d7/da/d17/d40/d55/d2f/d1f/d4e/f8b d7/d5e/de5/ff3 0 2024-04-24T23:14:02.671 INFO:tasks.workunit.client.1.smithi040.stdout:8/686: mknod d0/d38/d4c/da3/ce3 0 2024-04-24T23:14:02.672 INFO:tasks.workunit.client.1.smithi040.stdout:2/690: write da/d29/ffb [490250,64416] 0 2024-04-24T23:14:02.674 INFO:tasks.workunit.client.1.smithi040.stdout:6/612: unlink d7/d1a/d56/l65 0 2024-04-24T23:14:02.679 INFO:tasks.workunit.client.1.smithi040.stdout:7/677: symlink db/d12/d20/lf7 0 2024-04-24T23:14:02.684 INFO:tasks.workunit.client.1.smithi040.stdout:8/687: creat d0/d38/fe4 x:0 0 0 2024-04-24T23:14:02.684 INFO:tasks.workunit.client.1.smithi040.stdout:0/711: write df/d1e/d20/d21/d51/f57 [762560,57023] 0 2024-04-24T23:14:02.685 INFO:tasks.workunit.client.0.smithi031.stdout:7/864: dwrite d1/d2/dd/d33/d5e/d64/f91 [0,4194304] 0 2024-04-24T23:14:02.687 INFO:tasks.workunit.client.1.smithi040.stdout:6/613: mkdir d7/d1a/dbf 0 2024-04-24T23:14:02.688 INFO:tasks.workunit.client.1.smithi040.stdout:1/641: write da/d12/d18/d2a/d4d/fd2 [224015,117468] 0 2024-04-24T23:14:02.690 INFO:tasks.workunit.client.0.smithi031.stdout:7/865: creat d1/de4/d106/f12d x:0 0 0 2024-04-24T23:14:02.692 INFO:tasks.workunit.client.1.smithi040.stdout:5/601: rename d2/d7/d8/db/d1e/c87 to d2/d7/d8/d1f/d57/cc6 0 2024-04-24T23:14:02.695 INFO:tasks.workunit.client.1.smithi040.stdout:1/642: dwrite da/d12/d14/f2d [0,4194304] 0 2024-04-24T23:14:02.695 INFO:tasks.workunit.client.1.smithi040.stdout:7/678: rename db/d2f/f9d to db/d2f/ff8 0 2024-04-24T23:14:02.696 INFO:tasks.workunit.client.0.smithi031.stdout:7/866: truncate d1/d2/dd/d33/d5e/d64/d6d/fac 3495575 0 2024-04-24T23:14:02.696 INFO:tasks.workunit.client.1.smithi040.stdout:1/643: readlink da/d12/d14/l15 0 2024-04-24T23:14:02.697 INFO:tasks.workunit.client.1.smithi040.stdout:1/644: write da/d12/d14/f96 [1244809,124313] 0 2024-04-24T23:14:02.698 INFO:tasks.workunit.client.0.smithi031.stdout:7/867: dread - d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/f11d zero size 2024-04-24T23:14:02.700 INFO:tasks.workunit.client.1.smithi040.stdout:6/614: creat d7/d1a/fc0 x:0 0 0 2024-04-24T23:14:02.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/868: unlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d93/cb3 0 2024-04-24T23:14:02.706 INFO:tasks.workunit.client.1.smithi040.stdout:4/592: link da/d3f/l68 da/d3f/d40/d47/d75/d7c/ld1 0 2024-04-24T23:14:02.707 INFO:tasks.workunit.client.1.smithi040.stdout:4/593: chown da/d3f/d6b/d2b/f6d 0 1 2024-04-24T23:14:02.709 INFO:tasks.workunit.client.1.smithi040.stdout:7/679: creat db/d43/ff9 x:0 0 0 2024-04-24T23:14:02.709 INFO:tasks.workunit.client.1.smithi040.stdout:7/680: dread - db/d12/ff2 zero size 2024-04-24T23:14:02.712 INFO:tasks.workunit.client.1.smithi040.stdout:0/712: dread df/d1e/d89/fd7 [4194304,4194304] 0 2024-04-24T23:14:02.717 INFO:tasks.workunit.client.0.smithi031.stdout:7/869: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/f115 [0,4194304] 0 2024-04-24T23:14:02.718 INFO:tasks.workunit.client.1.smithi040.stdout:1/645: creat da/d12/d14/fdd x:0 0 0 2024-04-24T23:14:02.719 INFO:tasks.workunit.client.1.smithi040.stdout:1/646: fsync da/d12/d18/d2a/f79 0 2024-04-24T23:14:02.720 INFO:tasks.workunit.client.1.smithi040.stdout:6/615: creat d7/d1a/d9f/fc1 x:0 0 0 2024-04-24T23:14:02.723 INFO:tasks.workunit.client.0.smithi031.stdout:7/870: mknod d1/d2/dd/d33/d5e/d64/d6d/dae/dd7/c12e 0 2024-04-24T23:14:02.725 INFO:tasks.workunit.client.1.smithi040.stdout:2/691: dwrite da/d1c/d37/d39/d3f/f88 [0,4194304] 0 2024-04-24T23:14:02.730 INFO:tasks.workunit.client.1.smithi040.stdout:7/681: fdatasync db/d12/d62/d8a/f8b 0 2024-04-24T23:14:02.740 INFO:tasks.workunit.client.1.smithi040.stdout:4/594: write da/d3f/d40/d47/f93 [955591,103674] 0 2024-04-24T23:14:02.742 INFO:tasks.workunit.client.1.smithi040.stdout:3/700: rename d1/d2d/de/d30/fb9 to d1/d2d/fee 0 2024-04-24T23:14:02.745 INFO:tasks.workunit.client.1.smithi040.stdout:5/602: link d2/d7/fe d2/d7/d8/d6d/fc7 0 2024-04-24T23:14:02.748 INFO:tasks.workunit.client.1.smithi040.stdout:1/647: mknod da/d12/d18/cde 0 2024-04-24T23:14:02.750 INFO:tasks.workunit.client.0.smithi031.stdout:7/871: dread d1/d2/dd/f7d [0,4194304] 0 2024-04-24T23:14:02.754 INFO:tasks.workunit.client.1.smithi040.stdout:0/713: dwrite df/d1e/d20/d21/f9c [0,4194304] 0 2024-04-24T23:14:02.755 INFO:tasks.workunit.client.0.smithi031.stdout:7/872: creat d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f12f x:0 0 0 2024-04-24T23:14:02.756 INFO:tasks.workunit.client.1.smithi040.stdout:3/701: truncate d1/f79 4270481 0 2024-04-24T23:14:02.758 INFO:tasks.workunit.client.1.smithi040.stdout:6/616: dwrite d7/d1a/d20/d51/d1e/d46/f98 [0,4194304] 0 2024-04-24T23:14:02.761 INFO:tasks.workunit.client.1.smithi040.stdout:2/692: link da/d29/f4a da/d1c/d37/d39/d79/da3/f100 0 2024-04-24T23:14:02.762 INFO:tasks.workunit.client.1.smithi040.stdout:6/617: chown d7/d1a/d20/d51/d1e/d46/d69/d87/fad 2012 1 2024-04-24T23:14:02.763 INFO:tasks.workunit.client.1.smithi040.stdout:2/693: fsync da/d81/f97 0 2024-04-24T23:14:02.779 INFO:tasks.workunit.client.0.smithi031.stdout:7/873: dwrite d1/d2/d83/dc1/f102 [0,4194304] 0 2024-04-24T23:14:02.780 INFO:tasks.workunit.client.0.smithi031.stdout:7/874: readlink d1/d2/dd/d33/d5e/l12c 0 2024-04-24T23:14:02.781 INFO:tasks.workunit.client.0.smithi031.stdout:7/875: dread - d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f12f zero size 2024-04-24T23:14:02.788 INFO:tasks.workunit.client.0.smithi031.stdout:7/876: read d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f9b [142496,80036] 0 2024-04-24T23:14:02.791 INFO:tasks.workunit.client.0.smithi031.stdout:7/877: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/d130 0 2024-04-24T23:14:02.805 INFO:tasks.workunit.client.1.smithi040.stdout:8/688: sync 2024-04-24T23:14:02.805 INFO:tasks.workunit.client.1.smithi040.stdout:9/678: sync 2024-04-24T23:14:02.807 INFO:tasks.workunit.client.1.smithi040.stdout:0/714: mkdir df/d12/d58/dc3/dcd/de6 0 2024-04-24T23:14:02.807 INFO:tasks.workunit.client.1.smithi040.stdout:6/618: symlink d7/d1a/d20/d51/d1e/d46/d85/lc2 0 2024-04-24T23:14:02.807 INFO:tasks.workunit.client.0.smithi031.stdout:7/878: dread d1/d2/dd/d33/d5e/d64/d6d/fbb [0,4194304] 0 2024-04-24T23:14:02.808 INFO:tasks.workunit.client.1.smithi040.stdout:2/694: dread - da/d1c/d37/d8a/f90 zero size 2024-04-24T23:14:02.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/879: stat d1/d2/dd/d33/f77 0 2024-04-24T23:14:02.809 INFO:tasks.workunit.client.0.smithi031.stdout:7/880: chown d1/d2/dd/d33/d5e/d64/fce 483407 1 2024-04-24T23:14:02.810 INFO:tasks.workunit.client.0.smithi031.stdout:3/789: sync 2024-04-24T23:14:02.810 INFO:tasks.workunit.client.1.smithi040.stdout:4/595: dread da/f18 [0,4194304] 0 2024-04-24T23:14:02.811 INFO:tasks.workunit.client.1.smithi040.stdout:2/695: chown da/d81/de6 5724957 1 2024-04-24T23:14:02.811 INFO:tasks.workunit.client.1.smithi040.stdout:9/679: creat d7/d92/ff4 x:0 0 0 2024-04-24T23:14:02.811 INFO:tasks.workunit.client.1.smithi040.stdout:7/682: dread db/d2f/d37/d7e/f86 [0,4194304] 0 2024-04-24T23:14:02.811 INFO:tasks.workunit.client.0.smithi031.stdout:3/790: stat d6/d14/d1d/d69/f94 0 2024-04-24T23:14:02.811 INFO:tasks.workunit.client.1.smithi040.stdout:6/619: write d7/d1a/d20/d51/f17 [880421,130540] 0 2024-04-24T23:14:02.812 INFO:tasks.workunit.client.1.smithi040.stdout:7/683: readlink db/d2f/da8/le8 0 2024-04-24T23:14:02.815 INFO:tasks.workunit.client.1.smithi040.stdout:0/715: unlink df/d1e/d20/d21/cab 0 2024-04-24T23:14:02.816 INFO:tasks.workunit.client.1.smithi040.stdout:8/689: getdents d0/d38/d4c/dd5 0 2024-04-24T23:14:02.817 INFO:tasks.workunit.client.1.smithi040.stdout:0/716: chown df/d1e/d20/d21/d51/ddb 252135 1 2024-04-24T23:14:02.817 INFO:tasks.workunit.client.1.smithi040.stdout:7/684: mknod db/d12/d73/cfa 0 2024-04-24T23:14:02.817 INFO:tasks.workunit.client.0.smithi031.stdout:7/881: dread d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d2c/f4c [0,4194304] 0 2024-04-24T23:14:02.818 INFO:tasks.workunit.client.1.smithi040.stdout:0/717: write df/d12/d58/f72 [782129,85202] 0 2024-04-24T23:14:02.821 INFO:tasks.workunit.client.1.smithi040.stdout:6/620: link d7/de/d89/fb7 d7/d1a/d20/d51/d1e/fc3 0 2024-04-24T23:14:02.823 INFO:tasks.workunit.client.1.smithi040.stdout:2/696: dwrite da/d1c/d37/d39/d3f/f7b [0,4194304] 0 2024-04-24T23:14:02.823 INFO:tasks.workunit.client.1.smithi040.stdout:9/680: creat d7/ff5 x:0 0 0 2024-04-24T23:14:02.828 INFO:tasks.workunit.client.1.smithi040.stdout:6/621: creat d7/d1a/d20/d51/d1e/d5d/fc4 x:0 0 0 2024-04-24T23:14:02.830 INFO:tasks.workunit.client.1.smithi040.stdout:6/622: chown d7/fae 3168 1 2024-04-24T23:14:02.830 INFO:tasks.workunit.client.0.smithi031.stdout:7/882: getdents d1/d2/d83 0 2024-04-24T23:14:02.833 INFO:tasks.workunit.client.1.smithi040.stdout:0/718: symlink df/d12/d58/dc3/dde/le7 0 2024-04-24T23:14:02.833 INFO:tasks.workunit.client.1.smithi040.stdout:2/697: dwrite da/d1c/d37/d39/d3f/f7b [0,4194304] 0 2024-04-24T23:14:02.834 INFO:tasks.workunit.client.1.smithi040.stdout:9/681: dwrite d7/da/d17/d40/d55/d2f/d1f/fc5 [0,4194304] 0 2024-04-24T23:14:02.838 INFO:tasks.workunit.client.0.smithi031.stdout:7/883: dwrite d1/f13 [0,4194304] 0 2024-04-24T23:14:02.838 INFO:tasks.workunit.client.1.smithi040.stdout:9/682: chown d7/da/d17/d40/d55/d2f/d28/cd1 76 1 2024-04-24T23:14:02.840 INFO:tasks.workunit.client.1.smithi040.stdout:9/683: readlink d7/da/l13 0 2024-04-24T23:14:02.841 INFO:tasks.workunit.client.1.smithi040.stdout:6/623: dwrite d7/d1a/d20/d51/d1e/f86 [0,4194304] 0 2024-04-24T23:14:02.842 INFO:tasks.workunit.client.1.smithi040.stdout:2/698: rename da/d10/c23 to da/d29/d55/c101 0 2024-04-24T23:14:02.844 INFO:tasks.workunit.client.1.smithi040.stdout:8/690: getdents d0/d16/d1b/d58/d8f 0 2024-04-24T23:14:02.845 INFO:tasks.workunit.client.0.smithi031.stdout:7/884: mknod d1/d2/dd/d33/d5e/d64/d6d/dae/db1/c131 0 2024-04-24T23:14:02.845 INFO:tasks.workunit.client.1.smithi040.stdout:2/699: write da/d1c/f57 [2649594,89341] 0 2024-04-24T23:14:02.847 INFO:tasks.workunit.client.1.smithi040.stdout:9/684: rmdir d7/da/d17/d40/d55/d2f 39 2024-04-24T23:14:02.849 INFO:tasks.workunit.client.1.smithi040.stdout:6/624: symlink d7/d1a/lc5 0 2024-04-24T23:14:02.854 INFO:tasks.workunit.client.1.smithi040.stdout:7/685: dwrite db/d12/d20/d52/f70 [0,4194304] 0 2024-04-24T23:14:02.854 INFO:tasks.workunit.client.1.smithi040.stdout:0/719: rename df/d1e/d20/d21/d41/le2 to df/d12/d4b/le8 0 2024-04-24T23:14:02.854 INFO:tasks.workunit.client.0.smithi031.stdout:7/885: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/fc3 [0,4194304] 0 2024-04-24T23:14:02.856 INFO:tasks.workunit.client.1.smithi040.stdout:2/700: rmdir da/d10/d15/daa 39 2024-04-24T23:14:02.858 INFO:tasks.workunit.client.1.smithi040.stdout:9/685: chown d7/da/d17/d40/d55/d2f/d28/d67/f6a 1465201 1 2024-04-24T23:14:02.865 INFO:tasks.workunit.client.1.smithi040.stdout:5/603: sync 2024-04-24T23:14:02.865 INFO:tasks.workunit.client.1.smithi040.stdout:1/648: sync 2024-04-24T23:14:02.865 INFO:tasks.workunit.client.1.smithi040.stdout:0/720: creat df/de1/d6e/fe9 x:0 0 0 2024-04-24T23:14:02.866 INFO:tasks.workunit.client.0.smithi031.stdout:7/886: mkdir d1/de4/d100/d132 0 2024-04-24T23:14:02.866 INFO:tasks.workunit.client.1.smithi040.stdout:2/701: truncate da/d10/fa1 25746 0 2024-04-24T23:14:02.866 INFO:tasks.workunit.client.1.smithi040.stdout:9/686: symlink d7/da/d17/d40/d55/d2f/lf6 0 2024-04-24T23:14:02.868 INFO:tasks.workunit.client.1.smithi040.stdout:6/625: dread d7/de/d4e/f7a [0,4194304] 0 2024-04-24T23:14:02.868 INFO:tasks.workunit.client.1.smithi040.stdout:2/702: read - da/d81/de6/ff2 zero size 2024-04-24T23:14:02.870 INFO:tasks.workunit.client.1.smithi040.stdout:7/686: dwrite db/d2f/d37/d7e/fc3 [0,4194304] 0 2024-04-24T23:14:02.872 INFO:tasks.workunit.client.1.smithi040.stdout:8/691: getdents d0/d16/d1b/d58 0 2024-04-24T23:14:02.872 INFO:tasks.workunit.client.0.smithi031.stdout:7/887: mkdir d1/d2/dd/d33/d133 0 2024-04-24T23:14:02.874 INFO:tasks.workunit.client.1.smithi040.stdout:1/649: symlink da/d12/d14/ldf 0 2024-04-24T23:14:02.875 INFO:tasks.workunit.client.1.smithi040.stdout:0/721: mknod df/d12/d4b/cea 0 2024-04-24T23:14:02.875 INFO:tasks.workunit.client.1.smithi040.stdout:9/687: unlink d7/d5e/da1/fad 0 2024-04-24T23:14:02.881 INFO:tasks.workunit.client.1.smithi040.stdout:1/650: creat da/d12/d14/fe0 x:0 0 0 2024-04-24T23:14:02.882 INFO:tasks.workunit.client.1.smithi040.stdout:0/722: creat df/d1e/d20/d21/d41/feb x:0 0 0 2024-04-24T23:14:02.882 INFO:tasks.workunit.client.1.smithi040.stdout:8/692: read d0/d7e/dc1/d9e/d90/d91/f97 [427386,109954] 0 2024-04-24T23:14:02.883 INFO:tasks.workunit.client.1.smithi040.stdout:0/723: stat df/de1/d6e/fe9 0 2024-04-24T23:14:02.884 INFO:tasks.workunit.client.1.smithi040.stdout:7/687: creat db/d12/d62/de7/ffb x:0 0 0 2024-04-24T23:14:02.887 INFO:tasks.workunit.client.1.smithi040.stdout:9/688: rename d7/da/d17/d40/d55/d61 to d7/da/d17/d40/d55/d2f/df7 0 2024-04-24T23:14:02.888 INFO:tasks.workunit.client.1.smithi040.stdout:2/703: truncate da/d1c/d37/d39/d79/da3/fea 28564 0 2024-04-24T23:14:02.891 INFO:tasks.workunit.client.1.smithi040.stdout:0/724: chown df/d12/d49/c5c 2958 1 2024-04-24T23:14:02.894 INFO:tasks.workunit.client.1.smithi040.stdout:2/704: stat da/d1c/d37/d39/d79/caf 0 2024-04-24T23:14:02.896 INFO:tasks.workunit.client.1.smithi040.stdout:1/651: symlink da/d12/le1 0 2024-04-24T23:14:02.899 INFO:tasks.workunit.client.1.smithi040.stdout:0/725: fsync df/f63 0 2024-04-24T23:14:02.901 INFO:tasks.workunit.client.1.smithi040.stdout:7/688: mkdir db/d12/d20/d52/dfc 0 2024-04-24T23:14:02.908 INFO:tasks.workunit.client.0.smithi031.stdout:7/888: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/ffd [0,4194304] 0 2024-04-24T23:14:02.911 INFO:tasks.workunit.client.0.smithi031.stdout:7/889: write d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f5d [5242005,501] 0 2024-04-24T23:14:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:02 smithi040 ceph-mon[33128]: pgmap v34: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 95 MiB/s rd, 271 MiB/s wr, 568 op/s 2024-04-24T23:14:02.912 INFO:tasks.workunit.client.1.smithi040.stdout:7/689: symlink db/d12/d3f/lfd 0 2024-04-24T23:14:02.920 INFO:tasks.workunit.client.1.smithi040.stdout:8/693: truncate d0/d16/f7c 1439533 0 2024-04-24T23:14:02.922 INFO:tasks.workunit.client.1.smithi040.stdout:0/726: unlink df/d12/d58/dc3/dcd/fe0 0 2024-04-24T23:14:02.924 INFO:tasks.workunit.client.1.smithi040.stdout:7/690: unlink db/d43/c46 0 2024-04-24T23:14:02.925 INFO:tasks.workunit.client.1.smithi040.stdout:9/689: link d7/d92/cc1 d7/da/cf8 0 2024-04-24T23:14:02.927 INFO:tasks.workunit.client.1.smithi040.stdout:8/694: mkdir d0/d8/d22/db8/de5 0 2024-04-24T23:14:02.928 INFO:tasks.workunit.client.1.smithi040.stdout:7/691: read db/d12/d73/fbe [1769530,118828] 0 2024-04-24T23:14:02.930 INFO:tasks.workunit.client.1.smithi040.stdout:1/652: dwrite da/d12/d18/d2a/f9c [0,4194304] 0 2024-04-24T23:14:02.930 INFO:tasks.workunit.client.1.smithi040.stdout:4/596: sync 2024-04-24T23:14:02.931 INFO:tasks.workunit.client.1.smithi040.stdout:3/702: sync 2024-04-24T23:14:02.933 INFO:tasks.workunit.client.1.smithi040.stdout:8/695: write d0/d7e/dc1/d9e/d90/d91/fd3 [1182001,73279] 0 2024-04-24T23:14:02.933 INFO:tasks.workunit.client.1.smithi040.stdout:7/692: dread - db/d12/d3f/fc1 zero size 2024-04-24T23:14:02.934 INFO:tasks.workunit.client.1.smithi040.stdout:0/727: dwrite df/de1/d6e/fe9 [0,4194304] 0 2024-04-24T23:14:02.941 INFO:tasks.workunit.client.0.smithi031.stdout:7/890: dread d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/fb5 [0,4194304] 0 2024-04-24T23:14:02.943 INFO:tasks.workunit.client.1.smithi040.stdout:7/693: dwrite db/d12/d20/d56/fd6 [0,4194304] 0 2024-04-24T23:14:02.952 INFO:tasks.workunit.client.1.smithi040.stdout:3/703: creat d1/d9/d15/d45/d96/fef x:0 0 0 2024-04-24T23:14:02.957 INFO:tasks.workunit.client.1.smithi040.stdout:4/597: dread da/d3f/d6b/d2b/f39 [0,4194304] 0 2024-04-24T23:14:02.960 INFO:tasks.workunit.client.1.smithi040.stdout:0/728: rmdir df/d1e/d20 39 2024-04-24T23:14:02.964 INFO:tasks.workunit.client.1.smithi040.stdout:9/690: dwrite d7/da/d17/d40/d55/d2f/df7/d6c/fa8 [0,4194304] 0 2024-04-24T23:14:02.964 INFO:tasks.workunit.client.1.smithi040.stdout:7/694: rename db/d12/d3f/d7a/f9e to db/d12/d20/d52/d6c/d8f/da7/ffe 0 2024-04-24T23:14:02.965 INFO:tasks.workunit.client.1.smithi040.stdout:5/604: sync 2024-04-24T23:14:02.968 INFO:tasks.workunit.client.1.smithi040.stdout:3/704: rmdir d1/d9/d18/da7 39 2024-04-24T23:14:02.968 INFO:tasks.workunit.client.1.smithi040.stdout:7/695: write db/d12/d7d/fcd [345653,29190] 0 2024-04-24T23:14:02.969 INFO:tasks.workunit.client.1.smithi040.stdout:3/705: dread - d1/d9/d18/d36/d55/d7e/fc7 zero size 2024-04-24T23:14:02.970 INFO:tasks.workunit.client.1.smithi040.stdout:2/705: truncate da/d1c/d37/d39/d79/da3/fea 126225 0 2024-04-24T23:14:02.971 INFO:tasks.workunit.client.1.smithi040.stdout:2/706: fsync da/d81/de6/ff6 0 2024-04-24T23:14:02.972 INFO:tasks.workunit.client.1.smithi040.stdout:4/598: rmdir da/d3f/d40/d47 39 2024-04-24T23:14:02.973 INFO:tasks.workunit.client.1.smithi040.stdout:4/599: fsync da/d3f/d6b/fb1 0 2024-04-24T23:14:02.974 INFO:tasks.workunit.client.1.smithi040.stdout:8/696: dread d0/d8/d22/f62 [0,4194304] 0 2024-04-24T23:14:02.977 INFO:tasks.workunit.client.1.smithi040.stdout:0/729: rename df/d14/f96 to df/d12/d58/dc3/dde/fec 0 2024-04-24T23:14:02.978 INFO:tasks.workunit.client.1.smithi040.stdout:1/653: creat da/d12/d18/fe2 x:0 0 0 2024-04-24T23:14:02.979 INFO:tasks.workunit.client.1.smithi040.stdout:0/730: read df/d1e/f7b [1973568,66439] 0 2024-04-24T23:14:02.979 INFO:tasks.workunit.client.1.smithi040.stdout:1/654: readlink da/d12/d23/ld1 0 2024-04-24T23:14:02.979 INFO:tasks.workunit.client.1.smithi040.stdout:5/605: write d2/d2f/d14/f17 [4796636,55701] 0 2024-04-24T23:14:02.981 INFO:tasks.workunit.client.1.smithi040.stdout:4/600: creat da/d3f/d40/d41/fd2 x:0 0 0 2024-04-24T23:14:02.983 INFO:tasks.workunit.client.1.smithi040.stdout:4/601: read f9 [2772575,28623] 0 2024-04-24T23:14:02.983 INFO:tasks.workunit.client.0.smithi031.stdout:7/891: dread d1/d2/dd/d33/d5e/d64/f8e [0,4194304] 0 2024-04-24T23:14:02.984 INFO:tasks.workunit.client.0.smithi031.stdout:7/892: write d1/d2/f2a [2457552,20330] 0 2024-04-24T23:14:02.986 INFO:tasks.workunit.client.1.smithi040.stdout:0/731: mkdir df/d12/ded 0 2024-04-24T23:14:02.988 INFO:tasks.workunit.client.1.smithi040.stdout:3/706: mknod d1/d9/d18/da7/cf0 0 2024-04-24T23:14:02.990 INFO:tasks.workunit.client.1.smithi040.stdout:1/655: creat da/d35/dab/fe3 x:0 0 0 2024-04-24T23:14:02.991 INFO:tasks.workunit.client.1.smithi040.stdout:5/606: chown d2/d7/d8/db/c37 6867355 1 2024-04-24T23:14:02.991 INFO:tasks.workunit.client.1.smithi040.stdout:5/607: chown d2/d7/dc3 14 1 2024-04-24T23:14:02.997 INFO:tasks.workunit.client.1.smithi040.stdout:7/696: rename c9 to db/d12/d7d/dd2/cff 0 2024-04-24T23:14:02.998 INFO:tasks.workunit.client.1.smithi040.stdout:4/602: mkdir da/d3f/d40/d41/d8e/dd3 0 2024-04-24T23:14:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:02 smithi031 ceph-mon[26411]: pgmap v34: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 95 MiB/s rd, 271 MiB/s wr, 568 op/s 2024-04-24T23:14:03.001 INFO:tasks.workunit.client.1.smithi040.stdout:0/732: creat df/d12/d58/d5b/fee x:0 0 0 2024-04-24T23:14:03.006 INFO:tasks.workunit.client.1.smithi040.stdout:3/707: creat d1/d80/ff1 x:0 0 0 2024-04-24T23:14:03.007 INFO:tasks.workunit.client.1.smithi040.stdout:5/608: mknod d2/d7/d8/d6d/d89/cc8 0 2024-04-24T23:14:03.012 INFO:tasks.workunit.client.0.smithi031.stdout:7/893: truncate d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f5d 3105108 0 2024-04-24T23:14:03.013 INFO:tasks.workunit.client.1.smithi040.stdout:2/707: getdents da/d1c/d37/d39/d79/da3 0 2024-04-24T23:14:03.015 INFO:tasks.workunit.client.1.smithi040.stdout:1/656: mknod da/d12/d18/d2a/ce4 0 2024-04-24T23:14:03.018 INFO:tasks.workunit.client.1.smithi040.stdout:1/657: dread - da/d35/da3/faa zero size 2024-04-24T23:14:03.018 INFO:tasks.workunit.client.1.smithi040.stdout:0/733: dread df/d12/f4c [0,4194304] 0 2024-04-24T23:14:03.019 INFO:tasks.workunit.client.1.smithi040.stdout:8/697: dwrite d0/d52/f84 [0,4194304] 0 2024-04-24T23:14:03.022 INFO:tasks.workunit.client.1.smithi040.stdout:2/708: mkdir da/d1c/d37/d102 0 2024-04-24T23:14:03.023 INFO:tasks.workunit.client.1.smithi040.stdout:6/626: sync 2024-04-24T23:14:03.025 INFO:tasks.workunit.client.1.smithi040.stdout:2/709: dread da/d10/d4d/d6c/fad [4194304,4194304] 0 2024-04-24T23:14:03.026 INFO:tasks.workunit.client.0.smithi031.stdout:6/931: sync 2024-04-24T23:14:03.029 INFO:tasks.workunit.client.1.smithi040.stdout:9/691: rename d7/da/d17/d40/d55/d2f/d1f/d4e/l9e to d7/da/d17/d40/d55/d2f/d1f/d31/lf9 0 2024-04-24T23:14:03.030 INFO:tasks.workunit.client.0.smithi031.stdout:7/894: link d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/c22 d1/d2/dd/d33/d5e/d64/d6d/dae/dd7/d101/c134 0 2024-04-24T23:14:03.035 INFO:tasks.workunit.client.0.smithi031.stdout:6/932: unlink d13/d15/d1d/d23/d4d/d7b/le0 0 2024-04-24T23:14:03.036 INFO:tasks.workunit.client.1.smithi040.stdout:0/734: creat df/d12/d4b/db8/fef x:0 0 0 2024-04-24T23:14:03.038 INFO:tasks.workunit.client.0.smithi031.stdout:7/895: creat d1/d2/dd/d33/d5e/d64/d6d/dae/dd7/f135 x:0 0 0 2024-04-24T23:14:03.039 INFO:tasks.workunit.client.0.smithi031.stdout:7/896: write d1/de4/d106/f12d [260855,91032] 0 2024-04-24T23:14:03.040 INFO:tasks.workunit.client.0.smithi031.stdout:7/897: stat d1/ldb 0 2024-04-24T23:14:03.042 INFO:tasks.workunit.client.0.smithi031.stdout:7/898: dread - d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d2c/f11b zero size 2024-04-24T23:14:03.046 INFO:tasks.workunit.client.1.smithi040.stdout:4/603: rename da/d78/la3 to da/d3f/d40/d41/d62/ld4 0 2024-04-24T23:14:03.047 INFO:tasks.workunit.client.1.smithi040.stdout:1/658: mkdir da/d35/d38/d47/dbc/de5 0 2024-04-24T23:14:03.050 INFO:tasks.workunit.client.1.smithi040.stdout:7/697: getdents db/d12/d62/d8a/db3/dbf 0 2024-04-24T23:14:03.053 INFO:tasks.workunit.client.0.smithi031.stdout:6/933: getdents d13/d1b/d32/db4/d79/dd7/dfb 0 2024-04-24T23:14:03.055 INFO:tasks.workunit.client.0.smithi031.stdout:4/958: sync 2024-04-24T23:14:03.055 INFO:tasks.workunit.client.0.smithi031.stdout:2/788: sync 2024-04-24T23:14:03.055 INFO:tasks.workunit.client.0.smithi031.stdout:9/960: sync 2024-04-24T23:14:03.055 INFO:tasks.workunit.client.0.smithi031.stdout:3/791: sync 2024-04-24T23:14:03.055 INFO:tasks.workunit.client.0.smithi031.stdout:5/836: sync 2024-04-24T23:14:03.055 INFO:tasks.workunit.client.0.smithi031.stdout:1/840: sync 2024-04-24T23:14:03.055 INFO:tasks.workunit.client.0.smithi031.stdout:0/981: sync 2024-04-24T23:14:03.056 INFO:tasks.workunit.client.0.smithi031.stdout:0/982: chown da/d20/d4e/dc9 329 1 2024-04-24T23:14:03.059 INFO:tasks.workunit.client.0.smithi031.stdout:0/983: write da/d11/d6a/dba/dc0/f121 [322562,62810] 0 2024-04-24T23:14:03.060 INFO:tasks.workunit.client.1.smithi040.stdout:2/710: creat da/d1c/d37/d102/f103 x:0 0 0 2024-04-24T23:14:03.061 INFO:tasks.workunit.client.1.smithi040.stdout:2/711: chown da/d10/d15/f30 14 1 2024-04-24T23:14:03.065 INFO:tasks.workunit.client.1.smithi040.stdout:4/604: dread da/f37 [0,4194304] 0 2024-04-24T23:14:03.066 INFO:tasks.workunit.client.1.smithi040.stdout:3/708: rename d1/d2d/d44/c51 to d1/d2d/de/d30/cf2 0 2024-04-24T23:14:03.068 INFO:tasks.workunit.client.1.smithi040.stdout:6/627: truncate d7/d1a/d20/d51/f17 305973 0 2024-04-24T23:14:03.068 INFO:tasks.workunit.client.1.smithi040.stdout:3/709: chown d1/d9/fe2 26070 1 2024-04-24T23:14:03.069 INFO:tasks.workunit.client.0.smithi031.stdout:7/899: dwrite d1/dbe/fb7 [0,4194304] 0 2024-04-24T23:14:03.071 INFO:tasks.workunit.client.1.smithi040.stdout:4/605: dwrite da/d3f/d6b/fb1 [0,4194304] 0 2024-04-24T23:14:03.080 INFO:tasks.workunit.client.1.smithi040.stdout:5/609: getdents d2/d7/d8/d1f/da1 0 2024-04-24T23:14:03.081 INFO:tasks.workunit.client.1.smithi040.stdout:2/712: creat da/d1c/d37/d8a/f104 x:0 0 0 2024-04-24T23:14:03.084 INFO:tasks.workunit.client.1.smithi040.stdout:9/692: rename d7/d92/d9c/dc2 to d7/d5e/da1/dfa 0 2024-04-24T23:14:03.096 INFO:tasks.workunit.client.1.smithi040.stdout:5/610: creat d2/d7/d8/db/fc9 x:0 0 0 2024-04-24T23:14:03.098 INFO:tasks.workunit.client.1.smithi040.stdout:2/713: symlink da/d1c/d37/d39/d3f/d52/d78/dc7/dcc/l105 0 2024-04-24T23:14:03.105 INFO:tasks.workunit.client.1.smithi040.stdout:3/710: symlink d1/d80/dbf/ded/lf3 0 2024-04-24T23:14:03.105 INFO:tasks.workunit.client.1.smithi040.stdout:1/659: write da/d12/d18/fa6 [619513,70813] 0 2024-04-24T23:14:03.109 INFO:tasks.workunit.client.1.smithi040.stdout:6/628: link d7/d1a/d20/d51/d1e/d46/f98 d7/d1a/d20/d8c/fc6 0 2024-04-24T23:14:03.111 INFO:tasks.workunit.client.1.smithi040.stdout:1/660: dwrite da/d12/d18/d2a/f79 [0,4194304] 0 2024-04-24T23:14:03.115 INFO:tasks.workunit.client.1.smithi040.stdout:0/735: creat df/d1e/d20/ff0 x:0 0 0 2024-04-24T23:14:03.117 INFO:tasks.workunit.client.1.smithi040.stdout:8/698: sync 2024-04-24T23:14:03.123 INFO:tasks.workunit.client.1.smithi040.stdout:5/611: unlink d2/d7/d8/db/d1e/f82 0 2024-04-24T23:14:03.131 INFO:tasks.workunit.client.1.smithi040.stdout:7/698: write db/d12/d20/d52/f23 [905855,26703] 0 2024-04-24T23:14:03.133 INFO:tasks.workunit.client.1.smithi040.stdout:6/629: write d7/d1a/d20/d8c/fc6 [613317,18480] 0 2024-04-24T23:14:03.137 INFO:tasks.workunit.client.1.smithi040.stdout:1/661: mknod da/d12/d14/d58/ce6 0 2024-04-24T23:14:03.154 INFO:tasks.workunit.client.1.smithi040.stdout:4/606: dwrite da/d3f/d40/d47/d75/f97 [0,4194304] 0 2024-04-24T23:14:03.163 INFO:tasks.workunit.client.1.smithi040.stdout:7/699: creat db/d12/d7d/dd2/f100 x:0 0 0 2024-04-24T23:14:03.168 INFO:tasks.workunit.client.1.smithi040.stdout:5/612: dread d2/d2f/d14/f81 [0,4194304] 0 2024-04-24T23:14:03.171 INFO:tasks.workunit.client.1.smithi040.stdout:3/711: dwrite d1/d2a/d6a/f8e [4194304,4194304] 0 2024-04-24T23:14:03.173 INFO:tasks.workunit.client.1.smithi040.stdout:8/699: mknod d0/d8/ce6 0 2024-04-24T23:14:03.175 INFO:tasks.workunit.client.1.smithi040.stdout:2/714: creat da/f106 x:0 0 0 2024-04-24T23:14:03.178 INFO:tasks.workunit.client.1.smithi040.stdout:9/693: getdents d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4 0 2024-04-24T23:14:03.180 INFO:tasks.workunit.client.1.smithi040.stdout:4/607: rename da/d3f to da/d3f/d8d/db5/dd5 22 2024-04-24T23:14:03.182 INFO:tasks.workunit.client.1.smithi040.stdout:7/700: mkdir db/d43/db9/dd4/d101 0 2024-04-24T23:14:03.188 INFO:tasks.workunit.client.1.smithi040.stdout:1/662: mknod da/ce7 0 2024-04-24T23:14:03.193 INFO:tasks.workunit.client.1.smithi040.stdout:5/613: dread d2/d2f/d14/f61 [0,4194304] 0 2024-04-24T23:14:03.195 INFO:tasks.workunit.client.1.smithi040.stdout:0/736: link df/d1e/f7b df/d12/d58/d5b/ff1 0 2024-04-24T23:14:03.197 INFO:tasks.workunit.client.0.smithi031.stdout:6/934: mknod d13/d1b/d32/db4/d122/c13d 0 2024-04-24T23:14:03.198 INFO:tasks.workunit.client.0.smithi031.stdout:6/935: chown d13/d15/d1d/c95 11918 1 2024-04-24T23:14:03.199 INFO:tasks.workunit.client.1.smithi040.stdout:3/712: creat d1/d9/d18/d36/d55/d62/ff4 x:0 0 0 2024-04-24T23:14:03.200 INFO:tasks.workunit.client.0.smithi031.stdout:4/959: chown d3/da3/faf 778 1 2024-04-24T23:14:03.201 INFO:tasks.workunit.client.1.smithi040.stdout:8/700: mkdir d0/d7e/dc1/d9e/d90/de7 0 2024-04-24T23:14:03.203 INFO:tasks.workunit.client.1.smithi040.stdout:2/715: mknod da/d10/d7a/c107 0 2024-04-24T23:14:03.208 INFO:tasks.workunit.client.1.smithi040.stdout:4/608: symlink da/d78/ld6 0 2024-04-24T23:14:03.209 INFO:tasks.workunit.client.1.smithi040.stdout:4/609: chown da/d3f/d40/d47/d71/lc0 252 1 2024-04-24T23:14:03.211 INFO:tasks.workunit.client.1.smithi040.stdout:7/701: creat db/d12/d62/d8a/f102 x:0 0 0 2024-04-24T23:14:03.213 INFO:tasks.workunit.client.1.smithi040.stdout:7/702: chown db/d12/d20/d52/d44/f47 252 1 2024-04-24T23:14:03.215 INFO:tasks.workunit.client.1.smithi040.stdout:4/610: dwrite da/d78/dbb/fc2 [0,4194304] 0 2024-04-24T23:14:03.215 INFO:tasks.workunit.client.1.smithi040.stdout:1/663: symlink da/d12/d18/d2a/d8b/le8 0 2024-04-24T23:14:03.222 INFO:tasks.workunit.client.1.smithi040.stdout:0/737: mkdir df/d1e/d20/d21/d41/d8a/df2 0 2024-04-24T23:14:03.222 INFO:tasks.workunit.client.1.smithi040.stdout:6/630: dwrite d7/de/ff [0,4194304] 0 2024-04-24T23:14:03.223 INFO:tasks.workunit.client.1.smithi040.stdout:0/738: read - df/d12/d58/f7d zero size 2024-04-24T23:14:03.224 INFO:tasks.workunit.client.0.smithi031.stdout:6/936: dread d13/d1b/dd0/fd9 [0,4194304] 0 2024-04-24T23:14:03.226 INFO:tasks.workunit.client.1.smithi040.stdout:8/701: creat d0/d38/d4c/da3/fe8 x:0 0 0 2024-04-24T23:14:03.227 INFO:tasks.workunit.client.1.smithi040.stdout:8/702: fdatasync d0/d16/d1b/d58/fa5 0 2024-04-24T23:14:03.228 INFO:tasks.workunit.client.1.smithi040.stdout:2/716: mkdir da/d1c/d37/d39/d77/d108 0 2024-04-24T23:14:03.242 INFO:tasks.workunit.client.1.smithi040.stdout:7/703: mkdir db/d12/d62/d103 0 2024-04-24T23:14:03.246 INFO:tasks.workunit.client.1.smithi040.stdout:1/664: fdatasync da/d12/d23/f3c 0 2024-04-24T23:14:03.246 INFO:tasks.workunit.client.1.smithi040.stdout:5/614: dwrite d2/d7/fe [0,4194304] 0 2024-04-24T23:14:03.248 INFO:tasks.workunit.client.0.smithi031.stdout:0/984: symlink da/d18/d5a/d7d/d10d/l146 0 2024-04-24T23:14:03.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/985: read da/d18/d5a/d7d/f116 [594202,18503] 0 2024-04-24T23:14:03.251 INFO:tasks.workunit.client.1.smithi040.stdout:4/611: rename da/d3f/d40/d41/d8e/f96 to da/d3f/d6b/d3c/fd7 0 2024-04-24T23:14:03.253 INFO:tasks.workunit.client.1.smithi040.stdout:9/694: write d7/da/d17/d40/d55/d2f/d1f/d31/f36 [961672,47873] 0 2024-04-24T23:14:03.258 INFO:tasks.workunit.client.1.smithi040.stdout:4/612: dwrite da/d3f/d40/d41/f4e [4194304,4194304] 0 2024-04-24T23:14:03.259 INFO:tasks.workunit.client.1.smithi040.stdout:3/713: sync 2024-04-24T23:14:03.260 INFO:tasks.workunit.client.1.smithi040.stdout:4/613: chown da/d3f/d40/d41/f4e 1964 1 2024-04-24T23:14:03.260 INFO:tasks.workunit.client.0.smithi031.stdout:4/960: mkdir d3/d23/d37/d86/d113/d12a/d105/d138 0 2024-04-24T23:14:03.261 INFO:tasks.workunit.client.1.smithi040.stdout:1/665: dread da/d12/d18/d2a/fc7 [0,4194304] 0 2024-04-24T23:14:03.262 INFO:tasks.workunit.client.1.smithi040.stdout:8/703: truncate d0/d8/d55/f64 299747 0 2024-04-24T23:14:03.262 INFO:tasks.workunit.client.1.smithi040.stdout:8/704: stat d0/d16/d19/l60 0 2024-04-24T23:14:03.265 INFO:tasks.workunit.client.0.smithi031.stdout:9/961: dwrite d2/dae/f17 [0,4194304] 0 2024-04-24T23:14:03.266 INFO:tasks.workunit.client.1.smithi040.stdout:4/614: read da/f2f [905367,33003] 0 2024-04-24T23:14:03.269 INFO:tasks.workunit.client.0.smithi031.stdout:0/986: dwrite da/d86/da3/dcd/ddc/f118 [0,4194304] 0 2024-04-24T23:14:03.276 INFO:tasks.workunit.client.0.smithi031.stdout:1/841: dwrite d1/d3e/d46/dbd/f115 [4194304,4194304] 0 2024-04-24T23:14:03.276 INFO:tasks.workunit.client.0.smithi031.stdout:4/961: unlink d3/d23/d3b/fd6 0 2024-04-24T23:14:03.283 INFO:tasks.workunit.client.1.smithi040.stdout:1/666: mkdir da/d12/d18/d2a/d41/d4f/de9 0 2024-04-24T23:14:03.284 INFO:tasks.workunit.client.1.smithi040.stdout:1/667: write da/ff [2831745,59420] 0 2024-04-24T23:14:03.285 INFO:tasks.workunit.client.0.smithi031.stdout:3/792: link d6/df/d16/d57/de1/fe5 d6/fee 0 2024-04-24T23:14:03.286 INFO:tasks.workunit.client.0.smithi031.stdout:2/789: getdents d5/d8 0 2024-04-24T23:14:03.286 INFO:tasks.workunit.client.1.smithi040.stdout:8/705: fsync d0/d8/fc3 0 2024-04-24T23:14:03.286 INFO:tasks.workunit.client.0.smithi031.stdout:5/837: dwrite d0/d6/d3b/d4b/f51 [0,4194304] 0 2024-04-24T23:14:03.288 INFO:tasks.workunit.client.1.smithi040.stdout:4/615: mknod da/d45/d58/cd8 0 2024-04-24T23:14:03.293 INFO:tasks.workunit.client.1.smithi040.stdout:9/695: dread d7/da/d17/d40/d55/d2f/fca [0,4194304] 0 2024-04-24T23:14:03.294 INFO:tasks.workunit.client.0.smithi031.stdout:4/962: rename d3/d20/d25/l73 to d3/d23/d69/d3f/d123/l139 0 2024-04-24T23:14:03.297 INFO:tasks.workunit.client.1.smithi040.stdout:5/615: mkdir d2/d7/d8/d1f/d57/dca 0 2024-04-24T23:14:03.304 INFO:tasks.workunit.client.0.smithi031.stdout:5/838: fsync d0/d6/d3b/d4b/dc/d25/d55/f109 0 2024-04-24T23:14:03.311 INFO:tasks.workunit.client.1.smithi040.stdout:1/668: creat da/d12/d14/d58/fea x:0 0 0 2024-04-24T23:14:03.312 INFO:tasks.workunit.client.1.smithi040.stdout:8/706: mknod d0/d52/ce9 0 2024-04-24T23:14:03.316 INFO:tasks.workunit.client.0.smithi031.stdout:7/900: write d1/d2/dd/f74 [1954406,108167] 0 2024-04-24T23:14:03.319 INFO:tasks.workunit.client.1.smithi040.stdout:8/707: dwrite d0/d38/faa [0,4194304] 0 2024-04-24T23:14:03.321 INFO:tasks.workunit.client.1.smithi040.stdout:2/717: write da/d1c/d37/d39/f73 [117960,3280] 0 2024-04-24T23:14:03.321 INFO:tasks.workunit.client.1.smithi040.stdout:7/704: write db/d2f/d37/d7e/f86 [2171456,16099] 0 2024-04-24T23:14:03.323 INFO:tasks.workunit.client.1.smithi040.stdout:2/718: chown da/d10 76355 1 2024-04-24T23:14:03.323 INFO:tasks.workunit.client.1.smithi040.stdout:0/739: dwrite df/f63 [0,4194304] 0 2024-04-24T23:14:03.324 INFO:tasks.workunit.client.1.smithi040.stdout:8/708: chown d0/d8/d55 1576 1 2024-04-24T23:14:03.324 INFO:tasks.workunit.client.1.smithi040.stdout:4/616: creat da/d3f/d40/d41/fd9 x:0 0 0 2024-04-24T23:14:03.327 INFO:tasks.workunit.client.0.smithi031.stdout:6/937: dwrite d13/d15/d1d/d23/d24/d8b/f9b [0,4194304] 0 2024-04-24T23:14:03.329 INFO:tasks.workunit.client.0.smithi031.stdout:4/963: mkdir d3/d23/d37/d86/d113/d12a/d105/d138/d13a 0 2024-04-24T23:14:03.329 INFO:tasks.workunit.client.0.smithi031.stdout:9/962: write d2/dae/d14/d1d/d45/d56/f66 [1972659,84716] 0 2024-04-24T23:14:03.331 INFO:tasks.workunit.client.1.smithi040.stdout:7/705: read db/d12/d20/d52/d6c/d8f/db8/fc8 [2674219,118811] 0 2024-04-24T23:14:03.331 INFO:tasks.workunit.client.1.smithi040.stdout:6/631: rename d7/de/f5e to d7/fc7 0 2024-04-24T23:14:03.332 INFO:tasks.workunit.client.0.smithi031.stdout:0/987: getdents da/d18/d1a 0 2024-04-24T23:14:03.333 INFO:tasks.workunit.client.1.smithi040.stdout:5/616: chown d2/d7/d8/d64/d6b/l9a 2 1 2024-04-24T23:14:03.336 INFO:tasks.workunit.client.0.smithi031.stdout:5/839: symlink d0/d7/d74/dbd/d112/l11b 0 2024-04-24T23:14:03.339 INFO:tasks.workunit.client.0.smithi031.stdout:6/938: dread d13/d15/d1d/d23/d24/d8b/f9b [0,4194304] 0 2024-04-24T23:14:03.344 INFO:tasks.workunit.client.1.smithi040.stdout:1/669: creat da/d12/d18/d2a/d4d/feb x:0 0 0 2024-04-24T23:14:03.345 INFO:tasks.workunit.client.1.smithi040.stdout:3/714: dwrite d1/d2a/d8d/fca [0,4194304] 0 2024-04-24T23:14:03.346 INFO:tasks.workunit.client.0.smithi031.stdout:1/842: dread d1/da/d12/d65/dcf/d11/f20 [0,4194304] 0 2024-04-24T23:14:03.348 INFO:tasks.workunit.client.0.smithi031.stdout:4/964: fdatasync d3/d20/d2d/fd2 0 2024-04-24T23:14:03.350 INFO:tasks.workunit.client.1.smithi040.stdout:2/719: mkdir da/d10/d4d/dd7/d109 0 2024-04-24T23:14:03.351 INFO:tasks.workunit.client.0.smithi031.stdout:2/790: dwrite d5/fba [0,4194304] 0 2024-04-24T23:14:03.353 INFO:tasks.workunit.client.0.smithi031.stdout:5/840: readlink d0/d6/d3b/d4b/dc/d8b/l113 0 2024-04-24T23:14:03.355 INFO:tasks.workunit.client.0.smithi031.stdout:4/965: write d3/d20/d2d/d44/fb2 [3834447,56271] 0 2024-04-24T23:14:03.356 INFO:tasks.workunit.client.0.smithi031.stdout:7/901: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d10e/f136 x:0 0 0 2024-04-24T23:14:03.357 INFO:tasks.workunit.client.1.smithi040.stdout:2/720: dwrite da/f19 [4194304,4194304] 0 2024-04-24T23:14:03.359 INFO:tasks.workunit.client.0.smithi031.stdout:3/793: dwrite d6/d14/d91/f99 [0,4194304] 0 2024-04-24T23:14:03.360 INFO:tasks.workunit.client.1.smithi040.stdout:8/709: rename d0/d16/d1b/d3f/d44/l67 to d0/d38/d4c/lea 0 2024-04-24T23:14:03.364 INFO:tasks.workunit.client.0.smithi031.stdout:3/794: stat d6/df/d16/d2a/d95/d9b/d1a/l4e 0 2024-04-24T23:14:03.366 INFO:tasks.workunit.client.1.smithi040.stdout:2/721: dread da/d1c/d4f/f61 [0,4194304] 0 2024-04-24T23:14:03.368 INFO:tasks.workunit.client.0.smithi031.stdout:0/988: write da/d11/d6a/dba/ff7 [703883,20216] 0 2024-04-24T23:14:03.370 INFO:tasks.workunit.client.0.smithi031.stdout:0/989: chown da/d20/f13c 176 1 2024-04-24T23:14:03.373 INFO:tasks.workunit.client.0.smithi031.stdout:9/963: write d2/dae/d14/d1d/d45/d99/d9c/d100/f10e [301909,39090] 0 2024-04-24T23:14:03.382 INFO:tasks.workunit.client.0.smithi031.stdout:6/939: truncate d13/d15/d1d/d23/d24/d8b/fcd 1269287 0 2024-04-24T23:14:03.382 INFO:tasks.workunit.client.1.smithi040.stdout:6/632: symlink d7/d1a/d20/d51/d1e/lc8 0 2024-04-24T23:14:03.383 INFO:tasks.workunit.client.0.smithi031.stdout:1/843: truncate d1/d3e/d46/dbd/fdd 499820 0 2024-04-24T23:14:03.384 INFO:tasks.workunit.client.1.smithi040.stdout:6/633: write d7/d1a/d20/d51/d1e/d46/f98 [4020002,79468] 0 2024-04-24T23:14:03.384 INFO:tasks.workunit.client.0.smithi031.stdout:6/940: write d13/d15/d1d/d23/d24/d43/d68/db8/fcc [2813660,80669] 0 2024-04-24T23:14:03.386 INFO:tasks.workunit.client.1.smithi040.stdout:0/740: rename df/d1e/d20/d21/d41/d52 to df/d1e/d20/d21/d41/d8a/df3 0 2024-04-24T23:14:03.389 INFO:tasks.workunit.client.1.smithi040.stdout:0/741: chown df/d12/f22 3426169 1 2024-04-24T23:14:03.390 INFO:tasks.workunit.client.1.smithi040.stdout:2/722: chown da/d1c/d37/d39/d79/da3/fea 7764 1 2024-04-24T23:14:03.392 INFO:tasks.workunit.client.1.smithi040.stdout:7/706: dread db/d12/d20/d52/d44/f8d [0,4194304] 0 2024-04-24T23:14:03.392 INFO:tasks.workunit.client.0.smithi031.stdout:0/990: read da/d18/d1a/d37/f5e [198853,2355] 0 2024-04-24T23:14:03.393 INFO:tasks.workunit.client.1.smithi040.stdout:9/696: getdents d7/da/d17/d40/d55/d2f/d1f/d4e 0 2024-04-24T23:14:03.394 INFO:tasks.workunit.client.1.smithi040.stdout:7/707: write db/d12/ff2 [814126,126376] 0 2024-04-24T23:14:03.394 INFO:tasks.workunit.client.1.smithi040.stdout:7/708: fdatasync db/d43/db9/dd4/fdf 0 2024-04-24T23:14:03.396 INFO:tasks.workunit.client.1.smithi040.stdout:7/709: chown db/d12/d7d/ff6 37 1 2024-04-24T23:14:03.396 INFO:tasks.workunit.client.0.smithi031.stdout:9/964: chown d2/f32 1161707 1 2024-04-24T23:14:03.400 INFO:tasks.workunit.client.0.smithi031.stdout:4/966: mknod d3/d20/d2d/d44/d5b/c13b 0 2024-04-24T23:14:03.401 INFO:tasks.workunit.client.1.smithi040.stdout:3/715: dread d1/d9/d15/f53 [0,4194304] 0 2024-04-24T23:14:03.402 INFO:tasks.workunit.client.1.smithi040.stdout:4/617: dwrite da/d3f/d6b/f21 [0,4194304] 0 2024-04-24T23:14:03.403 INFO:tasks.workunit.client.1.smithi040.stdout:7/710: dwrite db/d12/d20/d52/f70 [0,4194304] 0 2024-04-24T23:14:03.403 INFO:tasks.workunit.client.0.smithi031.stdout:4/967: readlink d3/d20/d25/lf0 0 2024-04-24T23:14:03.404 INFO:tasks.workunit.client.1.smithi040.stdout:8/710: write d0/d16/d19/f9a [703699,23995] 0 2024-04-24T23:14:03.405 INFO:tasks.workunit.client.1.smithi040.stdout:8/711: chown d0/d38/fe4 162216 1 2024-04-24T23:14:03.405 INFO:tasks.workunit.client.1.smithi040.stdout:4/618: write da/d3f/d40/d41/d62/f6f [198346,93268] 0 2024-04-24T23:14:03.406 INFO:tasks.workunit.client.1.smithi040.stdout:7/711: chown db/d12/d20/d52/d44/lb1 0 1 2024-04-24T23:14:03.406 INFO:tasks.workunit.client.1.smithi040.stdout:4/619: stat da/f18 0 2024-04-24T23:14:03.407 INFO:tasks.workunit.client.1.smithi040.stdout:8/712: dread - d0/d7e/dc1/f70 zero size 2024-04-24T23:14:03.407 INFO:tasks.workunit.client.0.smithi031.stdout:5/841: dwrite d0/d6/d3b/d4b/dc/d8b/fa6 [4194304,4194304] 0 2024-04-24T23:14:03.408 INFO:tasks.workunit.client.1.smithi040.stdout:8/713: chown d0/d7e/dc1 55608925 1 2024-04-24T23:14:03.408 INFO:tasks.workunit.client.0.smithi031.stdout:0/991: dwrite da/d20/d8e/f11f [0,4194304] 0 2024-04-24T23:14:03.410 INFO:tasks.workunit.client.1.smithi040.stdout:7/712: write db/d12/d62/fdb [1919808,28630] 0 2024-04-24T23:14:03.413 INFO:tasks.workunit.client.1.smithi040.stdout:6/634: rename d7/d1a/d20/d51/d2f/dba/c4c to d7/d1a/d20/d51/d1e/d46/d69/d87/cc9 0 2024-04-24T23:14:03.413 INFO:tasks.workunit.client.1.smithi040.stdout:2/723: rmdir da/d1c/d37/d39/d3f 39 2024-04-24T23:14:03.414 INFO:tasks.workunit.client.1.smithi040.stdout:7/713: truncate db/d12/d62/d8a/db3/fef 776279 0 2024-04-24T23:14:03.415 INFO:tasks.workunit.client.0.smithi031.stdout:6/941: unlink d13/d1b/d32/f12e 0 2024-04-24T23:14:03.416 INFO:tasks.workunit.client.1.smithi040.stdout:7/714: write db/d12/d20/d52/d4b/f4f [84250,114081] 0 2024-04-24T23:14:03.417 INFO:tasks.workunit.client.0.smithi031.stdout:3/795: creat d6/df/d16/d53/fef x:0 0 0 2024-04-24T23:14:03.418 INFO:tasks.workunit.client.0.smithi031.stdout:6/942: write f11 [4478755,9977] 0 2024-04-24T23:14:03.422 INFO:tasks.workunit.client.1.smithi040.stdout:8/714: creat d0/d16/d19/feb x:0 0 0 2024-04-24T23:14:03.423 INFO:tasks.workunit.client.0.smithi031.stdout:6/943: chown d13/d15/d1d/d23/d2d/d9e 1047922 1 2024-04-24T23:14:03.423 INFO:tasks.workunit.client.0.smithi031.stdout:7/902: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/f78 [0,4194304] 0 2024-04-24T23:14:03.426 INFO:tasks.workunit.client.0.smithi031.stdout:9/965: creat d2/dae/d14/d78/d84/d4a/dab/f127 x:0 0 0 2024-04-24T23:14:03.427 INFO:tasks.workunit.client.1.smithi040.stdout:1/670: sync 2024-04-24T23:14:03.434 INFO:tasks.workunit.client.0.smithi031.stdout:4/968: creat d3/d23/d69/dce/f13c x:0 0 0 2024-04-24T23:14:03.435 INFO:tasks.workunit.client.1.smithi040.stdout:5/617: getdents d2/d62 0 2024-04-24T23:14:03.437 INFO:tasks.workunit.client.0.smithi031.stdout:2/791: write d5/d6/d68/f6b [1689596,33194] 0 2024-04-24T23:14:03.440 INFO:tasks.workunit.client.0.smithi031.stdout:5/842: symlink d0/d6/d32/l11c 0 2024-04-24T23:14:03.441 INFO:tasks.workunit.client.1.smithi040.stdout:3/716: dread d1/d2d/de/f2f [0,4194304] 0 2024-04-24T23:14:03.444 INFO:tasks.workunit.client.1.smithi040.stdout:7/715: creat db/d12/d20/d52/d4b/f104 x:0 0 0 2024-04-24T23:14:03.445 INFO:tasks.workunit.client.0.smithi031.stdout:0/992: rename da/d11/d6a/de8 to da/d18/d5a/d147 0 2024-04-24T23:14:03.448 INFO:tasks.workunit.client.1.smithi040.stdout:1/671: fdatasync da/d35/d38/d47/f80 0 2024-04-24T23:14:03.450 INFO:tasks.workunit.client.0.smithi031.stdout:3/796: symlink d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/lf0 0 2024-04-24T23:14:03.452 INFO:tasks.workunit.client.0.smithi031.stdout:5/843: dwrite d0/d6/d3b/d4b/dc/d86/dfd/f102 [0,4194304] 0 2024-04-24T23:14:03.452 INFO:tasks.workunit.client.0.smithi031.stdout:6/944: truncate f9 473249 0 2024-04-24T23:14:03.455 INFO:tasks.workunit.client.0.smithi031.stdout:9/966: chown d2/c55 33629423 1 2024-04-24T23:14:03.458 INFO:tasks.workunit.client.1.smithi040.stdout:6/635: dread d7/d1a/d20/f47 [0,4194304] 0 2024-04-24T23:14:03.460 INFO:tasks.workunit.client.0.smithi031.stdout:9/967: chown d2/dae/d14/d78/d84/d4a/d4b/d96/fa8 137187077 1 2024-04-24T23:14:03.461 INFO:tasks.workunit.client.0.smithi031.stdout:4/969: read - d3/d20/d25/d60/d63/fe3 zero size 2024-04-24T23:14:03.464 INFO:tasks.workunit.client.0.smithi031.stdout:1/844: creat d1/da/d12/d65/dcf/d11/d21/f118 x:0 0 0 2024-04-24T23:14:03.464 INFO:tasks.workunit.client.0.smithi031.stdout:4/970: chown d3/d23/d69/d3f/d9e/ld8 21 1 2024-04-24T23:14:03.464 INFO:tasks.workunit.client.1.smithi040.stdout:3/717: rmdir d1/d9/d18/d36/d61 39 2024-04-24T23:14:03.464 INFO:tasks.workunit.client.1.smithi040.stdout:2/724: stat da/d1c/d37/d39/d3f/l45 0 2024-04-24T23:14:03.464 INFO:tasks.workunit.client.1.smithi040.stdout:7/716: symlink db/d12/d20/d52/d6c/d8f/l105 0 2024-04-24T23:14:03.465 INFO:tasks.workunit.client.1.smithi040.stdout:3/718: chown d1/d2a/d6a/f35 65147 1 2024-04-24T23:14:03.470 INFO:tasks.workunit.client.1.smithi040.stdout:4/620: link da/d3f/d40/d47/c5b da/d3f/d40/d47/d75/d7c/cda 0 2024-04-24T23:14:03.473 INFO:tasks.workunit.client.0.smithi031.stdout:0/993: creat da/d18/f148 x:0 0 0 2024-04-24T23:14:03.475 INFO:tasks.workunit.client.0.smithi031.stdout:0/994: write da/d22/f96 [1830048,94806] 0 2024-04-24T23:14:03.476 INFO:tasks.workunit.client.1.smithi040.stdout:0/742: rename df/d1e/d20/d21/d51/d79/ld8 to df/d12/d49/lf4 0 2024-04-24T23:14:03.479 INFO:tasks.workunit.client.0.smithi031.stdout:5/844: mknod d0/d7/d56/d7c/d7e/c11d 0 2024-04-24T23:14:03.480 INFO:tasks.workunit.client.0.smithi031.stdout:5/845: dread - d0/d7/ff6 zero size 2024-04-24T23:14:03.481 INFO:tasks.workunit.client.0.smithi031.stdout:6/945: rmdir d13/d15/d1d/d23/d99 39 2024-04-24T23:14:03.482 INFO:tasks.workunit.client.0.smithi031.stdout:6/946: chown d13/d15/d1d/d23/d24/d43/f71 5751 1 2024-04-24T23:14:03.484 INFO:tasks.workunit.client.0.smithi031.stdout:6/947: write d13/d15/d1d/f130 [135815,113230] 0 2024-04-24T23:14:03.486 INFO:tasks.workunit.client.0.smithi031.stdout:6/948: chown d13/d15/d1d/d23/d24/d43/d68/f139 3700325 1 2024-04-24T23:14:03.492 INFO:tasks.workunit.client.1.smithi040.stdout:1/672: dread da/d35/d38/d47/fc1 [0,4194304] 0 2024-04-24T23:14:03.497 INFO:tasks.workunit.client.0.smithi031.stdout:1/845: truncate d1/d33/d36/f8f 475015 0 2024-04-24T23:14:03.498 INFO:tasks.workunit.client.1.smithi040.stdout:7/717: dread db/d12/d20/d52/d6c/d8f/db8/fc8 [0,4194304] 0 2024-04-24T23:14:03.499 INFO:tasks.workunit.client.0.smithi031.stdout:7/903: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/da0/f5f [0,4194304] 0 2024-04-24T23:14:03.509 INFO:tasks.workunit.client.0.smithi031.stdout:0/995: unlink da/d20/c98 0 2024-04-24T23:14:03.514 INFO:tasks.workunit.client.0.smithi031.stdout:3/797: mkdir d6/df/d16/d2a/d95/d9b/df1 0 2024-04-24T23:14:03.517 INFO:tasks.workunit.client.0.smithi031.stdout:6/949: mknod d13/d15/d1d/d23/d4d/d7b/d11b/c13e 0 2024-04-24T23:14:03.521 INFO:tasks.workunit.client.0.smithi031.stdout:1/846: creat d1/d3e/da3/d69/f119 x:0 0 0 2024-04-24T23:14:03.523 INFO:tasks.workunit.client.0.smithi031.stdout:2/792: link d5/d99/ld3 d5/d6/d29/d4f/l115 0 2024-04-24T23:14:03.527 INFO:tasks.workunit.client.0.smithi031.stdout:7/904: mknod d1/d2/dd/d33/d5e/d64/d6d/dae/c137 0 2024-04-24T23:14:03.529 INFO:tasks.workunit.client.0.smithi031.stdout:9/968: write d2/dae/d14/d1d/d45/f5e [1001840,30961] 0 2024-04-24T23:14:03.532 INFO:tasks.workunit.client.0.smithi031.stdout:5/846: getdents d0/d117 0 2024-04-24T23:14:03.533 INFO:tasks.workunit.client.0.smithi031.stdout:5/847: readlink d0/d6/d3b/d4b/d1d/l37 0 2024-04-24T23:14:03.535 INFO:tasks.workunit.client.0.smithi031.stdout:3/798: fsync d6/d14/f33 0 2024-04-24T23:14:03.539 INFO:tasks.workunit.client.0.smithi031.stdout:1/847: read d1/d3e/d46/dbd/fdd [59712,6027] 0 2024-04-24T23:14:03.541 INFO:tasks.workunit.client.0.smithi031.stdout:2/793: truncate d5/d28/d4e/fd6 4525202 0 2024-04-24T23:14:03.545 INFO:tasks.workunit.client.0.smithi031.stdout:7/905: dread d1/d2/dd/d33/f4e [0,4194304] 0 2024-04-24T23:14:03.549 INFO:tasks.workunit.client.0.smithi031.stdout:0/996: mknod da/d22/c149 0 2024-04-24T23:14:03.552 INFO:tasks.workunit.client.0.smithi031.stdout:9/969: mknod d2/d43/d9d/c128 0 2024-04-24T23:14:03.556 INFO:tasks.workunit.client.0.smithi031.stdout:1/848: dread d1/da/d12/d65/dcf/fe [0,4194304] 0 2024-04-24T23:14:03.557 INFO:tasks.workunit.client.1.smithi040.stdout:8/715: rmdir d0/d16/d19/da2/dd8 0 2024-04-24T23:14:03.559 INFO:tasks.workunit.client.0.smithi031.stdout:3/799: creat d6/d55/ff2 x:0 0 0 2024-04-24T23:14:03.567 INFO:tasks.workunit.client.1.smithi040.stdout:9/697: rename d7/da/d17/d40/d55/d2f/lf6 to d7/da/d17/d40/d55/d2f/df7/d6c/lfb 0 2024-04-24T23:14:03.573 INFO:tasks.workunit.client.0.smithi031.stdout:4/971: getdents d3/d23/d37/d86 0 2024-04-24T23:14:03.574 INFO:tasks.workunit.client.1.smithi040.stdout:1/673: truncate da/d12/d18/d2a/d4d/f75 2374123 0 2024-04-24T23:14:03.578 INFO:tasks.workunit.client.1.smithi040.stdout:7/718: write db/d12/d3f/d7a/fe1 [182611,102578] 0 2024-04-24T23:14:03.580 INFO:tasks.workunit.client.1.smithi040.stdout:8/716: rename d0/d8/d22/f62 to d0/d8/d22/fec 0 2024-04-24T23:14:03.581 INFO:tasks.workunit.client.1.smithi040.stdout:8/717: dread - d0/d28/db7/fbf zero size 2024-04-24T23:14:03.581 INFO:tasks.workunit.client.1.smithi040.stdout:8/718: dread - d0/d16/d1b/d47/fdc zero size 2024-04-24T23:14:03.582 INFO:tasks.workunit.client.0.smithi031.stdout:0/997: mkdir da/d20/d4d/d14a 0 2024-04-24T23:14:03.593 INFO:tasks.workunit.client.1.smithi040.stdout:4/621: creat da/d3f/d40/d41/d8e/dd3/fdb x:0 0 0 2024-04-24T23:14:03.594 INFO:tasks.workunit.client.0.smithi031.stdout:2/794: dwrite d5/d8/d15/f93 [0,4194304] 0 2024-04-24T23:14:03.596 INFO:tasks.workunit.client.1.smithi040.stdout:5/618: rename d2/d7/d8/d1f/d57/d3e/d48 to d2/d7/d8/d1f/d90/dcb 0 2024-04-24T23:14:03.596 INFO:tasks.workunit.client.1.smithi040.stdout:3/719: truncate d1/d9/d2e/f84 1246757 0 2024-04-24T23:14:03.600 INFO:tasks.workunit.client.0.smithi031.stdout:6/950: creat d13/f13f x:0 0 0 2024-04-24T23:14:03.603 INFO:tasks.workunit.client.1.smithi040.stdout:0/743: unlink df/d1e/d20/f43 0 2024-04-24T23:14:03.603 INFO:tasks.workunit.client.1.smithi040.stdout:0/744: write df/d12/f4c [1451665,107163] 0 2024-04-24T23:14:03.606 INFO:tasks.workunit.client.1.smithi040.stdout:7/719: creat db/d43/db9/dd4/f106 x:0 0 0 2024-04-24T23:14:03.606 INFO:tasks.workunit.client.0.smithi031.stdout:1/849: dwrite d1/d3e/da3/d69/f95 [0,4194304] 0 2024-04-24T23:14:03.606 INFO:tasks.workunit.client.1.smithi040.stdout:7/720: fdatasync db/d12/d20/d52/d6c/fc4 0 2024-04-24T23:14:03.607 INFO:tasks.workunit.client.0.smithi031.stdout:1/850: dread - d1/da/df0/fb7 zero size 2024-04-24T23:14:03.611 INFO:tasks.workunit.client.1.smithi040.stdout:6/636: link d7/d1a/d20/d51/d1e/f3d d7/fca 0 2024-04-24T23:14:03.612 INFO:tasks.workunit.client.1.smithi040.stdout:6/637: chown d7/d1a/d56/l4f 2292537 1 2024-04-24T23:14:03.617 INFO:tasks.workunit.client.0.smithi031.stdout:3/800: dwrite d6/df/d16/d57/de1/fe5 [0,4194304] 0 2024-04-24T23:14:03.619 INFO:tasks.workunit.client.1.smithi040.stdout:2/725: rename da/d10/f2b to da/d1c/d37/d39/d77/f10a 0 2024-04-24T23:14:03.625 INFO:tasks.workunit.client.0.smithi031.stdout:9/970: symlink d2/dae/d14/d1d/d45/d65/d117/l129 0 2024-04-24T23:14:03.625 INFO:tasks.workunit.client.1.smithi040.stdout:3/720: stat d1/d2d/de/d30/cf2 0 2024-04-24T23:14:03.630 INFO:tasks.workunit.client.0.smithi031.stdout:2/795: chown d5/d8/d45/d5b/d49/l47 91 1 2024-04-24T23:14:03.634 INFO:tasks.workunit.client.0.smithi031.stdout:6/951: mknod d13/d15/d1d/d23/d44/da0/da6/de6/c140 0 2024-04-24T23:14:03.641 INFO:tasks.workunit.client.1.smithi040.stdout:7/721: creat db/d2f/da8/f107 x:0 0 0 2024-04-24T23:14:03.642 INFO:tasks.workunit.client.0.smithi031.stdout:4/972: mknod d3/d23/d37/d86/d113/d12a/d105/d138/d13a/c13d 0 2024-04-24T23:14:03.644 INFO:tasks.workunit.client.0.smithi031.stdout:0/998: write da/d18/d5a/d7d/f116 [5192465,61590] 0 2024-04-24T23:14:03.645 INFO:tasks.workunit.client.0.smithi031.stdout:0/999: stat da/d18/d5a/d7d/ff5 0 2024-04-24T23:14:03.647 INFO:tasks.workunit.client.0.smithi031.stdout:7/906: link d1/d2/dd/d33/d5e/d64/le6 d1/de4/d106/l138 0 2024-04-24T23:14:03.649 INFO:tasks.workunit.client.1.smithi040.stdout:5/619: write d2/d7/d8/d1f/d57/d27/d65/f5a [5424458,97710] 0 2024-04-24T23:14:03.652 INFO:tasks.workunit.client.1.smithi040.stdout:1/674: sync 2024-04-24T23:14:03.652 INFO:tasks.workunit.client.1.smithi040.stdout:9/698: truncate d7/d5e/da1/fd0 278292 0 2024-04-24T23:14:03.653 INFO:tasks.workunit.client.1.smithi040.stdout:9/699: stat d7/d5e 0 2024-04-24T23:14:03.657 INFO:tasks.workunit.client.1.smithi040.stdout:3/721: creat d1/d9/d15/d45/daa/ff5 x:0 0 0 2024-04-24T23:14:03.657 INFO:tasks.workunit.client.0.smithi031.stdout:5/848: link d0/d7/d56/lfa d0/d7/d74/l11e 0 2024-04-24T23:14:03.660 INFO:tasks.workunit.client.1.smithi040.stdout:4/622: truncate da/d3f/d40/d41/d60/f42 606633 0 2024-04-24T23:14:03.660 INFO:tasks.workunit.client.1.smithi040.stdout:0/745: getdents df/d1e/d20/d21/d51/ddb 0 2024-04-24T23:14:03.660 INFO:tasks.workunit.client.1.smithi040.stdout:4/623: chown da/d3f/d6b/ccc 338 1 2024-04-24T23:14:03.662 INFO:tasks.workunit.client.0.smithi031.stdout:1/851: dwrite d1/da/d12/d7e/d7b/fb8 [0,4194304] 0 2024-04-24T23:14:03.664 INFO:tasks.workunit.client.1.smithi040.stdout:6/638: dread - d7/de/d89/f90 zero size 2024-04-24T23:14:03.666 INFO:tasks.workunit.client.0.smithi031.stdout:4/973: mkdir d3/d23/d69/d3f/d123/d13e 0 2024-04-24T23:14:03.667 INFO:tasks.workunit.client.1.smithi040.stdout:4/624: dwrite da/d1f/f9c [0,4194304] 0 2024-04-24T23:14:03.668 INFO:tasks.workunit.client.1.smithi040.stdout:8/719: rename d0/d8/d66/lad to d0/d38/d4c/led 0 2024-04-24T23:14:03.670 INFO:tasks.workunit.client.0.smithi031.stdout:7/907: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/f139 x:0 0 0 2024-04-24T23:14:03.674 INFO:tasks.workunit.client.1.smithi040.stdout:5/620: fdatasync d2/f9e 0 2024-04-24T23:14:03.676 INFO:tasks.workunit.client.1.smithi040.stdout:5/621: chown d2/d7/d8/db/d1e/l60 75048 1 2024-04-24T23:14:03.679 INFO:tasks.workunit.client.0.smithi031.stdout:6/952: write d13/d15/d1d/d23/d44/d47/fd6 [262899,15693] 0 2024-04-24T23:14:03.683 INFO:tasks.workunit.client.1.smithi040.stdout:2/726: mknod da/d10/d15/c10b 0 2024-04-24T23:14:03.684 INFO:tasks.workunit.client.0.smithi031.stdout:9/971: creat d2/dae/d14/d78/d84/d4a/dab/dad/db5/dfa/f12a x:0 0 0 2024-04-24T23:14:03.684 INFO:tasks.workunit.client.0.smithi031.stdout:5/849: read d0/f93 [1395021,88434] 0 2024-04-24T23:14:03.685 INFO:tasks.workunit.client.1.smithi040.stdout:2/727: dread - da/d29/d55/da7/fe3 zero size 2024-04-24T23:14:03.687 INFO:tasks.workunit.client.1.smithi040.stdout:3/722: write d1/d9/fe2 [1091110,41520] 0 2024-04-24T23:14:03.687 INFO:tasks.workunit.client.1.smithi040.stdout:7/722: dwrite db/d12/d62/d8a/db3/fed [0,4194304] 0 2024-04-24T23:14:03.689 INFO:tasks.workunit.client.1.smithi040.stdout:3/723: readlink d1/d9/d18/d36/d55/laf 0 2024-04-24T23:14:03.689 INFO:tasks.workunit.client.1.smithi040.stdout:7/723: stat db/d43/db9/dd4/fdf 0 2024-04-24T23:14:03.691 INFO:tasks.workunit.client.0.smithi031.stdout:1/852: creat d1/d3e/da2/f11a x:0 0 0 2024-04-24T23:14:03.692 INFO:tasks.workunit.client.1.smithi040.stdout:7/724: write db/d12/d20/d56/fd6 [3768098,29892] 0 2024-04-24T23:14:03.694 INFO:tasks.workunit.client.1.smithi040.stdout:6/639: truncate d7/d1a/f53 9795 0 2024-04-24T23:14:03.695 INFO:tasks.workunit.client.1.smithi040.stdout:7/725: chown db/d12/d20/d56/lec 4 1 2024-04-24T23:14:03.697 INFO:tasks.workunit.client.1.smithi040.stdout:6/640: write d7/fae [744143,122556] 0 2024-04-24T23:14:03.697 INFO:tasks.workunit.client.1.smithi040.stdout:5/622: dread d2/d7/d8/db/d1e/f28 [0,4194304] 0 2024-04-24T23:14:03.700 INFO:tasks.workunit.client.1.smithi040.stdout:3/724: dwrite d1/d2a/d6a/fe0 [0,4194304] 0 2024-04-24T23:14:03.702 INFO:tasks.workunit.client.0.smithi031.stdout:6/953: truncate d13/d1b/f6b 1700876 0 2024-04-24T23:14:03.703 INFO:tasks.workunit.client.1.smithi040.stdout:1/675: write da/d12/d14/d25/d5e/fa4 [4639150,50811] 0 2024-04-24T23:14:03.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/972: rmdir d2/d43/d6e 39 2024-04-24T23:14:03.712 INFO:tasks.workunit.client.1.smithi040.stdout:9/700: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/def/cfc 0 2024-04-24T23:14:03.713 INFO:tasks.workunit.client.0.smithi031.stdout:2/796: link d5/d8/d15/d56/d84/fa5 d5/d6/d29/d26/db0/dc2/dad/dbe/f116 0 2024-04-24T23:14:03.714 INFO:tasks.workunit.client.0.smithi031.stdout:2/797: write d5/d8/f98 [351166,78942] 0 2024-04-24T23:14:03.716 INFO:tasks.workunit.client.0.smithi031.stdout:5/850: truncate d0/d6/f40 1198559 0 2024-04-24T23:14:03.717 INFO:tasks.workunit.client.0.smithi031.stdout:5/851: fsync d0/d6/d3b/d4b/dc/f9a 0 2024-04-24T23:14:03.722 INFO:tasks.workunit.client.0.smithi031.stdout:7/908: write d1/d2/dd/d33/f62 [1826606,116878] 0 2024-04-24T23:14:03.723 INFO:tasks.workunit.client.1.smithi040.stdout:8/720: write d0/d16/d19/fb5 [828440,11259] 0 2024-04-24T23:14:03.725 INFO:tasks.workunit.client.1.smithi040.stdout:8/721: write d0/d16/d1b/f95 [1350478,97147] 0 2024-04-24T23:14:03.725 INFO:tasks.workunit.client.0.smithi031.stdout:4/974: dwrite d3/d20/d25/d60/d63/fbf [0,4194304] 0 2024-04-24T23:14:03.730 INFO:tasks.workunit.client.0.smithi031.stdout:3/801: getdents d6/df/d16/d2a/d95/d9b/d1a 0 2024-04-24T23:14:03.732 INFO:tasks.workunit.client.1.smithi040.stdout:6/641: mknod d7/d1a/d20/d8c/ccb 0 2024-04-24T23:14:03.733 INFO:tasks.workunit.client.0.smithi031.stdout:6/954: creat d13/d1b/dd0/f141 x:0 0 0 2024-04-24T23:14:03.737 INFO:tasks.workunit.client.1.smithi040.stdout:5/623: fdatasync d2/d7/d8/d1f/d31/f4e 0 2024-04-24T23:14:03.739 INFO:tasks.workunit.client.0.smithi031.stdout:9/973: chown d2/dae/f13 39079224 1 2024-04-24T23:14:03.743 INFO:tasks.workunit.client.1.smithi040.stdout:5/624: dwrite d2/d2f/d14/f17 [0,4194304] 0 2024-04-24T23:14:03.743 INFO:tasks.workunit.client.0.smithi031.stdout:5/852: dread d0/d7/d56/d7c/d7e/fb8 [0,4194304] 0 2024-04-24T23:14:03.743 INFO:tasks.workunit.client.0.smithi031.stdout:2/798: mkdir d5/d6/d29/d26/db0/dc7/d117 0 2024-04-24T23:14:03.744 INFO:tasks.workunit.client.1.smithi040.stdout:5/625: stat d2/d7/d8/db 0 2024-04-24T23:14:03.746 INFO:tasks.workunit.client.1.smithi040.stdout:4/625: fdatasync da/d3f/d40/d47/d71/fb3 0 2024-04-24T23:14:03.752 INFO:tasks.workunit.client.1.smithi040.stdout:1/676: mkdir da/d35/dec 0 2024-04-24T23:14:03.753 INFO:tasks.workunit.client.1.smithi040.stdout:1/677: truncate da/d35/d38/d47/f84 1604285 0 2024-04-24T23:14:03.754 INFO:tasks.workunit.client.1.smithi040.stdout:9/701: creat d7/d92/d97/ffd x:0 0 0 2024-04-24T23:14:03.756 INFO:tasks.workunit.client.0.smithi031.stdout:4/975: mknod d3/d23/d3b/df2/c13f 0 2024-04-24T23:14:03.757 INFO:tasks.workunit.client.1.smithi040.stdout:2/728: mkdir da/d1c/d37/d39/d77/d108/d10c 0 2024-04-24T23:14:03.758 INFO:tasks.workunit.client.1.smithi040.stdout:7/726: sync 2024-04-24T23:14:03.760 INFO:tasks.workunit.client.0.smithi031.stdout:7/909: rename d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f12f to d1/de4/d100/f13a 0 2024-04-24T23:14:03.762 INFO:tasks.workunit.client.1.smithi040.stdout:3/725: write d1/d9/d15/f1b [8494698,86225] 0 2024-04-24T23:14:03.762 INFO:tasks.workunit.client.1.smithi040.stdout:3/726: stat d1/d2a/d6a/f8c 0 2024-04-24T23:14:03.763 INFO:tasks.workunit.client.0.smithi031.stdout:7/910: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/d126/la7 0 2024-04-24T23:14:03.766 INFO:tasks.workunit.client.1.smithi040.stdout:8/722: readlink d0/d8/dc/d34/lb6 0 2024-04-24T23:14:03.766 INFO:tasks.workunit.client.1.smithi040.stdout:8/723: stat d0/d16/d1b/d58/fab 0 2024-04-24T23:14:03.767 INFO:tasks.workunit.client.0.smithi031.stdout:6/955: mkdir d13/d15/d1d/d23/d44/da0/da6/de6/d142 0 2024-04-24T23:14:03.768 INFO:tasks.workunit.client.1.smithi040.stdout:6/642: fdatasync d7/d1a/d20/d51/d1e/d5d/f93 0 2024-04-24T23:14:03.772 INFO:tasks.workunit.client.0.smithi031.stdout:1/853: dwrite d1/d3e/d46/f5c [0,4194304] 0 2024-04-24T23:14:03.773 INFO:tasks.workunit.client.0.smithi031.stdout:1/854: chown d1/da 49101 1 2024-04-24T23:14:03.775 INFO:tasks.workunit.client.0.smithi031.stdout:5/853: fsync d0/d6/d3b/d4b/d1d/d2e/f54 0 2024-04-24T23:14:03.781 INFO:tasks.workunit.client.1.smithi040.stdout:5/626: mkdir d2/d7/d8/db/da2/dcc 0 2024-04-24T23:14:03.785 INFO:tasks.workunit.client.1.smithi040.stdout:4/626: mknod da/d3f/d6b/d3c/cdc 0 2024-04-24T23:14:03.785 INFO:tasks.workunit.client.0.smithi031.stdout:3/802: dwrite d6/d55/fd0 [0,4194304] 0 2024-04-24T23:14:03.785 INFO:tasks.workunit.client.0.smithi031.stdout:9/974: write d2/dae/d14/f88 [307046,107549] 0 2024-04-24T23:14:03.788 INFO:tasks.workunit.client.1.smithi040.stdout:1/678: mkdir da/d12/d14/d58/ded 0 2024-04-24T23:14:03.791 INFO:tasks.workunit.client.1.smithi040.stdout:9/702: creat d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/ffe x:0 0 0 2024-04-24T23:14:03.792 INFO:tasks.workunit.client.0.smithi031.stdout:7/911: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/d126/d13b 0 2024-04-24T23:14:03.798 INFO:tasks.workunit.client.1.smithi040.stdout:0/746: getdents df/d1e/d20/d21/d41/d8a/df3 0 2024-04-24T23:14:03.799 INFO:tasks.workunit.client.1.smithi040.stdout:3/727: mknod d1/d9/d15/cf6 0 2024-04-24T23:14:03.800 INFO:tasks.workunit.client.1.smithi040.stdout:3/728: write d1/d9/d15/d45/d96/fef [448308,120733] 0 2024-04-24T23:14:03.801 INFO:tasks.workunit.client.1.smithi040.stdout:8/724: creat d0/d16/d19/d71/d76/fee x:0 0 0 2024-04-24T23:14:03.803 INFO:tasks.workunit.client.0.smithi031.stdout:5/854: creat d0/d7/d74/dbd/d112/f11f x:0 0 0 2024-04-24T23:14:03.804 INFO:tasks.workunit.client.0.smithi031.stdout:5/855: write d0/d6/f115 [902634,25649] 0 2024-04-24T23:14:03.804 INFO:tasks.workunit.client.1.smithi040.stdout:6/643: creat d7/d1a/d20/d51/d1e/d46/d69/d87/fcc x:0 0 0 2024-04-24T23:14:03.805 INFO:tasks.workunit.client.1.smithi040.stdout:6/644: chown d7 949461 1 2024-04-24T23:14:03.807 INFO:tasks.workunit.client.0.smithi031.stdout:2/799: mkdir d5/d8/d45/d5b/d49/dc4/def/d114/d118 0 2024-04-24T23:14:03.812 INFO:tasks.workunit.client.1.smithi040.stdout:5/627: symlink d2/d62/lcd 0 2024-04-24T23:14:03.812 INFO:tasks.workunit.client.0.smithi031.stdout:4/976: truncate d3/d23/d37/f7b 789168 0 2024-04-24T23:14:03.813 INFO:tasks.workunit.client.0.smithi031.stdout:6/956: write d13/d1b/f46 [669754,71627] 0 2024-04-24T23:14:03.815 INFO:tasks.workunit.client.0.smithi031.stdout:9/975: mkdir d2/d43/d9d/dc0/d12b 0 2024-04-24T23:14:03.817 INFO:tasks.workunit.client.1.smithi040.stdout:1/679: creat da/d12/d14/d25/fee x:0 0 0 2024-04-24T23:14:03.818 INFO:tasks.workunit.client.1.smithi040.stdout:7/727: dwrite db/d12/f16 [0,4194304] 0 2024-04-24T23:14:03.819 INFO:tasks.workunit.client.0.smithi031.stdout:3/803: mknod d6/d14/d1d/d59/cf3 0 2024-04-24T23:14:03.819 INFO:tasks.workunit.client.0.smithi031.stdout:1/855: dread d1/da/d12/f23 [4194304,4194304] 0 2024-04-24T23:14:03.822 INFO:tasks.workunit.client.0.smithi031.stdout:7/912: creat d1/d2/dd/d33/d5e/d64/f13c x:0 0 0 2024-04-24T23:14:03.823 INFO:tasks.workunit.client.1.smithi040.stdout:7/728: dread db/d12/f16 [0,4194304] 0 2024-04-24T23:14:03.824 INFO:tasks.workunit.client.1.smithi040.stdout:9/703: dwrite d7/da/d17/d40/d55/d2f/d28/d67/faf [0,4194304] 0 2024-04-24T23:14:03.829 INFO:tasks.workunit.client.0.smithi031.stdout:1/856: dwrite d1/d3e/d46/d74/f10c [0,4194304] 0 2024-04-24T23:14:03.829 INFO:tasks.workunit.client.0.smithi031.stdout:2/800: mkdir d5/d8/d45/d5b/d49/d6f/d119 0 2024-04-24T23:14:03.833 INFO:tasks.workunit.client.1.smithi040.stdout:0/747: symlink df/d1e/d20/d21/d51/d79/daa/lf5 0 2024-04-24T23:14:03.834 INFO:tasks.workunit.client.1.smithi040.stdout:3/729: mknod d1/d2d/d2b/cf7 0 2024-04-24T23:14:03.837 INFO:tasks.workunit.client.1.smithi040.stdout:3/730: truncate d1/d9/d15/d45/d96/fef 1215713 0 2024-04-24T23:14:03.839 INFO:tasks.workunit.client.1.smithi040.stdout:8/725: mkdir d0/d16/d1b/d47/def 0 2024-04-24T23:14:03.841 INFO:tasks.workunit.client.1.smithi040.stdout:6/645: mkdir d7/d1a/d20/d51/d1e/dcd 0 2024-04-24T23:14:03.843 INFO:tasks.workunit.client.0.smithi031.stdout:3/804: symlink d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/lf4 0 2024-04-24T23:14:03.844 INFO:tasks.workunit.client.1.smithi040.stdout:5/628: fsync d2/d7/d8/d1f/da1/fa8 0 2024-04-24T23:14:03.846 INFO:tasks.workunit.client.1.smithi040.stdout:1/680: creat da/d12/d14/d25/db9/fef x:0 0 0 2024-04-24T23:14:03.848 INFO:tasks.workunit.client.0.smithi031.stdout:7/913: read d1/d2/f46 [1973288,91066] 0 2024-04-24T23:14:03.849 INFO:tasks.workunit.client.0.smithi031.stdout:2/801: read - d5/d6/d29/d26/fa8 zero size 2024-04-24T23:14:03.851 INFO:tasks.workunit.client.0.smithi031.stdout:4/977: creat d3/d23/d69/d3f/d9e/d12b/f140 x:0 0 0 2024-04-24T23:14:03.852 INFO:tasks.workunit.client.1.smithi040.stdout:4/627: sync 2024-04-24T23:14:03.854 INFO:tasks.workunit.client.1.smithi040.stdout:2/729: rmdir da/d1c/d37/d39/d3f/d52/d78/dc7/ddc 0 2024-04-24T23:14:03.855 INFO:tasks.workunit.client.1.smithi040.stdout:2/730: stat da/d1c/d37/d39/d79/fe0 0 2024-04-24T23:14:03.855 INFO:tasks.workunit.client.1.smithi040.stdout:2/731: fsync da/d81/de6/ff2 0 2024-04-24T23:14:03.856 INFO:tasks.workunit.client.1.smithi040.stdout:0/748: creat df/d1e/da7/dd0/ff6 x:0 0 0 2024-04-24T23:14:03.856 INFO:tasks.workunit.client.1.smithi040.stdout:2/732: chown da/f106 1570 1 2024-04-24T23:14:03.857 INFO:tasks.workunit.client.1.smithi040.stdout:2/733: write da/f19 [9265877,42551] 0 2024-04-24T23:14:03.858 INFO:tasks.workunit.client.1.smithi040.stdout:3/731: read - d1/d9/d18/f66 zero size 2024-04-24T23:14:03.858 INFO:tasks.workunit.client.0.smithi031.stdout:7/914: dwrite d1/d2/dd/d33/d5e/d64/d6d/dae/db1/f109 [0,4194304] 0 2024-04-24T23:14:03.870 INFO:tasks.workunit.client.0.smithi031.stdout:6/957: write d13/d1b/d32/db4/fc6 [271746,33513] 0 2024-04-24T23:14:03.872 INFO:tasks.workunit.client.0.smithi031.stdout:9/976: write d2/d16/dbe/d7d/fac [1265876,94478] 0 2024-04-24T23:14:03.874 INFO:tasks.workunit.client.0.smithi031.stdout:2/802: symlink d5/d8/d45/d5b/d49/dc4/d104/l11a 0 2024-04-24T23:14:03.875 INFO:tasks.workunit.client.1.smithi040.stdout:7/729: mkdir db/d12/d62/d103/d108 0 2024-04-24T23:14:03.876 INFO:tasks.workunit.client.0.smithi031.stdout:3/805: dread d6/d14/d1d/d59/f6c [0,4194304] 0 2024-04-24T23:14:03.877 INFO:tasks.workunit.client.1.smithi040.stdout:4/628: rmdir da/d3f/d6b/d2b 39 2024-04-24T23:14:03.879 INFO:tasks.workunit.client.1.smithi040.stdout:9/704: mkdir d7/da/d17/d40/d55/d2f/d1f/d31/dff 0 2024-04-24T23:14:03.881 INFO:tasks.workunit.client.1.smithi040.stdout:0/749: fdatasync df/d12/f22 0 2024-04-24T23:14:03.883 INFO:tasks.workunit.client.1.smithi040.stdout:4/629: dwrite da/d3f/d40/d47/d75/daa/fc8 [0,4194304] 0 2024-04-24T23:14:03.885 INFO:tasks.workunit.client.1.smithi040.stdout:4/630: fdatasync da/d3f/d6b/d3c/d69/fa1 0 2024-04-24T23:14:03.886 INFO:tasks.workunit.client.0.smithi031.stdout:1/857: dwrite d1/da/d12/d7e/d7b/d4e/fd7 [0,4194304] 0 2024-04-24T23:14:03.887 INFO:tasks.workunit.client.0.smithi031.stdout:9/977: creat d2/dae/d14/d78/d84/d4a/dab/dad/f12c x:0 0 0 2024-04-24T23:14:03.889 INFO:tasks.workunit.client.0.smithi031.stdout:3/806: dread - d6/d14/d1d/d69/fc5 zero size 2024-04-24T23:14:03.889 INFO:tasks.workunit.client.1.smithi040.stdout:8/726: mknod d0/d8/d22/db8/dc2/cf0 0 2024-04-24T23:14:03.892 INFO:tasks.workunit.client.1.smithi040.stdout:2/734: rename da/d1c/d37/d39/d3f/cff to da/d1c/d37/d39/d3f/d52/d78/c10d 0 2024-04-24T23:14:03.895 INFO:tasks.workunit.client.0.smithi031.stdout:7/915: write d1/d2/dd/d33/d5e/d64/f99 [309299,26847] 0 2024-04-24T23:14:03.896 INFO:tasks.workunit.client.1.smithi040.stdout:6/646: mknod d7/cce 0 2024-04-24T23:14:03.899 INFO:tasks.workunit.client.1.smithi040.stdout:5/629: creat d2/d7/d8/d1f/d57/d27/d3c/da4/fce x:0 0 0 2024-04-24T23:14:03.902 INFO:tasks.workunit.client.0.smithi031.stdout:4/978: truncate d3/d5e/f8d 2521257 0 2024-04-24T23:14:03.904 INFO:tasks.workunit.client.0.smithi031.stdout:1/858: fdatasync d1/d31/f7c 0 2024-04-24T23:14:03.905 INFO:tasks.workunit.client.1.smithi040.stdout:1/681: truncate da/d12/d23/f30 556887 0 2024-04-24T23:14:03.907 INFO:tasks.workunit.client.0.smithi031.stdout:9/978: creat d2/dae/d14/d1d/f12d x:0 0 0 2024-04-24T23:14:03.909 INFO:tasks.workunit.client.1.smithi040.stdout:7/730: mkdir db/d12/d62/d109 0 2024-04-24T23:14:03.910 INFO:tasks.workunit.client.0.smithi031.stdout:3/807: symlink d6/d55/lf5 0 2024-04-24T23:14:03.910 INFO:tasks.workunit.client.1.smithi040.stdout:4/631: dread f9 [0,4194304] 0 2024-04-24T23:14:03.913 INFO:tasks.workunit.client.1.smithi040.stdout:4/632: dread f9 [0,4194304] 0 2024-04-24T23:14:03.913 INFO:tasks.workunit.client.0.smithi031.stdout:6/958: creat d13/d15/d1d/f143 x:0 0 0 2024-04-24T23:14:03.915 INFO:tasks.workunit.client.0.smithi031.stdout:2/803: creat d5/d6/f11b x:0 0 0 2024-04-24T23:14:03.916 INFO:tasks.workunit.client.0.smithi031.stdout:2/804: write d5/f5c [750386,23447] 0 2024-04-24T23:14:03.919 INFO:tasks.workunit.client.0.smithi031.stdout:7/916: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/c13d 0 2024-04-24T23:14:03.921 INFO:tasks.workunit.client.1.smithi040.stdout:0/750: truncate df/d12/d4b/fc6 959011 0 2024-04-24T23:14:03.929 INFO:tasks.workunit.client.1.smithi040.stdout:9/705: rename d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/cab to d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/c100 0 2024-04-24T23:14:03.931 INFO:tasks.workunit.client.0.smithi031.stdout:9/979: mkdir d2/d43/d9d/dbd/d12e 0 2024-04-24T23:14:03.933 INFO:tasks.workunit.client.0.smithi031.stdout:6/959: dread d13/d15/d1d/d23/d44/f80 [0,4194304] 0 2024-04-24T23:14:03.935 INFO:tasks.workunit.client.0.smithi031.stdout:3/808: creat d6/df/d16/d2a/d95/d9b/d29/db7/ff6 x:0 0 0 2024-04-24T23:14:03.937 INFO:tasks.workunit.client.1.smithi040.stdout:2/735: dwrite da/d10/d15/daa/fc3 [0,4194304] 0 2024-04-24T23:14:03.940 INFO:tasks.workunit.client.0.smithi031.stdout:3/809: dread d6/f48 [0,4194304] 0 2024-04-24T23:14:03.943 INFO:tasks.workunit.client.0.smithi031.stdout:2/805: symlink d5/d28/d4e/d5e/l11c 0 2024-04-24T23:14:03.944 INFO:tasks.workunit.client.1.smithi040.stdout:0/751: dread df/d1e/d20/d21/f9c [0,4194304] 0 2024-04-24T23:14:03.945 INFO:tasks.workunit.client.1.smithi040.stdout:2/736: dwrite da/d1c/d37/d39/d3f/f7b [0,4194304] 0 2024-04-24T23:14:03.948 INFO:tasks.workunit.client.0.smithi031.stdout:4/979: fsync d3/d20/d2d/d44/fe4 0 2024-04-24T23:14:03.951 INFO:tasks.workunit.client.1.smithi040.stdout:6/647: write d7/d1a/d20/d51/d1e/d46/d69/f8d [1171684,96571] 0 2024-04-24T23:14:03.951 INFO:tasks.workunit.client.1.smithi040.stdout:8/727: dwrite d0/d16/d1b/f40 [0,4194304] 0 2024-04-24T23:14:03.953 INFO:tasks.workunit.client.1.smithi040.stdout:9/706: dread d7/f3a [0,4194304] 0 2024-04-24T23:14:03.954 INFO:tasks.workunit.client.1.smithi040.stdout:9/707: chown d7/d92/ff4 10 1 2024-04-24T23:14:03.954 INFO:tasks.workunit.client.1.smithi040.stdout:6/648: stat d7/d1a/d20/d8c/db0 0 2024-04-24T23:14:03.955 INFO:tasks.workunit.client.1.smithi040.stdout:9/708: dread - d7/d5e/de5/ff3 zero size 2024-04-24T23:14:03.958 INFO:tasks.workunit.client.1.smithi040.stdout:3/732: creat d1/d9/d39/ff8 x:0 0 0 2024-04-24T23:14:03.963 INFO:tasks.workunit.client.0.smithi031.stdout:1/859: write d1/f18 [4756414,96836] 0 2024-04-24T23:14:03.965 INFO:tasks.workunit.client.0.smithi031.stdout:3/810: mknod d6/df/d16/d57/de1/cf7 0 2024-04-24T23:14:03.969 INFO:tasks.workunit.client.0.smithi031.stdout:7/917: mknod d1/c13e 0 2024-04-24T23:14:03.978 INFO:tasks.workunit.client.1.smithi040.stdout:4/633: dwrite da/d3f/d40/d41/f67 [0,4194304] 0 2024-04-24T23:14:03.979 INFO:tasks.workunit.client.1.smithi040.stdout:5/630: symlink d2/d7/d8/d64/lcf 0 2024-04-24T23:14:03.984 INFO:tasks.workunit.client.0.smithi031.stdout:3/811: mkdir d6/df/d16/d57/df8 0 2024-04-24T23:14:03.987 INFO:tasks.workunit.client.0.smithi031.stdout:6/960: dread d13/d15/d1d/d23/d44/d47/d69/fd4 [0,4194304] 0 2024-04-24T23:14:03.989 INFO:tasks.workunit.client.0.smithi031.stdout:2/806: dwrite d5/d8/f6c [0,4194304] 0 2024-04-24T23:14:03.989 INFO:tasks.workunit.client.0.smithi031.stdout:4/980: write d3/f4a [1757977,117220] 0 2024-04-24T23:14:03.991 INFO:tasks.workunit.client.1.smithi040.stdout:2/737: dread - da/d81/fbc zero size 2024-04-24T23:14:03.993 INFO:tasks.workunit.client.0.smithi031.stdout:9/980: write d2/f54 [2651327,6042] 0 2024-04-24T23:14:03.999 INFO:tasks.workunit.client.0.smithi031.stdout:1/860: getdents d1/da/d12/d65/dcf/d11/df8 0 2024-04-24T23:14:03.999 INFO:tasks.workunit.client.0.smithi031.stdout:4/981: dwrite d3/d20/d2d/f79 [0,4194304] 0 2024-04-24T23:14:04.002 INFO:tasks.workunit.client.0.smithi031.stdout:1/861: dread d1/d3e/d46/dbd/fdd [0,4194304] 0 2024-04-24T23:14:04.002 INFO:tasks.workunit.client.0.smithi031.stdout:3/812: creat d6/df/d16/d2a/d95/d9b/d60/ff9 x:0 0 0 2024-04-24T23:14:04.100 INFO:tasks.workunit.client.1.smithi040.stdout:6/649: rename d7/fc7 to d7/d1a/d20/d8c/fcf 0 2024-04-24T23:14:04.102 INFO:tasks.workunit.client.1.smithi040.stdout:1/682: truncate da/d12/d14/d25/db9/fc8 740236 0 2024-04-24T23:14:04.105 INFO:tasks.workunit.client.1.smithi040.stdout:9/709: mknod d7/da/d17/d40/d55/d2f/d28/d67/c101 0 2024-04-24T23:14:04.108 INFO:tasks.workunit.client.1.smithi040.stdout:1/683: dwrite da/d12/d14/fe0 [0,4194304] 0 2024-04-24T23:14:04.108 INFO:tasks.workunit.client.1.smithi040.stdout:3/733: creat d1/ff9 x:0 0 0 2024-04-24T23:14:04.110 INFO:tasks.workunit.client.0.smithi031.stdout:7/918: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/da0/dea/f103 [4194304,4194304] 0 2024-04-24T23:14:04.110 INFO:tasks.workunit.client.1.smithi040.stdout:1/684: truncate da/d12/d18/d2a/d4d/feb 471474 0 2024-04-24T23:14:04.121 INFO:tasks.workunit.client.0.smithi031.stdout:3/813: truncate d6/df/fae 1087114 0 2024-04-24T23:14:04.123 INFO:tasks.workunit.client.0.smithi031.stdout:2/807: dread d5/d6/d29/f12 [0,4194304] 0 2024-04-24T23:14:04.123 INFO:tasks.workunit.client.1.smithi040.stdout:4/634: unlink da/d45/d48/d6e/f92 0 2024-04-24T23:14:04.124 INFO:tasks.workunit.client.1.smithi040.stdout:4/635: chown da/d3f/d40/d41/d60/f34 18 1 2024-04-24T23:14:04.125 INFO:tasks.workunit.client.1.smithi040.stdout:5/631: dread d2/d7/d8/d1f/d57/d27/d3d/d99/fa6 [0,4194304] 0 2024-04-24T23:14:04.125 INFO:tasks.workunit.client.1.smithi040.stdout:0/752: symlink df/lf7 0 2024-04-24T23:14:04.127 INFO:tasks.workunit.client.1.smithi040.stdout:2/738: read da/d10/fa1 [23297,66233] 0 2024-04-24T23:14:04.127 INFO:tasks.workunit.client.1.smithi040.stdout:7/731: link db/d2f/d37/c98 db/d12/d7d/c10a 0 2024-04-24T23:14:04.128 INFO:tasks.workunit.client.1.smithi040.stdout:2/739: dread - da/d81/da8/fdd zero size 2024-04-24T23:14:04.129 INFO:tasks.workunit.client.1.smithi040.stdout:8/728: creat d0/d38/d4c/ddd/ff1 x:0 0 0 2024-04-24T23:14:04.132 INFO:tasks.workunit.client.1.smithi040.stdout:6/650: symlink d7/d1a/d20/d8c/ld0 0 2024-04-24T23:14:04.134 INFO:tasks.workunit.client.1.smithi040.stdout:2/740: dread da/f9b [0,4194304] 0 2024-04-24T23:14:04.141 INFO:tasks.workunit.client.1.smithi040.stdout:2/741: dwrite da/f106 [0,4194304] 0 2024-04-24T23:14:04.141 INFO:tasks.workunit.client.1.smithi040.stdout:3/734: creat d1/d2d/d72/ffa x:0 0 0 2024-04-24T23:14:04.142 INFO:tasks.workunit.client.1.smithi040.stdout:2/742: write da/d1c/f57 [4363323,38181] 0 2024-04-24T23:14:04.143 INFO:tasks.workunit.client.1.smithi040.stdout:9/710: dread d7/da/d17/d40/d55/d2f/d1f/d31/f98 [0,4194304] 0 2024-04-24T23:14:04.143 INFO:tasks.workunit.client.1.smithi040.stdout:1/685: creat da/d12/d14/d58/ff0 x:0 0 0 2024-04-24T23:14:04.144 INFO:tasks.workunit.client.1.smithi040.stdout:2/743: write da/d10/d15/f32 [635512,10329] 0 2024-04-24T23:14:04.147 INFO:tasks.workunit.client.1.smithi040.stdout:4/636: mkdir da/d45/d48/d6e/ddd 0 2024-04-24T23:14:04.152 INFO:tasks.workunit.client.1.smithi040.stdout:1/686: dwrite da/fd9 [0,4194304] 0 2024-04-24T23:14:04.152 INFO:tasks.workunit.client.1.smithi040.stdout:1/687: stat da/d12/d18/d2a/d32 0 2024-04-24T23:14:04.157 INFO:tasks.workunit.client.1.smithi040.stdout:1/688: read da/f71 [6994539,111359] 0 2024-04-24T23:14:04.158 INFO:tasks.workunit.client.1.smithi040.stdout:1/689: fdatasync da/d12/d18/d2a/d41/fc0 0 2024-04-24T23:14:04.160 INFO:tasks.workunit.client.1.smithi040.stdout:1/690: write da/d12/d18/d2a/d41/d4f/f9b [650916,95140] 0 2024-04-24T23:14:04.174 INFO:tasks.workunit.client.1.smithi040.stdout:2/744: unlink da/f106 0 2024-04-24T23:14:04.174 INFO:tasks.workunit.client.1.smithi040.stdout:2/745: fdatasync da/d10/d7a/f99 0 2024-04-24T23:14:04.175 INFO:tasks.workunit.client.1.smithi040.stdout:2/746: readlink da/d1c/l44 0 2024-04-24T23:14:04.175 INFO:tasks.workunit.client.1.smithi040.stdout:4/637: symlink da/d3f/d6b/d3c/d69/lde 0 2024-04-24T23:14:04.180 INFO:tasks.workunit.client.1.smithi040.stdout:5/632: dwrite d2/d7/d8/d1f/d57/d3e/f8a [0,4194304] 0 2024-04-24T23:14:04.182 INFO:tasks.workunit.client.1.smithi040.stdout:5/633: write d2/d7/d8/d1f/d57/d27/d3c/d43/f6f [2062041,13447] 0 2024-04-24T23:14:04.187 INFO:tasks.workunit.client.1.smithi040.stdout:5/634: dwrite d2/d2f/fa0 [4194304,4194304] 0 2024-04-24T23:14:04.193 INFO:tasks.workunit.client.1.smithi040.stdout:8/729: rename d0/d16/d1b/d58/l79 to d0/d8/dc/d34/da7/lf2 0 2024-04-24T23:14:04.194 INFO:tasks.workunit.client.1.smithi040.stdout:3/735: dwrite d1/d9/d39/f78 [0,4194304] 0 2024-04-24T23:14:04.206 INFO:tasks.workunit.client.1.smithi040.stdout:9/711: unlink d7/da/d17/d40/d55/d2f/d1f/d31/d3b/c94 0 2024-04-24T23:14:04.209 INFO:tasks.workunit.client.1.smithi040.stdout:7/732: truncate db/d2f/d37/d7e/f7f 1401997 0 2024-04-24T23:14:04.215 INFO:tasks.workunit.client.1.smithi040.stdout:4/638: rmdir da/d3f/d40/d41/d8e/dd3 39 2024-04-24T23:14:04.216 INFO:tasks.workunit.client.1.smithi040.stdout:4/639: chown da/d3f/d40/fb2 595397 1 2024-04-24T23:14:04.228 INFO:tasks.workunit.client.0.smithi031.stdout:9/981: mknod d2/d43/c12f 0 2024-04-24T23:14:04.229 INFO:tasks.workunit.client.1.smithi040.stdout:5/635: creat d2/d7/d8/db/d1e/fd0 x:0 0 0 2024-04-24T23:14:04.229 INFO:tasks.workunit.client.1.smithi040.stdout:9/712: dread d7/f59 [0,4194304] 0 2024-04-24T23:14:04.230 INFO:tasks.workunit.client.0.smithi031.stdout:4/982: fdatasync d3/d23/d69/f10b 0 2024-04-24T23:14:04.231 INFO:tasks.workunit.client.1.smithi040.stdout:9/713: chown d7/da/d17/d40/d55/d2f/d1f/d31/d3b/c81 99 1 2024-04-24T23:14:04.232 INFO:tasks.workunit.client.1.smithi040.stdout:9/714: fsync d7/da/d17/d40/d55/d2f/df7/d6c/f91 0 2024-04-24T23:14:04.233 INFO:tasks.workunit.client.1.smithi040.stdout:8/730: mknod d0/d7e/dc1/d9e/d90/cf3 0 2024-04-24T23:14:04.235 INFO:tasks.workunit.client.1.smithi040.stdout:8/731: fsync d0/d8/d66/fc0 0 2024-04-24T23:14:04.235 INFO:tasks.workunit.client.1.smithi040.stdout:3/736: mknod d1/d9/d18/d36/d55/d7e/cfb 0 2024-04-24T23:14:04.236 INFO:tasks.workunit.client.1.smithi040.stdout:2/747: write da/f9b [1166787,15437] 0 2024-04-24T23:14:04.237 INFO:tasks.workunit.client.0.smithi031.stdout:7/919: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/f13f x:0 0 0 2024-04-24T23:14:04.238 INFO:tasks.workunit.client.1.smithi040.stdout:2/748: chown da/d1c/d4f/f61 121528 1 2024-04-24T23:14:04.238 INFO:tasks.workunit.client.1.smithi040.stdout:9/715: dwrite d7/d5e/fd5 [0,4194304] 0 2024-04-24T23:14:04.239 INFO:tasks.workunit.client.1.smithi040.stdout:9/716: write d7/f37 [635443,61226] 0 2024-04-24T23:14:04.244 INFO:tasks.workunit.client.0.smithi031.stdout:6/961: write d13/d15/d1d/d23/d24/d8b/fcd [569094,66372] 0 2024-04-24T23:14:04.249 INFO:tasks.workunit.client.1.smithi040.stdout:4/640: creat da/d3f/d40/d47/d71/fdf x:0 0 0 2024-04-24T23:14:04.250 INFO:tasks.workunit.client.1.smithi040.stdout:4/641: chown da/d45/d58/f73 448044 1 2024-04-24T23:14:04.251 INFO:tasks.workunit.client.1.smithi040.stdout:4/642: chown da/d3f/d40/d47/d75/daa 41 1 2024-04-24T23:14:04.252 INFO:tasks.workunit.client.1.smithi040.stdout:0/753: getdents df 0 2024-04-24T23:14:04.253 INFO:tasks.workunit.client.1.smithi040.stdout:0/754: write df/d1e/d20/d61/fbb [1519222,128430] 0 2024-04-24T23:14:04.254 INFO:tasks.workunit.client.0.smithi031.stdout:9/982: dwrite d2/dae/d14/d1d/d45/f72 [0,4194304] 0 2024-04-24T23:14:04.255 INFO:tasks.workunit.client.1.smithi040.stdout:0/755: fdatasync df/d1e/d20/d21/d41/feb 0 2024-04-24T23:14:04.257 INFO:tasks.workunit.client.0.smithi031.stdout:9/983: readlink d2/dae/d14/d78/d84/da6/ld7 0 2024-04-24T23:14:04.260 INFO:tasks.workunit.client.0.smithi031.stdout:1/862: rename d1/ff to d1/da/d12/d65/dcf/f11b 0 2024-04-24T23:14:04.262 INFO:tasks.workunit.client.1.smithi040.stdout:3/737: dread d1/d2d/fc1 [4194304,4194304] 0 2024-04-24T23:14:04.264 INFO:tasks.workunit.client.1.smithi040.stdout:1/691: write da/d12/d18/f6c [657586,114082] 0 2024-04-24T23:14:04.264 INFO:tasks.workunit.client.1.smithi040.stdout:3/738: fdatasync d1/d9/d18/d36/fa2 0 2024-04-24T23:14:04.264 INFO:tasks.workunit.client.0.smithi031.stdout:2/808: dread d5/d28/d4e/fd6 [0,4194304] 0 2024-04-24T23:14:04.266 INFO:tasks.workunit.client.0.smithi031.stdout:2/809: stat d5/d99/da2 0 2024-04-24T23:14:04.266 INFO:tasks.workunit.client.1.smithi040.stdout:8/732: dread - d0/d16/d1b/d58/fab zero size 2024-04-24T23:14:04.266 INFO:tasks.workunit.client.1.smithi040.stdout:8/733: chown d0/d28/fbc 624289579 1 2024-04-24T23:14:04.268 INFO:tasks.workunit.client.0.smithi031.stdout:4/983: write d3/d20/d2d/f50 [2792995,66153] 0 2024-04-24T23:14:04.270 INFO:tasks.workunit.client.0.smithi031.stdout:4/984: readlink d3/d20/d2d/d44/d5b/lb1 0 2024-04-24T23:14:04.271 INFO:tasks.workunit.client.1.smithi040.stdout:5/636: write d2/d2f/d14/f61 [820294,100363] 0 2024-04-24T23:14:04.272 INFO:tasks.workunit.client.0.smithi031.stdout:9/984: dwrite d2/dae/d14/d78/d84/d4a/dab/dad/fc3 [0,4194304] 0 2024-04-24T23:14:04.272 INFO:tasks.workunit.client.0.smithi031.stdout:3/814: rename d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/fd9 to d6/d14/d71/dad/ded/ffa 0 2024-04-24T23:14:04.277 INFO:tasks.workunit.client.0.smithi031.stdout:2/810: dwrite d5/d6/d29/d26/db0/dc2/fd0 [0,4194304] 0 2024-04-24T23:14:04.279 INFO:tasks.workunit.client.0.smithi031.stdout:7/920: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/c140 0 2024-04-24T23:14:04.280 INFO:tasks.workunit.client.0.smithi031.stdout:7/921: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/da0/l30 0 2024-04-24T23:14:04.281 INFO:tasks.workunit.client.1.smithi040.stdout:9/717: dread - d7/da/d17/d40/fb4 zero size 2024-04-24T23:14:04.282 INFO:tasks.workunit.client.1.smithi040.stdout:6/651: getdents d7/d1a/d20/d8c 0 2024-04-24T23:14:04.284 INFO:tasks.workunit.client.1.smithi040.stdout:6/652: write d7/d1a/d20/d51/d1e/d46/fac [381434,124593] 0 2024-04-24T23:14:04.285 INFO:tasks.workunit.client.1.smithi040.stdout:7/733: mknod db/d12/d20/d52/c10b 0 2024-04-24T23:14:04.287 INFO:tasks.workunit.client.1.smithi040.stdout:6/653: truncate d7/d1a/d20/d51/daa/fb6 749377 0 2024-04-24T23:14:04.288 INFO:tasks.workunit.client.0.smithi031.stdout:3/815: unlink d6/l10 0 2024-04-24T23:14:04.288 INFO:tasks.workunit.client.1.smithi040.stdout:7/734: read db/d12/d20/d52/f23 [715782,12463] 0 2024-04-24T23:14:04.289 INFO:tasks.workunit.client.1.smithi040.stdout:6/654: write d7/d1a/d9f/fc1 [53782,92275] 0 2024-04-24T23:14:04.289 INFO:tasks.workunit.client.1.smithi040.stdout:4/643: truncate da/d3f/d6b/f79 807620 0 2024-04-24T23:14:04.290 INFO:tasks.workunit.client.0.smithi031.stdout:3/816: stat d6/df/d16/d2a/d95/d9b/d1a/d25/f54 0 2024-04-24T23:14:04.292 INFO:tasks.workunit.client.1.smithi040.stdout:6/655: truncate d7/d1a/d20/d51/d2f/dba/fa4 541891 0 2024-04-24T23:14:04.292 INFO:tasks.workunit.client.0.smithi031.stdout:1/863: fsync d1/d33/d36/f8f 0 2024-04-24T23:14:04.292 INFO:tasks.workunit.client.0.smithi031.stdout:1/864: readlink d1/d3e/d46/laa 0 2024-04-24T23:14:04.293 INFO:tasks.workunit.client.1.smithi040.stdout:6/656: read d7/d1a/d20/d51/d2f/f6f [862638,106176] 0 2024-04-24T23:14:04.298 INFO:tasks.workunit.client.0.smithi031.stdout:4/985: write d3/d20/d25/f11d [3224820,106641] 0 2024-04-24T23:14:04.298 INFO:tasks.workunit.client.1.smithi040.stdout:0/756: rename df/d1e/d20/d21/d51 to df/d1e/d20/d21/d41/d8a/df3/df8 0 2024-04-24T23:14:04.298 INFO:tasks.workunit.client.1.smithi040.stdout:6/657: read d7/d1a/d20/d51/d1e/d46/d69/f6b [1294495,30705] 0 2024-04-24T23:14:04.299 INFO:tasks.workunit.client.0.smithi031.stdout:2/811: fdatasync d5/d99/fdb 0 2024-04-24T23:14:04.300 INFO:tasks.workunit.client.0.smithi031.stdout:9/985: mkdir d2/d112/d130 0 2024-04-24T23:14:04.300 INFO:tasks.workunit.client.0.smithi031.stdout:2/812: chown d5/d8/d45/d5b/fbd 448239965 1 2024-04-24T23:14:04.303 INFO:tasks.workunit.client.1.smithi040.stdout:3/739: dread d1/d9/d15/f1d [0,4194304] 0 2024-04-24T23:14:04.304 INFO:tasks.workunit.client.1.smithi040.stdout:8/734: symlink d0/d16/d19/da2/lf4 0 2024-04-24T23:14:04.306 INFO:tasks.workunit.client.0.smithi031.stdout:1/865: read d1/f2 [124921,40814] 0 2024-04-24T23:14:04.308 INFO:tasks.workunit.client.0.smithi031.stdout:7/922: mknod d1/d121/c141 0 2024-04-24T23:14:04.309 INFO:tasks.workunit.client.1.smithi040.stdout:2/749: fsync da/d10/d7a/f8d 0 2024-04-24T23:14:04.312 INFO:tasks.workunit.client.1.smithi040.stdout:9/718: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/c102 0 2024-04-24T23:14:04.317 INFO:tasks.workunit.client.1.smithi040.stdout:4/644: rmdir da 39 2024-04-24T23:14:04.318 INFO:tasks.workunit.client.0.smithi031.stdout:3/817: getdents d6/df/d16/d2a/d95/d9b/d29/db7/de9 0 2024-04-24T23:14:04.319 INFO:tasks.workunit.client.1.smithi040.stdout:7/735: rmdir db/d12/d20/d52/d44 39 2024-04-24T23:14:04.323 INFO:tasks.workunit.client.0.smithi031.stdout:4/986: dread d3/d23/d3b/fe2 [0,4194304] 0 2024-04-24T23:14:04.324 INFO:tasks.workunit.client.1.smithi040.stdout:5/637: dwrite d2/d7/d8/d64/f68 [0,4194304] 0 2024-04-24T23:14:04.324 INFO:tasks.workunit.client.0.smithi031.stdout:2/813: write d5/d8/d15/f50 [745205,114954] 0 2024-04-24T23:14:04.325 INFO:tasks.workunit.client.0.smithi031.stdout:2/814: fsync d5/d6/d29/d26/f10e 0 2024-04-24T23:14:04.326 INFO:tasks.workunit.client.0.smithi031.stdout:9/986: mknod d2/dae/d14/d78/d84/d4a/dab/db7/df3/c131 0 2024-04-24T23:14:04.327 INFO:tasks.workunit.client.1.smithi040.stdout:0/757: creat df/de1/db6/ff9 x:0 0 0 2024-04-24T23:14:04.330 INFO:tasks.workunit.client.0.smithi031.stdout:1/866: mknod d1/da/d12/d65/dcf/d11/d21/d5a/dbb/dff/c11c 0 2024-04-24T23:14:04.332 INFO:tasks.workunit.client.0.smithi031.stdout:4/987: stat d3/d23/d37/ld1 0 2024-04-24T23:14:04.336 INFO:tasks.workunit.client.0.smithi031.stdout:4/988: dread - d3/d23/d37/f110 zero size 2024-04-24T23:14:04.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/818: dread d6/f7b [0,4194304] 0 2024-04-24T23:14:04.338 INFO:tasks.workunit.client.0.smithi031.stdout:4/989: write d3/d23/d69/d3f/d9e/d12b/f140 [304446,126041] 0 2024-04-24T23:14:04.339 INFO:tasks.workunit.client.1.smithi040.stdout:7/736: unlink db/d2f/d37/cf4 0 2024-04-24T23:14:04.341 INFO:tasks.workunit.client.0.smithi031.stdout:2/815: getdents d5/d99/d102 0 2024-04-24T23:14:04.342 INFO:tasks.workunit.client.0.smithi031.stdout:2/816: stat d5/d6/d29/d26/f92 0 2024-04-24T23:14:04.344 INFO:tasks.workunit.client.0.smithi031.stdout:4/990: mkdir d3/d20/d25/d60/d141 0 2024-04-24T23:14:04.346 INFO:tasks.workunit.client.0.smithi031.stdout:1/867: link d1/da/d12/d65/dcf/l88 d1/d3e/d46/df4/l11d 0 2024-04-24T23:14:04.348 INFO:tasks.workunit.client.1.smithi040.stdout:6/658: dread d7/fd [0,4194304] 0 2024-04-24T23:14:04.349 INFO:tasks.workunit.client.0.smithi031.stdout:2/817: rmdir d5/d8/d15/d56 39 2024-04-24T23:14:04.351 INFO:tasks.workunit.client.1.smithi040.stdout:8/735: link d0/d52/d9b/fe1 d0/d8/d66/ff5 0 2024-04-24T23:14:04.352 INFO:tasks.workunit.client.0.smithi031.stdout:4/991: creat d3/d20/f142 x:0 0 0 2024-04-24T23:14:04.355 INFO:tasks.workunit.client.0.smithi031.stdout:1/868: read d1/da/d12/f83 [484923,41833] 0 2024-04-24T23:14:04.356 INFO:tasks.workunit.client.1.smithi040.stdout:2/750: symlink da/d1c/d37/d8a/dfa/l10e 0 2024-04-24T23:14:04.359 INFO:tasks.workunit.client.1.smithi040.stdout:4/645: mkdir da/d45/d48/d6e/de0 0 2024-04-24T23:14:04.359 INFO:tasks.workunit.client.1.smithi040.stdout:4/646: read da/f37 [1021427,35630] 0 2024-04-24T23:14:04.361 INFO:tasks.workunit.client.0.smithi031.stdout:2/818: unlink d5/d6/d29/d26/db0/dc2/dad/cff 0 2024-04-24T23:14:04.362 INFO:tasks.workunit.client.0.smithi031.stdout:1/869: dwrite d1/da/d12/d65/fcb [0,4194304] 0 2024-04-24T23:14:04.362 INFO:tasks.workunit.client.1.smithi040.stdout:7/737: symlink db/d12/d3f/l10c 0 2024-04-24T23:14:04.367 INFO:tasks.workunit.client.1.smithi040.stdout:1/692: link da/d12/d14/d25/db9/fc8 da/d35/dec/ff1 0 2024-04-24T23:14:04.370 INFO:tasks.workunit.client.1.smithi040.stdout:0/758: dwrite df/d12/d1a/f4e [0,4194304] 0 2024-04-24T23:14:04.370 INFO:tasks.workunit.client.0.smithi031.stdout:4/992: creat d3/d5e/dc4/f143 x:0 0 0 2024-04-24T23:14:04.371 INFO:tasks.workunit.client.0.smithi031.stdout:3/819: getdents d6/d14 0 2024-04-24T23:14:04.372 INFO:tasks.workunit.client.0.smithi031.stdout:1/870: fsync d1/f34 0 2024-04-24T23:14:04.373 INFO:tasks.workunit.client.1.smithi040.stdout:6/659: creat d7/de/d84/fd1 x:0 0 0 2024-04-24T23:14:04.375 INFO:tasks.workunit.client.1.smithi040.stdout:8/736: fdatasync d0/d8/dc/f8c 0 2024-04-24T23:14:04.381 INFO:tasks.workunit.client.0.smithi031.stdout:3/820: creat d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/ffb x:0 0 0 2024-04-24T23:14:04.382 INFO:tasks.workunit.client.0.smithi031.stdout:1/871: dwrite d1/d33/fdc [0,4194304] 0 2024-04-24T23:14:04.386 INFO:tasks.workunit.client.0.smithi031.stdout:4/993: unlink d3/d20/d25/d60/d63/f126 0 2024-04-24T23:14:04.388 INFO:tasks.workunit.client.1.smithi040.stdout:5/638: rename d2/d7/d8/d1f/da1/fa8 to d2/d2f/fd1 0 2024-04-24T23:14:04.390 INFO:tasks.workunit.client.0.smithi031.stdout:2/819: mkdir d5/d8/d45/d5b/d49/d6f/d119/d11d 0 2024-04-24T23:14:04.391 INFO:tasks.workunit.client.0.smithi031.stdout:2/820: fdatasync d5/f5c 0 2024-04-24T23:14:04.393 INFO:tasks.workunit.client.0.smithi031.stdout:3/821: creat d6/d55/ffc x:0 0 0 2024-04-24T23:14:04.396 INFO:tasks.workunit.client.0.smithi031.stdout:1/872: unlink d1/da/d12/d7e/d7b/d4e/fd7 0 2024-04-24T23:14:04.396 INFO:tasks.workunit.client.0.smithi031.stdout:3/822: dread d6/d14/d1d/f8b [0,4194304] 0 2024-04-24T23:14:04.396 INFO:tasks.workunit.client.0.smithi031.stdout:1/873: fdatasync d1/f8 0 2024-04-24T23:14:04.398 INFO:tasks.workunit.client.1.smithi040.stdout:3/740: getdents d1 0 2024-04-24T23:14:04.398 INFO:tasks.workunit.client.1.smithi040.stdout:4/647: rmdir da/d45/d48/d6e 39 2024-04-24T23:14:04.407 INFO:tasks.workunit.client.0.smithi031.stdout:2/821: symlink d5/d6/d29/d26/l11e 0 2024-04-24T23:14:04.414 INFO:tasks.workunit.client.1.smithi040.stdout:5/639: truncate d2/d7/f36 558744 0 2024-04-24T23:14:04.416 INFO:tasks.workunit.client.1.smithi040.stdout:1/693: getdents da/d35/d38/d47/dbc/de5 0 2024-04-24T23:14:04.416 INFO:tasks.workunit.client.1.smithi040.stdout:1/694: stat da/d35/d38/d47/dbc/de5 0 2024-04-24T23:14:04.417 INFO:tasks.workunit.client.0.smithi031.stdout:3/823: symlink d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/lfd 0 2024-04-24T23:14:04.419 INFO:tasks.workunit.client.1.smithi040.stdout:0/759: unlink df/d12/d1a/d71/l15 0 2024-04-24T23:14:04.420 INFO:tasks.workunit.client.0.smithi031.stdout:4/994: write d3/d20/d2d/d44/d5b/d5d/f96 [2770291,116141] 0 2024-04-24T23:14:04.421 INFO:tasks.workunit.client.1.smithi040.stdout:3/741: fsync d1/d2a/f47 0 2024-04-24T23:14:04.425 INFO:tasks.workunit.client.1.smithi040.stdout:4/648: write da/d45/d58/f73 [1621140,71263] 0 2024-04-24T23:14:04.425 INFO:tasks.workunit.client.0.smithi031.stdout:3/824: dwrite d6/df/d16/d2a/d95/d9b/d60/ff9 [0,4194304] 0 2024-04-24T23:14:04.426 INFO:tasks.workunit.client.0.smithi031.stdout:1/874: symlink d1/da/d12/d65/dcf/d11/d21/d5a/l11e 0 2024-04-24T23:14:04.427 INFO:tasks.workunit.client.0.smithi031.stdout:3/825: fdatasync d6/d55/fd0 0 2024-04-24T23:14:04.427 INFO:tasks.workunit.client.1.smithi040.stdout:0/760: dwrite df/d1e/d20/d21/d41/d8a/df3/df8/da3/d8e/f99 [0,4194304] 0 2024-04-24T23:14:04.428 INFO:tasks.workunit.client.0.smithi031.stdout:1/875: fsync d1/da/d12/d7e/d7b/d4e/d98/ffb 0 2024-04-24T23:14:04.429 INFO:tasks.workunit.client.1.smithi040.stdout:6/660: dwrite d7/d1a/d20/d51/d1e/f22 [0,4194304] 0 2024-04-24T23:14:04.429 INFO:tasks.workunit.client.0.smithi031.stdout:4/995: rmdir d3/d20/d25/d7e 39 2024-04-24T23:14:04.435 INFO:tasks.workunit.client.0.smithi031.stdout:1/876: dwrite d1/d3e/da3/d69/f95 [0,4194304] 0 2024-04-24T23:14:04.438 INFO:tasks.workunit.client.1.smithi040.stdout:8/737: dwrite d0/d8/dc/d34/f3c [0,4194304] 0 2024-04-24T23:14:04.438 INFO:tasks.workunit.client.1.smithi040.stdout:0/761: chown df/d1e/d20/d21/c42 72 1 2024-04-24T23:14:04.445 INFO:tasks.workunit.client.1.smithi040.stdout:3/742: symlink d1/lfc 0 2024-04-24T23:14:04.445 INFO:tasks.workunit.client.0.smithi031.stdout:4/996: fdatasync d3/d20/d2d/d44/d5b/d5d/f97 0 2024-04-24T23:14:04.446 INFO:tasks.workunit.client.1.smithi040.stdout:2/751: dwrite da/d29/f42 [0,4194304] 0 2024-04-24T23:14:04.446 INFO:tasks.workunit.client.0.smithi031.stdout:4/997: dread - d3/d23/fc5 zero size 2024-04-24T23:14:04.448 INFO:tasks.workunit.client.1.smithi040.stdout:4/649: creat da/d3f/d40/d41/d62/fe1 x:0 0 0 2024-04-24T23:14:04.449 INFO:tasks.workunit.client.0.smithi031.stdout:2/822: truncate d5/d6/d29/d26/db0/dc2/f44 2451730 0 2024-04-24T23:14:04.451 INFO:tasks.workunit.client.1.smithi040.stdout:3/743: dread - d1/d9/d18/d36/d55/d62/ff4 zero size 2024-04-24T23:14:04.453 INFO:tasks.workunit.client.1.smithi040.stdout:1/695: dread da/d12/d23/f30 [0,4194304] 0 2024-04-24T23:14:04.462 INFO:tasks.workunit.client.1.smithi040.stdout:4/650: symlink da/d78/le2 0 2024-04-24T23:14:04.464 INFO:tasks.workunit.client.0.smithi031.stdout:2/823: creat d5/d8/d45/d5b/d49/dc4/de5/f11f x:0 0 0 2024-04-24T23:14:04.467 INFO:tasks.workunit.client.1.smithi040.stdout:2/752: symlink da/d10/d15/daa/l10f 0 2024-04-24T23:14:04.468 INFO:tasks.workunit.client.1.smithi040.stdout:3/744: fsync d1/d80/fcc 0 2024-04-24T23:14:04.469 INFO:tasks.workunit.client.1.smithi040.stdout:3/745: chown d1/d9/d2e/lbc 0 1 2024-04-24T23:14:04.470 INFO:tasks.workunit.client.0.smithi031.stdout:3/826: getdents d6/d14/db2 0 2024-04-24T23:14:04.471 INFO:tasks.workunit.client.0.smithi031.stdout:2/824: fsync d5/d3d/f62 0 2024-04-24T23:14:04.471 INFO:tasks.workunit.client.1.smithi040.stdout:8/738: creat d0/d38/d4c/dd5/ff6 x:0 0 0 2024-04-24T23:14:04.476 INFO:tasks.workunit.client.0.smithi031.stdout:2/825: mkdir d5/d6/d29/d26/d120 0 2024-04-24T23:14:04.476 INFO:tasks.workunit.client.0.smithi031.stdout:3/827: read d6/d14/f33 [1769291,107133] 0 2024-04-24T23:14:04.477 INFO:tasks.workunit.client.1.smithi040.stdout:9/719: sync 2024-04-24T23:14:04.477 INFO:tasks.workunit.client.1.smithi040.stdout:5/640: getdents d2/d7/d8/d6d 0 2024-04-24T23:14:04.477 INFO:tasks.workunit.client.0.smithi031.stdout:4/998: link d3/fb d3/d20/d2d/d44/d5b/f144 0 2024-04-24T23:14:04.478 INFO:tasks.workunit.client.1.smithi040.stdout:0/762: dread df/d1e/d20/d21/d41/d8a/df3/df8/d79/fae [0,4194304] 0 2024-04-24T23:14:04.481 INFO:tasks.workunit.client.0.smithi031.stdout:4/999: dread d3/d20/d25/d95/f112 [0,4194304] 0 2024-04-24T23:14:04.483 INFO:tasks.workunit.client.1.smithi040.stdout:4/651: dread da/d3f/d40/d41/d60/f42 [0,4194304] 0 2024-04-24T23:14:04.484 INFO:tasks.workunit.client.0.smithi031.stdout:2/826: dwrite d5/d8/d45/d5b/d49/f9f [0,4194304] 0 2024-04-24T23:14:04.490 INFO:tasks.workunit.client.0.smithi031.stdout:3/828: rename d6/df/d16/d2a/d95/d9b/d29/fea to d6/df/d16/d2a/d95/d9b/d29/db7/de9/ffe 0 2024-04-24T23:14:04.493 INFO:tasks.workunit.client.1.smithi040.stdout:7/738: sync 2024-04-24T23:14:04.494 INFO:tasks.workunit.client.1.smithi040.stdout:7/739: chown db/d12/d3f/l10c 842430 1 2024-04-24T23:14:04.494 INFO:tasks.workunit.client.0.smithi031.stdout:2/827: truncate d5/d8/d45/d5b/d49/f7e 3754252 0 2024-04-24T23:14:04.495 INFO:tasks.workunit.client.1.smithi040.stdout:7/740: write db/d12/d62/d8a/db3/fef [1507188,65309] 0 2024-04-24T23:14:04.495 INFO:tasks.workunit.client.0.smithi031.stdout:2/828: readlink d5/d28/d4e/d5e/l71 0 2024-04-24T23:14:04.496 INFO:tasks.workunit.client.1.smithi040.stdout:7/741: readlink db/d12/l26 0 2024-04-24T23:14:04.500 INFO:tasks.workunit.client.0.smithi031.stdout:2/829: rename d5/d28/f37 to d5/d6/d68/f121 0 2024-04-24T23:14:04.503 INFO:tasks.workunit.client.1.smithi040.stdout:6/661: write d7/d1a/d20/d51/d2a/f7c [930782,60105] 0 2024-04-24T23:14:04.505 INFO:tasks.workunit.client.0.smithi031.stdout:2/830: creat d5/d99/da2/f122 x:0 0 0 2024-04-24T23:14:04.509 INFO:tasks.workunit.client.1.smithi040.stdout:6/662: dwrite d7/d1a/d20/d51/d1e/d46/d69/d87/fad [0,4194304] 0 2024-04-24T23:14:04.522 INFO:tasks.workunit.client.1.smithi040.stdout:2/753: unlink da/d10/d4d/d6c/c6e 0 2024-04-24T23:14:04.523 INFO:tasks.workunit.client.1.smithi040.stdout:8/739: symlink d0/d7e/dc1/lf7 0 2024-04-24T23:14:04.525 INFO:tasks.workunit.client.1.smithi040.stdout:5/641: unlink d2/d7/d8/d64/d6b/c79 0 2024-04-24T23:14:04.526 INFO:tasks.workunit.client.1.smithi040.stdout:5/642: dread - d2/d7/d8/db/da2/fac zero size 2024-04-24T23:14:04.526 INFO:tasks.workunit.client.1.smithi040.stdout:4/652: mknod da/d78/dbb/ce3 0 2024-04-24T23:14:04.530 INFO:tasks.workunit.client.1.smithi040.stdout:2/754: rename da/d81/da8/fdd to da/d10/d4d/dd7/f110 0 2024-04-24T23:14:04.531 INFO:tasks.workunit.client.1.smithi040.stdout:2/755: chown da/d1c/d37/d39/d77/d108/d10c 0 1 2024-04-24T23:14:04.532 INFO:tasks.workunit.client.1.smithi040.stdout:8/740: creat d0/d8/dc/ff8 x:0 0 0 2024-04-24T23:14:04.533 INFO:tasks.workunit.client.1.smithi040.stdout:4/653: mkdir da/d3f/d40/d47/d75/de4 0 2024-04-24T23:14:04.534 INFO:tasks.workunit.client.1.smithi040.stdout:5/643: creat d2/d7/d8/d1f/d57/d27/d3c/da4/fd2 x:0 0 0 2024-04-24T23:14:04.535 INFO:tasks.workunit.client.1.smithi040.stdout:6/663: creat d7/d1a/d20/d8c/db0/fd2 x:0 0 0 2024-04-24T23:14:04.536 INFO:tasks.workunit.client.1.smithi040.stdout:1/696: getdents da/d12/d18/d2a/d8b 0 2024-04-24T23:14:04.536 INFO:tasks.workunit.client.1.smithi040.stdout:6/664: readlink d7/d1a/l76 0 2024-04-24T23:14:04.537 INFO:tasks.workunit.client.1.smithi040.stdout:0/763: link df/d12/f22 df/d12/d58/dc3/dcd/ffa 0 2024-04-24T23:14:04.538 INFO:tasks.workunit.client.1.smithi040.stdout:2/756: read da/d29/d55/f58 [506777,39797] 0 2024-04-24T23:14:04.540 INFO:tasks.workunit.client.1.smithi040.stdout:9/720: getdents d7/da/d17/d40/d55/d2f/df7 0 2024-04-24T23:14:04.540 INFO:tasks.workunit.client.1.smithi040.stdout:2/757: stat da/d1c/d37/d39/d3f/d52/f87 0 2024-04-24T23:14:04.541 INFO:tasks.workunit.client.1.smithi040.stdout:1/697: fdatasync da/d35/dab/f52 0 2024-04-24T23:14:04.542 INFO:tasks.workunit.client.1.smithi040.stdout:7/742: getdents db/d43 0 2024-04-24T23:14:04.542 INFO:tasks.workunit.client.1.smithi040.stdout:5/644: link d2/d7/d8/f34 d2/d7/d8/d1f/d90/fd3 0 2024-04-24T23:14:04.543 INFO:tasks.workunit.client.1.smithi040.stdout:6/665: mkdir d7/d1a/d20/d51/d1e/d5d/dbb/dd3 0 2024-04-24T23:14:04.544 INFO:tasks.workunit.client.1.smithi040.stdout:0/764: mknod df/d1e/da7/cfb 0 2024-04-24T23:14:04.544 INFO:tasks.workunit.client.1.smithi040.stdout:8/741: dwrite d0/d52/fb3 [0,4194304] 0 2024-04-24T23:14:04.544 INFO:tasks.workunit.client.1.smithi040.stdout:9/721: read - d7/da/d17/d40/d55/d2f/df7/d6c/f9b zero size 2024-04-24T23:14:04.545 INFO:tasks.workunit.client.1.smithi040.stdout:2/758: read - da/d10/d15/daa/fbb zero size 2024-04-24T23:14:04.546 INFO:tasks.workunit.client.1.smithi040.stdout:6/666: readlink d7/d1a/d20/d51/d1e/d5d/l43 0 2024-04-24T23:14:04.546 INFO:tasks.workunit.client.1.smithi040.stdout:8/742: chown d0/d38/c69 472006 1 2024-04-24T23:14:04.547 INFO:tasks.workunit.client.1.smithi040.stdout:7/743: mknod db/d12/d20/d56/c10d 0 2024-04-24T23:14:04.547 INFO:tasks.workunit.client.1.smithi040.stdout:0/765: chown df/d1e/d20/d21/d41/d8a/df3/df8/f8f 92 1 2024-04-24T23:14:04.549 INFO:tasks.workunit.client.1.smithi040.stdout:9/722: mknod d7/d5e/c103 0 2024-04-24T23:14:04.551 INFO:tasks.workunit.client.1.smithi040.stdout:6/667: creat d7/d1a/d20/d51/d1e/d5d/dbb/fd4 x:0 0 0 2024-04-24T23:14:04.552 INFO:tasks.workunit.client.1.smithi040.stdout:7/744: readlink db/d12/d20/d52/d44/l4d 0 2024-04-24T23:14:04.563 INFO:tasks.workunit.client.1.smithi040.stdout:2/759: rmdir da/d9f 39 2024-04-24T23:14:04.564 INFO:tasks.workunit.client.1.smithi040.stdout:6/668: mkdir d7/d1a/d20/d8c/dd5 0 2024-04-24T23:14:04.565 INFO:tasks.workunit.client.1.smithi040.stdout:9/723: mknod d7/da/d17/d40/d55/c104 0 2024-04-24T23:14:04.565 INFO:tasks.workunit.client.1.smithi040.stdout:0/766: dwrite df/d12/d58/f7d [0,4194304] 0 2024-04-24T23:14:04.567 INFO:tasks.workunit.client.1.smithi040.stdout:2/760: creat da/d1c/d37/d39/d3f/d52/f111 x:0 0 0 2024-04-24T23:14:04.568 INFO:tasks.workunit.client.1.smithi040.stdout:6/669: symlink d7/d1a/db5/ld6 0 2024-04-24T23:14:04.570 INFO:tasks.workunit.client.1.smithi040.stdout:0/767: symlink df/d12/d49/lfc 0 2024-04-24T23:14:04.571 INFO:tasks.workunit.client.1.smithi040.stdout:9/724: write d7/da/d17/d40/d55/fc6 [302804,93225] 0 2024-04-24T23:14:04.571 INFO:tasks.workunit.client.1.smithi040.stdout:6/670: creat d7/de/d84/fd7 x:0 0 0 2024-04-24T23:14:04.573 INFO:tasks.workunit.client.1.smithi040.stdout:2/761: rename da/d10/d4d/dd7/fdb to da/d29/d55/da7/f112 0 2024-04-24T23:14:04.573 INFO:tasks.workunit.client.1.smithi040.stdout:7/745: dwrite db/d43/db9/dd4/fdf [0,4194304] 0 2024-04-24T23:14:04.574 INFO:tasks.workunit.client.1.smithi040.stdout:0/768: creat df/d1e/da7/d6b/ffd x:0 0 0 2024-04-24T23:14:04.575 INFO:tasks.workunit.client.1.smithi040.stdout:2/762: chown da/d81/de6 6205025 1 2024-04-24T23:14:04.575 INFO:tasks.workunit.client.1.smithi040.stdout:6/671: write d7/f71 [1067632,123516] 0 2024-04-24T23:14:04.578 INFO:tasks.workunit.client.1.smithi040.stdout:0/769: write df/d12/d1a/f4e [7381283,27960] 0 2024-04-24T23:14:04.579 INFO:tasks.workunit.client.1.smithi040.stdout:9/725: creat d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/f105 x:0 0 0 2024-04-24T23:14:04.580 INFO:tasks.workunit.client.1.smithi040.stdout:0/770: chown df/d12/d4b/f6a 1111598 1 2024-04-24T23:14:04.581 INFO:tasks.workunit.client.1.smithi040.stdout:7/746: rename db/d12/f29 to db/d12/d62/de7/f10e 0 2024-04-24T23:14:04.582 INFO:tasks.workunit.client.1.smithi040.stdout:6/672: creat d7/d1a/fd8 x:0 0 0 2024-04-24T23:14:04.582 INFO:tasks.workunit.client.1.smithi040.stdout:7/747: fdatasync db/d12/d62/d8a/f102 0 2024-04-24T23:14:04.582 INFO:tasks.workunit.client.1.smithi040.stdout:2/763: mknod da/d1c/d37/d8a/dfa/c113 0 2024-04-24T23:14:04.583 INFO:tasks.workunit.client.1.smithi040.stdout:7/748: chown db/d2f/d37/d7e/fc3 237 1 2024-04-24T23:14:04.584 INFO:tasks.workunit.client.1.smithi040.stdout:0/771: read df/d1e/f40 [152755,39304] 0 2024-04-24T23:14:04.584 INFO:tasks.workunit.client.1.smithi040.stdout:2/764: rename da/d29 to da/d29/d55/d114 22 2024-04-24T23:14:04.584 INFO:tasks.workunit.client.1.smithi040.stdout:9/726: chown d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/lc8 180646 1 2024-04-24T23:14:04.588 INFO:tasks.workunit.client.1.smithi040.stdout:2/765: fdatasync da/d10/f31 0 2024-04-24T23:14:04.591 INFO:tasks.workunit.client.1.smithi040.stdout:9/727: creat d7/da/d17/d40/f106 x:0 0 0 2024-04-24T23:14:04.592 INFO:tasks.workunit.client.1.smithi040.stdout:7/749: getdents db/d12/d62/d109 0 2024-04-24T23:14:04.593 INFO:tasks.workunit.client.1.smithi040.stdout:9/728: read d7/f37 [3576718,126801] 0 2024-04-24T23:14:04.593 INFO:tasks.workunit.client.1.smithi040.stdout:7/750: read - db/d12/d62/de7/ffb zero size 2024-04-24T23:14:04.596 INFO:tasks.workunit.client.1.smithi040.stdout:9/729: truncate d7/da/d17/d40/d55/d2f/df7/d6c/fa8 6842593 0 2024-04-24T23:14:04.597 INFO:tasks.workunit.client.1.smithi040.stdout:0/772: link df/d1e/d20/d21/f9c df/d14/ffe 0 2024-04-24T23:14:04.600 INFO:tasks.workunit.client.1.smithi040.stdout:2/766: rename da/d10/d7a/fb9 to da/f115 0 2024-04-24T23:14:04.601 INFO:tasks.workunit.client.1.smithi040.stdout:7/751: mknod db/d12/d73/c10f 0 2024-04-24T23:14:04.604 INFO:tasks.workunit.client.1.smithi040.stdout:0/773: dread - df/d1e/d20/d21/d41/d8a/f9b zero size 2024-04-24T23:14:04.605 INFO:tasks.workunit.client.1.smithi040.stdout:0/774: fdatasync df/d1e/d20/d21/d41/d8a/df3/df8/da3/d8e/f99 0 2024-04-24T23:14:04.606 INFO:tasks.workunit.client.1.smithi040.stdout:2/767: creat da/d81/da8/f116 x:0 0 0 2024-04-24T23:14:04.607 INFO:tasks.workunit.client.1.smithi040.stdout:0/775: truncate df/d1e/d20/ff0 201397 0 2024-04-24T23:14:04.609 INFO:tasks.workunit.client.1.smithi040.stdout:7/752: unlink db/d2f/d37/f94 0 2024-04-24T23:14:04.614 INFO:tasks.workunit.client.1.smithi040.stdout:0/776: dwrite df/d1e/d20/ff0 [0,4194304] 0 2024-04-24T23:14:04.614 INFO:tasks.workunit.client.1.smithi040.stdout:4/654: dread da/d3f/d40/d41/d60/f82 [0,4194304] 0 2024-04-24T23:14:04.614 INFO:tasks.workunit.client.1.smithi040.stdout:7/753: creat db/d12/d62/de7/f110 x:0 0 0 2024-04-24T23:14:04.617 INFO:tasks.workunit.client.1.smithi040.stdout:7/754: read db/d2f/d37/d7e/fc3 [420694,19230] 0 2024-04-24T23:14:04.618 INFO:tasks.workunit.client.1.smithi040.stdout:4/655: fdatasync da/d3f/d40/d41/d60/f4d 0 2024-04-24T23:14:04.625 INFO:tasks.workunit.client.1.smithi040.stdout:7/755: write db/d2f/f6d [3335396,92248] 0 2024-04-24T23:14:04.631 INFO:tasks.workunit.client.1.smithi040.stdout:4/656: rename da/d3f/d40/d41/d60/f23 to da/d3f/d40/fe5 0 2024-04-24T23:14:04.632 INFO:tasks.workunit.client.1.smithi040.stdout:4/657: chown da/d3f/d40/d47/d75/c8f 828812092 1 2024-04-24T23:14:04.632 INFO:tasks.workunit.client.1.smithi040.stdout:7/756: creat db/f111 x:0 0 0 2024-04-24T23:14:04.635 INFO:tasks.workunit.client.1.smithi040.stdout:7/757: symlink db/d12/d7d/dd2/l112 0 2024-04-24T23:14:04.637 INFO:tasks.workunit.client.1.smithi040.stdout:4/658: creat da/d45/d48/d6e/d70/fe6 x:0 0 0 2024-04-24T23:14:04.641 INFO:tasks.workunit.client.1.smithi040.stdout:7/758: creat db/d12/d20/f113 x:0 0 0 2024-04-24T23:14:04.641 INFO:tasks.workunit.client.1.smithi040.stdout:7/759: chown db/d12/d62/d109 4042 1 2024-04-24T23:14:04.642 INFO:tasks.workunit.client.1.smithi040.stdout:7/760: chown db/d12/d20/d56/l75 14070 1 2024-04-24T23:14:04.643 INFO:tasks.workunit.client.1.smithi040.stdout:7/761: mknod db/d12/d20/d52/d44/c114 0 2024-04-24T23:14:04.646 INFO:tasks.workunit.client.1.smithi040.stdout:7/762: creat db/d43/db9/dd4/d101/f115 x:0 0 0 2024-04-24T23:14:04.649 INFO:tasks.workunit.client.1.smithi040.stdout:7/763: fsync db/d12/d20/d52/d6c/fae 0 2024-04-24T23:14:04.652 INFO:tasks.workunit.client.1.smithi040.stdout:7/764: readlink db/d12/d20/d52/d4b/le9 0 2024-04-24T23:14:04.652 INFO:tasks.workunit.client.1.smithi040.stdout:5/645: dread d2/d7/f36 [0,4194304] 0 2024-04-24T23:14:04.652 INFO:tasks.workunit.client.1.smithi040.stdout:3/746: dwrite d1/d9/d18/d36/f9e [0,4194304] 0 2024-04-24T23:14:04.653 INFO:tasks.workunit.client.1.smithi040.stdout:5/646: truncate d2/d7/d8/db/d1e/fd0 87058 0 2024-04-24T23:14:04.655 INFO:tasks.workunit.client.1.smithi040.stdout:7/765: rmdir db/d12/d62/d8a/db3 39 2024-04-24T23:14:04.660 INFO:tasks.workunit.client.1.smithi040.stdout:7/766: rename db/d12/d20/d52/d4b/f104 to db/d12/d62/d103/f116 0 2024-04-24T23:14:04.663 INFO:tasks.workunit.client.1.smithi040.stdout:3/747: creat d1/d9/d18/d36/ffd x:0 0 0 2024-04-24T23:14:04.665 INFO:tasks.workunit.client.1.smithi040.stdout:7/767: unlink db/d12/d20/d56/lec 0 2024-04-24T23:14:04.666 INFO:tasks.workunit.client.1.smithi040.stdout:3/748: fsync d1/d2d/d2b/f4d 0 2024-04-24T23:14:04.666 INFO:tasks.workunit.client.1.smithi040.stdout:3/749: readlink d1/d2d/d44/lbb 0 2024-04-24T23:14:04.669 INFO:tasks.workunit.client.1.smithi040.stdout:1/698: truncate da/d12/d55/f8e 4186125 0 2024-04-24T23:14:04.671 INFO:tasks.workunit.client.0.smithi031.stdout:1/877: truncate d1/d33/fdc 2724862 0 2024-04-24T23:14:04.672 INFO:tasks.workunit.client.0.smithi031.stdout:2/831: write d5/d8/d15/f5f [172426,8794] 0 2024-04-24T23:14:04.680 INFO:tasks.workunit.client.0.smithi031.stdout:1/878: dwrite d1/da/d12/d65/dcf/d11/f13 [4194304,4194304] 0 2024-04-24T23:14:04.701 INFO:tasks.workunit.client.1.smithi040.stdout:8/743: sync 2024-04-24T23:14:04.703 INFO:tasks.workunit.client.0.smithi031.stdout:2/832: dwrite d5/d8/d45/f8d [0,4194304] 0 2024-04-24T23:14:04.705 INFO:tasks.workunit.client.0.smithi031.stdout:2/833: chown d5/d28/d4e/d5e/l71 8259922 1 2024-04-24T23:14:04.712 INFO:tasks.workunit.client.1.smithi040.stdout:7/768: symlink db/d12/d62/d8a/db3/dbf/l117 0 2024-04-24T23:14:04.717 INFO:tasks.workunit.client.1.smithi040.stdout:4/659: write f5 [753189,91798] 0 2024-04-24T23:14:04.720 INFO:tasks.workunit.client.1.smithi040.stdout:3/750: creat d1/d9/d18/d36/d61/ffe x:0 0 0 2024-04-24T23:14:04.720 INFO:tasks.workunit.client.1.smithi040.stdout:8/744: truncate d0/d8/f29 2049351 0 2024-04-24T23:14:04.721 INFO:tasks.workunit.client.1.smithi040.stdout:6/673: dwrite d7/d1a/d20/d51/d1e/d5d/f31 [0,4194304] 0 2024-04-24T23:14:04.721 INFO:tasks.workunit.client.1.smithi040.stdout:7/769: mkdir db/d2f/d37/d7e/d118 0 2024-04-24T23:14:04.723 INFO:tasks.workunit.client.1.smithi040.stdout:9/730: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f48 [4194304,4194304] 0 2024-04-24T23:14:04.726 INFO:tasks.workunit.client.1.smithi040.stdout:4/660: fdatasync da/d3f/d6b/f79 0 2024-04-24T23:14:04.726 INFO:tasks.workunit.client.1.smithi040.stdout:6/674: write d7/d1a/d9f/fb4 [324646,88151] 0 2024-04-24T23:14:04.731 INFO:tasks.workunit.client.1.smithi040.stdout:8/745: creat d0/d52/d9b/ff9 x:0 0 0 2024-04-24T23:14:04.733 INFO:tasks.workunit.client.1.smithi040.stdout:1/699: dwrite da/d12/d23/f45 [0,4194304] 0 2024-04-24T23:14:04.734 INFO:tasks.workunit.client.1.smithi040.stdout:4/661: truncate da/d3f/d6b/f21 4637171 0 2024-04-24T23:14:04.737 INFO:tasks.workunit.client.1.smithi040.stdout:8/746: creat d0/d16/d1b/d47/ffa x:0 0 0 2024-04-24T23:14:04.737 INFO:tasks.workunit.client.1.smithi040.stdout:4/662: creat da/d1f/fe7 x:0 0 0 2024-04-24T23:14:04.740 INFO:tasks.workunit.client.1.smithi040.stdout:1/700: rename da/d12/d23/ld1 to da/d12/d14/d58/lf2 0 2024-04-24T23:14:04.741 INFO:tasks.workunit.client.1.smithi040.stdout:7/770: link db/d12/d20/d52/d4b/fa1 db/d43/db9/dd4/d101/f119 0 2024-04-24T23:14:04.742 INFO:tasks.workunit.client.1.smithi040.stdout:8/747: dread d0/d16/d1b/f40 [0,4194304] 0 2024-04-24T23:14:04.744 INFO:tasks.workunit.client.1.smithi040.stdout:4/663: creat da/d78/dbb/fe8 x:0 0 0 2024-04-24T23:14:04.745 INFO:tasks.workunit.client.1.smithi040.stdout:6/675: getdents d7/d1a/d20/d51/d1e/d5d 0 2024-04-24T23:14:04.747 INFO:tasks.workunit.client.1.smithi040.stdout:7/771: mkdir db/d2f/d11a 0 2024-04-24T23:14:04.749 INFO:tasks.workunit.client.1.smithi040.stdout:8/748: fsync d0/d8/dc/f8c 0 2024-04-24T23:14:04.751 INFO:tasks.workunit.client.1.smithi040.stdout:6/676: creat d7/d1a/d20/d51/d1e/fd9 x:0 0 0 2024-04-24T23:14:04.754 INFO:tasks.workunit.client.1.smithi040.stdout:4/664: mkdir da/d3f/d40/d41/d8e/de9 0 2024-04-24T23:14:04.755 INFO:tasks.workunit.client.1.smithi040.stdout:3/751: dread d1/d2a/d8d/fca [0,4194304] 0 2024-04-24T23:14:04.755 INFO:tasks.workunit.client.1.smithi040.stdout:7/772: mkdir db/d12/d20/d52/dfc/d11b 0 2024-04-24T23:14:04.762 INFO:tasks.workunit.client.1.smithi040.stdout:7/773: rmdir db/d12/d20/d52/d6c 39 2024-04-24T23:14:04.766 INFO:tasks.workunit.client.1.smithi040.stdout:6/677: rmdir d7/d1a/d20/d51/d1e/dcd 0 2024-04-24T23:14:04.770 INFO:tasks.workunit.client.1.smithi040.stdout:3/752: creat d1/d9/d39/fff x:0 0 0 2024-04-24T23:14:04.770 INFO:tasks.workunit.client.1.smithi040.stdout:6/678: rmdir d7/d1a/d20/d51 39 2024-04-24T23:14:04.772 INFO:tasks.workunit.client.1.smithi040.stdout:7/774: mknod db/d12/d20/d52/c11c 0 2024-04-24T23:14:04.774 INFO:tasks.workunit.client.1.smithi040.stdout:9/731: dread d7/da/d17/d40/d55/d2f/d28/f88 [0,4194304] 0 2024-04-24T23:14:04.774 INFO:tasks.workunit.client.1.smithi040.stdout:4/665: dread da/d3f/d6b/d2b/f55 [0,4194304] 0 2024-04-24T23:14:04.775 INFO:tasks.workunit.client.1.smithi040.stdout:6/679: truncate d7/de/d84/f91 112485 0 2024-04-24T23:14:04.775 INFO:tasks.workunit.client.1.smithi040.stdout:3/753: dread d1/d9/d18/da7/fad [0,4194304] 0 2024-04-24T23:14:04.777 INFO:tasks.workunit.client.1.smithi040.stdout:8/749: write d0/d8/f29 [2143085,123499] 0 2024-04-24T23:14:04.777 INFO:tasks.workunit.client.1.smithi040.stdout:9/732: chown d7/da/d17/d40/d55/d2f/c2e 649773650 1 2024-04-24T23:14:04.780 INFO:tasks.workunit.client.1.smithi040.stdout:3/754: unlink d1/d2d/fee 0 2024-04-24T23:14:04.780 INFO:tasks.workunit.client.1.smithi040.stdout:8/750: readlink d0/d16/d19/d71/d76/laf 0 2024-04-24T23:14:04.781 INFO:tasks.workunit.client.1.smithi040.stdout:7/775: dwrite db/f111 [0,4194304] 0 2024-04-24T23:14:04.784 INFO:tasks.workunit.client.1.smithi040.stdout:7/776: dread - db/d12/d62/de7/f110 zero size 2024-04-24T23:14:04.786 INFO:tasks.workunit.client.1.smithi040.stdout:8/751: truncate d0/d8/d22/d81/f74 5098343 0 2024-04-24T23:14:04.787 INFO:tasks.workunit.client.1.smithi040.stdout:4/666: write da/d3f/d40/d41/d60/f42 [1241112,99599] 0 2024-04-24T23:14:04.790 INFO:tasks.workunit.client.1.smithi040.stdout:3/755: dread d1/d2a/d6a/f3a [0,4194304] 0 2024-04-24T23:14:04.790 INFO:tasks.workunit.client.1.smithi040.stdout:6/680: dwrite d7/d1a/d20/d51/d1e/d46/d69/f8d [0,4194304] 0 2024-04-24T23:14:04.791 INFO:tasks.workunit.client.1.smithi040.stdout:4/667: truncate da/d3f/fb0 4416367 0 2024-04-24T23:14:04.792 INFO:tasks.workunit.client.1.smithi040.stdout:7/777: chown db/c11 315155 1 2024-04-24T23:14:04.795 INFO:tasks.workunit.client.1.smithi040.stdout:6/681: chown d7/f60 478273 1 2024-04-24T23:14:04.796 INFO:tasks.workunit.client.1.smithi040.stdout:3/756: mkdir d1/d9/d2e/d100 0 2024-04-24T23:14:04.799 INFO:tasks.workunit.client.1.smithi040.stdout:9/733: dread d7/da/d17/d40/d55/d2f/df7/d6c/fa8 [4194304,4194304] 0 2024-04-24T23:14:04.800 INFO:tasks.workunit.client.1.smithi040.stdout:4/668: dread - da/d3f/d40/d41/d8e/dd3/fdb zero size 2024-04-24T23:14:04.801 INFO:tasks.workunit.client.1.smithi040.stdout:3/757: symlink d1/d2a/d6a/d94/l101 0 2024-04-24T23:14:04.801 INFO:tasks.workunit.client.1.smithi040.stdout:8/752: link d0/d52/l7f d0/d8/d22/db8/lfb 0 2024-04-24T23:14:04.804 INFO:tasks.workunit.client.1.smithi040.stdout:9/734: fdatasync d7/f3a 0 2024-04-24T23:14:04.804 INFO:tasks.workunit.client.1.smithi040.stdout:8/753: write d0/d8/dc/d34/f3c [3826088,127407] 0 2024-04-24T23:14:04.804 INFO:tasks.workunit.client.1.smithi040.stdout:3/758: chown d1/d9/d39/cc4 2634599 1 2024-04-24T23:14:04.806 INFO:tasks.workunit.client.1.smithi040.stdout:9/735: stat d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/def 0 2024-04-24T23:14:04.806 INFO:tasks.workunit.client.1.smithi040.stdout:3/759: write d1/d9/f21 [3826506,49347] 0 2024-04-24T23:14:04.806 INFO:tasks.workunit.client.1.smithi040.stdout:6/682: creat d7/d1a/d20/d51/fda x:0 0 0 2024-04-24T23:14:04.808 INFO:tasks.workunit.client.1.smithi040.stdout:6/683: fdatasync d7/d1a/d20/d51/d2f/f6f 0 2024-04-24T23:14:04.810 INFO:tasks.workunit.client.1.smithi040.stdout:7/778: write db/f1e [839270,61734] 0 2024-04-24T23:14:04.811 INFO:tasks.workunit.client.1.smithi040.stdout:7/779: stat db/d43/db9/dd4 0 2024-04-24T23:14:04.814 INFO:tasks.workunit.client.1.smithi040.stdout:9/736: dread - d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f60 zero size 2024-04-24T23:14:04.815 INFO:tasks.workunit.client.1.smithi040.stdout:9/737: stat d7/da/d17/d40/lec 0 2024-04-24T23:14:04.816 INFO:tasks.workunit.client.1.smithi040.stdout:3/760: dread d1/d9/f17 [0,4194304] 0 2024-04-24T23:14:04.817 INFO:tasks.workunit.client.1.smithi040.stdout:7/780: creat db/d12/d20/d56/f11d x:0 0 0 2024-04-24T23:14:04.818 INFO:tasks.workunit.client.1.smithi040.stdout:4/669: getdents da/d3f/d40/d47/d75/daa 0 2024-04-24T23:14:04.819 INFO:tasks.workunit.client.1.smithi040.stdout:7/781: chown db/d2f/d37/cba 1466 1 2024-04-24T23:14:04.820 INFO:tasks.workunit.client.1.smithi040.stdout:3/761: creat d1/d2a/d8d/d7c/f102 x:0 0 0 2024-04-24T23:14:04.820 INFO:tasks.workunit.client.1.smithi040.stdout:6/684: creat d7/d1a/d20/d51/fdb x:0 0 0 2024-04-24T23:14:04.823 INFO:tasks.workunit.client.1.smithi040.stdout:7/782: unlink db/d2f/fe0 0 2024-04-24T23:14:04.824 INFO:tasks.workunit.client.1.smithi040.stdout:6/685: chown d7/d1a/d20/d51/d2a 871095754 1 2024-04-24T23:14:04.824 INFO:tasks.workunit.client.1.smithi040.stdout:8/754: dwrite d0/d16/d1b/d58/d8f/db1/fe2 [0,4194304] 0 2024-04-24T23:14:04.826 INFO:tasks.workunit.client.1.smithi040.stdout:7/783: write db/d12/d62/fdb [2940424,42456] 0 2024-04-24T23:14:04.828 INFO:tasks.workunit.client.1.smithi040.stdout:3/762: unlink d1/d2a/d6a/d94/f98 0 2024-04-24T23:14:04.829 INFO:tasks.workunit.client.1.smithi040.stdout:8/755: chown d0/d8/dc/ff8 0 1 2024-04-24T23:14:04.830 INFO:tasks.workunit.client.1.smithi040.stdout:6/686: mknod d7/d1a/d20/d8c/db0/cdc 0 2024-04-24T23:14:04.831 INFO:tasks.workunit.client.1.smithi040.stdout:3/763: fdatasync d1/d9/d15/d45/daa/ff5 0 2024-04-24T23:14:04.832 INFO:tasks.workunit.client.1.smithi040.stdout:4/670: dwrite da/d3f/d6b/d3c/d69/fa1 [0,4194304] 0 2024-04-24T23:14:04.834 INFO:tasks.workunit.client.1.smithi040.stdout:8/756: dread d0/d8/dc/f96 [0,4194304] 0 2024-04-24T23:14:04.835 INFO:tasks.workunit.client.1.smithi040.stdout:2/768: sync 2024-04-24T23:14:04.839 INFO:tasks.workunit.client.1.smithi040.stdout:9/738: dwrite d7/f57 [0,4194304] 0 2024-04-24T23:14:04.840 INFO:tasks.workunit.client.1.smithi040.stdout:8/757: mkdir d0/d16/d1b/d58/d8f/dfc 0 2024-04-24T23:14:04.841 INFO:tasks.workunit.client.1.smithi040.stdout:8/758: fdatasync d0/d16/d1b/d47/ffa 0 2024-04-24T23:14:04.842 INFO:tasks.workunit.client.1.smithi040.stdout:7/784: link f6 db/d12/d20/d52/d4b/f11e 0 2024-04-24T23:14:04.845 INFO:tasks.workunit.client.1.smithi040.stdout:8/759: chown d0/d16/d19/fb5 977802356 1 2024-04-24T23:14:04.845 INFO:tasks.workunit.client.1.smithi040.stdout:2/769: unlink da/d10/d4d/da4/cb5 0 2024-04-24T23:14:04.847 INFO:tasks.workunit.client.1.smithi040.stdout:4/671: mkdir da/d45/d48/d6e/ddd/dea 0 2024-04-24T23:14:04.847 INFO:tasks.workunit.client.1.smithi040.stdout:3/764: dwrite d1/d2a/d6a/f35 [8388608,4194304] 0 2024-04-24T23:14:04.850 INFO:tasks.workunit.client.1.smithi040.stdout:7/785: creat db/d2f/d37/f11f x:0 0 0 2024-04-24T23:14:04.850 INFO:tasks.workunit.client.1.smithi040.stdout:8/760: mkdir d0/d7e/dc1/d9e/d90/dfd 0 2024-04-24T23:14:04.853 INFO:tasks.workunit.client.1.smithi040.stdout:3/765: symlink d1/d9/d2e/l103 0 2024-04-24T23:14:04.853 INFO:tasks.workunit.client.1.smithi040.stdout:4/672: mkdir da/d3f/d40/d41/d8e/db8/deb 0 2024-04-24T23:14:04.854 INFO:tasks.workunit.client.1.smithi040.stdout:9/739: creat d7/da/f107 x:0 0 0 2024-04-24T23:14:04.854 INFO:tasks.workunit.client.1.smithi040.stdout:8/761: stat d0/d16/d19/da2/lf4 0 2024-04-24T23:14:04.855 INFO:tasks.workunit.client.1.smithi040.stdout:2/770: mknod da/d10/d4d/c117 0 2024-04-24T23:14:04.856 INFO:tasks.workunit.client.1.smithi040.stdout:2/771: stat da/d1c/l44 0 2024-04-24T23:14:04.858 INFO:tasks.workunit.client.1.smithi040.stdout:3/766: mknod d1/d9/d2e/dcf/c104 0 2024-04-24T23:14:04.858 INFO:tasks.workunit.client.1.smithi040.stdout:6/687: dwrite d7/de/d4e/f7a [0,4194304] 0 2024-04-24T23:14:04.861 INFO:tasks.workunit.client.1.smithi040.stdout:8/762: creat d0/d38/d4c/da3/ffe x:0 0 0 2024-04-24T23:14:04.862 INFO:tasks.workunit.client.1.smithi040.stdout:4/673: mknod da/d45/d48/d6e/de0/cec 0 2024-04-24T23:14:04.863 INFO:tasks.workunit.client.1.smithi040.stdout:4/674: chown da/d45/d58/cd8 0 1 2024-04-24T23:14:04.864 INFO:tasks.workunit.client.1.smithi040.stdout:3/767: symlink d1/d2d/de/l105 0 2024-04-24T23:14:04.866 INFO:tasks.workunit.client.1.smithi040.stdout:3/768: readlink d1/d2a/d8d/d7c/l9f 0 2024-04-24T23:14:04.866 INFO:tasks.workunit.client.1.smithi040.stdout:2/772: mknod da/d10/c118 0 2024-04-24T23:14:04.867 INFO:tasks.workunit.client.1.smithi040.stdout:7/786: dwrite db/d12/d20/d52/d6c/d8f/da7/ffe [4194304,4194304] 0 2024-04-24T23:14:04.867 INFO:tasks.workunit.client.1.smithi040.stdout:4/675: mknod da/d3f/d6b/d2b/ced 0 2024-04-24T23:14:04.869 INFO:tasks.workunit.client.1.smithi040.stdout:8/763: creat d0/d8/da0/fff x:0 0 0 2024-04-24T23:14:04.869 INFO:tasks.workunit.client.1.smithi040.stdout:4/676: readlink da/d45/d48/l4f 0 2024-04-24T23:14:04.870 INFO:tasks.workunit.client.1.smithi040.stdout:2/773: chown da/d1c/d37/d39/d77/cae 14013 1 2024-04-24T23:14:04.870 INFO:tasks.workunit.client.1.smithi040.stdout:0/777: sync 2024-04-24T23:14:04.870 INFO:tasks.workunit.client.1.smithi040.stdout:1/701: sync 2024-04-24T23:14:04.870 INFO:tasks.workunit.client.1.smithi040.stdout:5/647: sync 2024-04-24T23:14:04.873 INFO:tasks.workunit.client.1.smithi040.stdout:7/787: read db/d12/d20/d52/f28 [2694702,77855] 0 2024-04-24T23:14:04.873 INFO:tasks.workunit.client.1.smithi040.stdout:1/702: chown da/d12/d23/lcc 2 1 2024-04-24T23:14:04.873 INFO:tasks.workunit.client.1.smithi040.stdout:3/769: rename d1/d9/d18/d36/d55/d7e to d1/d2d/d72/d106 0 2024-04-24T23:14:04.874 INFO:tasks.workunit.client.1.smithi040.stdout:0/778: write df/de1/f9e [1469410,40682] 0 2024-04-24T23:14:04.874 INFO:tasks.workunit.client.1.smithi040.stdout:2/774: write da/d29/d55/fa9 [5184358,97531] 0 2024-04-24T23:14:04.876 INFO:tasks.workunit.client.1.smithi040.stdout:0/779: chown df/d14/l76 0 1 2024-04-24T23:14:04.877 INFO:tasks.workunit.client.1.smithi040.stdout:2/775: readlink da/d29/l53 0 2024-04-24T23:14:04.878 INFO:tasks.workunit.client.1.smithi040.stdout:8/764: unlink d0/d16/d1b/d58/fc6 0 2024-04-24T23:14:04.879 INFO:tasks.workunit.client.1.smithi040.stdout:0/780: truncate df/d12/d1a/f4e 9382968 0 2024-04-24T23:14:04.880 INFO:tasks.workunit.client.1.smithi040.stdout:8/765: chown d0/d16/d19/d71/c7b 1595 1 2024-04-24T23:14:04.880 INFO:tasks.workunit.client.1.smithi040.stdout:0/781: chown df/f63 16419937 1 2024-04-24T23:14:04.880 INFO:tasks.workunit.client.1.smithi040.stdout:4/677: creat da/d45/d48/d6e/d70/fee x:0 0 0 2024-04-24T23:14:04.881 INFO:tasks.workunit.client.1.smithi040.stdout:5/648: creat d2/d7/d8/d1f/d57/d27/d3d/fd4 x:0 0 0 2024-04-24T23:14:04.883 INFO:tasks.workunit.client.1.smithi040.stdout:3/770: mkdir d1/d2d/de/d30/d37/d107 0 2024-04-24T23:14:04.884 INFO:tasks.workunit.client.1.smithi040.stdout:9/740: write d7/da/d17/d40/f42 [2106567,78974] 0 2024-04-24T23:14:04.889 INFO:tasks.workunit.client.1.smithi040.stdout:8/766: fsync d0/d8/d22/f73 0 2024-04-24T23:14:04.892 INFO:tasks.workunit.client.1.smithi040.stdout:4/678: dread da/d3f/d40/d47/fa4 [0,4194304] 0 2024-04-24T23:14:04.893 INFO:tasks.workunit.client.1.smithi040.stdout:5/649: creat d2/d7/d8/d1f/d57/d27/d3c/d9b/fd5 x:0 0 0 2024-04-24T23:14:04.893 INFO:tasks.workunit.client.1.smithi040.stdout:4/679: chown da/d3f/d40/d47/d75/f97 20773189 1 2024-04-24T23:14:04.896 INFO:tasks.workunit.client.1.smithi040.stdout:9/741: mknod d7/da/d17/d40/d55/d2f/d28/c108 0 2024-04-24T23:14:04.896 INFO:tasks.workunit.client.1.smithi040.stdout:9/742: readlink d7/ld8 0 2024-04-24T23:14:04.897 INFO:tasks.workunit.client.1.smithi040.stdout:6/688: dread d7/d1a/d20/d51/d1e/d46/d69/f6b [0,4194304] 0 2024-04-24T23:14:04.898 INFO:tasks.workunit.client.1.smithi040.stdout:2/776: rename da/d1c/d37/d39/d3f/l46 to da/d29/d55/l119 0 2024-04-24T23:14:04.901 INFO:tasks.workunit.client.1.smithi040.stdout:8/767: creat d0/d16/d1b/d58/f100 x:0 0 0 2024-04-24T23:14:04.902 INFO:tasks.workunit.client.1.smithi040.stdout:7/788: write db/d12/d20/d52/d4b/fa1 [5013390,90821] 0 2024-04-24T23:14:04.904 INFO:tasks.workunit.client.1.smithi040.stdout:4/680: creat da/d3f/d40/d47/d75/d7c/fef x:0 0 0 2024-04-24T23:14:04.905 INFO:tasks.workunit.client.1.smithi040.stdout:5/650: mkdir d2/d7/d8/d1f/d57/d27/d3d/dd6 0 2024-04-24T23:14:04.905 INFO:tasks.workunit.client.1.smithi040.stdout:6/689: dwrite d7/d1a/d20/d51/d1e/d46/fac [0,4194304] 0 2024-04-24T23:14:04.906 INFO:tasks.workunit.client.1.smithi040.stdout:7/789: truncate db/d12/ff2 1173289 0 2024-04-24T23:14:04.907 INFO:tasks.workunit.client.1.smithi040.stdout:3/771: write d1/d2d/fc1 [6940985,85605] 0 2024-04-24T23:14:04.908 INFO:tasks.workunit.client.1.smithi040.stdout:4/681: chown da/d78/f91 1796 1 2024-04-24T23:14:04.911 INFO:tasks.workunit.client.1.smithi040.stdout:1/703: getdents da/d12/d14 0 2024-04-24T23:14:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:04 smithi040 ceph-mon[33128]: pgmap v35: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 65 MiB/s rd, 178 MiB/s wr, 378 op/s 2024-04-24T23:14:04.912 INFO:tasks.workunit.client.1.smithi040.stdout:2/777: symlink da/d1c/d37/d39/d77/d108/d10c/l11a 0 2024-04-24T23:14:04.914 INFO:tasks.workunit.client.1.smithi040.stdout:0/782: link df/d1e/da7/c93 df/d1e/d20/d21/d41/d8a/df3/df8/d79/cff 0 2024-04-24T23:14:04.917 INFO:tasks.workunit.client.1.smithi040.stdout:1/704: creat da/d12/d18/d2a/d41/ff3 x:0 0 0 2024-04-24T23:14:04.917 INFO:tasks.workunit.client.1.smithi040.stdout:8/768: dwrite d0/d8/d66/f78 [0,4194304] 0 2024-04-24T23:14:04.918 INFO:tasks.workunit.client.1.smithi040.stdout:3/772: truncate d1/d2d/de/f34 756114 0 2024-04-24T23:14:04.921 INFO:tasks.workunit.client.1.smithi040.stdout:5/651: write d2/d2f/d14/f86 [4918906,103742] 0 2024-04-24T23:14:04.923 INFO:tasks.workunit.client.1.smithi040.stdout:6/690: rename d7/d1a/d20/d51/daa/lb9 to d7/d1a/d20/d51/d1e/d5d/dbb/ldd 0 2024-04-24T23:14:04.927 INFO:tasks.workunit.client.1.smithi040.stdout:6/691: stat d7/d1a/d20/d8c/dd5 0 2024-04-24T23:14:04.927 INFO:tasks.workunit.client.1.smithi040.stdout:7/790: truncate db/f111 1535254 0 2024-04-24T23:14:04.927 INFO:tasks.workunit.client.1.smithi040.stdout:4/682: dwrite da/d3f/d6b/d2b/f35 [0,4194304] 0 2024-04-24T23:14:04.928 INFO:tasks.workunit.client.1.smithi040.stdout:1/705: symlink da/d12/d18/d2a/d32/db3/lf4 0 2024-04-24T23:14:04.928 INFO:tasks.workunit.client.1.smithi040.stdout:2/778: mkdir da/d1c/d37/d39/d3f/d52/d78/df5/d11b 0 2024-04-24T23:14:04.931 INFO:tasks.workunit.client.1.smithi040.stdout:5/652: rmdir d2/d7/d8/d1f 39 2024-04-24T23:14:04.931 INFO:tasks.workunit.client.1.smithi040.stdout:0/783: dwrite df/d1e/d20/d61/f86 [0,4194304] 0 2024-04-24T23:14:04.935 INFO:tasks.workunit.client.1.smithi040.stdout:5/653: chown d2/d7/d8/db/d1e/l60 2577488 1 2024-04-24T23:14:04.936 INFO:tasks.workunit.client.1.smithi040.stdout:6/692: stat d7/de/d84/ca7 0 2024-04-24T23:14:04.938 INFO:tasks.workunit.client.1.smithi040.stdout:1/706: read da/d12/d14/d58/fc2 [7065824,66801] 0 2024-04-24T23:14:04.939 INFO:tasks.workunit.client.1.smithi040.stdout:2/779: mknod da/d1c/d37/d39/d3f/d52/c11c 0 2024-04-24T23:14:04.944 INFO:tasks.workunit.client.1.smithi040.stdout:6/693: fsync d7/de/f9e 0 2024-04-24T23:14:04.945 INFO:tasks.workunit.client.1.smithi040.stdout:8/769: getdents d0/d16/d1b/d58 0 2024-04-24T23:14:04.947 INFO:tasks.workunit.client.1.smithi040.stdout:1/707: unlink da/d12/d18/d2a/d32/db3/lf4 0 2024-04-24T23:14:04.948 INFO:tasks.workunit.client.1.smithi040.stdout:4/683: creat da/d3f/d40/d41/d8e/ff0 x:0 0 0 2024-04-24T23:14:04.951 INFO:tasks.workunit.client.1.smithi040.stdout:8/770: mkdir d0/d52/d9b/d101 0 2024-04-24T23:14:04.954 INFO:tasks.workunit.client.1.smithi040.stdout:1/708: mkdir da/d35/d38/d47/dbc/de5/df5 0 2024-04-24T23:14:04.956 INFO:tasks.workunit.client.1.smithi040.stdout:6/694: getdents d7/d1a/d20/d51/d1e/d46 0 2024-04-24T23:14:04.957 INFO:tasks.workunit.client.1.smithi040.stdout:4/684: dwrite da/d3f/d40/d41/d60/f42 [0,4194304] 0 2024-04-24T23:14:04.957 INFO:tasks.workunit.client.1.smithi040.stdout:0/784: write f6 [4065767,44621] 0 2024-04-24T23:14:04.959 INFO:tasks.workunit.client.1.smithi040.stdout:3/773: dread d1/fd [4194304,4194304] 0 2024-04-24T23:14:04.959 INFO:tasks.workunit.client.1.smithi040.stdout:5/654: truncate d2/d2f/d14/f61 773082 0 2024-04-24T23:14:04.962 INFO:tasks.workunit.client.1.smithi040.stdout:4/685: read - da/d45/d48/d6e/d70/fbe zero size 2024-04-24T23:14:04.965 INFO:tasks.workunit.client.1.smithi040.stdout:7/791: dwrite db/d2f/ff8 [0,4194304] 0 2024-04-24T23:14:04.967 INFO:tasks.workunit.client.1.smithi040.stdout:1/709: rename da/ce to da/d12/d14/d25/cf6 0 2024-04-24T23:14:04.969 INFO:tasks.workunit.client.1.smithi040.stdout:2/780: dread da/d10/d15/f32 [0,4194304] 0 2024-04-24T23:14:04.970 INFO:tasks.workunit.client.1.smithi040.stdout:4/686: creat da/d3f/d40/d47/d75/daa/ff1 x:0 0 0 2024-04-24T23:14:04.971 INFO:tasks.workunit.client.1.smithi040.stdout:2/781: chown da/d1c/d37/d39/d3f/db2 2870 1 2024-04-24T23:14:04.972 INFO:tasks.workunit.client.1.smithi040.stdout:0/785: creat df/d12/d49/f100 x:0 0 0 2024-04-24T23:14:04.977 INFO:tasks.workunit.client.1.smithi040.stdout:1/710: symlink da/d12/d18/d2a/d8b/dd3/lf7 0 2024-04-24T23:14:04.978 INFO:tasks.workunit.client.1.smithi040.stdout:4/687: creat da/d3f/d40/d47/ff2 x:0 0 0 2024-04-24T23:14:04.979 INFO:tasks.workunit.client.1.smithi040.stdout:6/695: write d7/de/d84/f91 [70857,113242] 0 2024-04-24T23:14:04.980 INFO:tasks.workunit.client.1.smithi040.stdout:2/782: rename da/d29/d55/c86 to da/d1c/d37/d102/c11d 0 2024-04-24T23:14:04.982 INFO:tasks.workunit.client.1.smithi040.stdout:7/792: link db/d12/d20/d52/d6c/d8f/da7/ffe db/d2f/d37/f120 0 2024-04-24T23:14:04.983 INFO:tasks.workunit.client.1.smithi040.stdout:3/774: write d1/d2a/d6a/f3a [1833184,30993] 0 2024-04-24T23:14:04.984 INFO:tasks.workunit.client.1.smithi040.stdout:5/655: creat d2/d7/d8/d1f/d57/d27/fd7 x:0 0 0 2024-04-24T23:14:04.985 INFO:tasks.workunit.client.1.smithi040.stdout:4/688: mkdir da/d3f/d6b/d3c/d69/df3 0 2024-04-24T23:14:04.986 INFO:tasks.workunit.client.1.smithi040.stdout:4/689: chown da/d78/fa7 163359244 1 2024-04-24T23:14:04.986 INFO:tasks.workunit.client.1.smithi040.stdout:2/783: creat da/d1c/d37/f11e x:0 0 0 2024-04-24T23:14:04.988 INFO:tasks.workunit.client.1.smithi040.stdout:2/784: chown da/d1c/d37/ld1 15149 1 2024-04-24T23:14:04.988 INFO:tasks.workunit.client.1.smithi040.stdout:6/696: rename d7/d1a/d20/d51/d1e/d5d/dbb to d7/d1a/d20/d51/d2f/dba/dde 0 2024-04-24T23:14:04.990 INFO:tasks.workunit.client.1.smithi040.stdout:7/793: mkdir db/d43/db9/dd4/d101/d121 0 2024-04-24T23:14:04.991 INFO:tasks.workunit.client.1.smithi040.stdout:6/697: dread d7/d1a/d9f/fb4 [0,4194304] 0 2024-04-24T23:14:04.992 INFO:tasks.workunit.client.1.smithi040.stdout:3/775: truncate d1/d9/d15/d45/f4b 217828 0 2024-04-24T23:14:04.993 INFO:tasks.workunit.client.1.smithi040.stdout:5/656: truncate d2/d7/d8/db/f96 376057 0 2024-04-24T23:14:04.994 INFO:tasks.workunit.client.1.smithi040.stdout:4/690: mkdir da/d3f/d40/d47/d75/d7c/df4 0 2024-04-24T23:14:04.995 INFO:tasks.workunit.client.1.smithi040.stdout:2/785: mknod da/d10/d4d/d6c/c11f 0 2024-04-24T23:14:04.996 INFO:tasks.workunit.client.1.smithi040.stdout:4/691: truncate da/d45/d48/d6e/d70/fcb 158893 0 2024-04-24T23:14:04.997 INFO:tasks.workunit.client.1.smithi040.stdout:4/692: stat da/d3f/d6b/d3c/d69/fa1 0 2024-04-24T23:14:04.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:04 smithi031 ceph-mon[26411]: pgmap v35: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 65 MiB/s rd, 178 MiB/s wr, 378 op/s 2024-04-24T23:14:05.000 INFO:tasks.workunit.client.1.smithi040.stdout:2/786: creat da/d1c/d37/d39/d3f/d52/f120 x:0 0 0 2024-04-24T23:14:05.002 INFO:tasks.workunit.client.1.smithi040.stdout:0/786: link df/d12/d1a/l35 df/d1e/d20/l101 0 2024-04-24T23:14:05.003 INFO:tasks.workunit.client.1.smithi040.stdout:6/698: dwrite d7/d1a/d20/d51/d1e/f72 [0,4194304] 0 2024-04-24T23:14:05.006 INFO:tasks.workunit.client.1.smithi040.stdout:1/711: getdents da/d12/d14 0 2024-04-24T23:14:05.009 INFO:tasks.workunit.client.1.smithi040.stdout:2/787: readlink da/d1c/d4f/l9e 0 2024-04-24T23:14:05.012 INFO:tasks.workunit.client.1.smithi040.stdout:3/776: write d1/f16 [4849905,3052] 0 2024-04-24T23:14:05.014 INFO:tasks.workunit.client.1.smithi040.stdout:3/777: fdatasync d1/d2a/d6a/f35 0 2024-04-24T23:14:05.014 INFO:tasks.workunit.client.1.smithi040.stdout:3/778: read - d1/d2d/de/d30/f58 zero size 2024-04-24T23:14:05.017 INFO:tasks.workunit.client.1.smithi040.stdout:1/712: creat da/d12/d18/d2a/d32/ff8 x:0 0 0 2024-04-24T23:14:05.017 INFO:tasks.workunit.client.1.smithi040.stdout:7/794: write db/d2f/f35 [1206882,87272] 0 2024-04-24T23:14:05.017 INFO:tasks.workunit.client.1.smithi040.stdout:1/713: chown da/d12/d18/cde 1249947345 1 2024-04-24T23:14:05.017 INFO:tasks.workunit.client.1.smithi040.stdout:5/657: dread d2/d7/d8/f1b [0,4194304] 0 2024-04-24T23:14:05.018 INFO:tasks.workunit.client.1.smithi040.stdout:2/788: rmdir da/d1c/d37/d39/d3f/d52 39 2024-04-24T23:14:05.018 INFO:tasks.workunit.client.1.smithi040.stdout:1/714: dread - da/d12/d14/fdd zero size 2024-04-24T23:14:05.021 INFO:tasks.workunit.client.1.smithi040.stdout:3/779: creat d1/d2d/d44/f108 x:0 0 0 2024-04-24T23:14:05.022 INFO:tasks.workunit.client.1.smithi040.stdout:7/795: mknod db/d12/d3f/c122 0 2024-04-24T23:14:05.024 INFO:tasks.workunit.client.1.smithi040.stdout:5/658: unlink d2/d7/d8/db/da2/fac 0 2024-04-24T23:14:05.025 INFO:tasks.workunit.client.1.smithi040.stdout:6/699: creat d7/d1a/d20/d51/fdf x:0 0 0 2024-04-24T23:14:05.030 INFO:tasks.workunit.client.1.smithi040.stdout:2/789: dwrite da/d1c/d37/fc4 [0,4194304] 0 2024-04-24T23:14:05.030 INFO:tasks.workunit.client.1.smithi040.stdout:4/693: dwrite da/d3f/d40/d41/d60/f34 [0,4194304] 0 2024-04-24T23:14:05.030 INFO:tasks.workunit.client.1.smithi040.stdout:0/787: dread df/d12/d1a/d71/f28 [4194304,4194304] 0 2024-04-24T23:14:05.032 INFO:tasks.workunit.client.1.smithi040.stdout:4/694: fsync da/d45/d58/fa6 0 2024-04-24T23:14:05.032 INFO:tasks.workunit.client.1.smithi040.stdout:2/790: dread - da/d81/de6/fcd zero size 2024-04-24T23:14:05.035 INFO:tasks.workunit.client.1.smithi040.stdout:9/743: sync 2024-04-24T23:14:05.051 INFO:tasks.workunit.client.1.smithi040.stdout:3/780: write d1/d2a/d6a/fa5 [437375,94571] 0 2024-04-24T23:14:05.054 INFO:tasks.workunit.client.1.smithi040.stdout:5/659: rmdir d2/dae 0 2024-04-24T23:14:05.056 INFO:tasks.workunit.client.1.smithi040.stdout:4/695: write da/d3f/d40/d41/d8e/dd3/fdb [55880,123978] 0 2024-04-24T23:14:05.058 INFO:tasks.workunit.client.1.smithi040.stdout:7/796: dread db/d12/d3f/d7a/fe1 [4194304,4194304] 0 2024-04-24T23:14:05.058 INFO:tasks.workunit.client.1.smithi040.stdout:9/744: mknod d7/da/d17/d40/d55/d2f/d1f/c109 0 2024-04-24T23:14:05.059 INFO:tasks.workunit.client.1.smithi040.stdout:3/781: write d1/d9/d15/d45/daa/fdf [450473,27080] 0 2024-04-24T23:14:05.060 INFO:tasks.workunit.client.1.smithi040.stdout:1/715: dwrite f2 [4194304,4194304] 0 2024-04-24T23:14:05.062 INFO:tasks.workunit.client.1.smithi040.stdout:0/788: rename df/d12/d58/c91 to df/d12/d58/dc3/c102 0 2024-04-24T23:14:05.063 INFO:tasks.workunit.client.1.smithi040.stdout:4/696: rmdir da 39 2024-04-24T23:14:05.063 INFO:tasks.workunit.client.1.smithi040.stdout:2/791: mknod da/d1c/d37/d39/d3f/d52/d78/dc7/dcc/c121 0 2024-04-24T23:14:05.063 INFO:tasks.workunit.client.1.smithi040.stdout:1/716: chown da/d12/d14/ldf 4802926 1 2024-04-24T23:14:05.065 INFO:tasks.workunit.client.1.smithi040.stdout:5/660: creat d2/d7/d8/d1f/d90/dcb/fd8 x:0 0 0 2024-04-24T23:14:05.065 INFO:tasks.workunit.client.1.smithi040.stdout:3/782: symlink d1/d2a/d6a/d94/l109 0 2024-04-24T23:14:05.068 INFO:tasks.workunit.client.1.smithi040.stdout:7/797: symlink db/d12/d62/d103/d108/l123 0 2024-04-24T23:14:05.068 INFO:tasks.workunit.client.1.smithi040.stdout:1/717: write da/d12/d14/f24 [8851556,49659] 0 2024-04-24T23:14:05.071 INFO:tasks.workunit.client.1.smithi040.stdout:7/798: read db/d12/d20/d52/d44/f8d [3634139,121183] 0 2024-04-24T23:14:05.072 INFO:tasks.workunit.client.1.smithi040.stdout:2/792: mknod da/d1c/d37/d39/d3f/c122 0 2024-04-24T23:14:05.075 INFO:tasks.workunit.client.1.smithi040.stdout:1/718: mknod da/d12/d18/d2a/d4d/cf9 0 2024-04-24T23:14:05.075 INFO:tasks.workunit.client.1.smithi040.stdout:7/799: dread db/f1e [0,4194304] 0 2024-04-24T23:14:05.077 INFO:tasks.workunit.client.1.smithi040.stdout:2/793: unlink da/d29/ffb 0 2024-04-24T23:14:05.078 INFO:tasks.workunit.client.1.smithi040.stdout:4/697: creat da/d3f/d40/ff5 x:0 0 0 2024-04-24T23:14:05.078 INFO:tasks.workunit.client.1.smithi040.stdout:0/789: getdents df/d1e/d20/d21/d41/d8a/df3/df8/da3/d8e 0 2024-04-24T23:14:05.079 INFO:tasks.workunit.client.1.smithi040.stdout:9/745: dwrite d7/da/d17/d40/d55/d2f/f6f [4194304,4194304] 0 2024-04-24T23:14:05.079 INFO:tasks.workunit.client.1.smithi040.stdout:1/719: rmdir da/d12/d18/d2a/d8b 39 2024-04-24T23:14:05.080 INFO:tasks.workunit.client.1.smithi040.stdout:0/790: readlink df/d12/d58/d5b/l98 0 2024-04-24T23:14:05.082 INFO:tasks.workunit.client.1.smithi040.stdout:2/794: unlink da/d10/d15/f30 0 2024-04-24T23:14:05.083 INFO:tasks.workunit.client.1.smithi040.stdout:4/698: stat da/d1f/c4a 0 2024-04-24T23:14:05.086 INFO:tasks.workunit.client.1.smithi040.stdout:2/795: fdatasync da/d1c/d37/fc4 0 2024-04-24T23:14:05.092 INFO:tasks.workunit.client.1.smithi040.stdout:9/746: write d7/da/d17/d40/d55/d2f/d1f/f5a [4780234,101102] 0 2024-04-24T23:14:05.094 INFO:tasks.workunit.client.1.smithi040.stdout:4/699: unlink da/d3f/d40/d41/d8e/lbc 0 2024-04-24T23:14:05.096 INFO:tasks.workunit.client.1.smithi040.stdout:7/800: dwrite db/d12/d3f/d7a/fca [0,4194304] 0 2024-04-24T23:14:05.099 INFO:tasks.workunit.client.1.smithi040.stdout:1/720: dwrite da/d12/d18/d2a/d41/f62 [0,4194304] 0 2024-04-24T23:14:05.102 INFO:tasks.workunit.client.1.smithi040.stdout:3/783: dwrite d1/d2d/de/d30/d37/fc6 [0,4194304] 0 2024-04-24T23:14:05.105 INFO:tasks.workunit.client.1.smithi040.stdout:9/747: fdatasync d7/f37 0 2024-04-24T23:14:05.109 INFO:tasks.workunit.client.1.smithi040.stdout:0/791: dwrite df/d14/ffe [0,4194304] 0 2024-04-24T23:14:05.112 INFO:tasks.workunit.client.1.smithi040.stdout:2/796: dwrite da/d1c/d37/d39/d3f/d52/f111 [0,4194304] 0 2024-04-24T23:14:05.147 INFO:tasks.workunit.client.1.smithi040.stdout:8/771: sync 2024-04-24T23:14:05.162 INFO:tasks.workunit.client.1.smithi040.stdout:0/792: unlink df/d1e/da7/cfb 0 2024-04-24T23:14:05.167 INFO:tasks.workunit.client.1.smithi040.stdout:1/721: dwrite da/f71 [8388608,4194304] 0 2024-04-24T23:14:05.167 INFO:tasks.workunit.client.1.smithi040.stdout:4/700: dwrite da/d3f/d40/d41/d8e/dd3/fdb [0,4194304] 0 2024-04-24T23:14:05.170 INFO:tasks.workunit.client.1.smithi040.stdout:8/772: write d0/d7e/dc1/d9e/d90/d91/f97 [4190430,86835] 0 2024-04-24T23:14:05.173 INFO:tasks.workunit.client.1.smithi040.stdout:3/784: dwrite d1/d9/d18/f66 [0,4194304] 0 2024-04-24T23:14:05.173 INFO:tasks.workunit.client.1.smithi040.stdout:4/701: chown da/d3f/d40/d41/la5 30553 1 2024-04-24T23:14:05.179 INFO:tasks.workunit.client.1.smithi040.stdout:1/722: dwrite da/d12/d18/d2a/d4d/feb [0,4194304] 0 2024-04-24T23:14:05.181 INFO:tasks.workunit.client.1.smithi040.stdout:3/785: dwrite d1/d2a/d6a/fc9 [0,4194304] 0 2024-04-24T23:14:05.189 INFO:tasks.workunit.client.1.smithi040.stdout:3/786: dwrite d1/d9/d15/d45/daa/ff5 [0,4194304] 0 2024-04-24T23:14:05.208 INFO:tasks.workunit.client.1.smithi040.stdout:9/748: rename d7/da/l24 to d7/da/l10a 0 2024-04-24T23:14:05.209 INFO:tasks.workunit.client.1.smithi040.stdout:9/749: stat d7/da/d17/d40/d55/d2f/d1f/d31/d3b/c81 0 2024-04-24T23:14:05.211 INFO:tasks.workunit.client.1.smithi040.stdout:9/750: write d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f89 [536900,88325] 0 2024-04-24T23:14:05.212 INFO:tasks.workunit.client.1.smithi040.stdout:9/751: stat d7/da/d17/d40/d55/d2f/df7/fe9 0 2024-04-24T23:14:05.221 INFO:tasks.workunit.client.1.smithi040.stdout:8/773: unlink d0/d16/d19/d71/d76/l99 0 2024-04-24T23:14:05.222 INFO:tasks.workunit.client.1.smithi040.stdout:8/774: chown d0/d38/d4c/da3/fe8 6108376 1 2024-04-24T23:14:05.225 INFO:tasks.workunit.client.1.smithi040.stdout:8/775: dread d0/d8/dc/d34/f3c [0,4194304] 0 2024-04-24T23:14:05.229 INFO:tasks.workunit.client.1.smithi040.stdout:1/723: stat da/d12/d18/c57 0 2024-04-24T23:14:05.229 INFO:tasks.workunit.client.1.smithi040.stdout:8/776: dread - d0/d16/d1b/d58/f100 zero size 2024-04-24T23:14:05.231 INFO:tasks.workunit.client.1.smithi040.stdout:0/793: symlink df/d12/d1a/d71/l103 0 2024-04-24T23:14:05.233 INFO:tasks.workunit.client.1.smithi040.stdout:9/752: unlink d7/da/d17/d40/d55/c104 0 2024-04-24T23:14:05.235 INFO:tasks.workunit.client.1.smithi040.stdout:4/702: mknod da/d3f/d6b/cf6 0 2024-04-24T23:14:05.236 INFO:tasks.workunit.client.1.smithi040.stdout:8/777: fdatasync d0/d28/f39 0 2024-04-24T23:14:05.236 INFO:tasks.workunit.client.1.smithi040.stdout:6/700: sync 2024-04-24T23:14:05.240 INFO:tasks.workunit.client.1.smithi040.stdout:9/753: dwrite d7/da/d17/d40/d55/d2f/df7/d6c/d8c/fde [0,4194304] 0 2024-04-24T23:14:05.240 INFO:tasks.workunit.client.1.smithi040.stdout:0/794: rmdir df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dcf 39 2024-04-24T23:14:05.240 INFO:tasks.workunit.client.1.smithi040.stdout:8/778: truncate d0/d16/d1b/d47/fdc 769900 0 2024-04-24T23:14:05.247 INFO:tasks.workunit.client.1.smithi040.stdout:4/703: mknod da/d3f/d40/d41/d8e/dd3/cf7 0 2024-04-24T23:14:05.248 INFO:tasks.workunit.client.1.smithi040.stdout:4/704: dread - da/d3f/d6b/d3c/fc5 zero size 2024-04-24T23:14:05.249 INFO:tasks.workunit.client.1.smithi040.stdout:4/705: fdatasync da/d3f/d40/d41/d60/f24 0 2024-04-24T23:14:05.250 INFO:tasks.workunit.client.1.smithi040.stdout:3/787: link d1/d2d/d44/lbb d1/d2d/de/l10a 0 2024-04-24T23:14:05.253 INFO:tasks.workunit.client.1.smithi040.stdout:0/795: chown df/d12/d49/lf4 4296 1 2024-04-24T23:14:05.254 INFO:tasks.workunit.client.1.smithi040.stdout:9/754: dread d7/d5e/da1/fd0 [0,4194304] 0 2024-04-24T23:14:05.255 INFO:tasks.workunit.client.1.smithi040.stdout:8/779: write d0/d8/dc/d34/f3c [580120,56431] 0 2024-04-24T23:14:05.255 INFO:tasks.workunit.client.1.smithi040.stdout:4/706: read - da/d45/fac zero size 2024-04-24T23:14:05.256 INFO:tasks.workunit.client.1.smithi040.stdout:3/788: creat d1/d9/d15/d45/daa/f10b x:0 0 0 2024-04-24T23:14:05.264 INFO:tasks.workunit.client.1.smithi040.stdout:6/701: link d7/d1a/d20/d51/d1e/d46/d69/d87/cc9 d7/de/d89/ce0 0 2024-04-24T23:14:05.265 INFO:tasks.workunit.client.1.smithi040.stdout:4/707: creat da/d3f/d40/d9b/ff8 x:0 0 0 2024-04-24T23:14:05.266 INFO:tasks.workunit.client.1.smithi040.stdout:4/708: chown da/d78/f8c 107900981 1 2024-04-24T23:14:05.267 INFO:tasks.workunit.client.1.smithi040.stdout:2/797: write da/d1c/f27 [1739293,92349] 0 2024-04-24T23:14:05.270 INFO:tasks.workunit.client.1.smithi040.stdout:3/789: link d1/d2a/d6a/c76 d1/d2d/de/d30/dde/c10c 0 2024-04-24T23:14:05.272 INFO:tasks.workunit.client.1.smithi040.stdout:4/709: dwrite da/d3f/d6b/fb1 [0,4194304] 0 2024-04-24T23:14:05.273 INFO:tasks.workunit.client.1.smithi040.stdout:8/780: getdents d0/d38/d4c/dd5 0 2024-04-24T23:14:05.274 INFO:tasks.workunit.client.1.smithi040.stdout:4/710: chown da/d45/d48/d6e/d70 24529 1 2024-04-24T23:14:05.274 INFO:tasks.workunit.client.1.smithi040.stdout:8/781: stat d0/d8/dc/d34/c65 0 2024-04-24T23:14:05.275 INFO:tasks.workunit.client.1.smithi040.stdout:2/798: rename da/d1c/d37/d39/d3f/db2/cf9 to da/d10/d4d/dd7/c123 0 2024-04-24T23:14:05.275 INFO:tasks.workunit.client.1.smithi040.stdout:8/782: write d0/d52/fb3 [852966,73316] 0 2024-04-24T23:14:05.278 INFO:tasks.workunit.client.1.smithi040.stdout:7/801: sync 2024-04-24T23:14:05.278 INFO:tasks.workunit.client.1.smithi040.stdout:5/661: sync 2024-04-24T23:14:05.280 INFO:tasks.workunit.client.1.smithi040.stdout:4/711: read - da/d3f/d6b/d3c/fd7 zero size 2024-04-24T23:14:05.281 INFO:tasks.workunit.client.1.smithi040.stdout:7/802: fdatasync db/d2f/d37/fa6 0 2024-04-24T23:14:05.281 INFO:tasks.workunit.client.1.smithi040.stdout:1/724: write da/d12/d18/d2a/d41/f70 [691488,1202] 0 2024-04-24T23:14:05.282 INFO:tasks.workunit.client.1.smithi040.stdout:2/799: truncate da/d1c/d37/d39/d3f/d52/d78/dc7/fee 140052 0 2024-04-24T23:14:05.283 INFO:tasks.workunit.client.1.smithi040.stdout:1/725: chown da/ff 345 1 2024-04-24T23:14:05.284 INFO:tasks.workunit.client.1.smithi040.stdout:7/803: fsync db/d12/d73/fbe 0 2024-04-24T23:14:05.284 INFO:tasks.workunit.client.1.smithi040.stdout:5/662: chown d2/d7/d8/d1f/d57/d27/d3c/da4/fd2 1304893 1 2024-04-24T23:14:05.285 INFO:tasks.workunit.client.1.smithi040.stdout:1/726: dread - da/d35/d38/fda zero size 2024-04-24T23:14:05.288 INFO:tasks.workunit.client.1.smithi040.stdout:7/804: truncate db/d2f/d37/f11f 595123 0 2024-04-24T23:14:05.290 INFO:tasks.workunit.client.1.smithi040.stdout:6/702: write d7/d1a/f53 [137201,29314] 0 2024-04-24T23:14:05.291 INFO:tasks.workunit.client.1.smithi040.stdout:5/663: write d2/d7/d8/d1f/d57/d27/d3c/d9b/fd5 [682067,107363] 0 2024-04-24T23:14:05.292 INFO:tasks.workunit.client.1.smithi040.stdout:8/783: write d0/d16/f57 [892195,2807] 0 2024-04-24T23:14:05.292 INFO:tasks.workunit.client.1.smithi040.stdout:4/712: write da/d78/fa7 [488195,24615] 0 2024-04-24T23:14:05.300 INFO:tasks.workunit.client.1.smithi040.stdout:4/713: chown da/fb 810 1 2024-04-24T23:14:05.302 INFO:tasks.workunit.client.1.smithi040.stdout:0/796: dwrite df/d12/d4b/f6a [0,4194304] 0 2024-04-24T23:14:05.305 INFO:tasks.workunit.client.1.smithi040.stdout:9/755: dwrite d7/da/d17/d40/d55/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:14:05.306 INFO:tasks.workunit.client.1.smithi040.stdout:5/664: write d2/d7/d8/d1f/d57/d27/d65/f5a [441202,65431] 0 2024-04-24T23:14:05.308 INFO:tasks.workunit.client.1.smithi040.stdout:2/800: dwrite da/d1c/d37/d102/f103 [0,4194304] 0 2024-04-24T23:14:05.308 INFO:tasks.workunit.client.1.smithi040.stdout:4/714: fdatasync da/d3f/d40/d41/d8e/ff0 0 2024-04-24T23:14:05.310 INFO:tasks.workunit.client.1.smithi040.stdout:3/790: dread d1/d2d/d44/f52 [0,4194304] 0 2024-04-24T23:14:05.313 INFO:tasks.workunit.client.1.smithi040.stdout:4/715: chown da/d3f/d6b/d2b/faf 1 1 2024-04-24T23:14:05.313 INFO:tasks.workunit.client.1.smithi040.stdout:8/784: dwrite d0/d8/d55/f83 [0,4194304] 0 2024-04-24T23:14:05.315 INFO:tasks.workunit.client.1.smithi040.stdout:4/716: dread - da/d45/d48/d6e/d70/fee zero size 2024-04-24T23:14:05.324 INFO:tasks.workunit.client.1.smithi040.stdout:8/785: dwrite d0/d16/d19/fb5 [0,4194304] 0 2024-04-24T23:14:05.405 INFO:tasks.workunit.client.1.smithi040.stdout:1/727: symlink da/d35/d38/d47/dbc/de5/lfa 0 2024-04-24T23:14:05.408 INFO:tasks.workunit.client.1.smithi040.stdout:6/703: symlink d7/de/d4e/le1 0 2024-04-24T23:14:05.411 INFO:tasks.workunit.client.1.smithi040.stdout:0/797: creat df/d12/d58/d5b/f104 x:0 0 0 2024-04-24T23:14:05.412 INFO:tasks.workunit.client.1.smithi040.stdout:0/798: chown df/d1e/f60 119679 1 2024-04-24T23:14:05.413 INFO:tasks.workunit.client.1.smithi040.stdout:5/665: creat d2/d7/d8/d6d/d89/fd9 x:0 0 0 2024-04-24T23:14:05.442 INFO:tasks.workunit.client.1.smithi040.stdout:8/786: sync 2024-04-24T23:14:05.475 INFO:tasks.workunit.client.1.smithi040.stdout:3/791: dread d1/d2d/fcd [0,4194304] 0 2024-04-24T23:14:05.476 INFO:tasks.workunit.client.1.smithi040.stdout:3/792: write d1/d9/d15/f1b [7913913,20446] 0 2024-04-24T23:14:05.477 INFO:tasks.workunit.client.1.smithi040.stdout:3/793: write d1/fd [8121957,129787] 0 2024-04-24T23:14:05.482 INFO:tasks.workunit.client.1.smithi040.stdout:7/805: getdents db/d12/d62/d109 0 2024-04-24T23:14:05.484 INFO:tasks.workunit.client.1.smithi040.stdout:1/728: creat da/d12/d14/d25/d5e/ffb x:0 0 0 2024-04-24T23:14:05.485 INFO:tasks.workunit.client.1.smithi040.stdout:6/704: creat d7/d1a/db5/fe2 x:0 0 0 2024-04-24T23:14:05.488 INFO:tasks.workunit.client.1.smithi040.stdout:0/799: dread - df/de1/db6/fbc zero size 2024-04-24T23:14:05.490 INFO:tasks.workunit.client.1.smithi040.stdout:1/729: dwrite da/d12/d14/d58/ff0 [0,4194304] 0 2024-04-24T23:14:05.544 INFO:tasks.workunit.client.1.smithi040.stdout:3/794: rmdir d1/d2a/d8d 39 2024-04-24T23:14:05.545 INFO:tasks.workunit.client.1.smithi040.stdout:3/795: dread - d1/d9/d2e/dcf/fe6 zero size 2024-04-24T23:14:05.547 INFO:tasks.workunit.client.1.smithi040.stdout:4/717: link da/d3f/d40/d47/f93 da/d3f/d40/d47/d75/d7c/ff9 0 2024-04-24T23:14:05.550 INFO:tasks.workunit.client.1.smithi040.stdout:6/705: chown d7/d1a/d20/d51/d1e/d46/d69/d87/l75 921561 1 2024-04-24T23:14:05.552 INFO:tasks.workunit.client.1.smithi040.stdout:3/796: dwrite d1/d80/dbf/fd5 [0,4194304] 0 2024-04-24T23:14:05.552 INFO:tasks.workunit.client.1.smithi040.stdout:0/800: write df/d1e/d89/fd7 [2415341,39048] 0 2024-04-24T23:14:05.559 INFO:tasks.workunit.client.1.smithi040.stdout:4/718: mknod da/d3f/d40/d41/cfa 0 2024-04-24T23:14:05.560 INFO:tasks.workunit.client.1.smithi040.stdout:6/706: rmdir d7/d1a/d20/d51/d2f/dba/dde 39 2024-04-24T23:14:05.567 INFO:tasks.workunit.client.1.smithi040.stdout:9/756: getdents d7/da/d17/d40/d55/d2f/d1f/d31 0 2024-04-24T23:14:05.572 INFO:tasks.workunit.client.1.smithi040.stdout:0/801: getdents df/d1e/d20/d21/d41/d8a/df3/df8/ddb 0 2024-04-24T23:14:05.581 INFO:tasks.workunit.client.1.smithi040.stdout:4/719: creat da/d3f/d40/d41/d8e/ffb x:0 0 0 2024-04-24T23:14:05.586 INFO:tasks.workunit.client.1.smithi040.stdout:4/720: creat da/d45/d48/d6e/d70/ffc x:0 0 0 2024-04-24T23:14:05.586 INFO:tasks.workunit.client.1.smithi040.stdout:4/721: fsync da/d45/d58/fa6 0 2024-04-24T23:14:05.588 INFO:tasks.workunit.client.1.smithi040.stdout:0/802: getdents df/d1e/da7 0 2024-04-24T23:14:05.595 INFO:tasks.workunit.client.1.smithi040.stdout:2/801: symlink da/d10/d15/l124 0 2024-04-24T23:14:05.596 INFO:tasks.workunit.client.1.smithi040.stdout:2/802: readlink da/d1c/d37/d39/d77/d108/d10c/l11a 0 2024-04-24T23:14:05.596 INFO:tasks.workunit.client.1.smithi040.stdout:2/803: fdatasync da/d81/de6/ff2 0 2024-04-24T23:14:05.692 INFO:tasks.workunit.client.1.smithi040.stdout:6/707: sync 2024-04-24T23:14:05.694 INFO:tasks.workunit.client.1.smithi040.stdout:6/708: mknod d7/d1a/d20/d51/d2a/ce3 0 2024-04-24T23:14:05.698 INFO:tasks.workunit.client.1.smithi040.stdout:6/709: symlink d7/d1a/d20/d51/d1e/d46/d69/le4 0 2024-04-24T23:14:05.700 INFO:tasks.workunit.client.1.smithi040.stdout:6/710: readlink d7/d1a/d56/l59 0 2024-04-24T23:14:05.701 INFO:tasks.workunit.client.1.smithi040.stdout:6/711: dread - d7/d1a/d20/d51/d1e/d46/fa0 zero size 2024-04-24T23:14:05.704 INFO:tasks.workunit.client.1.smithi040.stdout:6/712: creat d7/d1a/d20/d51/d1e/d5d/fe5 x:0 0 0 2024-04-24T23:14:05.706 INFO:tasks.workunit.client.1.smithi040.stdout:6/713: fdatasync d7/d1a/d20/d51/d1e/d46/d69/f8d 0 2024-04-24T23:14:05.712 INFO:tasks.workunit.client.1.smithi040.stdout:6/714: dwrite d7/d1a/d20/d51/d2f/f5b [0,4194304] 0 2024-04-24T23:14:05.714 INFO:tasks.workunit.client.1.smithi040.stdout:7/806: rename db/d2f/f6d to db/d12/d62/d8a/f124 0 2024-04-24T23:14:05.717 INFO:tasks.workunit.client.1.smithi040.stdout:6/715: dread d7/d1a/d20/d51/d1e/fc3 [0,4194304] 0 2024-04-24T23:14:05.718 INFO:tasks.workunit.client.1.smithi040.stdout:6/716: readlink d7/d1a/d20/d51/d1e/l50 0 2024-04-24T23:14:05.718 INFO:tasks.workunit.client.1.smithi040.stdout:6/717: read - d7/d1a/d56/faf zero size 2024-04-24T23:14:05.721 INFO:tasks.workunit.client.1.smithi040.stdout:6/718: creat d7/d1a/d20/d51/daa/fe6 x:0 0 0 2024-04-24T23:14:05.724 INFO:tasks.workunit.client.1.smithi040.stdout:6/719: read d7/d1a/d20/d8c/fcf [999588,98257] 0 2024-04-24T23:14:05.727 INFO:tasks.workunit.client.1.smithi040.stdout:6/720: symlink d7/d1a/dbf/le7 0 2024-04-24T23:14:05.730 INFO:tasks.workunit.client.1.smithi040.stdout:2/804: sync 2024-04-24T23:14:05.731 INFO:tasks.workunit.client.1.smithi040.stdout:2/805: dread - da/d10/d15/daa/fbb zero size 2024-04-24T23:14:05.775 INFO:tasks.workunit.client.1.smithi040.stdout:2/806: chown da/d1c/d37/d39/d79/da3/f100 1664 1 2024-04-24T23:14:05.775 INFO:tasks.workunit.client.1.smithi040.stdout:2/807: symlink da/d1c/d37/d39/d3f/d52/l125 0 2024-04-24T23:14:05.775 INFO:tasks.workunit.client.1.smithi040.stdout:7/807: dread db/d12/d3f/d7a/fe1 [0,4194304] 0 2024-04-24T23:14:05.775 INFO:tasks.workunit.client.1.smithi040.stdout:5/666: dwrite d2/f9e [0,4194304] 0 2024-04-24T23:14:05.775 INFO:tasks.workunit.client.1.smithi040.stdout:5/667: chown d2/d2f/d14/f86 1 1 2024-04-24T23:14:05.775 INFO:tasks.workunit.client.1.smithi040.stdout:7/808: mknod db/d12/d62/d8a/db3/c125 0 2024-04-24T23:14:05.775 INFO:tasks.workunit.client.1.smithi040.stdout:5/668: creat d2/d7/d8/d1f/d90/dcb/fda x:0 0 0 2024-04-24T23:14:05.775 INFO:tasks.workunit.client.1.smithi040.stdout:4/722: truncate da/d3f/d40/d47/f93 654957 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:4/723: creat da/d3f/d8d/ffd x:0 0 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:4/724: chown da/d45/d48/d6e 154086 1 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:3/797: truncate d1/d2a/d6a/f3a 259015 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:9/757: write d7/f37 [684488,65801] 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:2/808: rename da/d10/d4d/d6c/fad to da/d1c/d37/f126 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:3/798: getdents d1/d9/d39 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:3/799: unlink d1/d9/f27 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:3/800: stat d1/d9/d39/ff8 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:3/801: chown d1/d9/d18/f66 227607084 1 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:3/802: write d1/d9/d18/d36/d55/d62/ff4 [618012,5427] 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:9/758: rmdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/df2 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:9/759: stat d7/da/d17/d40/d55/d2f/d1f/d4e 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:2/809: dread da/d1c/d37/d39/d79/da3/f100 [0,4194304] 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:8/787: mkdir d0/d16/d19/d102 0 2024-04-24T23:14:05.776 INFO:tasks.workunit.client.1.smithi040.stdout:2/810: readlink da/d10/d4d/d6c/la5 0 2024-04-24T23:14:05.777 INFO:tasks.workunit.client.1.smithi040.stdout:0/803: truncate df/de1/d6e/fe9 441981 0 2024-04-24T23:14:05.779 INFO:tasks.workunit.client.0.smithi031.stdout:9/987: sync 2024-04-24T23:14:05.779 INFO:tasks.workunit.client.0.smithi031.stdout:5/856: sync 2024-04-24T23:14:05.779 INFO:tasks.workunit.client.1.smithi040.stdout:2/811: creat da/d1c/d37/d8a/f127 x:0 0 0 2024-04-24T23:14:05.780 INFO:tasks.workunit.client.1.smithi040.stdout:0/804: truncate df/d1e/f40 1140669 0 2024-04-24T23:14:05.781 INFO:tasks.workunit.client.1.smithi040.stdout:0/805: readlink df/ld5 0 2024-04-24T23:14:05.783 INFO:tasks.workunit.client.1.smithi040.stdout:3/803: rename d1/d9/d15 to d1/d9/d18/d36/d10d 0 2024-04-24T23:14:05.785 INFO:tasks.workunit.client.0.smithi031.stdout:9/988: creat d2/dae/d14/d1d/d45/d99/d9c/d100/f132 x:0 0 0 2024-04-24T23:14:05.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/857: fsync d0/d6/d3b/d4f/fce 0 2024-04-24T23:14:05.788 INFO:tasks.workunit.client.1.smithi040.stdout:0/806: fdatasync df/d1e/d20/d21/d41/d8a/df3/f87 0 2024-04-24T23:14:05.789 INFO:tasks.workunit.client.1.smithi040.stdout:0/807: chown df/d1e/d20/d61/le4 226396 1 2024-04-24T23:14:05.790 INFO:tasks.workunit.client.0.smithi031.stdout:9/989: mknod d2/d43/d9d/dc0/c133 0 2024-04-24T23:14:05.790 INFO:tasks.workunit.client.1.smithi040.stdout:0/808: chown df/f34 1 1 2024-04-24T23:14:05.794 INFO:tasks.workunit.client.0.smithi031.stdout:5/858: symlink d0/d7/d74/de2/l120 0 2024-04-24T23:14:05.797 INFO:tasks.workunit.client.1.smithi040.stdout:1/730: mkdir da/d12/d18/d2a/dfc 0 2024-04-24T23:14:05.798 INFO:tasks.workunit.client.0.smithi031.stdout:5/859: read d0/d6/f40 [1092662,66681] 0 2024-04-24T23:14:05.799 INFO:tasks.workunit.client.0.smithi031.stdout:5/860: chown d0/d6/d3b/l78 152304179 1 2024-04-24T23:14:05.800 INFO:tasks.workunit.client.0.smithi031.stdout:5/861: chown d0/d6/l3c 685057 1 2024-04-24T23:14:05.800 INFO:tasks.workunit.client.0.smithi031.stdout:9/990: chown d2/dae/d85/fc4 483 1 2024-04-24T23:14:05.800 INFO:tasks.workunit.client.1.smithi040.stdout:2/812: link da/d81/de6/fcd da/d1c/d37/d39/d3f/d52/d78/df5/d11b/f128 0 2024-04-24T23:14:05.802 INFO:tasks.workunit.client.1.smithi040.stdout:3/804: creat d1/d9/d18/f10e x:0 0 0 2024-04-24T23:14:05.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/731: mkdir da/d12/d18/d2a/d4d/dfd 0 2024-04-24T23:14:05.807 INFO:tasks.workunit.client.0.smithi031.stdout:9/991: rename d2/d16/de0 to d2/dae/d14/d78/d84/d4a/d4b/dcb/d134 0 2024-04-24T23:14:05.809 INFO:tasks.workunit.client.0.smithi031.stdout:9/992: dread d2/dae/d14/d78/d84/d4a/f91 [0,4194304] 0 2024-04-24T23:14:05.810 INFO:tasks.workunit.client.1.smithi040.stdout:2/813: rmdir da/d1c/d37/d39/d3f/d52/d78/df5 39 2024-04-24T23:14:05.812 INFO:tasks.workunit.client.1.smithi040.stdout:3/805: mkdir d1/d2d/de/d30/d10f 0 2024-04-24T23:14:05.815 INFO:tasks.workunit.client.0.smithi031.stdout:9/993: mknod d2/dae/d14/d78/d84/c135 0 2024-04-24T23:14:05.815 INFO:tasks.workunit.client.1.smithi040.stdout:6/721: write d7/d1a/d20/d51/d1e/d5d/f93 [719743,118021] 0 2024-04-24T23:14:05.815 INFO:tasks.workunit.client.1.smithi040.stdout:0/809: link df/d12/d1a/d71/f36 df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dbe/f105 0 2024-04-24T23:14:05.816 INFO:tasks.workunit.client.1.smithi040.stdout:6/722: chown d7/de/d84/fd7 86 1 2024-04-24T23:14:05.816 INFO:tasks.workunit.client.1.smithi040.stdout:0/810: dread - df/d1e/d20/d21/d41/feb zero size 2024-04-24T23:14:05.816 INFO:tasks.workunit.client.0.smithi031.stdout:5/862: link d0/d6/d3b/d4b/d1d/d2e/d28/l97 d0/d7/d74/dbd/d112/l121 0 2024-04-24T23:14:05.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/863: read d0/d6/d3b/f68 [2849214,33502] 0 2024-04-24T23:14:05.819 INFO:tasks.workunit.client.0.smithi031.stdout:5/864: chown d0/d6/d32/d6b/f10b 3335949 1 2024-04-24T23:14:05.820 INFO:tasks.workunit.client.1.smithi040.stdout:1/732: creat da/d12/d23/dac/ffe x:0 0 0 2024-04-24T23:14:05.821 INFO:tasks.workunit.client.1.smithi040.stdout:0/811: unlink df/d12/d4b/db8/cce 0 2024-04-24T23:14:05.821 INFO:tasks.workunit.client.0.smithi031.stdout:9/994: mknod d2/dae/d14/d1d/d45/c136 0 2024-04-24T23:14:05.825 INFO:tasks.workunit.client.0.smithi031.stdout:5/865: creat d0/d6/d3b/f122 x:0 0 0 2024-04-24T23:14:05.826 INFO:tasks.workunit.client.1.smithi040.stdout:6/723: rename d7/fd to d7/d1a/d20/d51/d1e/fe8 0 2024-04-24T23:14:05.829 INFO:tasks.workunit.client.0.smithi031.stdout:5/866: creat d0/d7/d56/d7c/d7e/f123 x:0 0 0 2024-04-24T23:14:05.833 INFO:tasks.workunit.client.1.smithi040.stdout:7/809: dwrite db/d12/d62/d8a/fd5 [0,4194304] 0 2024-04-24T23:14:05.834 INFO:tasks.workunit.client.1.smithi040.stdout:0/812: mkdir df/d1e/d20/d21/d41/d8a/df3/df8/da3/d8e/d106 0 2024-04-24T23:14:05.835 INFO:tasks.workunit.client.1.smithi040.stdout:6/724: truncate d7/d1a/d20/d8c/db0/fd2 580227 0 2024-04-24T23:14:05.835 INFO:tasks.workunit.client.0.smithi031.stdout:5/867: creat d0/d6/d3b/d4b/d1d/d2e/dc6/f124 x:0 0 0 2024-04-24T23:14:05.836 INFO:tasks.workunit.client.0.smithi031.stdout:5/868: dread - d0/d6/d3b/d4b/dc/d25/f101 zero size 2024-04-24T23:14:05.836 INFO:tasks.workunit.client.1.smithi040.stdout:5/669: dwrite d2/d7/d8/d64/d6b/f92 [0,4194304] 0 2024-04-24T23:14:05.837 INFO:tasks.workunit.client.1.smithi040.stdout:7/810: readlink db/d12/d20/d52/d4b/le9 0 2024-04-24T23:14:05.839 INFO:tasks.workunit.client.0.smithi031.stdout:9/995: creat d2/d43/f137 x:0 0 0 2024-04-24T23:14:05.840 INFO:tasks.workunit.client.1.smithi040.stdout:4/725: dwrite da/d45/d48/fb4 [0,4194304] 0 2024-04-24T23:14:05.843 INFO:tasks.workunit.client.1.smithi040.stdout:4/726: dread - da/d3f/d40/d41/fd2 zero size 2024-04-24T23:14:05.844 INFO:tasks.workunit.client.0.smithi031.stdout:5/869: mkdir d0/d6/d3b/d4b/dc/de3/d125 0 2024-04-24T23:14:05.850 INFO:tasks.workunit.client.0.smithi031.stdout:5/870: mkdir d0/d6/d3b/d4b/d1d/d2e/d126 0 2024-04-24T23:14:05.854 INFO:tasks.workunit.client.0.smithi031.stdout:9/996: mkdir d2/dae/d138 0 2024-04-24T23:14:05.856 INFO:tasks.workunit.client.1.smithi040.stdout:8/788: dwrite d0/d8/d22/f6a [4194304,4194304] 0 2024-04-24T23:14:05.856 INFO:tasks.workunit.client.1.smithi040.stdout:9/760: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f9d [0,4194304] 0 2024-04-24T23:14:05.859 INFO:tasks.workunit.client.1.smithi040.stdout:9/761: chown d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/def/cfc 1140089 1 2024-04-24T23:14:05.861 INFO:tasks.workunit.client.1.smithi040.stdout:5/670: mknod d2/d7/d8/d1f/d90/cdb 0 2024-04-24T23:14:05.863 INFO:tasks.workunit.client.0.smithi031.stdout:9/997: read d2/d16/f20 [3921407,26570] 0 2024-04-24T23:14:05.866 INFO:tasks.workunit.client.1.smithi040.stdout:7/811: read db/d12/d3f/fa4 [92716,55801] 0 2024-04-24T23:14:05.868 INFO:tasks.workunit.client.1.smithi040.stdout:7/812: read db/d2f/d37/fd8 [202219,89957] 0 2024-04-24T23:14:05.868 INFO:tasks.workunit.client.1.smithi040.stdout:7/813: stat db/d12/d20/d52/f68 0 2024-04-24T23:14:05.870 INFO:tasks.workunit.client.0.smithi031.stdout:2/834: sync 2024-04-24T23:14:05.870 INFO:tasks.workunit.client.0.smithi031.stdout:3/829: sync 2024-04-24T23:14:05.870 INFO:tasks.workunit.client.0.smithi031.stdout:7/923: sync 2024-04-24T23:14:05.870 INFO:tasks.workunit.client.0.smithi031.stdout:6/962: sync 2024-04-24T23:14:05.870 INFO:tasks.workunit.client.0.smithi031.stdout:1/879: sync 2024-04-24T23:14:05.871 INFO:tasks.workunit.client.1.smithi040.stdout:8/789: creat d0/d16/d1b/d47/f103 x:0 0 0 2024-04-24T23:14:05.872 INFO:tasks.workunit.client.1.smithi040.stdout:4/727: dread da/f2f [0,4194304] 0 2024-04-24T23:14:05.872 INFO:tasks.workunit.client.0.smithi031.stdout:2/835: read d5/fba [861589,42506] 0 2024-04-24T23:14:05.873 INFO:tasks.workunit.client.1.smithi040.stdout:6/725: mknod d7/d1a/d20/d51/ce9 0 2024-04-24T23:14:05.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/924: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/f87 100 1 2024-04-24T23:14:05.874 INFO:tasks.workunit.client.1.smithi040.stdout:6/726: dread - d7/de/f9e zero size 2024-04-24T23:14:05.877 INFO:tasks.workunit.client.1.smithi040.stdout:9/762: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/d10b 0 2024-04-24T23:14:05.878 INFO:tasks.workunit.client.1.smithi040.stdout:9/763: fsync d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d 0 2024-04-24T23:14:05.883 INFO:tasks.workunit.client.1.smithi040.stdout:0/813: dread df/d12/d58/dc3/dcd/ffa [0,4194304] 0 2024-04-24T23:14:05.886 INFO:tasks.workunit.client.1.smithi040.stdout:7/814: mknod db/d12/d20/d52/d6c/c126 0 2024-04-24T23:14:05.887 INFO:tasks.workunit.client.1.smithi040.stdout:2/814: truncate da/d1c/d37/d102/f103 2055806 0 2024-04-24T23:14:05.889 INFO:tasks.workunit.client.1.smithi040.stdout:3/806: sync 2024-04-24T23:14:05.889 INFO:tasks.workunit.client.1.smithi040.stdout:8/790: mkdir d0/d16/d19/d71/d76/d104 0 2024-04-24T23:14:05.890 INFO:tasks.workunit.client.0.smithi031.stdout:2/836: truncate d5/d6/f16 4497403 0 2024-04-24T23:14:05.892 INFO:tasks.workunit.client.1.smithi040.stdout:7/815: dwrite db/d12/d20/d52/d4b/fa1 [0,4194304] 0 2024-04-24T23:14:05.893 INFO:tasks.workunit.client.0.smithi031.stdout:7/925: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/d142 0 2024-04-24T23:14:05.894 INFO:tasks.workunit.client.1.smithi040.stdout:7/816: read - db/d43/db9/fe3 zero size 2024-04-24T23:14:05.895 INFO:tasks.workunit.client.1.smithi040.stdout:4/728: rmdir da/d3f/d6b 39 2024-04-24T23:14:05.895 INFO:tasks.workunit.client.0.smithi031.stdout:6/963: dread d13/f3f [0,4194304] 0 2024-04-24T23:14:05.898 INFO:tasks.workunit.client.1.smithi040.stdout:6/727: symlink d7/d1a/d20/d51/d2a/lea 0 2024-04-24T23:14:05.898 INFO:tasks.workunit.client.1.smithi040.stdout:3/807: dwrite d1/d9/d2e/fa1 [0,4194304] 0 2024-04-24T23:14:05.899 INFO:tasks.workunit.client.1.smithi040.stdout:7/817: read - db/d12/d62/d103/f116 zero size 2024-04-24T23:14:05.900 INFO:tasks.workunit.client.1.smithi040.stdout:6/728: chown d7/fae 7348 1 2024-04-24T23:14:05.903 INFO:tasks.workunit.client.0.smithi031.stdout:3/830: dread d6/d14/d71/dad/fb9 [0,4194304] 0 2024-04-24T23:14:05.903 INFO:tasks.workunit.client.1.smithi040.stdout:1/733: dwrite da/d12/d18/d2a/fc7 [0,4194304] 0 2024-04-24T23:14:05.904 INFO:tasks.workunit.client.0.smithi031.stdout:3/831: write d6/df/d16/d2a/d95/d9b/d60/ff9 [2041116,72172] 0 2024-04-24T23:14:05.907 INFO:tasks.workunit.client.0.smithi031.stdout:2/837: rename d5/d8/d45/d5b/d49/dc4/def to d5/d6/d29/d26/db0/dc7/d117/d123 0 2024-04-24T23:14:05.909 INFO:tasks.workunit.client.1.smithi040.stdout:5/671: rename d2/d62/fb5 to d2/d7/d8/db/da2/dcc/fdc 0 2024-04-24T23:14:05.910 INFO:tasks.workunit.client.0.smithi031.stdout:7/926: fdatasync d1/d2/dd/f7d 0 2024-04-24T23:14:05.912 INFO:tasks.workunit.client.1.smithi040.stdout:6/729: dwrite d7/d1a/db5/fe2 [0,4194304] 0 2024-04-24T23:14:05.915 INFO:tasks.workunit.client.0.smithi031.stdout:6/964: creat d13/d1b/d32/d9c/ddb/f144 x:0 0 0 2024-04-24T23:14:05.921 INFO:tasks.workunit.client.1.smithi040.stdout:2/815: creat da/d10/d4d/dd7/f129 x:0 0 0 2024-04-24T23:14:05.924 INFO:tasks.workunit.client.0.smithi031.stdout:2/838: dread d5/d6/d29/d26/db0/dc2/dad/dbe/f116 [0,4194304] 0 2024-04-24T23:14:05.926 INFO:tasks.workunit.client.0.smithi031.stdout:3/832: mkdir d6/df/d16/d2a/d95/dff 0 2024-04-24T23:14:05.936 INFO:tasks.workunit.client.1.smithi040.stdout:7/818: truncate db/d12/d73/f9c 4566674 0 2024-04-24T23:14:05.937 INFO:tasks.workunit.client.0.smithi031.stdout:6/965: creat d13/d15/d1d/d23/d44/d47/d69/f145 x:0 0 0 2024-04-24T23:14:05.938 INFO:tasks.workunit.client.1.smithi040.stdout:1/734: mknod da/d12/d18/d2a/d41/d4f/cff 0 2024-04-24T23:14:05.941 INFO:tasks.workunit.client.0.smithi031.stdout:1/880: write d1/da/d12/d65/fb6 [389070,52493] 0 2024-04-24T23:14:05.942 INFO:tasks.workunit.client.0.smithi031.stdout:2/839: chown d5/d8/d45/d5b/d49/f7e 462 1 2024-04-24T23:14:05.947 INFO:tasks.workunit.client.1.smithi040.stdout:5/672: dread d2/d7/d8/f34 [4194304,4194304] 0 2024-04-24T23:14:05.948 INFO:tasks.workunit.client.0.smithi031.stdout:5/871: creat d0/d6/d3b/d4b/d1d/d2e/d28/d77/f127 x:0 0 0 2024-04-24T23:14:05.948 INFO:tasks.workunit.client.1.smithi040.stdout:0/814: dwrite df/d12/f22 [0,4194304] 0 2024-04-24T23:14:05.949 INFO:tasks.workunit.client.0.smithi031.stdout:5/872: readlink d0/d6/d3b/d4f/lcb 0 2024-04-24T23:14:05.950 INFO:tasks.workunit.client.1.smithi040.stdout:0/815: stat df/d1e/d20/d21/d41/la4 0 2024-04-24T23:14:05.950 INFO:tasks.workunit.client.1.smithi040.stdout:6/730: symlink d7/d1a/leb 0 2024-04-24T23:14:05.951 INFO:tasks.workunit.client.1.smithi040.stdout:6/731: chown d7/d1a/d20/d51/d2f/dba/c4d 67 1 2024-04-24T23:14:05.952 INFO:tasks.workunit.client.0.smithi031.stdout:1/881: mkdir d1/d3e/d46/df4/d11f 0 2024-04-24T23:14:05.955 INFO:tasks.workunit.client.0.smithi031.stdout:1/882: write d1/f8 [1940327,54596] 0 2024-04-24T23:14:05.956 INFO:tasks.workunit.client.0.smithi031.stdout:9/998: dwrite d2/dae/d14/d78/d84/d4a/dab/db8/fee [0,4194304] 0 2024-04-24T23:14:05.961 INFO:tasks.workunit.client.1.smithi040.stdout:8/791: creat d0/d16/d1b/d58/d8f/db1/dde/f105 x:0 0 0 2024-04-24T23:14:05.964 INFO:tasks.workunit.client.0.smithi031.stdout:7/927: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/f3c [2508056,124231] 0 2024-04-24T23:14:05.967 INFO:tasks.workunit.client.0.smithi031.stdout:7/928: read d1/d2/f46 [3588637,87007] 0 2024-04-24T23:14:05.968 INFO:tasks.workunit.client.1.smithi040.stdout:7/819: rename db/d12/d20/d52/d44/dd7 to db/d12/d20/d52/d4b/d127 0 2024-04-24T23:14:05.970 INFO:tasks.workunit.client.0.smithi031.stdout:1/883: write d1/da/d12/d65/dcf/d11/f10a [1323952,22548] 0 2024-04-24T23:14:05.970 INFO:tasks.workunit.client.1.smithi040.stdout:1/735: rmdir da/d35/d38/d47/dbc/de5 39 2024-04-24T23:14:05.971 INFO:tasks.workunit.client.0.smithi031.stdout:3/833: write d6/f7b [1775108,109861] 0 2024-04-24T23:14:05.971 INFO:tasks.workunit.client.1.smithi040.stdout:1/736: truncate da/d12/d14/d58/fea 596738 0 2024-04-24T23:14:05.972 INFO:tasks.workunit.client.1.smithi040.stdout:1/737: stat da/d12/f1c 0 2024-04-24T23:14:05.974 INFO:tasks.workunit.client.1.smithi040.stdout:5/673: truncate d2/d7/d8/d1f/d57/f44 8499479 0 2024-04-24T23:14:05.975 INFO:tasks.workunit.client.1.smithi040.stdout:5/674: dread - d2/d7/d8/d1f/d57/f91 zero size 2024-04-24T23:14:05.978 INFO:tasks.workunit.client.1.smithi040.stdout:6/732: creat d7/d1a/d20/d51/d1e/fec x:0 0 0 2024-04-24T23:14:05.979 INFO:tasks.workunit.client.0.smithi031.stdout:7/929: creat d1/d2/deb/d127/f143 x:0 0 0 2024-04-24T23:14:05.980 INFO:tasks.workunit.client.1.smithi040.stdout:2/816: symlink da/d9f/l12a 0 2024-04-24T23:14:05.981 INFO:tasks.workunit.client.0.smithi031.stdout:5/873: write d0/d6/d3b/d4b/dc/d25/f69 [856510,76365] 0 2024-04-24T23:14:05.981 INFO:tasks.workunit.client.1.smithi040.stdout:2/817: chown f8 15286043 1 2024-04-24T23:14:05.982 INFO:tasks.workunit.client.0.smithi031.stdout:1/884: unlink d1/da/d61/c92 0 2024-04-24T23:14:05.983 INFO:tasks.workunit.client.1.smithi040.stdout:2/818: dread da/d1c/d37/d39/d79/da3/fb3 [0,4194304] 0 2024-04-24T23:14:05.983 INFO:tasks.workunit.client.1.smithi040.stdout:2/819: chown da/d29/f6b 2114609 1 2024-04-24T23:14:05.984 INFO:tasks.workunit.client.1.smithi040.stdout:2/820: write da/d1c/d37/d8a/f127 [624920,53497] 0 2024-04-24T23:14:05.985 INFO:tasks.workunit.client.0.smithi031.stdout:9/999: creat d2/dae/d14/d78/d84/d4a/d4b/dcb/d134/d11d/f139 x:0 0 0 2024-04-24T23:14:05.986 INFO:tasks.workunit.client.1.smithi040.stdout:8/792: creat d0/d8/da0/f106 x:0 0 0 2024-04-24T23:14:05.987 INFO:tasks.workunit.client.0.smithi031.stdout:7/930: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/da0/f5f [4194304,4194304] 0 2024-04-24T23:14:05.988 INFO:tasks.workunit.client.0.smithi031.stdout:2/840: link d5/d8/d15/d65/c6a d5/d6/d29/d26/db0/dc7/da7/c124 0 2024-04-24T23:14:05.989 INFO:tasks.workunit.client.0.smithi031.stdout:2/841: stat d5/d8/d45/d5b/cb9 0 2024-04-24T23:14:05.990 INFO:tasks.workunit.client.1.smithi040.stdout:4/729: sync 2024-04-24T23:14:05.991 INFO:tasks.workunit.client.1.smithi040.stdout:9/764: getdents d7 0 2024-04-24T23:14:05.992 INFO:tasks.workunit.client.1.smithi040.stdout:4/730: read - da/d3f/d40/ff5 zero size 2024-04-24T23:14:05.992 INFO:tasks.workunit.client.1.smithi040.stdout:9/765: dread - d7/da/d17/d40/fb4 zero size 2024-04-24T23:14:05.994 INFO:tasks.workunit.client.0.smithi031.stdout:7/931: unlink d1/d2/dd/d33/fcb 0 2024-04-24T23:14:05.995 INFO:tasks.workunit.client.1.smithi040.stdout:1/738: mknod da/d12/d18/d2a/d41/d4f/c100 0 2024-04-24T23:14:05.996 INFO:tasks.workunit.client.1.smithi040.stdout:1/739: chown da/d12/d14/d25/db9/lcd 791494 1 2024-04-24T23:14:05.998 INFO:tasks.workunit.client.1.smithi040.stdout:0/816: symlink df/d1e/l107 0 2024-04-24T23:14:05.999 INFO:tasks.workunit.client.1.smithi040.stdout:0/817: stat df/d1e/d20/d61/fb0 0 2024-04-24T23:14:05.999 INFO:tasks.workunit.client.0.smithi031.stdout:5/874: mknod d0/d6/d3b/d4b/d1d/d2e/c128 0 2024-04-24T23:14:06.000 INFO:tasks.workunit.client.1.smithi040.stdout:8/793: dread d0/d52/fb3 [0,4194304] 0 2024-04-24T23:14:06.005 INFO:tasks.workunit.client.0.smithi031.stdout:2/842: symlink d5/d8/d15/d56/d84/l125 0 2024-04-24T23:14:06.006 INFO:tasks.workunit.client.1.smithi040.stdout:2/821: truncate da/d10/f71 1256914 0 2024-04-24T23:14:06.010 INFO:tasks.workunit.client.1.smithi040.stdout:3/808: getdents d1/d2d 0 2024-04-24T23:14:06.012 INFO:tasks.workunit.client.0.smithi031.stdout:2/843: mknod d5/d8/d45/d5b/d49/d2f/df8/c126 0 2024-04-24T23:14:06.016 INFO:tasks.workunit.client.0.smithi031.stdout:1/885: link d1/da/d12/d65/dcf/d11/fee d1/f120 0 2024-04-24T23:14:06.016 INFO:tasks.workunit.client.1.smithi040.stdout:9/766: creat d7/d5e/da1/dfa/f10c x:0 0 0 2024-04-24T23:14:06.016 INFO:tasks.workunit.client.0.smithi031.stdout:7/932: dwrite d1/dbe/fe0 [0,4194304] 0 2024-04-24T23:14:06.017 INFO:tasks.workunit.client.1.smithi040.stdout:5/675: dread d2/d7/d8/d1f/d57/d27/f54 [0,4194304] 0 2024-04-24T23:14:06.017 INFO:tasks.workunit.client.0.smithi031.stdout:3/834: dread d6/df/d16/d2a/d95/d9b/d1a/d25/f3d [4194304,4194304] 0 2024-04-24T23:14:06.018 INFO:tasks.workunit.client.1.smithi040.stdout:9/767: write d7/da/d17/d40/d55/d2f/d1f/d31/fe0 [481630,85186] 0 2024-04-24T23:14:06.021 INFO:tasks.workunit.client.1.smithi040.stdout:1/740: readlink da/d35/d38/d47/dbc/de5/lfa 0 2024-04-24T23:14:06.025 INFO:tasks.workunit.client.1.smithi040.stdout:8/794: mkdir d0/d16/d1b/d58/d107 0 2024-04-24T23:14:06.027 INFO:tasks.workunit.client.0.smithi031.stdout:1/886: dwrite d1/da/fe0 [0,4194304] 0 2024-04-24T23:14:06.027 INFO:tasks.workunit.client.1.smithi040.stdout:5/676: dwrite d2/d7/d8/d1f/d57/d27/d3d/fd4 [0,4194304] 0 2024-04-24T23:14:06.028 INFO:tasks.workunit.client.0.smithi031.stdout:3/835: creat d6/df/d16/d57/f100 x:0 0 0 2024-04-24T23:14:06.030 INFO:tasks.workunit.client.0.smithi031.stdout:2/844: dread d5/d6/d29/d26/f63 [0,4194304] 0 2024-04-24T23:14:06.032 INFO:tasks.workunit.client.0.smithi031.stdout:7/933: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/fc3 [4194304,4194304] 0 2024-04-24T23:14:06.033 INFO:tasks.workunit.client.1.smithi040.stdout:0/818: dwrite df/d1e/d20/d21/fd9 [4194304,4194304] 0 2024-04-24T23:14:06.034 INFO:tasks.workunit.client.1.smithi040.stdout:2/822: mknod da/d1c/d37/d102/c12b 0 2024-04-24T23:14:06.036 INFO:tasks.workunit.client.0.smithi031.stdout:1/887: write d1/d3e/da3/d69/f113 [773881,84072] 0 2024-04-24T23:14:06.039 INFO:tasks.workunit.client.0.smithi031.stdout:5/875: dwrite d0/faa [0,4194304] 0 2024-04-24T23:14:06.040 INFO:tasks.workunit.client.0.smithi031.stdout:7/934: stat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/c108 0 2024-04-24T23:14:06.043 INFO:tasks.workunit.client.0.smithi031.stdout:7/935: stat d1/d2/dd/d33/d5e/d64/d6d/dae/db1/c131 0 2024-04-24T23:14:06.043 INFO:tasks.workunit.client.1.smithi040.stdout:3/809: rename d1/d9/d2e/d100 to d1/d9/d2e/dcf/d110 0 2024-04-24T23:14:06.043 INFO:tasks.workunit.client.0.smithi031.stdout:7/936: chown d1/df8 197035 1 2024-04-24T23:14:06.044 INFO:tasks.workunit.client.1.smithi040.stdout:3/810: chown d1/d9/d18/d36/ffd 25 1 2024-04-24T23:14:06.049 INFO:tasks.workunit.client.0.smithi031.stdout:2/845: mkdir d5/d8/d45/d127 0 2024-04-24T23:14:06.049 INFO:tasks.workunit.client.0.smithi031.stdout:3/836: chown d6/d14/d1d/d69/le7 443 1 2024-04-24T23:14:06.050 INFO:tasks.workunit.client.1.smithi040.stdout:9/768: unlink d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/c95 0 2024-04-24T23:14:06.050 INFO:tasks.workunit.client.0.smithi031.stdout:1/888: fdatasync d1/f2 0 2024-04-24T23:14:06.058 INFO:tasks.workunit.client.0.smithi031.stdout:3/837: mkdir d6/df/d16/d2a/d95/d9b/d29/db7/de9/d101 0 2024-04-24T23:14:06.060 INFO:tasks.workunit.client.0.smithi031.stdout:1/889: dwrite d1/da/d12/f114 [0,4194304] 0 2024-04-24T23:14:06.060 INFO:tasks.workunit.client.1.smithi040.stdout:0/819: creat df/d12/d58/dc3/dcd/f108 x:0 0 0 2024-04-24T23:14:06.061 INFO:tasks.workunit.client.1.smithi040.stdout:8/795: dwrite d0/d16/d1b/d3f/fa8 [4194304,4194304] 0 2024-04-24T23:14:06.061 INFO:tasks.workunit.client.0.smithi031.stdout:1/890: chown d1/d3e/d46/ce6 0 1 2024-04-24T23:14:06.062 INFO:tasks.workunit.client.1.smithi040.stdout:2/823: mknod da/d10/d4d/da4/c12c 0 2024-04-24T23:14:06.063 INFO:tasks.workunit.client.0.smithi031.stdout:1/891: write d1/d3e/d46/f5c [4784982,38589] 0 2024-04-24T23:14:06.064 INFO:tasks.workunit.client.0.smithi031.stdout:7/937: link d1/d2/f2a d1/dbe/f144 0 2024-04-24T23:14:06.066 INFO:tasks.workunit.client.1.smithi040.stdout:7/820: getdents db/d43/db9/dd4/d101 0 2024-04-24T23:14:06.069 INFO:tasks.workunit.client.1.smithi040.stdout:7/821: write db/d12/d20/d52/d4b/f4f [223858,21396] 0 2024-04-24T23:14:06.070 INFO:tasks.workunit.client.0.smithi031.stdout:3/838: mknod d6/df/d16/d2a/d95/d9b/d29/c102 0 2024-04-24T23:14:06.070 INFO:tasks.workunit.client.0.smithi031.stdout:5/876: dread d0/d6/d3b/d4b/d1d/d2e/dc6/f31 [0,4194304] 0 2024-04-24T23:14:06.071 INFO:tasks.workunit.client.1.smithi040.stdout:4/731: link da/d3f/d40/d47/c5b da/d3f/d40/d41/d8e/db8/cfe 0 2024-04-24T23:14:06.072 INFO:tasks.workunit.client.1.smithi040.stdout:2/824: dwrite da/d1c/d37/d8a/f104 [0,4194304] 0 2024-04-24T23:14:06.074 INFO:tasks.workunit.client.1.smithi040.stdout:7/822: chown db/d12/d20/lb0 6 1 2024-04-24T23:14:06.074 INFO:tasks.workunit.client.0.smithi031.stdout:2/846: creat d5/d8/d15/d56/f128 x:0 0 0 2024-04-24T23:14:06.077 INFO:tasks.workunit.client.0.smithi031.stdout:1/892: truncate d1/da/d12/d7e/faf 939974 0 2024-04-24T23:14:06.081 INFO:tasks.workunit.client.1.smithi040.stdout:6/733: getdents d7/d1a/d9f 0 2024-04-24T23:14:06.081 INFO:tasks.workunit.client.1.smithi040.stdout:3/811: dwrite d1/d9/d18/d36/d55/f89 [0,4194304] 0 2024-04-24T23:14:06.083 INFO:tasks.workunit.client.1.smithi040.stdout:3/812: chown d1/d9/d39/cc4 2 1 2024-04-24T23:14:06.086 INFO:tasks.workunit.client.1.smithi040.stdout:4/732: dwrite da/d3f/f5a [0,4194304] 0 2024-04-24T23:14:06.087 INFO:tasks.workunit.client.0.smithi031.stdout:1/893: rmdir d1/d3e/d46/d74 39 2024-04-24T23:14:06.089 INFO:tasks.workunit.client.1.smithi040.stdout:4/733: write da/d3f/fb0 [2137638,59424] 0 2024-04-24T23:14:06.093 INFO:tasks.workunit.client.1.smithi040.stdout:8/796: creat d0/d16/d19/f108 x:0 0 0 2024-04-24T23:14:06.093 INFO:tasks.workunit.client.0.smithi031.stdout:2/847: mknod d5/d8/d45/d5b/d49/d6f/d119/d11d/c129 0 2024-04-24T23:14:06.094 INFO:tasks.workunit.client.1.smithi040.stdout:6/734: dwrite d7/d1a/d20/d51/d1e/f72 [4194304,4194304] 0 2024-04-24T23:14:06.101 INFO:tasks.workunit.client.1.smithi040.stdout:2/825: rmdir da/d10/d4d/da4 39 2024-04-24T23:14:06.101 INFO:tasks.workunit.client.1.smithi040.stdout:9/769: write d7/da/d17/d40/d55/d2f/d1f/d4e/d83/fb3 [224146,111385] 0 2024-04-24T23:14:06.103 INFO:tasks.workunit.client.1.smithi040.stdout:1/741: truncate da/d12/d14/d25/d5e/fa4 3795899 0 2024-04-24T23:14:06.104 INFO:tasks.workunit.client.1.smithi040.stdout:9/770: chown d7/da/d17/d40/d55/d2f/d28/ca3 1903705 1 2024-04-24T23:14:06.105 INFO:tasks.workunit.client.0.smithi031.stdout:2/848: mknod d5/d8/d45/d5b/d49/c12a 0 2024-04-24T23:14:06.109 INFO:tasks.workunit.client.1.smithi040.stdout:0/820: creat df/dd4/f109 x:0 0 0 2024-04-24T23:14:06.110 INFO:tasks.workunit.client.1.smithi040.stdout:5/677: dwrite d2/d7/d8/d1f/d57/d27/d3c/f95 [0,4194304] 0 2024-04-24T23:14:06.112 INFO:tasks.workunit.client.1.smithi040.stdout:0/821: chown df/d12/d58/f72 355598 1 2024-04-24T23:14:06.112 INFO:tasks.workunit.client.1.smithi040.stdout:8/797: truncate d0/d8/dc/f63 2018186 0 2024-04-24T23:14:06.113 INFO:tasks.workunit.client.1.smithi040.stdout:4/734: read da/d45/d48/d6e/fbd [116752,86283] 0 2024-04-24T23:14:06.114 INFO:tasks.workunit.client.1.smithi040.stdout:2/826: rename da/d10/d4d/d6c/la5 to da/d1c/d37/d39/d3f/db2/l12d 0 2024-04-24T23:14:06.118 INFO:tasks.workunit.client.1.smithi040.stdout:9/771: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/c10d 0 2024-04-24T23:14:06.122 INFO:tasks.workunit.client.1.smithi040.stdout:3/813: dread d1/d2a/d6a/f8e [4194304,4194304] 0 2024-04-24T23:14:06.125 INFO:tasks.workunit.client.0.smithi031.stdout:5/877: dwrite d0/d6/d3b/d4b/d1d/d2e/f52 [4194304,4194304] 0 2024-04-24T23:14:06.127 INFO:tasks.workunit.client.1.smithi040.stdout:5/678: fdatasync d2/d7/d8/db/d1e/f46 0 2024-04-24T23:14:06.133 INFO:tasks.workunit.client.1.smithi040.stdout:8/798: creat d0/d7e/dc1/d9e/f109 x:0 0 0 2024-04-24T23:14:06.134 INFO:tasks.workunit.client.1.smithi040.stdout:0/822: dread df/d1e/d20/fb2 [0,4194304] 0 2024-04-24T23:14:06.135 INFO:tasks.workunit.client.1.smithi040.stdout:2/827: readlink da/d29/d55/l119 0 2024-04-24T23:14:06.138 INFO:tasks.workunit.client.1.smithi040.stdout:7/823: dwrite db/d12/d20/d52/d44/fab [0,4194304] 0 2024-04-24T23:14:06.140 INFO:tasks.workunit.client.0.smithi031.stdout:1/894: dread d1/d33/f47 [0,4194304] 0 2024-04-24T23:14:06.141 INFO:tasks.workunit.client.1.smithi040.stdout:9/772: creat d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/def/f10e x:0 0 0 2024-04-24T23:14:06.143 INFO:tasks.workunit.client.0.smithi031.stdout:1/895: dread d1/f85 [0,4194304] 0 2024-04-24T23:14:06.144 INFO:tasks.workunit.client.1.smithi040.stdout:3/814: creat d1/d2d/de/f111 x:0 0 0 2024-04-24T23:14:06.146 INFO:tasks.workunit.client.1.smithi040.stdout:6/735: dread d7/d1a/d20/f23 [0,4194304] 0 2024-04-24T23:14:06.147 INFO:tasks.workunit.client.1.smithi040.stdout:9/773: truncate d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/def/f10e 680075 0 2024-04-24T23:14:06.149 INFO:tasks.workunit.client.1.smithi040.stdout:5/679: dread - d2/d2f/fd1 zero size 2024-04-24T23:14:06.150 INFO:tasks.workunit.client.1.smithi040.stdout:6/736: truncate d7/d1a/d20/d51/fdf 933744 0 2024-04-24T23:14:06.151 INFO:tasks.workunit.client.1.smithi040.stdout:5/680: chown d2/lf 246 1 2024-04-24T23:14:06.151 INFO:tasks.workunit.client.1.smithi040.stdout:4/735: symlink da/d3f/d40/d41/d8e/de9/lff 0 2024-04-24T23:14:06.152 INFO:tasks.workunit.client.1.smithi040.stdout:3/815: dwrite d1/d2d/fc1 [4194304,4194304] 0 2024-04-24T23:14:06.162 INFO:tasks.workunit.client.1.smithi040.stdout:9/774: dread d7/da/d17/d40/d55/d2f/d1f/d31/f36 [0,4194304] 0 2024-04-24T23:14:06.163 INFO:tasks.workunit.client.0.smithi031.stdout:1/896: dread d1/da/d12/d65/dcf/d11/d21/f28 [0,4194304] 0 2024-04-24T23:14:06.164 INFO:tasks.workunit.client.0.smithi031.stdout:1/897: chown d1/da/d12/d7e/d7b/lfd 218 1 2024-04-24T23:14:06.166 INFO:tasks.workunit.client.1.smithi040.stdout:7/824: mknod db/d43/db9/dd4/d101/c128 0 2024-04-24T23:14:06.172 INFO:tasks.workunit.client.1.smithi040.stdout:4/736: write da/d3f/d6b/d3c/fc5 [136307,84076] 0 2024-04-24T23:14:06.174 INFO:tasks.workunit.client.1.smithi040.stdout:4/737: write da/d3f/d40/d41/d60/f34 [4307217,85163] 0 2024-04-24T23:14:06.175 INFO:tasks.workunit.client.1.smithi040.stdout:4/738: readlink da/d3f/d40/d41/d8e/de9/lff 0 2024-04-24T23:14:06.180 INFO:tasks.workunit.client.1.smithi040.stdout:0/823: rename df/d1e/f7b to df/d12/d4b/db8/f10a 0 2024-04-24T23:14:06.187 INFO:tasks.workunit.client.1.smithi040.stdout:7/825: fdatasync db/d12/d62/d8a/f124 0 2024-04-24T23:14:06.188 INFO:tasks.workunit.client.1.smithi040.stdout:7/826: fsync db/d12/ff2 0 2024-04-24T23:14:06.188 INFO:tasks.workunit.client.0.smithi031.stdout:2/849: mkdir d5/d8/d15/d65/d12b 0 2024-04-24T23:14:06.194 INFO:tasks.workunit.client.0.smithi031.stdout:5/878: creat d0/d7/d74/f129 x:0 0 0 2024-04-24T23:14:06.195 INFO:tasks.workunit.client.1.smithi040.stdout:1/742: dread da/f7e [0,4194304] 0 2024-04-24T23:14:06.195 INFO:tasks.workunit.client.1.smithi040.stdout:1/743: fsync da/d12/d14/d58/ff0 0 2024-04-24T23:14:06.200 INFO:tasks.workunit.client.0.smithi031.stdout:2/850: link d5/d8/f98 d5/d6/d29/d26/db0/dc7/d117/d123/d114/f12c 0 2024-04-24T23:14:06.205 INFO:tasks.workunit.client.0.smithi031.stdout:2/851: mknod d5/d6/d29/d26/db0/c12d 0 2024-04-24T23:14:06.207 INFO:tasks.workunit.client.1.smithi040.stdout:6/737: rename d7/d1a/lc5 to d7/d1a/d20/d51/d2f/dba/dde/led 0 2024-04-24T23:14:06.207 INFO:tasks.workunit.client.1.smithi040.stdout:9/775: write d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f49 [1380428,120056] 0 2024-04-24T23:14:06.208 INFO:tasks.workunit.client.1.smithi040.stdout:0/824: chown df/d1e/d20/f3f 6 1 2024-04-24T23:14:06.209 INFO:tasks.workunit.client.0.smithi031.stdout:2/852: rename d5/d8/l105 to d5/d6/d29/d4f/l12e 0 2024-04-24T23:14:06.210 INFO:tasks.workunit.client.0.smithi031.stdout:2/853: chown d5/f13 1988 1 2024-04-24T23:14:06.211 INFO:tasks.workunit.client.1.smithi040.stdout:2/828: truncate da/d10/d4d/da4/fd9 1191887 0 2024-04-24T23:14:06.214 INFO:tasks.workunit.client.1.smithi040.stdout:0/825: dwrite df/d12/d1a/f4e [0,4194304] 0 2024-04-24T23:14:06.217 INFO:tasks.workunit.client.1.smithi040.stdout:4/739: write da/d45/d48/d6e/fbd [1397924,49439] 0 2024-04-24T23:14:06.219 INFO:tasks.workunit.client.1.smithi040.stdout:5/681: dwrite d2/d7/d8/db/f96 [0,4194304] 0 2024-04-24T23:14:06.220 INFO:tasks.workunit.client.0.smithi031.stdout:5/879: write d0/d6/d3b/d4b/d1d/d2e/d28/f30 [1123090,116514] 0 2024-04-24T23:14:06.220 INFO:tasks.workunit.client.1.smithi040.stdout:6/738: rename d7/d1a/d20/d51/d2f/dba/f9c to d7/d1a/fee 0 2024-04-24T23:14:06.221 INFO:tasks.workunit.client.1.smithi040.stdout:5/682: chown d2/d62/fb2 115 1 2024-04-24T23:14:06.221 INFO:tasks.workunit.client.0.smithi031.stdout:5/880: chown d0/d6/d3b/d4b/dc/fea 190263 1 2024-04-24T23:14:06.222 INFO:tasks.workunit.client.1.smithi040.stdout:3/816: write d1/d9/d2e/f84 [1699065,102485] 0 2024-04-24T23:14:06.222 INFO:tasks.workunit.client.1.smithi040.stdout:6/739: fsync d7/d1a/d20/d51/d1e/f72 0 2024-04-24T23:14:06.223 INFO:tasks.workunit.client.1.smithi040.stdout:9/776: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/c10f 0 2024-04-24T23:14:06.223 INFO:tasks.workunit.client.1.smithi040.stdout:5/683: readlink d2/d7/d8/d1f/d31/l71 0 2024-04-24T23:14:06.224 INFO:tasks.workunit.client.1.smithi040.stdout:2/829: symlink da/d1c/d37/l12e 0 2024-04-24T23:14:06.230 INFO:tasks.workunit.client.0.smithi031.stdout:5/881: creat d0/d6/d3b/d4b/dc/d86/dfd/da8/f12a x:0 0 0 2024-04-24T23:14:06.231 INFO:tasks.workunit.client.0.smithi031.stdout:5/882: readlink d0/d7/d56/d7c/d7e/l66 0 2024-04-24T23:14:06.232 INFO:tasks.workunit.client.0.smithi031.stdout:5/883: chown d0/d6/d3b/cd4 5 1 2024-04-24T23:14:06.233 INFO:tasks.workunit.client.1.smithi040.stdout:8/799: getdents d0/d16/d1b 0 2024-04-24T23:14:06.235 INFO:tasks.workunit.client.0.smithi031.stdout:2/854: mkdir d5/d8/d45/d5b/d12f 0 2024-04-24T23:14:06.236 INFO:tasks.workunit.client.1.smithi040.stdout:4/740: creat da/d3f/d40/d47/d75/f100 x:0 0 0 2024-04-24T23:14:06.241 INFO:tasks.workunit.client.0.smithi031.stdout:2/855: creat d5/d6/d29/d26/db0/dc7/d117/d123/f130 x:0 0 0 2024-04-24T23:14:06.242 INFO:tasks.workunit.client.1.smithi040.stdout:3/817: mkdir d1/d9/d39/d112 0 2024-04-24T23:14:06.243 INFO:tasks.workunit.client.0.smithi031.stdout:2/856: chown d5/d8/d45/d5b/d49/dc4/de5/f11f 0 1 2024-04-24T23:14:06.246 INFO:tasks.workunit.client.1.smithi040.stdout:6/740: creat d7/d1a/d20/d51/d1e/d46/d85/fef x:0 0 0 2024-04-24T23:14:06.249 INFO:tasks.workunit.client.1.smithi040.stdout:3/818: dwrite d1/d9/d18/d36/d61/ffe [0,4194304] 0 2024-04-24T23:14:06.252 INFO:tasks.workunit.client.0.smithi031.stdout:2/857: fdatasync d5/d8/d15/d65/f8e 0 2024-04-24T23:14:06.253 INFO:tasks.workunit.client.1.smithi040.stdout:6/741: dread d7/d1a/d20/d51/d1e/d5d/f93 [0,4194304] 0 2024-04-24T23:14:06.255 INFO:tasks.workunit.client.1.smithi040.stdout:8/800: mknod d0/d16/d1b/d3f/d44/c10a 0 2024-04-24T23:14:06.255 INFO:tasks.workunit.client.1.smithi040.stdout:0/826: dread df/d12/d1a/fad [0,4194304] 0 2024-04-24T23:14:06.256 INFO:tasks.workunit.client.1.smithi040.stdout:8/801: readlink d0/d16/d1b/lc4 0 2024-04-24T23:14:06.256 INFO:tasks.workunit.client.0.smithi031.stdout:5/884: dwrite d0/d6/f40 [0,4194304] 0 2024-04-24T23:14:06.259 INFO:tasks.workunit.client.1.smithi040.stdout:4/741: dread da/d3f/fb0 [0,4194304] 0 2024-04-24T23:14:06.262 INFO:tasks.workunit.client.0.smithi031.stdout:2/858: unlink d5/d6/d29/d26/db0/c12d 0 2024-04-24T23:14:06.264 INFO:tasks.workunit.client.0.smithi031.stdout:5/885: creat d0/d6/d3b/d4b/dc/d8b/f12b x:0 0 0 2024-04-24T23:14:06.264 INFO:tasks.workunit.client.1.smithi040.stdout:2/830: dwrite da/d10/f84 [0,4194304] 0 2024-04-24T23:14:06.266 INFO:tasks.workunit.client.1.smithi040.stdout:2/831: chown da/d1c/c1d 584 1 2024-04-24T23:14:06.267 INFO:tasks.workunit.client.0.smithi031.stdout:2/859: symlink d5/d8/d45/d5b/d49/dc4/de5/l131 0 2024-04-24T23:14:06.271 INFO:tasks.workunit.client.1.smithi040.stdout:0/827: mknod df/d12/d58/dc3/c10b 0 2024-04-24T23:14:06.276 INFO:tasks.workunit.client.0.smithi031.stdout:5/886: link d0/d6/d3b/d4b/dc/d86/dfd/da8/f12a d0/d7/d74/f12c 0 2024-04-24T23:14:06.278 INFO:tasks.workunit.client.1.smithi040.stdout:6/742: write d7/d1a/f42 [967243,1267] 0 2024-04-24T23:14:06.278 INFO:tasks.workunit.client.1.smithi040.stdout:5/684: creat d2/d7/d8/d1f/d57/fdd x:0 0 0 2024-04-24T23:14:06.279 INFO:tasks.workunit.client.1.smithi040.stdout:5/685: chown d2/d7/d8/d1f/d90/dcb/l5d 994664 1 2024-04-24T23:14:06.283 INFO:tasks.workunit.client.0.smithi031.stdout:5/887: mkdir d0/d6/d3b/d4b/d1d/d12d 0 2024-04-24T23:14:06.285 INFO:tasks.workunit.client.1.smithi040.stdout:8/802: mknod d0/d16/c10b 0 2024-04-24T23:14:06.285 INFO:tasks.workunit.client.0.smithi031.stdout:2/860: dwrite d5/d8/d45/d5b/d49/f7e [0,4194304] 0 2024-04-24T23:14:06.286 INFO:tasks.workunit.client.1.smithi040.stdout:4/742: fsync da/d3f/d40/d47/f93 0 2024-04-24T23:14:06.287 INFO:tasks.workunit.client.1.smithi040.stdout:4/743: fsync da/d3f/d40/d41/d8e/dd3/fdb 0 2024-04-24T23:14:06.289 INFO:tasks.workunit.client.1.smithi040.stdout:4/744: write da/d3f/d6b/d2b/f46 [3778524,114428] 0 2024-04-24T23:14:06.289 INFO:tasks.workunit.client.1.smithi040.stdout:3/819: unlink d1/d9/d39/f78 0 2024-04-24T23:14:06.289 INFO:tasks.workunit.client.0.smithi031.stdout:5/888: dread - d0/d6/d3b/d4b/d1d/d2e/dc6/fcd zero size 2024-04-24T23:14:06.295 INFO:tasks.workunit.client.0.smithi031.stdout:5/889: dread d0/d6/d3b/d4b/dc/d86/dfd/f102 [0,4194304] 0 2024-04-24T23:14:06.297 INFO:tasks.workunit.client.1.smithi040.stdout:4/745: dwrite da/d3f/d40/d41/f4e [8388608,4194304] 0 2024-04-24T23:14:06.297 INFO:tasks.workunit.client.1.smithi040.stdout:8/803: mknod d0/d28/db7/c10c 0 2024-04-24T23:14:06.299 INFO:tasks.workunit.client.1.smithi040.stdout:4/746: chown da/d3f/d40/d41/d62/ccd 1609230593 1 2024-04-24T23:14:06.299 INFO:tasks.workunit.client.1.smithi040.stdout:8/804: write d0/d16/f57 [13698,114926] 0 2024-04-24T23:14:06.306 INFO:tasks.workunit.client.0.smithi031.stdout:2/861: dwrite d5/d8/d15/d56/d84/d75/f91 [0,4194304] 0 2024-04-24T23:14:06.308 INFO:tasks.workunit.client.0.smithi031.stdout:5/890: link d0/d6/d3b/d4b/dc/d8b/lb0 d0/d7/d74/de2/l12e 0 2024-04-24T23:14:06.310 INFO:tasks.workunit.client.1.smithi040.stdout:2/832: rename da/d1c/d37/d39/d3f/d52/d78/dc7/dcc/c121 to da/d10/d15/c12f 0 2024-04-24T23:14:06.311 INFO:tasks.workunit.client.1.smithi040.stdout:2/833: chown da/d29/f69 234 1 2024-04-24T23:14:06.314 INFO:tasks.workunit.client.0.smithi031.stdout:5/891: dread d0/d6/f40 [0,4194304] 0 2024-04-24T23:14:06.314 INFO:tasks.workunit.client.0.smithi031.stdout:2/862: unlink d5/d8/d15/d65/f8e 0 2024-04-24T23:14:06.315 INFO:tasks.workunit.client.1.smithi040.stdout:5/686: dread d2/d7/d8/db/f75 [0,4194304] 0 2024-04-24T23:14:06.315 INFO:tasks.workunit.client.0.smithi031.stdout:5/892: stat d0/d6/d3b/d4b/dc/d25 0 2024-04-24T23:14:06.316 INFO:tasks.workunit.client.1.smithi040.stdout:5/687: truncate d2/d7/d8/d1f/d57/fdd 971402 0 2024-04-24T23:14:06.317 INFO:tasks.workunit.client.0.smithi031.stdout:5/893: write d0/d6/d3b/d4b/dc/d25/f101 [925594,63586] 0 2024-04-24T23:14:06.319 INFO:tasks.workunit.client.0.smithi031.stdout:2/863: creat d5/d8/d15/d56/d84/f132 x:0 0 0 2024-04-24T23:14:06.321 INFO:tasks.workunit.client.1.smithi040.stdout:0/828: rename df/d12/d1a/d71/fa2 to df/d1e/d20/d21/d41/d8a/df3/df8/d79/f10c 0 2024-04-24T23:14:06.321 INFO:tasks.workunit.client.1.smithi040.stdout:0/829: stat df/dd4/f109 0 2024-04-24T23:14:06.321 INFO:tasks.workunit.client.1.smithi040.stdout:0/830: readlink df/de1/l37 0 2024-04-24T23:14:06.322 INFO:tasks.workunit.client.1.smithi040.stdout:3/820: write d1/d9/d2e/fac [627238,55001] 0 2024-04-24T23:14:06.322 INFO:tasks.workunit.client.1.smithi040.stdout:0/831: stat df/d12/d49/cc7 0 2024-04-24T23:14:06.324 INFO:tasks.workunit.client.1.smithi040.stdout:2/834: dread da/f11 [4194304,4194304] 0 2024-04-24T23:14:06.325 INFO:tasks.workunit.client.1.smithi040.stdout:2/835: dread - da/d81/de6/ff6 zero size 2024-04-24T23:14:06.325 INFO:tasks.workunit.client.0.smithi031.stdout:5/894: rename d0/d6/d3b/d4b/d1d/d2e/d28/f30 to d0/d6/d3b/d4f/f12f 0 2024-04-24T23:14:06.326 INFO:tasks.workunit.client.1.smithi040.stdout:4/747: mknod da/d3f/c101 0 2024-04-24T23:14:06.327 INFO:tasks.workunit.client.1.smithi040.stdout:2/836: dread da/d1c/d37/d39/d79/da3/fea [0,4194304] 0 2024-04-24T23:14:06.327 INFO:tasks.workunit.client.0.smithi031.stdout:2/864: mknod d5/d8/d15/d65/c133 0 2024-04-24T23:14:06.328 INFO:tasks.workunit.client.0.smithi031.stdout:2/865: read - d5/d6/d29/d26/db0/dc7/d117/d123/f130 zero size 2024-04-24T23:14:06.329 INFO:tasks.workunit.client.1.smithi040.stdout:5/688: creat d2/d7/d8/db/fde x:0 0 0 2024-04-24T23:14:06.331 INFO:tasks.workunit.client.0.smithi031.stdout:5/895: mkdir d0/d6/d3b/d4b/dc/d86/d91/d130 0 2024-04-24T23:14:06.332 INFO:tasks.workunit.client.1.smithi040.stdout:4/748: dwrite da/d45/d48/d6e/d70/fcb [0,4194304] 0 2024-04-24T23:14:06.333 INFO:tasks.workunit.client.0.smithi031.stdout:2/866: symlink d5/d6/d29/d26/db0/dc7/d117/d123/d114/l134 0 2024-04-24T23:14:06.333 INFO:tasks.workunit.client.1.smithi040.stdout:0/832: mkdir df/de1/d6e/d10d 0 2024-04-24T23:14:06.334 INFO:tasks.workunit.client.1.smithi040.stdout:3/821: creat d1/d9/d2e/f113 x:0 0 0 2024-04-24T23:14:06.336 INFO:tasks.workunit.client.0.smithi031.stdout:2/867: fdatasync d5/d28/d4e/fd6 0 2024-04-24T23:14:06.339 INFO:tasks.workunit.client.1.smithi040.stdout:7/827: sync 2024-04-24T23:14:06.347 INFO:tasks.workunit.client.1.smithi040.stdout:0/833: creat df/d1e/d20/d61/f10e x:0 0 0 2024-04-24T23:14:06.349 INFO:tasks.workunit.client.1.smithi040.stdout:7/828: mknod db/d43/c129 0 2024-04-24T23:14:06.350 INFO:tasks.workunit.client.1.smithi040.stdout:7/829: chown db/d12/d62/d8a/db3/dbf 477 1 2024-04-24T23:14:06.351 INFO:tasks.workunit.client.1.smithi040.stdout:4/749: mkdir da/d3f/d40/d47/d75/de4/d102 0 2024-04-24T23:14:06.351 INFO:tasks.workunit.client.1.smithi040.stdout:7/830: chown db/lee 104 1 2024-04-24T23:14:06.352 INFO:tasks.workunit.client.1.smithi040.stdout:7/831: stat db/d12/d7d/dd2/f100 0 2024-04-24T23:14:06.355 INFO:tasks.workunit.client.1.smithi040.stdout:0/834: link df/de1/db6/fdc df/d1e/d20/d21/d41/d8a/f10f 0 2024-04-24T23:14:06.356 INFO:tasks.workunit.client.1.smithi040.stdout:0/835: write df/d1e/da7/f7f [1125108,28929] 0 2024-04-24T23:14:06.356 INFO:tasks.workunit.client.0.smithi031.stdout:5/896: dread d0/d6/d3b/d4b/d1d/d2e/dc6/f21 [4194304,4194304] 0 2024-04-24T23:14:06.356 INFO:tasks.workunit.client.0.smithi031.stdout:2/868: rename d5/d6 to d5/d8/d45/d5b/d49/d2f/d135 0 2024-04-24T23:14:06.357 INFO:tasks.workunit.client.0.smithi031.stdout:5/897: chown d0/d6/d3b/d4b/dc/d25/c5b 0 1 2024-04-24T23:14:06.360 INFO:tasks.workunit.client.1.smithi040.stdout:3/822: getdents d1/d2d/d2b 0 2024-04-24T23:14:06.361 INFO:tasks.workunit.client.1.smithi040.stdout:3/823: chown d1/d2d/fc3 42416253 1 2024-04-24T23:14:06.361 INFO:tasks.workunit.client.1.smithi040.stdout:2/837: dread da/d10/f34 [0,4194304] 0 2024-04-24T23:14:06.361 INFO:tasks.workunit.client.0.smithi031.stdout:2/869: dread - d5/d8/d15/d56/f8f zero size 2024-04-24T23:14:06.363 INFO:tasks.workunit.client.0.smithi031.stdout:6/966: sync 2024-04-24T23:14:06.363 INFO:tasks.workunit.client.0.smithi031.stdout:7/938: sync 2024-04-24T23:14:06.364 INFO:tasks.workunit.client.1.smithi040.stdout:0/836: mknod df/d1e/d20/d61/c110 0 2024-04-24T23:14:06.369 INFO:tasks.workunit.client.1.smithi040.stdout:5/689: dwrite d2/d7/d8/db/da2/dcc/fdc [0,4194304] 0 2024-04-24T23:14:06.375 INFO:tasks.workunit.client.1.smithi040.stdout:7/832: write db/d12/f3e [1242619,79476] 0 2024-04-24T23:14:06.380 INFO:tasks.workunit.client.0.smithi031.stdout:5/898: write d0/d6/d3b/d4b/dc/f80 [512036,110905] 0 2024-04-24T23:14:06.381 INFO:tasks.workunit.client.1.smithi040.stdout:4/750: link da/d45/d48/l4f da/d45/l103 0 2024-04-24T23:14:06.381 INFO:tasks.workunit.client.1.smithi040.stdout:4/751: fsync da/d3f/d40/fb2 0 2024-04-24T23:14:06.381 INFO:tasks.workunit.client.0.smithi031.stdout:6/967: creat d13/d15/dea/f146 x:0 0 0 2024-04-24T23:14:06.382 INFO:tasks.workunit.client.1.smithi040.stdout:4/752: fdatasync da/d3f/d6b/d2b/f46 0 2024-04-24T23:14:06.384 INFO:tasks.workunit.client.1.smithi040.stdout:5/690: truncate d2/d7/d8/f34 4187286 0 2024-04-24T23:14:06.384 INFO:tasks.workunit.client.0.smithi031.stdout:2/870: link d5/f87 d5/d28/d4e/d5e/f136 0 2024-04-24T23:14:06.385 INFO:tasks.workunit.client.1.smithi040.stdout:2/838: fsync da/f72 0 2024-04-24T23:14:06.389 INFO:tasks.workunit.client.1.smithi040.stdout:3/824: link d1/d9/d18/f66 d1/d9/d18/d36/d55/d62/f114 0 2024-04-24T23:14:06.390 INFO:tasks.workunit.client.0.smithi031.stdout:5/899: dread d0/d6/d3b/d4b/d1d/f9e [0,4194304] 0 2024-04-24T23:14:06.390 INFO:tasks.workunit.client.1.smithi040.stdout:7/833: creat db/d2f/da8/f12a x:0 0 0 2024-04-24T23:14:06.396 INFO:tasks.workunit.client.1.smithi040.stdout:4/753: mkdir da/d3f/d40/d47/d75/d104 0 2024-04-24T23:14:06.397 INFO:tasks.workunit.client.0.smithi031.stdout:7/939: getdents d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/df2 0 2024-04-24T23:14:06.397 INFO:tasks.workunit.client.1.smithi040.stdout:4/754: write da/d3f/d6b/d3c/fc5 [737463,63178] 0 2024-04-24T23:14:06.398 INFO:tasks.workunit.client.1.smithi040.stdout:5/691: dread d2/d7/d8/db/d1e/f46 [0,4194304] 0 2024-04-24T23:14:06.399 INFO:tasks.workunit.client.0.smithi031.stdout:6/968: write d13/d15/d1d/d23/d24/d8b/f126 [1008494,51254] 0 2024-04-24T23:14:06.400 INFO:tasks.workunit.client.1.smithi040.stdout:0/837: dread df/d1e/d20/d21/d41/d8a/df3/df8/d79/f10c [0,4194304] 0 2024-04-24T23:14:06.404 INFO:tasks.workunit.client.1.smithi040.stdout:7/834: write db/d43/db9/fe3 [136887,113581] 0 2024-04-24T23:14:06.405 INFO:tasks.workunit.client.0.smithi031.stdout:2/871: link d5/d99/da2/f122 d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/d117/f137 0 2024-04-24T23:14:06.408 INFO:tasks.workunit.client.0.smithi031.stdout:6/969: dwrite d13/d15/d1d/d23/d24/d8b/f126 [0,4194304] 0 2024-04-24T23:14:06.414 INFO:tasks.workunit.client.0.smithi031.stdout:5/900: dwrite d0/d6/f40 [0,4194304] 0 2024-04-24T23:14:06.414 INFO:tasks.workunit.client.1.smithi040.stdout:2/839: mknod da/d1c/d37/d39/d3f/d52/d78/df5/c130 0 2024-04-24T23:14:06.419 INFO:tasks.workunit.client.0.smithi031.stdout:6/970: read d13/d15/d1d/d23/d4d/ff4 [448025,53279] 0 2024-04-24T23:14:06.426 INFO:tasks.workunit.client.0.smithi031.stdout:5/901: dwrite d0/d6/d3b/d4b/dc/de3/f104 [0,4194304] 0 2024-04-24T23:14:06.429 INFO:tasks.workunit.client.0.smithi031.stdout:1/898: sync 2024-04-24T23:14:06.429 INFO:tasks.workunit.client.0.smithi031.stdout:3/839: sync 2024-04-24T23:14:06.429 INFO:tasks.workunit.client.1.smithi040.stdout:7/835: rename db/d43 to db/d12/d20/d52/d4b/d127/d12b 0 2024-04-24T23:14:06.430 INFO:tasks.workunit.client.1.smithi040.stdout:4/755: write da/d3f/d40/d47/fa4 [360674,45957] 0 2024-04-24T23:14:06.431 INFO:tasks.workunit.client.1.smithi040.stdout:5/692: mkdir d2/d2f/d14/ddf 0 2024-04-24T23:14:06.433 INFO:tasks.workunit.client.1.smithi040.stdout:4/756: write da/d3f/d40/d41/d62/f6f [1025312,7964] 0 2024-04-24T23:14:06.434 INFO:tasks.workunit.client.1.smithi040.stdout:1/744: sync 2024-04-24T23:14:06.435 INFO:tasks.workunit.client.1.smithi040.stdout:5/693: write d2/d7/d8/d1f/d57/d27/d3d/fd4 [261505,30138] 0 2024-04-24T23:14:06.435 INFO:tasks.workunit.client.1.smithi040.stdout:3/825: truncate d1/d9/d18/d36/d10d/f1b 7722357 0 2024-04-24T23:14:06.436 INFO:tasks.workunit.client.0.smithi031.stdout:6/971: mkdir d13/d1b/de3/d147 0 2024-04-24T23:14:06.436 INFO:tasks.workunit.client.1.smithi040.stdout:4/757: chown da/d3f/d40/d41/d8e/dd3/cf7 182 1 2024-04-24T23:14:06.437 INFO:tasks.workunit.client.0.smithi031.stdout:7/940: dread d1/d2/f2d [0,4194304] 0 2024-04-24T23:14:06.437 INFO:tasks.workunit.client.0.smithi031.stdout:6/972: readlink d13/d15/d1d/d23/d24/d63/l10e 0 2024-04-24T23:14:06.437 INFO:tasks.workunit.client.1.smithi040.stdout:5/694: write d2/d7/d8/d64/f68 [3120527,126349] 0 2024-04-24T23:14:06.439 INFO:tasks.workunit.client.0.smithi031.stdout:5/902: mkdir d0/d6/d3b/d4b/dc/d8b/d131 0 2024-04-24T23:14:06.441 INFO:tasks.workunit.client.0.smithi031.stdout:5/903: readlink d0/d6/d3b/d4b/d1d/d2e/d28/l99 0 2024-04-24T23:14:06.443 INFO:tasks.workunit.client.1.smithi040.stdout:7/836: symlink db/d12/d20/d52/d4b/d127/d12b/db9/l12c 0 2024-04-24T23:14:06.443 INFO:tasks.workunit.client.1.smithi040.stdout:4/758: dwrite da/d3f/d40/d41/fd9 [0,4194304] 0 2024-04-24T23:14:06.445 INFO:tasks.workunit.client.0.smithi031.stdout:6/973: dread - d13/d15/d1d/d23/d4d/d7b/d11b/f133 zero size 2024-04-24T23:14:06.446 INFO:tasks.workunit.client.0.smithi031.stdout:2/872: dwrite d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/f10b [0,4194304] 0 2024-04-24T23:14:06.450 INFO:tasks.workunit.client.0.smithi031.stdout:1/899: mkdir d1/d121 0 2024-04-24T23:14:06.451 INFO:tasks.workunit.client.0.smithi031.stdout:1/900: readlink d1/da/d12/d7e/d7b/l102 0 2024-04-24T23:14:06.453 INFO:tasks.workunit.client.0.smithi031.stdout:1/901: read d1/d33/d41/d9b/da8/fa9 [274787,1057] 0 2024-04-24T23:14:06.455 INFO:tasks.workunit.client.0.smithi031.stdout:3/840: rename d6/l78 to d6/d14/d91/l103 0 2024-04-24T23:14:06.456 INFO:tasks.workunit.client.0.smithi031.stdout:3/841: stat d6/d14/d1d/d59/cf3 0 2024-04-24T23:14:06.457 INFO:tasks.workunit.client.0.smithi031.stdout:3/842: read d6/d14/d1d/f8b [218274,4339] 0 2024-04-24T23:14:06.458 INFO:tasks.workunit.client.1.smithi040.stdout:3/826: fsync d1/d2a/f6d 0 2024-04-24T23:14:06.461 INFO:tasks.workunit.client.0.smithi031.stdout:6/974: mknod d13/d1b/dd0/c148 0 2024-04-24T23:14:06.462 INFO:tasks.workunit.client.1.smithi040.stdout:0/838: truncate df/d1e/da7/f65 4060141 0 2024-04-24T23:14:06.465 INFO:tasks.workunit.client.0.smithi031.stdout:7/941: truncate d1/d2/dd/d33/d5e/d64/d6d/da1/fcf 174741 0 2024-04-24T23:14:06.465 INFO:tasks.workunit.client.0.smithi031.stdout:2/873: fsync d5/d8/d45/d5b/d49/d2f/d135/d29/d4f/f77 0 2024-04-24T23:14:06.466 INFO:tasks.workunit.client.1.smithi040.stdout:5/695: symlink d2/d7/d8/d1f/d57/d27/d3d/d99/le0 0 2024-04-24T23:14:06.469 INFO:tasks.workunit.client.1.smithi040.stdout:2/840: link da/d10/d7a/f8d da/d29/d55/f131 0 2024-04-24T23:14:06.470 INFO:tasks.workunit.client.0.smithi031.stdout:3/843: fdatasync f0 0 2024-04-24T23:14:06.476 INFO:tasks.workunit.client.1.smithi040.stdout:3/827: rmdir d1/d9/d39 39 2024-04-24T23:14:06.476 INFO:tasks.workunit.client.1.smithi040.stdout:3/828: readlink d1/d9/l1e 0 2024-04-24T23:14:06.477 INFO:tasks.workunit.client.1.smithi040.stdout:3/829: stat d1/d9/d18/da7/fc5 0 2024-04-24T23:14:06.481 INFO:tasks.workunit.client.1.smithi040.stdout:1/745: symlink da/d12/d18/d2a/d4d/dfd/l101 0 2024-04-24T23:14:06.482 INFO:tasks.workunit.client.0.smithi031.stdout:1/902: creat d1/da/d12/d65/dcf/d11/df8/f122 x:0 0 0 2024-04-24T23:14:06.487 INFO:tasks.workunit.client.1.smithi040.stdout:1/746: dwrite da/d12/d18/d2a/d4d/fd2 [0,4194304] 0 2024-04-24T23:14:06.489 INFO:tasks.workunit.client.1.smithi040.stdout:4/759: creat da/d3f/d40/d47/d75/d104/f105 x:0 0 0 2024-04-24T23:14:06.491 INFO:tasks.workunit.client.0.smithi031.stdout:3/844: mknod d6/d14/d71/dad/c104 0 2024-04-24T23:14:06.493 INFO:tasks.workunit.client.0.smithi031.stdout:3/845: dread - d6/d14/d71/dad/ded/ffa zero size 2024-04-24T23:14:06.494 INFO:tasks.workunit.client.0.smithi031.stdout:6/975: rename l3 to d13/d1b/d32/db4/d79/l149 0 2024-04-24T23:14:06.495 INFO:tasks.workunit.client.1.smithi040.stdout:1/747: dwrite da/d12/d14/d25/fee [0,4194304] 0 2024-04-24T23:14:06.497 INFO:tasks.workunit.client.0.smithi031.stdout:3/846: dread d6/d55/fd0 [0,4194304] 0 2024-04-24T23:14:06.501 INFO:tasks.workunit.client.1.smithi040.stdout:0/839: write df/d1e/d20/d21/d41/d8a/f9b [680748,27646] 0 2024-04-24T23:14:06.503 INFO:tasks.workunit.client.1.smithi040.stdout:0/840: readlink df/d12/d58/d5b/l7c 0 2024-04-24T23:14:06.503 INFO:tasks.workunit.client.0.smithi031.stdout:2/874: link d5/d28/f54 d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/da7/f138 0 2024-04-24T23:14:06.504 INFO:tasks.workunit.client.0.smithi031.stdout:7/942: write d1/d2/dd/d33/d5e/d64/d6d/de1/da0/f11c [740031,76102] 0 2024-04-24T23:14:06.506 INFO:tasks.workunit.client.0.smithi031.stdout:7/943: write d1/de4/d106/f12d [917215,79119] 0 2024-04-24T23:14:06.508 INFO:tasks.workunit.client.0.smithi031.stdout:6/976: unlink d13/d15/d1d/d23/d24/d63/f11c 0 2024-04-24T23:14:06.513 INFO:tasks.workunit.client.0.smithi031.stdout:2/875: dread d5/d8/d45/d5b/d49/d6f/fb6 [0,4194304] 0 2024-04-24T23:14:06.513 INFO:tasks.workunit.client.0.smithi031.stdout:3/847: truncate d6/df/d16/d2a/d95/d9b/d1a/f22 6485236 0 2024-04-24T23:14:06.520 INFO:tasks.workunit.client.0.smithi031.stdout:6/977: creat d13/d15/d1d/d23/d44/da0/da6/de6/d102/f14a x:0 0 0 2024-04-24T23:14:06.521 INFO:tasks.workunit.client.0.smithi031.stdout:1/903: link d1/d31/l6b d1/d3e/da3/d69/l123 0 2024-04-24T23:14:06.522 INFO:tasks.workunit.client.0.smithi031.stdout:2/876: fsync d5/d3d/f62 0 2024-04-24T23:14:06.536 INFO:tasks.workunit.client.1.smithi040.stdout:9/777: sync 2024-04-24T23:14:06.536 INFO:tasks.workunit.client.1.smithi040.stdout:6/743: sync 2024-04-24T23:14:06.536 INFO:tasks.workunit.client.1.smithi040.stdout:8/805: sync 2024-04-24T23:14:06.537 INFO:tasks.workunit.client.0.smithi031.stdout:2/877: chown d5/f87 108 1 2024-04-24T23:14:06.537 INFO:tasks.workunit.client.1.smithi040.stdout:9/778: chown d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8 437 1 2024-04-24T23:14:06.538 INFO:tasks.workunit.client.0.smithi031.stdout:2/878: chown d5/d8/d45/d5b/d49/d2f/d135/d29/d4f 237290548 1 2024-04-24T23:14:06.538 INFO:tasks.workunit.client.1.smithi040.stdout:9/779: readlink d7/da/d17/d40/d55/d2f/df7/d6c/d8c/ldb 0 2024-04-24T23:14:06.538 INFO:tasks.workunit.client.0.smithi031.stdout:2/879: chown d5/d28/d43/f67 171548 1 2024-04-24T23:14:06.539 INFO:tasks.workunit.client.1.smithi040.stdout:9/780: stat d7/da/d17/d40/d55/d2f/d28/cd1 0 2024-04-24T23:14:06.547 INFO:tasks.workunit.client.0.smithi031.stdout:6/978: creat d13/d1b/d32/f14b x:0 0 0 2024-04-24T23:14:06.548 INFO:tasks.workunit.client.0.smithi031.stdout:6/979: chown d13/d1b/dd0/f141 21 1 2024-04-24T23:14:06.549 INFO:tasks.workunit.client.0.smithi031.stdout:7/944: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/f98 [0,4194304] 0 2024-04-24T23:14:06.550 INFO:tasks.workunit.client.0.smithi031.stdout:2/880: fdatasync d5/d8/d45/d5b/d49/d2f/d135/d68/f121 0 2024-04-24T23:14:06.556 INFO:tasks.workunit.client.0.smithi031.stdout:3/848: write d6/d14/f33 [152485,96454] 0 2024-04-24T23:14:06.558 INFO:tasks.workunit.client.1.smithi040.stdout:7/837: sync 2024-04-24T23:14:06.560 INFO:tasks.workunit.client.0.smithi031.stdout:6/980: symlink d13/d1b/d32/d9c/ddb/l14c 0 2024-04-24T23:14:06.562 INFO:tasks.workunit.client.0.smithi031.stdout:6/981: chown d13/d15/d1d/d23/d44/d47/d69/dd5/feb 291529 1 2024-04-24T23:14:06.562 INFO:tasks.workunit.client.0.smithi031.stdout:2/881: dread - d5/d8/d15/d56/d84/f106 zero size 2024-04-24T23:14:06.563 INFO:tasks.workunit.client.0.smithi031.stdout:1/904: write d1/da/d12/d65/dcf/d11/f38 [1537015,81161] 0 2024-04-24T23:14:06.565 INFO:tasks.workunit.client.0.smithi031.stdout:1/905: fsync d1/da/d12/d65/dcf/d11/d21/f118 0 2024-04-24T23:14:06.565 INFO:tasks.workunit.client.1.smithi040.stdout:5/696: mknod d2/d2f/d14/ce1 0 2024-04-24T23:14:06.565 INFO:tasks.workunit.client.1.smithi040.stdout:5/697: chown d2/d2f 411 1 2024-04-24T23:14:06.566 INFO:tasks.workunit.client.0.smithi031.stdout:1/906: read d1/d31/f39 [4055722,116874] 0 2024-04-24T23:14:06.567 INFO:tasks.workunit.client.0.smithi031.stdout:3/849: creat d6/df/d16/d2a/d95/d9b/d89/f105 x:0 0 0 2024-04-24T23:14:06.568 INFO:tasks.workunit.client.1.smithi040.stdout:4/760: creat da/d45/f106 x:0 0 0 2024-04-24T23:14:06.571 INFO:tasks.workunit.client.0.smithi031.stdout:7/945: creat d1/f145 x:0 0 0 2024-04-24T23:14:06.574 INFO:tasks.workunit.client.0.smithi031.stdout:2/882: chown d5/d8/d45/d5b/d49/d2f/d135/d29/f5a 30178223 1 2024-04-24T23:14:06.575 INFO:tasks.workunit.client.1.smithi040.stdout:1/748: mkdir da/d12/d18/d2a/d32/d102 0 2024-04-24T23:14:06.578 INFO:tasks.workunit.client.1.smithi040.stdout:0/841: fsync df/d12/d58/f59 0 2024-04-24T23:14:06.579 INFO:tasks.workunit.client.1.smithi040.stdout:0/842: readlink df/d12/d58/dc3/dde/le7 0 2024-04-24T23:14:06.579 INFO:tasks.workunit.client.1.smithi040.stdout:0/843: chown df/d1e/d89/l9d 1024 1 2024-04-24T23:14:06.581 INFO:tasks.workunit.client.0.smithi031.stdout:3/850: mkdir d6/df/d16/d2a/d95/d9b/d29/db7/d106 0 2024-04-24T23:14:06.582 INFO:tasks.workunit.client.1.smithi040.stdout:8/806: dread d0/d8/dc/f96 [0,4194304] 0 2024-04-24T23:14:06.583 INFO:tasks.workunit.client.1.smithi040.stdout:6/744: creat d7/d1a/d20/d51/d1e/d46/d85/ff0 x:0 0 0 2024-04-24T23:14:06.584 INFO:tasks.workunit.client.1.smithi040.stdout:9/781: symlink d7/d92/l110 0 2024-04-24T23:14:06.585 INFO:tasks.workunit.client.0.smithi031.stdout:7/946: rmdir d1/d2/deb/d127 39 2024-04-24T23:14:06.586 INFO:tasks.workunit.client.0.smithi031.stdout:7/947: dread - d1/f10d zero size 2024-04-24T23:14:06.586 INFO:tasks.workunit.client.1.smithi040.stdout:3/830: dwrite d1/d9/d18/d36/d55/f5d [0,4194304] 0 2024-04-24T23:14:06.588 INFO:tasks.workunit.client.1.smithi040.stdout:7/838: rmdir db/d12/d62/d8a/db3 39 2024-04-24T23:14:06.590 INFO:tasks.workunit.client.1.smithi040.stdout:7/839: dread db/d12/d20/d52/d4b/d127/d12b/db9/fe3 [0,4194304] 0 2024-04-24T23:14:06.590 INFO:tasks.workunit.client.0.smithi031.stdout:6/982: write d13/d1b/d32/db4/d79/dd7/dfb/f10f [754630,85143] 0 2024-04-24T23:14:06.592 INFO:tasks.workunit.client.1.smithi040.stdout:5/698: chown d2/d7/d8/d1f/d57/cc6 4 1 2024-04-24T23:14:06.595 INFO:tasks.workunit.client.1.smithi040.stdout:4/761: mkdir da/d45/d58/d107 0 2024-04-24T23:14:06.598 INFO:tasks.workunit.client.1.smithi040.stdout:2/841: creat da/d10/d4d/f132 x:0 0 0 2024-04-24T23:14:06.602 INFO:tasks.workunit.client.0.smithi031.stdout:7/948: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/c146 0 2024-04-24T23:14:06.603 INFO:tasks.workunit.client.1.smithi040.stdout:0/844: fsync df/d1e/d20/d21/d41/d8a/df3/df8/d79/fae 0 2024-04-24T23:14:06.604 INFO:tasks.workunit.client.1.smithi040.stdout:0/845: fsync df/de1/db6/ff9 0 2024-04-24T23:14:06.604 INFO:tasks.workunit.client.0.smithi031.stdout:2/883: symlink d5/d8/d45/d5b/d49/d2f/d135/d29/d26/d120/l139 0 2024-04-24T23:14:06.606 INFO:tasks.workunit.client.1.smithi040.stdout:8/807: unlink d0/d8/dc/f17 0 2024-04-24T23:14:06.606 INFO:tasks.workunit.client.0.smithi031.stdout:6/983: rmdir d13/d1b 39 2024-04-24T23:14:06.608 INFO:tasks.workunit.client.1.smithi040.stdout:6/745: fdatasync d7/de/d89/fb7 0 2024-04-24T23:14:06.610 INFO:tasks.workunit.client.0.smithi031.stdout:1/907: creat d1/da/d12/d65/dcf/d11/d21/d5a/f124 x:0 0 0 2024-04-24T23:14:06.612 INFO:tasks.workunit.client.0.smithi031.stdout:3/851: mkdir d6/df/d16/d2a/d95/d9b/d29/db7/d106/d107 0 2024-04-24T23:14:06.614 INFO:tasks.workunit.client.1.smithi040.stdout:3/831: rmdir d1/d2a/d6a 39 2024-04-24T23:14:06.615 INFO:tasks.workunit.client.1.smithi040.stdout:1/749: dwrite da/d35/dec/ff1 [0,4194304] 0 2024-04-24T23:14:06.616 INFO:tasks.workunit.client.0.smithi031.stdout:3/852: dread d6/d14/d71/dad/fb9 [0,4194304] 0 2024-04-24T23:14:06.616 INFO:tasks.workunit.client.1.smithi040.stdout:9/782: truncate d7/da/d17/d40/d55/d2f/d1f/d4e/f68 1505024 0 2024-04-24T23:14:06.617 INFO:tasks.workunit.client.0.smithi031.stdout:2/884: symlink d5/d8/d15/d56/d84/l13a 0 2024-04-24T23:14:06.620 INFO:tasks.workunit.client.1.smithi040.stdout:7/840: mknod db/d12/d20/d52/d4b/d127/d12b/c12d 0 2024-04-24T23:14:06.624 INFO:tasks.workunit.client.1.smithi040.stdout:4/762: mknod da/d45/c108 0 2024-04-24T23:14:06.626 INFO:tasks.workunit.client.0.smithi031.stdout:2/885: fdatasync d5/d28/d43/f55 0 2024-04-24T23:14:06.628 INFO:tasks.workunit.client.0.smithi031.stdout:6/984: link d13/d15/d1d/d23/d44/d47/d69/faa d13/d15/d1d/d23/d44/d47/d10c/f14d 0 2024-04-24T23:14:06.631 INFO:tasks.workunit.client.0.smithi031.stdout:1/908: creat d1/d106/f125 x:0 0 0 2024-04-24T23:14:06.633 INFO:tasks.workunit.client.0.smithi031.stdout:7/949: truncate d1/dbe/f144 3227371 0 2024-04-24T23:14:06.635 INFO:tasks.workunit.client.0.smithi031.stdout:1/909: dread d1/da/d12/d65/dcf/d11/d21/f28 [0,4194304] 0 2024-04-24T23:14:06.635 INFO:tasks.workunit.client.0.smithi031.stdout:3/853: write d6/d14/d1d/f9d [836924,9895] 0 2024-04-24T23:14:06.638 INFO:tasks.workunit.client.0.smithi031.stdout:2/886: fsync d5/d8/d45/d5b/d49/d2f/d135/d68/f121 0 2024-04-24T23:14:06.645 INFO:tasks.workunit.client.0.smithi031.stdout:3/854: symlink d6/d14/l108 0 2024-04-24T23:14:06.646 INFO:tasks.workunit.client.0.smithi031.stdout:5/904: sync 2024-04-24T23:14:06.647 INFO:tasks.workunit.client.1.smithi040.stdout:1/750: rename da/d12/d14/fc3 to da/d12/d18/d2a/d4d/f103 0 2024-04-24T23:14:06.651 INFO:tasks.workunit.client.0.smithi031.stdout:1/910: dread d1/d3e/d46/dbd/fce [4194304,4194304] 0 2024-04-24T23:14:06.653 INFO:tasks.workunit.client.0.smithi031.stdout:6/985: mknod d13/d15/d1d/d23/d99/c14e 0 2024-04-24T23:14:06.654 INFO:tasks.workunit.client.0.smithi031.stdout:2/887: mkdir d5/d8/d45/d5b/d49/dc4/d13b 0 2024-04-24T23:14:06.656 INFO:tasks.workunit.client.1.smithi040.stdout:7/841: symlink db/d2f/da8/l12e 0 2024-04-24T23:14:06.657 INFO:tasks.workunit.client.1.smithi040.stdout:8/808: dwrite d0/d16/d19/f35 [0,4194304] 0 2024-04-24T23:14:06.657 INFO:tasks.workunit.client.0.smithi031.stdout:7/950: rename d1/d2/dd/c59 to d1/dbe/c147 0 2024-04-24T23:14:06.661 INFO:tasks.workunit.client.1.smithi040.stdout:8/809: dread d0/d7e/dc1/d77/fcd [0,4194304] 0 2024-04-24T23:14:06.661 INFO:tasks.workunit.client.1.smithi040.stdout:4/763: symlink da/d78/l109 0 2024-04-24T23:14:06.663 INFO:tasks.workunit.client.1.smithi040.stdout:9/783: write d7/da/d17/d40/d55/d2f/d1f/d31/f35 [536106,74433] 0 2024-04-24T23:14:06.670 INFO:tasks.workunit.client.0.smithi031.stdout:6/986: unlink d13/d15/d1d/d23/d2d/f41 0 2024-04-24T23:14:06.670 INFO:tasks.workunit.client.1.smithi040.stdout:0/846: fdatasync df/d1e/da7/f65 0 2024-04-24T23:14:06.674 INFO:tasks.workunit.client.1.smithi040.stdout:8/810: dread d0/d38/faa [0,4194304] 0 2024-04-24T23:14:06.678 INFO:tasks.workunit.client.0.smithi031.stdout:3/855: mknod d6/df/c109 0 2024-04-24T23:14:06.681 INFO:tasks.workunit.client.1.smithi040.stdout:8/811: dwrite d0/d52/d9b/ff9 [0,4194304] 0 2024-04-24T23:14:06.682 INFO:tasks.workunit.client.1.smithi040.stdout:3/832: rename d1/d80/dbf/ded to d1/d9/d39/d112/d115 0 2024-04-24T23:14:06.683 INFO:tasks.workunit.client.0.smithi031.stdout:5/905: mkdir d0/d6/d3b/d4b/dc/d8b/d131/d132 0 2024-04-24T23:14:06.683 INFO:tasks.workunit.client.1.smithi040.stdout:5/699: dwrite d2/d2f/d14/f76 [0,4194304] 0 2024-04-24T23:14:06.684 INFO:tasks.workunit.client.0.smithi031.stdout:1/911: write d1/da/d12/d65/dcf/d11/d21/d5a/f76 [4339970,57038] 0 2024-04-24T23:14:06.684 INFO:tasks.workunit.client.1.smithi040.stdout:2/842: truncate da/d29/f42 2759136 0 2024-04-24T23:14:06.686 INFO:tasks.workunit.client.1.smithi040.stdout:1/751: rmdir da/d12/d18/d2a/d4d 39 2024-04-24T23:14:06.688 INFO:tasks.workunit.client.1.smithi040.stdout:1/752: write da/d12/d14/fe0 [4759402,92110] 0 2024-04-24T23:14:06.690 INFO:tasks.workunit.client.0.smithi031.stdout:6/987: truncate d13/d15/d1d/d23/d24/d43/d68/db8/fd1 591982 0 2024-04-24T23:14:06.691 INFO:tasks.workunit.client.0.smithi031.stdout:7/951: write d1/d2/dd/d33/d5e/d64/fce [210013,110847] 0 2024-04-24T23:14:06.691 INFO:tasks.workunit.client.1.smithi040.stdout:0/847: dread df/d1e/d20/ff0 [0,4194304] 0 2024-04-24T23:14:06.697 INFO:tasks.workunit.client.0.smithi031.stdout:2/888: dwrite d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/fbb [0,4194304] 0 2024-04-24T23:14:06.703 INFO:tasks.workunit.client.0.smithi031.stdout:3/856: fsync d6/df/f61 0 2024-04-24T23:14:06.707 INFO:tasks.workunit.client.0.smithi031.stdout:1/912: creat d1/da/d12/d7e/d7b/d4e/f126 x:0 0 0 2024-04-24T23:14:06.709 INFO:tasks.workunit.client.0.smithi031.stdout:6/988: dread - d13/d1b/d32/d9c/ddb/f144 zero size 2024-04-24T23:14:06.711 INFO:tasks.workunit.client.0.smithi031.stdout:7/952: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/f69 [1857750,101832] 0 2024-04-24T23:14:06.715 INFO:tasks.workunit.client.1.smithi040.stdout:6/746: creat d7/d1a/d20/ff1 x:0 0 0 2024-04-24T23:14:06.715 INFO:tasks.workunit.client.1.smithi040.stdout:6/747: dread - d7/d1a/d9f/fa9 zero size 2024-04-24T23:14:06.716 INFO:tasks.workunit.client.1.smithi040.stdout:6/748: write d7/d1a/f42 [158233,82833] 0 2024-04-24T23:14:06.717 INFO:tasks.workunit.client.1.smithi040.stdout:6/749: stat d7/d1a/d20/d51/fdb 0 2024-04-24T23:14:06.717 INFO:tasks.workunit.client.1.smithi040.stdout:6/750: fdatasync d7/d1a/d20/d51/d1e/d46/f98 0 2024-04-24T23:14:06.718 INFO:tasks.workunit.client.0.smithi031.stdout:5/906: rename d0/d6/d3b/d4b/f96 to d0/d7/d74/de2/f133 0 2024-04-24T23:14:06.722 INFO:tasks.workunit.client.0.smithi031.stdout:1/913: mknod d1/d3e/d46/df4/c127 0 2024-04-24T23:14:06.725 INFO:tasks.workunit.client.0.smithi031.stdout:6/989: mkdir d13/d15/d1d/d23/d24/d8b/d14f 0 2024-04-24T23:14:06.726 INFO:tasks.workunit.client.0.smithi031.stdout:5/907: dwrite d0/d6/d3b/d4b/dc/d25/d55/f109 [0,4194304] 0 2024-04-24T23:14:06.727 INFO:tasks.workunit.client.1.smithi040.stdout:2/843: fdatasync da/d29/f7f 0 2024-04-24T23:14:06.728 INFO:tasks.workunit.client.0.smithi031.stdout:5/908: chown d0/d7/d56/d7c/ff5 932634576 1 2024-04-24T23:14:06.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/990: fsync d13/d15/d1d/d23/d24/d43/d68/db8/fcc 0 2024-04-24T23:14:06.730 INFO:tasks.workunit.client.0.smithi031.stdout:7/953: mkdir d1/d2/dd/d33/d5e/d64/d6d/dae/dd7/d148 0 2024-04-24T23:14:06.732 INFO:tasks.workunit.client.0.smithi031.stdout:2/889: write d5/d8/d45/d5b/d49/d2f/d135/fc1 [4392289,74234] 0 2024-04-24T23:14:06.733 INFO:tasks.workunit.client.1.smithi040.stdout:1/753: creat da/d12/d14/d58/f104 x:0 0 0 2024-04-24T23:14:06.734 INFO:tasks.workunit.client.0.smithi031.stdout:3/857: mknod d6/df/d16/d2a/d95/dff/c10a 0 2024-04-24T23:14:06.743 INFO:tasks.workunit.client.0.smithi031.stdout:5/909: fsync d0/d6/d3b/d4f/ff2 0 2024-04-24T23:14:06.746 INFO:tasks.workunit.client.1.smithi040.stdout:4/764: link da/d3f/d40/d41/d60/f42 da/d3f/d6b/d3c/d69/df3/f10a 0 2024-04-24T23:14:06.747 INFO:tasks.workunit.client.1.smithi040.stdout:9/784: mkdir d7/da/d17/d40/d55/d2f/d1f/d31/dff/d111 0 2024-04-24T23:14:06.748 INFO:tasks.workunit.client.0.smithi031.stdout:6/991: creat d13/d15/d1d/d23/d99/f150 x:0 0 0 2024-04-24T23:14:06.751 INFO:tasks.workunit.client.0.smithi031.stdout:3/858: creat d6/df/d16/d57/de1/f10b x:0 0 0 2024-04-24T23:14:06.751 INFO:tasks.workunit.client.1.smithi040.stdout:7/842: write db/d2f/d37/d7e/f86 [2030381,59318] 0 2024-04-24T23:14:06.753 INFO:tasks.workunit.client.1.smithi040.stdout:9/785: dwrite d7/da/d17/d40/f42 [0,4194304] 0 2024-04-24T23:14:06.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/890: rename d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/cf7 to d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/c13c 0 2024-04-24T23:14:06.756 INFO:tasks.workunit.client.0.smithi031.stdout:1/914: symlink d1/da/d12/d65/dcf/l128 0 2024-04-24T23:14:06.758 INFO:tasks.workunit.client.0.smithi031.stdout:6/992: dwrite d13/d1b/d32/d9c/ddb/f144 [0,4194304] 0 2024-04-24T23:14:06.760 INFO:tasks.workunit.client.1.smithi040.stdout:6/751: mkdir d7/d1a/d20/d51/d2f/df2 0 2024-04-24T23:14:06.763 INFO:tasks.workunit.client.1.smithi040.stdout:8/812: mknod d0/d16/d1b/d47/def/c10d 0 2024-04-24T23:14:06.764 INFO:tasks.workunit.client.0.smithi031.stdout:3/859: dwrite d6/df/d16/d53/fef [0,4194304] 0 2024-04-24T23:14:06.766 INFO:tasks.workunit.client.0.smithi031.stdout:7/954: unlink d1/c1b 0 2024-04-24T23:14:06.768 INFO:tasks.workunit.client.1.smithi040.stdout:2/844: truncate da/d1c/d37/d39/d77/f68 494874 0 2024-04-24T23:14:06.769 INFO:tasks.workunit.client.1.smithi040.stdout:2/845: fsync da/d1c/d37/d39/f76 0 2024-04-24T23:14:06.770 INFO:tasks.workunit.client.0.smithi031.stdout:7/955: write d1/d2/dd/d33/d5e/d64/d6d/dae/dd7/f135 [849696,28872] 0 2024-04-24T23:14:06.772 INFO:tasks.workunit.client.0.smithi031.stdout:7/956: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d65/f98 286 1 2024-04-24T23:14:06.773 INFO:tasks.workunit.client.0.smithi031.stdout:2/891: unlink d5/d8/d45/d5b/d49/d2f/d135/d29/d4f/la4 0 2024-04-24T23:14:06.778 INFO:tasks.workunit.client.1.smithi040.stdout:4/765: unlink da/d3f/d40/d41/d60/f82 0 2024-04-24T23:14:06.780 INFO:tasks.workunit.client.1.smithi040.stdout:7/843: fdatasync db/d12/d20/d52/d6c/fb2 0 2024-04-24T23:14:06.782 INFO:tasks.workunit.client.0.smithi031.stdout:7/957: readlink d1/d2/l7b 0 2024-04-24T23:14:06.786 INFO:tasks.workunit.client.1.smithi040.stdout:8/813: rename d0/d38/d4c/lce to d0/d7e/dc1/d9e/d90/d91/l10e 0 2024-04-24T23:14:06.792 INFO:tasks.workunit.client.1.smithi040.stdout:5/700: sync 2024-04-24T23:14:06.792 INFO:tasks.workunit.client.1.smithi040.stdout:3/833: sync 2024-04-24T23:14:06.794 INFO:tasks.workunit.client.0.smithi031.stdout:3/860: link d6/df/d16/d2a/l44 d6/d14/l10c 0 2024-04-24T23:14:06.796 INFO:tasks.workunit.client.1.smithi040.stdout:0/848: creat df/d1e/d20/d21/d41/d8a/df3/df8/f111 x:0 0 0 2024-04-24T23:14:06.799 INFO:tasks.workunit.client.0.smithi031.stdout:6/993: dread d13/d15/d1d/d23/d44/da0/da6/fb6 [0,4194304] 0 2024-04-24T23:14:06.801 INFO:tasks.workunit.client.1.smithi040.stdout:4/766: dread da/d3f/d40/d41/d8e/dd3/fdb [0,4194304] 0 2024-04-24T23:14:06.804 INFO:tasks.workunit.client.0.smithi031.stdout:5/910: dwrite d0/d6/d3b/d4b/d1d/d2e/dc6/f6f [0,4194304] 0 2024-04-24T23:14:06.807 INFO:tasks.workunit.client.0.smithi031.stdout:3/861: rename d6/d14/l6a to d6/d14/d1d/d69/l10d 0 2024-04-24T23:14:06.807 INFO:tasks.workunit.client.1.smithi040.stdout:1/754: write da/d35/d38/d47/f80 [2144226,85797] 0 2024-04-24T23:14:06.809 INFO:tasks.workunit.client.1.smithi040.stdout:6/752: mknod d7/d1a/d20/d51/cf3 0 2024-04-24T23:14:06.813 INFO:tasks.workunit.client.0.smithi031.stdout:7/958: getdents d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d2c 0 2024-04-24T23:14:06.815 INFO:tasks.workunit.client.0.smithi031.stdout:1/915: dwrite d1/d3e/da2/fb2 [0,4194304] 0 2024-04-24T23:14:06.822 INFO:tasks.workunit.client.1.smithi040.stdout:3/834: creat d1/d9/d2e/dcf/d110/f116 x:0 0 0 2024-04-24T23:14:06.822 INFO:tasks.workunit.client.0.smithi031.stdout:2/892: write d5/d28/d4e/fd6 [1058931,11261] 0 2024-04-24T23:14:06.823 INFO:tasks.workunit.client.1.smithi040.stdout:9/786: dwrite d7/da/d17/d40/d55/d2f/d28/f6b [0,4194304] 0 2024-04-24T23:14:06.824 INFO:tasks.workunit.client.0.smithi031.stdout:6/994: rename d13/d15/d1d/d23/d2d/c7e to d13/d15/d1d/d23/d24/d63/c151 0 2024-04-24T23:14:06.826 INFO:tasks.workunit.client.1.smithi040.stdout:0/849: creat df/d12/d58/d5b/f112 x:0 0 0 2024-04-24T23:14:06.826 INFO:tasks.workunit.client.0.smithi031.stdout:6/995: write d13/d1b/d32/db4/d79/dd7/dfb/f10f [1446215,125909] 0 2024-04-24T23:14:06.827 INFO:tasks.workunit.client.0.smithi031.stdout:6/996: write d13/d15/dea/f146 [503274,78101] 0 2024-04-24T23:14:06.828 INFO:tasks.workunit.client.1.smithi040.stdout:7/844: mknod db/d12/d20/d52/dfc/d11b/c12f 0 2024-04-24T23:14:06.829 INFO:tasks.workunit.client.0.smithi031.stdout:1/916: fsync d1/d33/d36/f44 0 2024-04-24T23:14:06.832 INFO:tasks.workunit.client.1.smithi040.stdout:9/787: dwrite d7/d92/ff1 [0,4194304] 0 2024-04-24T23:14:06.837 INFO:tasks.workunit.client.0.smithi031.stdout:1/917: dwrite d1/da/d12/d65/dcf/d11/f38 [0,4194304] 0 2024-04-24T23:14:06.838 INFO:tasks.workunit.client.0.smithi031.stdout:1/918: chown d1/da/df0/fb7 147672407 1 2024-04-24T23:14:06.838 INFO:tasks.workunit.client.0.smithi031.stdout:5/911: write d0/d7/fc9 [1000235,93426] 0 2024-04-24T23:14:06.844 INFO:tasks.workunit.client.1.smithi040.stdout:8/814: mkdir d0/d16/d19/d102/d10f 0 2024-04-24T23:14:06.846 INFO:tasks.workunit.client.1.smithi040.stdout:6/753: write d7/d1a/d20/d51/d1e/f49 [5106690,100426] 0 2024-04-24T23:14:06.846 INFO:tasks.workunit.client.1.smithi040.stdout:5/701: mknod d2/d7/d8/d1f/d57/ce2 0 2024-04-24T23:14:06.848 INFO:tasks.workunit.client.1.smithi040.stdout:2/846: creat da/d1c/d37/d39/d3f/d52/d78/f133 x:0 0 0 2024-04-24T23:14:06.849 INFO:tasks.workunit.client.1.smithi040.stdout:3/835: rmdir d1/d2d/d2b 39 2024-04-24T23:14:06.851 INFO:tasks.workunit.client.1.smithi040.stdout:8/815: dread d0/d16/f57 [0,4194304] 0 2024-04-24T23:14:06.858 INFO:tasks.workunit.client.1.smithi040.stdout:9/788: unlink d7/da/d17/d40/d55/d2f/df7/d6c/f91 0 2024-04-24T23:14:06.859 INFO:tasks.workunit.client.0.smithi031.stdout:6/997: mkdir d13/d15/d1d/d23/d44/da0/da6/de6/d142/d152 0 2024-04-24T23:14:06.861 INFO:tasks.workunit.client.0.smithi031.stdout:2/893: write d5/d8/f98 [951843,64212] 0 2024-04-24T23:14:06.862 INFO:tasks.workunit.client.0.smithi031.stdout:2/894: dread - d5/d28/d43/f57 zero size 2024-04-24T23:14:06.866 INFO:tasks.workunit.client.0.smithi031.stdout:3/862: link d6/d14/l21 d6/d14/d71/l10e 0 2024-04-24T23:14:06.869 INFO:tasks.workunit.client.1.smithi040.stdout:4/767: dwrite da/d3f/d40/d41/d8e/db8/fc4 [0,4194304] 0 2024-04-24T23:14:06.870 INFO:tasks.workunit.client.1.smithi040.stdout:1/755: truncate da/d35/d38/d47/fc1 1834491 0 2024-04-24T23:14:06.874 INFO:tasks.workunit.client.1.smithi040.stdout:4/768: truncate da/d3f/d40/d47/d75/d7c/fef 461243 0 2024-04-24T23:14:06.874 INFO:tasks.workunit.client.1.smithi040.stdout:3/836: mknod d1/d9/d18/d36/d55/c117 0 2024-04-24T23:14:06.875 INFO:tasks.workunit.client.1.smithi040.stdout:0/850: creat df/d1e/d89/ddf/f113 x:0 0 0 2024-04-24T23:14:06.876 INFO:tasks.workunit.client.1.smithi040.stdout:4/769: truncate da/d3f/d40/d41/d62/fe1 301557 0 2024-04-24T23:14:06.877 INFO:tasks.workunit.client.1.smithi040.stdout:0/851: chown df/d1e/d20/d21/f9c 1973846 1 2024-04-24T23:14:06.879 INFO:tasks.workunit.client.1.smithi040.stdout:8/816: symlink d0/d16/d1b/d3f/l110 0 2024-04-24T23:14:06.879 INFO:tasks.workunit.client.1.smithi040.stdout:2/847: dwrite da/d10/fef [0,4194304] 0 2024-04-24T23:14:06.880 INFO:tasks.workunit.client.1.smithi040.stdout:4/770: dread da/d3f/d40/d41/d62/fe1 [0,4194304] 0 2024-04-24T23:14:06.880 INFO:tasks.workunit.client.1.smithi040.stdout:7/845: symlink db/d12/l130 0 2024-04-24T23:14:06.881 INFO:tasks.workunit.client.1.smithi040.stdout:9/789: creat d7/da/d17/d40/d55/d2f/d28/d67/f112 x:0 0 0 2024-04-24T23:14:06.882 INFO:tasks.workunit.client.1.smithi040.stdout:4/771: stat da/d3f/d6b/d2b/f35 0 2024-04-24T23:14:06.883 INFO:tasks.workunit.client.1.smithi040.stdout:6/754: creat d7/d1a/d20/d8c/dd5/ff4 x:0 0 0 2024-04-24T23:14:06.885 INFO:tasks.workunit.client.1.smithi040.stdout:4/772: chown da/d3f/d40/l64 322553795 1 2024-04-24T23:14:06.887 INFO:tasks.workunit.client.1.smithi040.stdout:6/755: readlink d7/d1a/d20/d51/d2a/lea 0 2024-04-24T23:14:06.888 INFO:tasks.workunit.client.1.smithi040.stdout:3/837: creat d1/d9/d18/da7/f118 x:0 0 0 2024-04-24T23:14:06.890 INFO:tasks.workunit.client.1.smithi040.stdout:8/817: dwrite d0/d8/dc/d34/f3c [0,4194304] 0 2024-04-24T23:14:06.892 INFO:tasks.workunit.client.1.smithi040.stdout:8/818: readlink d0/d16/d19/d71/d76/laf 0 2024-04-24T23:14:06.896 INFO:tasks.workunit.client.1.smithi040.stdout:9/790: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/f35 [0,4194304] 0 2024-04-24T23:14:06.898 INFO:tasks.workunit.client.1.smithi040.stdout:7/846: rename db/d12/d7d/ff6 to db/d12/d3f/d7a/f131 0 2024-04-24T23:14:06.900 INFO:tasks.workunit.client.1.smithi040.stdout:9/791: rename d7/da/d17/d40/d55 to d7/da/d17/d40/d55/d2f/ded/d113 22 2024-04-24T23:14:06.903 INFO:tasks.workunit.client.1.smithi040.stdout:6/756: mkdir d7/d1a/d20/d8c/df5 0 2024-04-24T23:14:06.905 INFO:tasks.workunit.client.1.smithi040.stdout:6/757: chown d7/d1a/d20/d51/f26 241170 1 2024-04-24T23:14:06.905 INFO:tasks.workunit.client.1.smithi040.stdout:5/702: getdents d2/d7/d8/d64/d6b 0 2024-04-24T23:14:06.907 INFO:tasks.workunit.client.1.smithi040.stdout:5/703: readlink d2/d7/d8/l2c 0 2024-04-24T23:14:06.907 INFO:tasks.workunit.client.1.smithi040.stdout:4/773: mkdir da/d3f/d6b/d3c/d69/df3/d10b 0 2024-04-24T23:14:06.907 INFO:tasks.workunit.client.1.smithi040.stdout:1/756: dread da/d12/d55/f8e [0,4194304] 0 2024-04-24T23:14:06.909 INFO:tasks.workunit.client.1.smithi040.stdout:3/838: mknod d1/d9/d39/c119 0 2024-04-24T23:14:06.910 INFO:tasks.workunit.client.1.smithi040.stdout:0/852: creat df/d1e/d20/f114 x:0 0 0 2024-04-24T23:14:06.910 INFO:tasks.workunit.client.1.smithi040.stdout:2/848: creat da/d1c/d37/d39/d3f/d52/d78/f134 x:0 0 0 2024-04-24T23:14:06.910 INFO:tasks.workunit.client.1.smithi040.stdout:8/819: getdents d0/d8/d22/db8/de5 0 2024-04-24T23:14:06.911 INFO:tasks.workunit.client.1.smithi040.stdout:0/853: chown df/d1e/d89/ddf 133263 1 2024-04-24T23:14:06.911 INFO:tasks.workunit.client.1.smithi040.stdout:6/758: mkdir d7/d1a/d9f/df6 0 2024-04-24T23:14:06.912 INFO:tasks.workunit.client.1.smithi040.stdout:5/704: mknod d2/d7/d8/d1f/d57/d27/d3c/d9b/ce3 0 2024-04-24T23:14:06.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:06 smithi040 ceph-mon[33128]: pgmap v36: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 73 MiB/s rd, 197 MiB/s wr, 445 op/s 2024-04-24T23:14:06.912 INFO:tasks.workunit.client.1.smithi040.stdout:8/820: write d0/d8/d22/f6a [5069333,51076] 0 2024-04-24T23:14:06.920 INFO:tasks.workunit.client.1.smithi040.stdout:0/854: write df/d1e/d20/d61/f86 [3945618,12602] 0 2024-04-24T23:14:06.921 INFO:tasks.workunit.client.1.smithi040.stdout:8/821: chown d0/d16/d1b/d3f/l110 16090 1 2024-04-24T23:14:06.927 INFO:tasks.workunit.client.1.smithi040.stdout:5/705: dwrite d2/d7/d8/d6d/fc7 [0,4194304] 0 2024-04-24T23:14:06.928 INFO:tasks.workunit.client.1.smithi040.stdout:0/855: dwrite df/d12/d58/dc3/dcd/f108 [0,4194304] 0 2024-04-24T23:14:06.984 INFO:tasks.workunit.client.1.smithi040.stdout:1/757: sync 2024-04-24T23:14:06.994 INFO:tasks.workunit.client.1.smithi040.stdout:1/758: dread da/d35/dab/f52 [0,4194304] 0 2024-04-24T23:14:06.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:06 smithi031 ceph-mon[26411]: pgmap v36: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 73 MiB/s rd, 197 MiB/s wr, 445 op/s 2024-04-24T23:14:07.033 INFO:tasks.workunit.client.0.smithi031.stdout:5/912: dwrite d0/d6/d3b/d4f/f98 [0,4194304] 0 2024-04-24T23:14:07.034 INFO:tasks.workunit.client.1.smithi040.stdout:4/774: mkdir da/d3f/d40/d47/d75/daa/d10c 0 2024-04-24T23:14:07.034 INFO:tasks.workunit.client.0.smithi031.stdout:2/895: rename d5/d28/d4e/f111 to d5/d8/d45/d5b/d49/d2f/d135/d29/d26/d120/f13d 0 2024-04-24T23:14:07.037 INFO:tasks.workunit.client.1.smithi040.stdout:2/849: mknod da/d1c/d37/d39/d3f/d52/d78/df5/c135 0 2024-04-24T23:14:07.039 INFO:tasks.workunit.client.0.smithi031.stdout:1/919: getdents d1/d3e/da3 0 2024-04-24T23:14:07.042 INFO:tasks.workunit.client.1.smithi040.stdout:2/850: read da/d1c/d37/f126 [3047797,110187] 0 2024-04-24T23:14:07.043 INFO:tasks.workunit.client.1.smithi040.stdout:2/851: chown da/d1c/d37/d8a/f127 0 1 2024-04-24T23:14:07.043 INFO:tasks.workunit.client.1.smithi040.stdout:8/822: rename d0/d7e/dc1/l8b to d0/d7e/dc1/d87/l111 0 2024-04-24T23:14:07.045 INFO:tasks.workunit.client.0.smithi031.stdout:3/863: creat d6/df/d16/d2a/d95/d9b/d1a/d25/d83/f10f x:0 0 0 2024-04-24T23:14:07.047 INFO:tasks.workunit.client.0.smithi031.stdout:6/998: dread d13/d1b/d32/d9c/fc4 [4194304,4194304] 0 2024-04-24T23:14:07.047 INFO:tasks.workunit.client.0.smithi031.stdout:3/864: chown d6/df/d16/d2a/d95/d9b/d60 2 1 2024-04-24T23:14:07.048 INFO:tasks.workunit.client.0.smithi031.stdout:5/913: creat d0/d6/d3b/d4b/d1d/d2e/d28/deb/f134 x:0 0 0 2024-04-24T23:14:07.049 INFO:tasks.workunit.client.1.smithi040.stdout:9/792: dread d7/da/d17/d40/d55/d2f/d28/f29 [0,4194304] 0 2024-04-24T23:14:07.049 INFO:tasks.workunit.client.0.smithi031.stdout:6/999: stat d13/d15/d1d/d23/d44/d47/d69/dd5/feb 0 2024-04-24T23:14:07.050 INFO:tasks.workunit.client.1.smithi040.stdout:0/856: fdatasync df/d12/d4b/db8/f10a 0 2024-04-24T23:14:07.050 INFO:tasks.workunit.client.1.smithi040.stdout:0/857: readlink df/lf7 0 2024-04-24T23:14:07.051 INFO:tasks.workunit.client.0.smithi031.stdout:2/896: truncate d5/d8/f10 412954 0 2024-04-24T23:14:07.052 INFO:tasks.workunit.client.1.smithi040.stdout:1/759: fsync da/d12/d14/f3f 0 2024-04-24T23:14:07.053 INFO:tasks.workunit.client.1.smithi040.stdout:4/775: creat da/d45/d48/d6e/ddd/f10d x:0 0 0 2024-04-24T23:14:07.055 INFO:tasks.workunit.client.0.smithi031.stdout:3/865: creat d6/d14/d1d/f110 x:0 0 0 2024-04-24T23:14:07.057 INFO:tasks.workunit.client.1.smithi040.stdout:1/760: dwrite da/ff [0,4194304] 0 2024-04-24T23:14:07.057 INFO:tasks.workunit.client.1.smithi040.stdout:6/759: creat d7/d1a/d20/d51/d2f/df2/ff7 x:0 0 0 2024-04-24T23:14:07.059 INFO:tasks.workunit.client.0.smithi031.stdout:5/914: truncate d0/d7/fa3 898449 0 2024-04-24T23:14:07.061 INFO:tasks.workunit.client.0.smithi031.stdout:2/897: fsync d5/d28/d4e/fa1 0 2024-04-24T23:14:07.063 INFO:tasks.workunit.client.1.smithi040.stdout:8/823: dread d0/d16/d19/f5f [0,4194304] 0 2024-04-24T23:14:07.064 INFO:tasks.workunit.client.0.smithi031.stdout:1/920: rmdir d1/d121 0 2024-04-24T23:14:07.066 INFO:tasks.workunit.client.0.smithi031.stdout:3/866: symlink d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/l111 0 2024-04-24T23:14:07.067 INFO:tasks.workunit.client.1.smithi040.stdout:7/847: dwrite db/d2f/f9f [0,4194304] 0 2024-04-24T23:14:07.070 INFO:tasks.workunit.client.0.smithi031.stdout:5/915: creat d0/d7/d56/d7c/d7e/f135 x:0 0 0 2024-04-24T23:14:07.072 INFO:tasks.workunit.client.1.smithi040.stdout:9/793: mkdir d7/da/d17/d40/d55/d2f/d1f/d31/dff/d114 0 2024-04-24T23:14:07.075 INFO:tasks.workunit.client.1.smithi040.stdout:0/858: rename df/d12/d1a/l78 to df/d1e/d20/d21/d41/l115 0 2024-04-24T23:14:07.077 INFO:tasks.workunit.client.0.smithi031.stdout:2/898: creat d5/d8/d45/d5b/d49/d2f/d135/d68/f13e x:0 0 0 2024-04-24T23:14:07.078 INFO:tasks.workunit.client.0.smithi031.stdout:2/899: chown d5/d28/fca 506 1 2024-04-24T23:14:07.082 INFO:tasks.workunit.client.0.smithi031.stdout:1/921: symlink d1/da/d12/d7e/d7b/d4e/l129 0 2024-04-24T23:14:07.082 INFO:tasks.workunit.client.1.smithi040.stdout:3/839: dwrite d1/d9/d18/d36/d55/d62/f8b [0,4194304] 0 2024-04-24T23:14:07.083 INFO:tasks.workunit.client.1.smithi040.stdout:4/776: mknod da/d78/dbb/c10e 0 2024-04-24T23:14:07.085 INFO:tasks.workunit.client.0.smithi031.stdout:5/916: rmdir d0/d6/d32/d6b 39 2024-04-24T23:14:07.091 INFO:tasks.workunit.client.1.smithi040.stdout:6/760: mkdir d7/d1a/d20/d51/d2f/df8 0 2024-04-24T23:14:07.092 INFO:tasks.workunit.client.0.smithi031.stdout:1/922: write d1/d3e/d46/dbd/fce [7381273,79367] 0 2024-04-24T23:14:07.094 INFO:tasks.workunit.client.0.smithi031.stdout:1/923: fdatasync d1/da/d12/d7e/d7b/d4e/d98/ffb 0 2024-04-24T23:14:07.096 INFO:tasks.workunit.client.0.smithi031.stdout:5/917: dread - d0/d6/d3b/d4b/dc/d86/dfd/fe7 zero size 2024-04-24T23:14:07.097 INFO:tasks.workunit.client.1.smithi040.stdout:8/824: creat d0/d8/dc/d34/da7/f112 x:0 0 0 2024-04-24T23:14:07.099 INFO:tasks.workunit.client.1.smithi040.stdout:8/825: write d0/d7e/dc1/f80 [1200257,125273] 0 2024-04-24T23:14:07.102 INFO:tasks.workunit.client.0.smithi031.stdout:5/918: fsync d0/d6/d3b/d4b/dc/f26 0 2024-04-24T23:14:07.104 INFO:tasks.workunit.client.1.smithi040.stdout:7/848: creat db/d12/d3f/d7a/f132 x:0 0 0 2024-04-24T23:14:07.106 INFO:tasks.workunit.client.0.smithi031.stdout:1/924: symlink d1/d3e/d46/l12a 0 2024-04-24T23:14:07.107 INFO:tasks.workunit.client.1.smithi040.stdout:0/859: mknod df/d1e/d89/c116 0 2024-04-24T23:14:07.110 INFO:tasks.workunit.client.1.smithi040.stdout:0/860: read df/de1/f9e [1266486,45602] 0 2024-04-24T23:14:07.111 INFO:tasks.workunit.client.1.smithi040.stdout:0/861: write df/d12/d58/f72 [1639927,109720] 0 2024-04-24T23:14:07.114 INFO:tasks.workunit.client.1.smithi040.stdout:4/777: mknod da/d3f/d40/d47/d75/d104/c10f 0 2024-04-24T23:14:07.115 INFO:tasks.workunit.client.0.smithi031.stdout:5/919: rmdir d0/d6/d3b/d4b/dc/d8b 39 2024-04-24T23:14:07.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/867: write d6/d14/d1d/d59/fe6 [297913,6556] 0 2024-04-24T23:14:07.116 INFO:tasks.workunit.client.0.smithi031.stdout:2/900: write d5/d8/d15/d56/f8f [752260,83340] 0 2024-04-24T23:14:07.117 INFO:tasks.workunit.client.1.smithi040.stdout:1/761: symlink da/d35/d38/d47/dbc/l105 0 2024-04-24T23:14:07.118 INFO:tasks.workunit.client.0.smithi031.stdout:5/920: chown d0/d6/d3b/d4b/d1d/d2e/d28/d77/lcf 594670449 1 2024-04-24T23:14:07.120 INFO:tasks.workunit.client.1.smithi040.stdout:2/852: creat da/d1c/d37/d39/d3f/d52/d78/f136 x:0 0 0 2024-04-24T23:14:07.120 INFO:tasks.workunit.client.1.smithi040.stdout:2/853: dread - da/d10/d4d/dd7/f129 zero size 2024-04-24T23:14:07.121 INFO:tasks.workunit.client.1.smithi040.stdout:5/706: getdents d2/d7/d8/d1f/d90 0 2024-04-24T23:14:07.126 INFO:tasks.workunit.client.1.smithi040.stdout:8/826: rename d0/d7e/dc1/d77 to d0/d52/d9b/d101/d113 0 2024-04-24T23:14:07.130 INFO:tasks.workunit.client.1.smithi040.stdout:7/849: write db/d12/d3f/d7a/fe1 [6726954,56233] 0 2024-04-24T23:14:07.133 INFO:tasks.workunit.client.1.smithi040.stdout:8/827: dwrite d0/d16/d1b/d47/f103 [0,4194304] 0 2024-04-24T23:14:07.133 INFO:tasks.workunit.client.0.smithi031.stdout:7/959: sync 2024-04-24T23:14:07.134 INFO:tasks.workunit.client.1.smithi040.stdout:8/828: chown d0/d8/d22/db8/dc2 16 1 2024-04-24T23:14:07.134 INFO:tasks.workunit.client.1.smithi040.stdout:8/829: chown d0/d16/d19/d71/c7b 2059955617 1 2024-04-24T23:14:07.135 INFO:tasks.workunit.client.1.smithi040.stdout:3/840: unlink d1/d2a/d8d/f99 0 2024-04-24T23:14:07.138 INFO:tasks.workunit.client.1.smithi040.stdout:4/778: creat da/d3f/d40/d9b/f110 x:0 0 0 2024-04-24T23:14:07.142 INFO:tasks.workunit.client.0.smithi031.stdout:7/960: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d10e/f136 [0,4194304] 0 2024-04-24T23:14:07.145 INFO:tasks.workunit.client.0.smithi031.stdout:7/961: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/f139 [578425,42083] 0 2024-04-24T23:14:07.147 INFO:tasks.workunit.client.1.smithi040.stdout:1/762: dwrite da/d12/d18/d2a/d4d/d7d/fa9 [0,4194304] 0 2024-04-24T23:14:07.148 INFO:tasks.workunit.client.1.smithi040.stdout:8/830: dread d0/d28/f39 [0,4194304] 0 2024-04-24T23:14:07.149 INFO:tasks.workunit.client.1.smithi040.stdout:5/707: dread d2/d7/d8/d1f/d57/d27/d3d/f7a [0,4194304] 0 2024-04-24T23:14:07.158 INFO:tasks.workunit.client.1.smithi040.stdout:4/779: truncate da/d45/d48/d6e/d70/f72 2617401 0 2024-04-24T23:14:07.158 INFO:tasks.workunit.client.1.smithi040.stdout:4/780: dread - da/d45/d48/d6e/d70/ffc zero size 2024-04-24T23:14:07.161 INFO:tasks.workunit.client.1.smithi040.stdout:3/841: dread d1/d2a/f9a [0,4194304] 0 2024-04-24T23:14:07.161 INFO:tasks.workunit.client.1.smithi040.stdout:9/794: getdents d7/d5e/de5 0 2024-04-24T23:14:07.164 INFO:tasks.workunit.client.1.smithi040.stdout:4/781: dwrite da/d45/d58/fa6 [0,4194304] 0 2024-04-24T23:14:07.166 INFO:tasks.workunit.client.1.smithi040.stdout:1/763: unlink da/d12/d14/l8d 0 2024-04-24T23:14:07.169 INFO:tasks.workunit.client.1.smithi040.stdout:8/831: creat d0/d16/d1b/d3f/f114 x:0 0 0 2024-04-24T23:14:07.170 INFO:tasks.workunit.client.1.smithi040.stdout:7/850: symlink db/d12/d62/d8a/l133 0 2024-04-24T23:14:07.170 INFO:tasks.workunit.client.1.smithi040.stdout:6/761: sync 2024-04-24T23:14:07.174 INFO:tasks.workunit.client.1.smithi040.stdout:8/832: dwrite d0/d8/da0/f106 [0,4194304] 0 2024-04-24T23:14:07.176 INFO:tasks.workunit.client.1.smithi040.stdout:0/862: creat df/d12/d1a/d71/f117 x:0 0 0 2024-04-24T23:14:07.185 INFO:tasks.workunit.client.1.smithi040.stdout:3/842: symlink d1/d9/d18/d36/d55/d62/l11a 0 2024-04-24T23:14:07.212 INFO:tasks.workunit.client.0.smithi031.stdout:1/925: symlink d1/da/d12/d65/dcf/d11/d21/l12b 0 2024-04-24T23:14:07.215 INFO:tasks.workunit.client.0.smithi031.stdout:2/901: mknod d5/d8/d45/d5b/d49/d6f/d119/d11d/c13f 0 2024-04-24T23:14:07.218 INFO:tasks.workunit.client.0.smithi031.stdout:7/962: unlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/de5/f11d 0 2024-04-24T23:14:07.219 INFO:tasks.workunit.client.0.smithi031.stdout:1/926: dread - d1/d3e/d46/fd3 zero size 2024-04-24T23:14:07.223 INFO:tasks.workunit.client.0.smithi031.stdout:5/921: symlink d0/d6/d3b/d4b/dc/de3/d125/l136 0 2024-04-24T23:14:07.227 INFO:tasks.workunit.client.0.smithi031.stdout:1/927: write d1/da/d12/d7e/d7b/d4e/f94 [1748673,15624] 0 2024-04-24T23:14:07.228 INFO:tasks.workunit.client.0.smithi031.stdout:7/963: truncate d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f/f9b 189978 0 2024-04-24T23:14:07.230 INFO:tasks.workunit.client.0.smithi031.stdout:5/922: read - d0/d6/d3b/d4b/dc/d25/fe4 zero size 2024-04-24T23:14:07.230 INFO:tasks.workunit.client.0.smithi031.stdout:5/923: readlink d0/d7/d74/de2/l120 0 2024-04-24T23:14:07.231 INFO:tasks.workunit.client.0.smithi031.stdout:1/928: dread - d1/da/d10/d52/f10b zero size 2024-04-24T23:14:07.234 INFO:tasks.workunit.client.1.smithi040.stdout:7/851: creat db/d12/d3f/d7a/f134 x:0 0 0 2024-04-24T23:14:07.235 INFO:tasks.workunit.client.0.smithi031.stdout:7/964: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/l37 0 2024-04-24T23:14:07.235 INFO:tasks.workunit.client.1.smithi040.stdout:0/863: creat df/dd4/f118 x:0 0 0 2024-04-24T23:14:07.236 INFO:tasks.workunit.client.1.smithi040.stdout:2/854: link da/d1c/d37/d39/d3f/l8b da/d1c/d37/d39/d77/d108/d10c/l137 0 2024-04-24T23:14:07.237 INFO:tasks.workunit.client.1.smithi040.stdout:1/764: write da/d12/d23/f30 [1136243,112126] 0 2024-04-24T23:14:07.238 INFO:tasks.workunit.client.0.smithi031.stdout:1/929: dread d1/d33/d36/f44 [0,4194304] 0 2024-04-24T23:14:07.238 INFO:tasks.workunit.client.1.smithi040.stdout:7/852: write db/d12/d62/de7/f110 [349039,58802] 0 2024-04-24T23:14:07.240 INFO:tasks.workunit.client.1.smithi040.stdout:0/864: write df/d1e/d20/d21/fd9 [1960021,85813] 0 2024-04-24T23:14:07.241 INFO:tasks.workunit.client.0.smithi031.stdout:7/965: creat d1/de4/d100/d132/f149 x:0 0 0 2024-04-24T23:14:07.243 INFO:tasks.workunit.client.1.smithi040.stdout:0/865: fdatasync df/d12/f4c 0 2024-04-24T23:14:07.245 INFO:tasks.workunit.client.1.smithi040.stdout:8/833: dread d0/d7e/dc1/f80 [0,4194304] 0 2024-04-24T23:14:07.245 INFO:tasks.workunit.client.1.smithi040.stdout:0/866: chown f6 255477 1 2024-04-24T23:14:07.246 INFO:tasks.workunit.client.1.smithi040.stdout:7/853: fdatasync db/d12/d20/d52/d4b/d127/d12b/db9/dd4/d101/f115 0 2024-04-24T23:14:07.247 INFO:tasks.workunit.client.1.smithi040.stdout:5/708: dwrite d2/d7/d8/d1f/d57/d27/d3c/d43/f6a [4194304,4194304] 0 2024-04-24T23:14:07.247 INFO:tasks.workunit.client.1.smithi040.stdout:8/834: read - d0/d16/d1b/d3f/f114 zero size 2024-04-24T23:14:07.248 INFO:tasks.workunit.client.1.smithi040.stdout:7/854: chown db/d12/d20/d52/d4b/le9 410 1 2024-04-24T23:14:07.249 INFO:tasks.workunit.client.1.smithi040.stdout:4/782: rmdir da/d3f/d40/d41/d8e/db8/deb 0 2024-04-24T23:14:07.249 INFO:tasks.workunit.client.0.smithi031.stdout:7/966: dwrite d1/d2/dd/d33/d5e/d64/f99 [0,4194304] 0 2024-04-24T23:14:07.249 INFO:tasks.workunit.client.0.smithi031.stdout:5/924: dread d0/d6/d3b/d4b/f7a [0,4194304] 0 2024-04-24T23:14:07.251 INFO:tasks.workunit.client.0.smithi031.stdout:1/930: unlink d1/da/d12/d7e/faf 0 2024-04-24T23:14:07.251 INFO:tasks.workunit.client.0.smithi031.stdout:5/925: chown d0/d6/d3b/d4b/dc/d86/cbb 0 1 2024-04-24T23:14:07.253 INFO:tasks.workunit.client.1.smithi040.stdout:2/855: mkdir da/d10/d138 0 2024-04-24T23:14:07.258 INFO:tasks.workunit.client.0.smithi031.stdout:3/868: dwrite d6/df/d16/f35 [4194304,4194304] 0 2024-04-24T23:14:07.259 INFO:tasks.workunit.client.1.smithi040.stdout:4/783: dwrite da/d1f/fe7 [0,4194304] 0 2024-04-24T23:14:07.263 INFO:tasks.workunit.client.0.smithi031.stdout:3/869: chown d6/df/d16/d57/ld4 191979 1 2024-04-24T23:14:07.263 INFO:tasks.workunit.client.0.smithi031.stdout:2/902: dwrite d5/d8/d45/d5b/d49/d2f/d135/d29/f72 [0,4194304] 0 2024-04-24T23:14:07.265 INFO:tasks.workunit.client.1.smithi040.stdout:8/835: readlink d0/d38/d4c/led 0 2024-04-24T23:14:07.266 INFO:tasks.workunit.client.1.smithi040.stdout:8/836: chown d0/d7e/dc1/d87/l9d 1 1 2024-04-24T23:14:07.267 INFO:tasks.workunit.client.1.smithi040.stdout:8/837: write d0/d38/d4c/da3/fe8 [902243,102721] 0 2024-04-24T23:14:07.267 INFO:tasks.workunit.client.1.smithi040.stdout:7/855: mknod db/d12/d20/d52/d6c/c135 0 2024-04-24T23:14:07.268 INFO:tasks.workunit.client.1.smithi040.stdout:8/838: read - d0/d38/d4c/dd5/ff6 zero size 2024-04-24T23:14:07.274 INFO:tasks.workunit.client.0.smithi031.stdout:5/926: getdents d0/d6/d3b/d4b/dc/d8b/d131/d132 0 2024-04-24T23:14:07.275 INFO:tasks.workunit.client.0.smithi031.stdout:5/927: chown d0/d7/d56/lfa 19094 1 2024-04-24T23:14:07.276 INFO:tasks.workunit.client.1.smithi040.stdout:9/795: sync 2024-04-24T23:14:07.277 INFO:tasks.workunit.client.0.smithi031.stdout:5/928: fsync d0/d7/d56/fa7 0 2024-04-24T23:14:07.277 INFO:tasks.workunit.client.0.smithi031.stdout:3/870: mknod d6/df/d16/d2a/d95/d9b/d60/c112 0 2024-04-24T23:14:07.277 INFO:tasks.workunit.client.1.smithi040.stdout:1/765: getdents da/d35/dd6 0 2024-04-24T23:14:07.278 INFO:tasks.workunit.client.1.smithi040.stdout:4/784: symlink da/d3f/d8d/db5/l111 0 2024-04-24T23:14:07.279 INFO:tasks.workunit.client.1.smithi040.stdout:9/796: chown d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/c10d 2 1 2024-04-24T23:14:07.281 INFO:tasks.workunit.client.1.smithi040.stdout:1/766: read da/d12/f54 [2878567,12728] 0 2024-04-24T23:14:07.287 INFO:tasks.workunit.client.0.smithi031.stdout:5/929: rmdir d0/d6/d3b 39 2024-04-24T23:14:07.288 INFO:tasks.workunit.client.0.smithi031.stdout:3/871: dread - d6/d14/d1d/d59/fc0 zero size 2024-04-24T23:14:07.293 INFO:tasks.workunit.client.1.smithi040.stdout:8/839: unlink d0/d16/d1b/d3f/fa8 0 2024-04-24T23:14:07.294 INFO:tasks.workunit.client.0.smithi031.stdout:3/872: mkdir d6/d55/d113 0 2024-04-24T23:14:07.295 INFO:tasks.workunit.client.1.smithi040.stdout:3/843: link d1/d9/d18/d36/d10d/d45/cd3 d1/d2d/de/d30/d37/d107/c11b 0 2024-04-24T23:14:07.300 INFO:tasks.workunit.client.0.smithi031.stdout:1/931: write d1/da/d12/d7e/d7b/f93 [707202,94221] 0 2024-04-24T23:14:07.304 INFO:tasks.workunit.client.1.smithi040.stdout:9/797: unlink d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f89 0 2024-04-24T23:14:07.305 INFO:tasks.workunit.client.0.smithi031.stdout:3/873: unlink d6/df/ce3 0 2024-04-24T23:14:07.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/967: write d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d2c/f4c [1200550,108948] 0 2024-04-24T23:14:07.306 INFO:tasks.workunit.client.1.smithi040.stdout:0/867: write df/d1e/da7/d6b/f73 [87223,76516] 0 2024-04-24T23:14:07.308 INFO:tasks.workunit.client.0.smithi031.stdout:3/874: truncate d6/d14/d1d/d59/fe6 1055657 0 2024-04-24T23:14:07.309 INFO:tasks.workunit.client.1.smithi040.stdout:5/709: creat d2/d7/d8/d1f/d57/d27/fe4 x:0 0 0 2024-04-24T23:14:07.312 INFO:tasks.workunit.client.0.smithi031.stdout:2/903: write d5/d8/d15/d56/d84/fa5 [1950268,98252] 0 2024-04-24T23:14:07.313 INFO:tasks.workunit.client.0.smithi031.stdout:7/968: creat d1/de4/f14a x:0 0 0 2024-04-24T23:14:07.315 INFO:tasks.workunit.client.0.smithi031.stdout:1/932: truncate d1/da/d12/d65/fa6 6076287 0 2024-04-24T23:14:07.317 INFO:tasks.workunit.client.1.smithi040.stdout:8/840: rmdir d0/d16/d19/d71/d76 39 2024-04-24T23:14:07.325 INFO:tasks.workunit.client.0.smithi031.stdout:7/969: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/df2/f14b x:0 0 0 2024-04-24T23:14:07.325 INFO:tasks.workunit.client.0.smithi031.stdout:5/930: write d0/d6/d3b/d4b/d1d/fa4 [589298,57530] 0 2024-04-24T23:14:07.326 INFO:tasks.workunit.client.1.smithi040.stdout:4/785: dwrite da/d3f/d6b/d3c/d69/df3/f10a [0,4194304] 0 2024-04-24T23:14:07.327 INFO:tasks.workunit.client.0.smithi031.stdout:7/970: readlink d1/d2/dd/d33/d5e/d64/d6d/de1/d84/l8f 0 2024-04-24T23:14:07.336 INFO:tasks.workunit.client.1.smithi040.stdout:1/767: symlink da/d12/l106 0 2024-04-24T23:14:07.336 INFO:tasks.workunit.client.1.smithi040.stdout:2/856: truncate da/f40 2242126 0 2024-04-24T23:14:07.338 INFO:tasks.workunit.client.1.smithi040.stdout:2/857: readlink da/d10/lb0 0 2024-04-24T23:14:07.338 INFO:tasks.workunit.client.1.smithi040.stdout:1/768: chown da/d12/d14/d25/d5e/l9d 0 1 2024-04-24T23:14:07.338 INFO:tasks.workunit.client.1.smithi040.stdout:9/798: dwrite d7/da/d17/d40/d55/d2f/d28/f29 [4194304,4194304] 0 2024-04-24T23:14:07.339 INFO:tasks.workunit.client.0.smithi031.stdout:2/904: dwrite d5/d99/da2/fd1 [0,4194304] 0 2024-04-24T23:14:07.340 INFO:tasks.workunit.client.1.smithi040.stdout:0/868: mkdir df/d12/d58/d5b/d119 0 2024-04-24T23:14:07.341 INFO:tasks.workunit.client.1.smithi040.stdout:1/769: fsync da/d12/d18/d2a/d41/fc0 0 2024-04-24T23:14:07.343 INFO:tasks.workunit.client.1.smithi040.stdout:1/770: write da/d12/d18/f6c [975673,26797] 0 2024-04-24T23:14:07.344 INFO:tasks.workunit.client.1.smithi040.stdout:1/771: chown da/d12/d14/f96 70748 1 2024-04-24T23:14:07.345 INFO:tasks.workunit.client.0.smithi031.stdout:3/875: creat d6/df/d16/d2a/d95/d9b/d1a/d25/f114 x:0 0 0 2024-04-24T23:14:07.347 INFO:tasks.workunit.client.0.smithi031.stdout:7/971: mkdir d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d14c 0 2024-04-24T23:14:07.348 INFO:tasks.workunit.client.1.smithi040.stdout:3/844: mknod d1/d2d/de/d30/d10f/c11c 0 2024-04-24T23:14:07.349 INFO:tasks.workunit.client.0.smithi031.stdout:1/933: link d1/da/d61/d80/f84 d1/da/d12/d7e/dfc/d10e/f12c 0 2024-04-24T23:14:07.350 INFO:tasks.workunit.client.0.smithi031.stdout:7/972: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/lc7 10967346 1 2024-04-24T23:14:07.360 INFO:tasks.workunit.client.0.smithi031.stdout:3/876: truncate d6/df/d16/d2a/d95/d9b/f7c 2046211 0 2024-04-24T23:14:07.360 INFO:tasks.workunit.client.1.smithi040.stdout:0/869: rmdir df/d12/d58/dc3 39 2024-04-24T23:14:07.361 INFO:tasks.workunit.client.0.smithi031.stdout:1/934: dread - d1/d1d/f105 zero size 2024-04-24T23:14:07.362 INFO:tasks.workunit.client.0.smithi031.stdout:1/935: stat d1/da/d12/d65/dcf/d11/d21/d5a/d103 0 2024-04-24T23:14:07.362 INFO:tasks.workunit.client.1.smithi040.stdout:7/856: getdents db/d12/d20/d52/d4b/d127/d12b/db7 0 2024-04-24T23:14:07.364 INFO:tasks.workunit.client.1.smithi040.stdout:8/841: dwrite d0/d7e/dc1/f8e [0,4194304] 0 2024-04-24T23:14:07.365 INFO:tasks.workunit.client.0.smithi031.stdout:7/973: rename d1/d2/dd/d33/d5e/d64/d6d/de1/da0/d4f to d1/d2/dd/d33/d5e/d64/d6d/de1/da0/ddc/d14d 0 2024-04-24T23:14:07.367 INFO:tasks.workunit.client.0.smithi031.stdout:5/931: link d0/d6/d3b/d4f/lcb d0/d6/d3b/d4b/d1d/d2e/dc6/l137 0 2024-04-24T23:14:07.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/974: fsync d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/df2/f14b 0 2024-04-24T23:14:07.369 INFO:tasks.workunit.client.0.smithi031.stdout:7/975: write d1/d2/dd/d33/d5e/d64/f99 [1406865,53665] 0 2024-04-24T23:14:07.372 INFO:tasks.workunit.client.1.smithi040.stdout:1/772: rename da/d12/d14/d25 to da/d12/d18/d2a/d4d/d107 0 2024-04-24T23:14:07.374 INFO:tasks.workunit.client.0.smithi031.stdout:2/905: write d5/d8/d45/d5b/d49/f103 [469839,100545] 0 2024-04-24T23:14:07.375 INFO:tasks.workunit.client.0.smithi031.stdout:3/877: creat d6/df/d16/d53/f115 x:0 0 0 2024-04-24T23:14:07.376 INFO:tasks.workunit.client.1.smithi040.stdout:6/762: sync 2024-04-24T23:14:07.376 INFO:tasks.workunit.client.1.smithi040.stdout:3/845: truncate d1/d2d/de/d30/d37/f74 1109908 0 2024-04-24T23:14:07.380 INFO:tasks.workunit.client.0.smithi031.stdout:5/932: symlink d0/d7/d74/dbd/l138 0 2024-04-24T23:14:07.381 INFO:tasks.workunit.client.0.smithi031.stdout:5/933: fdatasync d0/d6/d3b/d4b/dc/d86/dc3/f118 0 2024-04-24T23:14:07.383 INFO:tasks.workunit.client.1.smithi040.stdout:2/858: symlink da/d10/d4d/dd7/d109/l139 0 2024-04-24T23:14:07.386 INFO:tasks.workunit.client.1.smithi040.stdout:2/859: read da/d1c/d37/d39/f76 [4714116,38158] 0 2024-04-24T23:14:07.386 INFO:tasks.workunit.client.0.smithi031.stdout:1/936: dread d1/da/d10/d52/fdf [0,4194304] 0 2024-04-24T23:14:07.387 INFO:tasks.workunit.client.0.smithi031.stdout:2/906: mkdir d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/d140 0 2024-04-24T23:14:07.389 INFO:tasks.workunit.client.0.smithi031.stdout:2/907: fdatasync d5/d8/d45/d5b/d49/d2f/d135/d29/d26/f10e 0 2024-04-24T23:14:07.389 INFO:tasks.workunit.client.1.smithi040.stdout:0/870: unlink df/d12/d1a/f4e 0 2024-04-24T23:14:07.391 INFO:tasks.workunit.client.0.smithi031.stdout:7/976: dread d1/d2/f46 [0,4194304] 0 2024-04-24T23:14:07.392 INFO:tasks.workunit.client.1.smithi040.stdout:5/710: sync 2024-04-24T23:14:07.394 INFO:tasks.workunit.client.1.smithi040.stdout:5/711: truncate d2/d7/d8/d1f/d57/d27/fe4 379558 0 2024-04-24T23:14:07.395 INFO:tasks.workunit.client.1.smithi040.stdout:8/842: mkdir d0/d7e/dc1/d87/d115 0 2024-04-24T23:14:07.397 INFO:tasks.workunit.client.1.smithi040.stdout:8/843: write d0/d38/d4c/ddd/ff1 [629939,55285] 0 2024-04-24T23:14:07.397 INFO:tasks.workunit.client.1.smithi040.stdout:5/712: truncate d2/d7/d8/db/d1e/fd0 930598 0 2024-04-24T23:14:07.398 INFO:tasks.workunit.client.1.smithi040.stdout:8/844: chown d0/d16/d19/da2/lf4 9763479 1 2024-04-24T23:14:07.400 INFO:tasks.workunit.client.1.smithi040.stdout:6/763: mknod d7/d1a/dbf/cf9 0 2024-04-24T23:14:07.402 INFO:tasks.workunit.client.1.smithi040.stdout:0/871: dread df/d12/f1f [0,4194304] 0 2024-04-24T23:14:07.403 INFO:tasks.workunit.client.1.smithi040.stdout:0/872: dread - df/d1e/d20/d61/f10e zero size 2024-04-24T23:14:07.404 INFO:tasks.workunit.client.0.smithi031.stdout:2/908: dread d5/f46 [0,4194304] 0 2024-04-24T23:14:07.406 INFO:tasks.workunit.client.1.smithi040.stdout:4/786: creat da/d3f/d40/d41/d8e/f112 x:0 0 0 2024-04-24T23:14:07.407 INFO:tasks.workunit.client.0.smithi031.stdout:5/934: dread d0/d6/d3b/d4b/fac [0,4194304] 0 2024-04-24T23:14:07.409 INFO:tasks.workunit.client.1.smithi040.stdout:0/873: dwrite df/d12/f4c [4194304,4194304] 0 2024-04-24T23:14:07.409 INFO:tasks.workunit.client.1.smithi040.stdout:9/799: creat d7/da/d17/d40/d55/f115 x:0 0 0 2024-04-24T23:14:07.410 INFO:tasks.workunit.client.1.smithi040.stdout:9/800: chown d7/d5e/cba 50868 1 2024-04-24T23:14:07.411 INFO:tasks.workunit.client.0.smithi031.stdout:7/977: write d1/d2/dd/d33/f77 [2388839,27531] 0 2024-04-24T23:14:07.414 INFO:tasks.workunit.client.0.smithi031.stdout:1/937: dwrite d1/d3e/da3/fc7 [0,4194304] 0 2024-04-24T23:14:07.414 INFO:tasks.workunit.client.1.smithi040.stdout:2/860: creat da/d1c/d37/d39/d79/da3/f13a x:0 0 0 2024-04-24T23:14:07.416 INFO:tasks.workunit.client.0.smithi031.stdout:7/978: chown d1/d2/d83/dc1/f102 53 1 2024-04-24T23:14:07.417 INFO:tasks.workunit.client.0.smithi031.stdout:3/878: rmdir d6/df/d16/d2a/d95/d9b/d29/db7/d106/d107 0 2024-04-24T23:14:07.417 INFO:tasks.workunit.client.0.smithi031.stdout:7/979: chown d1/d2/dd/d33/d5e/d64/d6d/da1/cff 3802679 1 2024-04-24T23:14:07.418 INFO:tasks.workunit.client.1.smithi040.stdout:7/857: mknod db/d12/c136 0 2024-04-24T23:14:07.421 INFO:tasks.workunit.client.0.smithi031.stdout:2/909: dread d5/d8/d45/d5b/d49/f9f [0,4194304] 0 2024-04-24T23:14:07.426 INFO:tasks.workunit.client.1.smithi040.stdout:6/764: rename d7/d1a/d20/ff1 to d7/d1a/dbf/ffa 0 2024-04-24T23:14:07.427 INFO:tasks.workunit.client.0.smithi031.stdout:7/980: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/la9 155291086 1 2024-04-24T23:14:07.433 INFO:tasks.workunit.client.0.smithi031.stdout:5/935: read d0/d6/d32/d6b/fa0 [1560368,80067] 0 2024-04-24T23:14:07.436 INFO:tasks.workunit.client.1.smithi040.stdout:9/801: mknod d7/da/d17/d40/d55/d2f/d28/d67/c116 0 2024-04-24T23:14:07.437 INFO:tasks.workunit.client.1.smithi040.stdout:9/802: chown d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc 13 1 2024-04-24T23:14:07.438 INFO:tasks.workunit.client.1.smithi040.stdout:9/803: write d7/da/d17/d40/d55/fc6 [281185,81353] 0 2024-04-24T23:14:07.444 INFO:tasks.workunit.client.0.smithi031.stdout:3/879: mkdir d6/df/d16/d2a/d95/d9b/d116 0 2024-04-24T23:14:07.446 INFO:tasks.workunit.client.1.smithi040.stdout:7/858: symlink db/d12/d20/d52/d4b/d127/d12b/l137 0 2024-04-24T23:14:07.446 INFO:tasks.workunit.client.0.smithi031.stdout:1/938: write d1/da/d12/d65/dcf/f11b [1234307,10629] 0 2024-04-24T23:14:07.447 INFO:tasks.workunit.client.0.smithi031.stdout:7/981: mknod d1/d2/dd/d33/d5e/d64/d6d/da1/c14e 0 2024-04-24T23:14:07.448 INFO:tasks.workunit.client.0.smithi031.stdout:2/910: dwrite d5/d8/d45/d5b/d49/d2f/d135/d29/d26/fd5 [0,4194304] 0 2024-04-24T23:14:07.449 INFO:tasks.workunit.client.1.smithi040.stdout:9/804: dread d7/f37 [0,4194304] 0 2024-04-24T23:14:07.450 INFO:tasks.workunit.client.1.smithi040.stdout:9/805: stat d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f50 0 2024-04-24T23:14:07.451 INFO:tasks.workunit.client.0.smithi031.stdout:5/936: creat d0/d6/d3b/d4b/dc/d86/dc3/f139 x:0 0 0 2024-04-24T23:14:07.452 INFO:tasks.workunit.client.1.smithi040.stdout:8/845: mknod d0/d7e/dc1/d9e/d90/de7/c116 0 2024-04-24T23:14:07.453 INFO:tasks.workunit.client.0.smithi031.stdout:3/880: mknod d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/c117 0 2024-04-24T23:14:07.455 INFO:tasks.workunit.client.0.smithi031.stdout:2/911: fsync d5/d28/d4e/d5e/f136 0 2024-04-24T23:14:07.456 INFO:tasks.workunit.client.1.smithi040.stdout:6/765: creat d7/d1a/d20/d51/d1e/d5d/ffb x:0 0 0 2024-04-24T23:14:07.457 INFO:tasks.workunit.client.1.smithi040.stdout:6/766: chown f2 45438086 1 2024-04-24T23:14:07.457 INFO:tasks.workunit.client.0.smithi031.stdout:5/937: dread d0/d6/f115 [0,4194304] 0 2024-04-24T23:14:07.458 INFO:tasks.workunit.client.0.smithi031.stdout:7/982: mknod d1/d2/dd/d33/d5e/d64/d6d/de1/da0/ddc/c14f 0 2024-04-24T23:14:07.459 INFO:tasks.workunit.client.1.smithi040.stdout:4/787: creat da/d45/d58/d107/f113 x:0 0 0 2024-04-24T23:14:07.464 INFO:tasks.workunit.client.0.smithi031.stdout:7/983: dread d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d81/dfa/f139 [0,4194304] 0 2024-04-24T23:14:07.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/938: fsync d0/d6/d3b/d4b/dc/d86/dc3/f139 0 2024-04-24T23:14:07.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/939: readlink d0/d7/d74/ld8 0 2024-04-24T23:14:07.471 INFO:tasks.workunit.client.0.smithi031.stdout:2/912: symlink d5/d8/d45/d5b/d49/d2f/df8/l141 0 2024-04-24T23:14:07.472 INFO:tasks.workunit.client.1.smithi040.stdout:1/773: getdents da/d12/d23 0 2024-04-24T23:14:07.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/939: truncate d1/d3e/d46/dbd/fce 1920524 0 2024-04-24T23:14:07.480 INFO:tasks.workunit.client.1.smithi040.stdout:8/846: fdatasync d0/d16/d1b/d58/d8f/fd0 0 2024-04-24T23:14:07.480 INFO:tasks.workunit.client.1.smithi040.stdout:2/861: write da/d1c/d37/d39/f4e [1547276,70225] 0 2024-04-24T23:14:07.481 INFO:tasks.workunit.client.1.smithi040.stdout:2/862: dread - da/d1c/d37/d39/d3f/d52/d78/f134 zero size 2024-04-24T23:14:07.482 INFO:tasks.workunit.client.1.smithi040.stdout:2/863: write da/d1c/d37/f11e [753697,49795] 0 2024-04-24T23:14:07.547 INFO:tasks.workunit.client.1.smithi040.stdout:0/874: rename df/d12/d4b/fc6 to df/de1/f11a 0 2024-04-24T23:14:07.550 INFO:tasks.workunit.client.0.smithi031.stdout:3/881: getdents d6/d14/db2 0 2024-04-24T23:14:07.551 INFO:tasks.workunit.client.1.smithi040.stdout:3/846: getdents d1/d9/d18/d36/d10d 0 2024-04-24T23:14:07.555 INFO:tasks.workunit.client.0.smithi031.stdout:2/913: creat d5/d8/d45/d5b/d49/dc4/de5/f142 x:0 0 0 2024-04-24T23:14:07.556 INFO:tasks.workunit.client.1.smithi040.stdout:6/767: dread d7/d1a/d20/d51/d1e/fc3 [0,4194304] 0 2024-04-24T23:14:07.565 INFO:tasks.workunit.client.0.smithi031.stdout:3/882: symlink d6/df/d16/d2a/d95/d9b/d29/db7/l118 0 2024-04-24T23:14:07.567 INFO:tasks.workunit.client.1.smithi040.stdout:5/713: getdents d2/d62 0 2024-04-24T23:14:07.569 INFO:tasks.workunit.client.0.smithi031.stdout:5/940: dwrite d0/d6/d3b/d4b/d1d/d2e/dc6/fc8 [0,4194304] 0 2024-04-24T23:14:07.571 INFO:tasks.workunit.client.0.smithi031.stdout:2/914: creat d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/dad/dbe/f143 x:0 0 0 2024-04-24T23:14:07.577 INFO:tasks.workunit.client.0.smithi031.stdout:5/941: fdatasync d0/d6/d3b/d4b/d1d/d2e/dc6/f36 0 2024-04-24T23:14:07.581 INFO:tasks.workunit.client.0.smithi031.stdout:2/915: creat d5/d99/f144 x:0 0 0 2024-04-24T23:14:07.582 INFO:tasks.workunit.client.1.smithi040.stdout:7/859: truncate db/d12/d7d/fbb 3747401 0 2024-04-24T23:14:07.583 INFO:tasks.workunit.client.1.smithi040.stdout:7/860: readlink db/d12/d20/d52/d4b/l69 0 2024-04-24T23:14:07.583 INFO:tasks.workunit.client.0.smithi031.stdout:2/916: chown d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/d140 279 1 2024-04-24T23:14:07.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/940: creat d1/d33/d41/f12d x:0 0 0 2024-04-24T23:14:07.602 INFO:tasks.workunit.client.0.smithi031.stdout:1/941: dread d1/d3e/da3/f77 [0,4194304] 0 2024-04-24T23:14:07.605 INFO:tasks.workunit.client.0.smithi031.stdout:1/942: fdatasync d1/d3e/d46/fd3 0 2024-04-24T23:14:07.608 INFO:tasks.workunit.client.0.smithi031.stdout:3/883: dwrite d6/df/d16/d53/fa8 [0,4194304] 0 2024-04-24T23:14:07.615 INFO:tasks.workunit.client.0.smithi031.stdout:3/884: read d6/df/d16/f1c [861459,119851] 0 2024-04-24T23:14:07.616 INFO:tasks.workunit.client.0.smithi031.stdout:5/942: write d0/d6/d3b/d4b/dc/d25/d55/f75 [1340462,63994] 0 2024-04-24T23:14:07.616 INFO:tasks.workunit.client.0.smithi031.stdout:1/943: symlink d1/d33/d36/de8/l12e 0 2024-04-24T23:14:07.619 INFO:tasks.workunit.client.0.smithi031.stdout:1/944: write d1/d3e/da3/d69/f113 [1311102,56762] 0 2024-04-24T23:14:07.619 INFO:tasks.workunit.client.0.smithi031.stdout:5/943: read d0/d6/d3b/d4b/dc/d25/d55/f109 [12910,96528] 0 2024-04-24T23:14:07.619 INFO:tasks.workunit.client.0.smithi031.stdout:2/917: write d5/d3d/f90 [601974,57438] 0 2024-04-24T23:14:07.628 INFO:tasks.workunit.client.0.smithi031.stdout:3/885: truncate d6/df/d16/d2a/d95/d9b/d1a/f97 722667 0 2024-04-24T23:14:07.634 INFO:tasks.workunit.client.0.smithi031.stdout:3/886: dwrite d6/df/d16/d53/fa8 [0,4194304] 0 2024-04-24T23:14:07.645 INFO:tasks.workunit.client.0.smithi031.stdout:1/945: dread d1/d33/d41/d9b/fcd [0,4194304] 0 2024-04-24T23:14:07.650 INFO:tasks.workunit.client.0.smithi031.stdout:5/944: write d0/d7/fa3 [1199500,37302] 0 2024-04-24T23:14:07.651 INFO:tasks.workunit.client.0.smithi031.stdout:2/918: write d5/d8/f10 [464284,117154] 0 2024-04-24T23:14:07.655 INFO:tasks.workunit.client.0.smithi031.stdout:1/946: mknod d1/da/d10/d52/c12f 0 2024-04-24T23:14:07.656 INFO:tasks.workunit.client.1.smithi040.stdout:2/864: mknod da/d1c/d37/d39/d3f/db2/c13b 0 2024-04-24T23:14:07.657 INFO:tasks.workunit.client.0.smithi031.stdout:5/945: dwrite d0/d7/d56/fa7 [0,4194304] 0 2024-04-24T23:14:07.657 INFO:tasks.workunit.client.0.smithi031.stdout:3/887: getdents d6/df/d16/d2a/d95/dff 0 2024-04-24T23:14:07.659 INFO:tasks.workunit.client.1.smithi040.stdout:4/788: rename da/d3f/d6b/f79 to da/d78/f114 0 2024-04-24T23:14:07.663 INFO:tasks.workunit.client.1.smithi040.stdout:0/875: unlink df/d1e/da7/f7f 0 2024-04-24T23:14:07.666 INFO:tasks.workunit.client.0.smithi031.stdout:3/888: symlink d6/d14/d71/dad/l119 0 2024-04-24T23:14:07.667 INFO:tasks.workunit.client.1.smithi040.stdout:4/789: dwrite da/d3f/d40/d47/d71/fdf [0,4194304] 0 2024-04-24T23:14:07.667 INFO:tasks.workunit.client.1.smithi040.stdout:8/847: write d0/d16/d1b/f94 [1016265,99410] 0 2024-04-24T23:14:07.668 INFO:tasks.workunit.client.0.smithi031.stdout:5/946: truncate d0/d7/d56/f61 2678748 0 2024-04-24T23:14:07.670 INFO:tasks.workunit.client.1.smithi040.stdout:8/848: dread d0/d16/d1b/d47/fdc [0,4194304] 0 2024-04-24T23:14:07.671 INFO:tasks.workunit.client.0.smithi031.stdout:1/947: dwrite d1/da/d12/d65/dcf/d11/d21/f28 [4194304,4194304] 0 2024-04-24T23:14:07.672 INFO:tasks.workunit.client.1.smithi040.stdout:9/806: link d7/da/d17/d40/d55/d2f/d1f/d4e/d83/c6e d7/d92/c117 0 2024-04-24T23:14:07.675 INFO:tasks.workunit.client.1.smithi040.stdout:6/768: dread d7/d1a/d20/d51/f26 [0,4194304] 0 2024-04-24T23:14:07.676 INFO:tasks.workunit.client.1.smithi040.stdout:8/849: dwrite d0/d52/f84 [0,4194304] 0 2024-04-24T23:14:07.677 INFO:tasks.workunit.client.1.smithi040.stdout:6/769: truncate d7/d1a/f53 1085255 0 2024-04-24T23:14:07.678 INFO:tasks.workunit.client.1.smithi040.stdout:8/850: chown d0/d7e/dc1/d87/l9d 3796 1 2024-04-24T23:14:07.681 INFO:tasks.workunit.client.0.smithi031.stdout:2/919: dwrite d5/d99/f10d [0,4194304] 0 2024-04-24T23:14:07.682 INFO:tasks.workunit.client.1.smithi040.stdout:2/865: readlink da/d1c/d4f/l9e 0 2024-04-24T23:14:07.682 INFO:tasks.workunit.client.0.smithi031.stdout:5/947: chown d0/d6/d3b/d4b/d1d/d2e/dc6/l137 231443001 1 2024-04-24T23:14:07.682 INFO:tasks.workunit.client.0.smithi031.stdout:1/948: rmdir d1/da/d12/d65 39 2024-04-24T23:14:07.685 INFO:tasks.workunit.client.1.smithi040.stdout:1/774: rename da/d35/da3/cb1 to da/d12/d23/c108 0 2024-04-24T23:14:07.690 INFO:tasks.workunit.client.1.smithi040.stdout:0/876: readlink df/d1e/d20/l101 0 2024-04-24T23:14:07.692 INFO:tasks.workunit.client.1.smithi040.stdout:3/847: symlink d1/d9/d18/l11d 0 2024-04-24T23:14:07.695 INFO:tasks.workunit.client.0.smithi031.stdout:2/920: dwrite d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/da7/f138 [4194304,4194304] 0 2024-04-24T23:14:07.703 INFO:tasks.workunit.client.0.smithi031.stdout:2/921: creat d5/d8/d45/d127/f145 x:0 0 0 2024-04-24T23:14:07.707 INFO:tasks.workunit.client.1.smithi040.stdout:6/770: symlink d7/d1a/d20/d51/d1e/d46/d85/lfc 0 2024-04-24T23:14:07.711 INFO:tasks.workunit.client.0.smithi031.stdout:2/922: symlink d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/l146 0 2024-04-24T23:14:07.714 INFO:tasks.workunit.client.0.smithi031.stdout:2/923: read d5/d8/d15/f50 [743555,39393] 0 2024-04-24T23:14:07.715 INFO:tasks.workunit.client.1.smithi040.stdout:0/877: symlink df/de1/l11b 0 2024-04-24T23:14:07.715 INFO:tasks.workunit.client.1.smithi040.stdout:1/775: dread da/d12/f42 [0,4194304] 0 2024-04-24T23:14:07.718 INFO:tasks.workunit.client.0.smithi031.stdout:3/889: dwrite d6/df/d16/d2a/d95/d9b/d1a/fdc [0,4194304] 0 2024-04-24T23:14:07.722 INFO:tasks.workunit.client.1.smithi040.stdout:4/790: write da/f2f [1746986,127035] 0 2024-04-24T23:14:07.724 INFO:tasks.workunit.client.1.smithi040.stdout:3/848: dwrite d1/d2a/d6a/fa5 [0,4194304] 0 2024-04-24T23:14:07.726 INFO:tasks.workunit.client.1.smithi040.stdout:5/714: truncate d2/d7/d8/d64/d6b/f92 81632 0 2024-04-24T23:14:07.728 INFO:tasks.workunit.client.0.smithi031.stdout:1/949: write d1/d3e/d46/d74/fb5 [348901,87097] 0 2024-04-24T23:14:07.729 INFO:tasks.workunit.client.0.smithi031.stdout:2/924: creat d5/d8/d15/d65/d12b/f147 x:0 0 0 2024-04-24T23:14:07.730 INFO:tasks.workunit.client.1.smithi040.stdout:8/851: mknod d0/d16/d1b/d58/d107/c117 0 2024-04-24T23:14:07.730 INFO:tasks.workunit.client.0.smithi031.stdout:5/948: dwrite d0/d6/d3b/d4b/d1d/d2e/fa1 [0,4194304] 0 2024-04-24T23:14:07.732 INFO:tasks.workunit.client.0.smithi031.stdout:5/949: readlink d0/d7/d74/ld8 0 2024-04-24T23:14:07.733 INFO:tasks.workunit.client.0.smithi031.stdout:5/950: stat d0/d6/d3b/d4b/dc/d86/dfd/f102 0 2024-04-24T23:14:07.741 INFO:tasks.workunit.client.1.smithi040.stdout:7/861: rename db/d12/cc2 to db/d12/d20/d52/d4b/d127/c138 0 2024-04-24T23:14:07.741 INFO:tasks.workunit.client.1.smithi040.stdout:2/866: write da/d10/d15/f2e [998761,88896] 0 2024-04-24T23:14:07.744 INFO:tasks.workunit.client.0.smithi031.stdout:2/925: creat d5/d8/d45/d5b/d49/d2f/d135/d68/f148 x:0 0 0 2024-04-24T23:14:07.747 INFO:tasks.workunit.client.0.smithi031.stdout:5/951: stat d0/cb9 0 2024-04-24T23:14:07.750 INFO:tasks.workunit.client.0.smithi031.stdout:2/926: dread d5/d8/d45/d5b/d49/d2f/d135/d29/d26/fd5 [0,4194304] 0 2024-04-24T23:14:07.753 INFO:tasks.workunit.client.1.smithi040.stdout:1/776: dwrite da/d12/f42 [8388608,4194304] 0 2024-04-24T23:14:07.756 INFO:tasks.workunit.client.1.smithi040.stdout:4/791: mkdir da/d78/d115 0 2024-04-24T23:14:07.757 INFO:tasks.workunit.client.1.smithi040.stdout:6/771: write d7/de/d4e/f8e [947680,65391] 0 2024-04-24T23:14:07.761 INFO:tasks.workunit.client.0.smithi031.stdout:5/952: creat d0/d6/d3b/d4b/dc/d86/dc3/f13a x:0 0 0 2024-04-24T23:14:07.762 INFO:tasks.workunit.client.1.smithi040.stdout:5/715: unlink d2/d2f/d14/f17 0 2024-04-24T23:14:07.763 INFO:tasks.workunit.client.1.smithi040.stdout:4/792: dwrite da/d3f/d40/d9b/ff8 [0,4194304] 0 2024-04-24T23:14:07.764 INFO:tasks.workunit.client.0.smithi031.stdout:3/890: dwrite d6/df/d16/d2a/d95/d9b/d29/db7/de9/ffe [0,4194304] 0 2024-04-24T23:14:07.764 INFO:tasks.workunit.client.1.smithi040.stdout:5/716: write d2/d7/d8/d1f/d57/d3e/f8a [1491201,109823] 0 2024-04-24T23:14:07.765 INFO:tasks.workunit.client.0.smithi031.stdout:3/891: fsync d6/df/d16/d53/fef 0 2024-04-24T23:14:07.766 INFO:tasks.workunit.client.1.smithi040.stdout:8/852: mknod d0/d8/d22/db8/dc2/c118 0 2024-04-24T23:14:07.767 INFO:tasks.workunit.client.1.smithi040.stdout:0/878: write df/de1/db6/fbc [285553,8398] 0 2024-04-24T23:14:07.772 INFO:tasks.workunit.client.0.smithi031.stdout:5/953: creat d0/d6/d3b/d4b/dc/d25/d55/f13b x:0 0 0 2024-04-24T23:14:07.774 INFO:tasks.workunit.client.1.smithi040.stdout:2/867: mknod da/d1c/d4f/c13c 0 2024-04-24T23:14:07.778 INFO:tasks.workunit.client.0.smithi031.stdout:3/892: unlink d6/df/d16/d57/lb5 0 2024-04-24T23:14:07.778 INFO:tasks.workunit.client.1.smithi040.stdout:9/807: getdents d7/da/d17/d40/d55/d2f/d1f/d31/d3b 0 2024-04-24T23:14:07.779 INFO:tasks.workunit.client.0.smithi031.stdout:1/950: dread d1/da/d12/d65/dcf/fe [0,4194304] 0 2024-04-24T23:14:07.783 INFO:tasks.workunit.client.1.smithi040.stdout:3/849: mkdir d1/d2d/d2b/d11e 0 2024-04-24T23:14:07.786 INFO:tasks.workunit.client.0.smithi031.stdout:3/893: rename d6/df/d16/d53/f88 to d6/d55/f11a 0 2024-04-24T23:14:07.788 INFO:tasks.workunit.client.1.smithi040.stdout:3/850: dwrite d1/d9/d18/d36/d55/d62/f8b [0,4194304] 0 2024-04-24T23:14:07.788 INFO:tasks.workunit.client.0.smithi031.stdout:1/951: rmdir d1/d33/d36/de8 39 2024-04-24T23:14:07.793 INFO:tasks.workunit.client.0.smithi031.stdout:2/927: dwrite d5/d28/d4e/fa1 [0,4194304] 0 2024-04-24T23:14:07.794 INFO:tasks.workunit.client.0.smithi031.stdout:3/894: mkdir d6/d14/d1d/d59/d11b 0 2024-04-24T23:14:07.797 INFO:tasks.workunit.client.0.smithi031.stdout:1/952: dread d1/d33/d41/d9b/da8/fa9 [4194304,4194304] 0 2024-04-24T23:14:07.801 INFO:tasks.workunit.client.0.smithi031.stdout:5/954: dread d0/d6/d32/d6b/fa0 [0,4194304] 0 2024-04-24T23:14:07.802 INFO:tasks.workunit.client.0.smithi031.stdout:5/955: read - d0/d7/d74/f129 zero size 2024-04-24T23:14:07.804 INFO:tasks.workunit.client.0.smithi031.stdout:2/928: mkdir d5/d8/d45/d149 0 2024-04-24T23:14:07.809 INFO:tasks.workunit.client.1.smithi040.stdout:8/853: creat d0/d16/d1b/d58/d8f/db1/f119 x:0 0 0 2024-04-24T23:14:07.810 INFO:tasks.workunit.client.0.smithi031.stdout:1/953: symlink d1/d3e/d46/dbd/l130 0 2024-04-24T23:14:07.812 INFO:tasks.workunit.client.1.smithi040.stdout:7/862: symlink db/d12/d62/d109/l139 0 2024-04-24T23:14:07.813 INFO:tasks.workunit.client.0.smithi031.stdout:5/956: creat d0/d7/d56/d7c/f13c x:0 0 0 2024-04-24T23:14:07.816 INFO:tasks.workunit.client.0.smithi031.stdout:3/895: write d6/d14/d1d/fcd [4803020,93179] 0 2024-04-24T23:14:07.817 INFO:tasks.workunit.client.1.smithi040.stdout:9/808: creat d7/da/d17/d40/d55/d2f/df7/f118 x:0 0 0 2024-04-24T23:14:07.824 INFO:tasks.workunit.client.1.smithi040.stdout:3/851: unlink d1/d9/c31 0 2024-04-24T23:14:07.825 INFO:tasks.workunit.client.0.smithi031.stdout:3/896: fsync d6/df/d16/d2a/d95/d9b/fe4 0 2024-04-24T23:14:07.827 INFO:tasks.workunit.client.1.smithi040.stdout:4/793: mkdir da/d3f/d40/d41/d8e/d116 0 2024-04-24T23:14:07.828 INFO:tasks.workunit.client.1.smithi040.stdout:5/717: mkdir d2/d2f/d14/ddf/de5 0 2024-04-24T23:14:07.829 INFO:tasks.workunit.client.1.smithi040.stdout:0/879: mkdir df/d14/d11c 0 2024-04-24T23:14:07.829 INFO:tasks.workunit.client.0.smithi031.stdout:1/954: write d1/d33/d36/fa7 [1385212,71813] 0 2024-04-24T23:14:07.832 INFO:tasks.workunit.client.1.smithi040.stdout:8/854: mkdir d0/d38/d4c/d11a 0 2024-04-24T23:14:07.832 INFO:tasks.workunit.client.0.smithi031.stdout:5/957: write d0/d6/d3b/d4b/d1d/d2e/d28/d77/ff7 [3225477,31012] 0 2024-04-24T23:14:07.835 INFO:tasks.workunit.client.0.smithi031.stdout:3/897: fdatasync d6/df/d16/d2a/d95/d9b/fe4 0 2024-04-24T23:14:07.836 INFO:tasks.workunit.client.1.smithi040.stdout:6/772: dwrite d7/d1a/d20/d51/d1e/fe8 [4194304,4194304] 0 2024-04-24T23:14:07.837 INFO:tasks.workunit.client.0.smithi031.stdout:3/898: dread - d6/df/d16/d2a/d95/d9b/fe4 zero size 2024-04-24T23:14:07.837 INFO:tasks.workunit.client.1.smithi040.stdout:6/773: chown d7/fca 75738 1 2024-04-24T23:14:07.838 INFO:tasks.workunit.client.1.smithi040.stdout:3/852: dread d1/d2a/d6a/f8e [4194304,4194304] 0 2024-04-24T23:14:07.839 INFO:tasks.workunit.client.1.smithi040.stdout:6/774: fdatasync d7/d1a/d20/d51/fdf 0 2024-04-24T23:14:07.840 INFO:tasks.workunit.client.0.smithi031.stdout:2/929: getdents d5/d28/d4e 0 2024-04-24T23:14:07.843 INFO:tasks.workunit.client.1.smithi040.stdout:1/777: rename da/d35/da3 to da/d35/d38/d47/dbc/d109 0 2024-04-24T23:14:07.845 INFO:tasks.workunit.client.1.smithi040.stdout:9/809: creat d7/d92/d9c/f119 x:0 0 0 2024-04-24T23:14:07.846 INFO:tasks.workunit.client.1.smithi040.stdout:9/810: dread - d7/da/d17/d40/d55/d2f/d1f/d31/d3b/fe3 zero size 2024-04-24T23:14:07.846 INFO:tasks.workunit.client.1.smithi040.stdout:2/868: write da/d29/d55/da7/fd6 [1504742,70777] 0 2024-04-24T23:14:07.847 INFO:tasks.workunit.client.1.smithi040.stdout:2/869: chown da/d1c/d37/d39/d3f/d52/d78/c8c 11241 1 2024-04-24T23:14:07.849 INFO:tasks.workunit.client.0.smithi031.stdout:3/899: creat d6/df/d16/d2a/f11c x:0 0 0 2024-04-24T23:14:07.852 INFO:tasks.workunit.client.0.smithi031.stdout:2/930: unlink d5/d8/d15/d56/f128 0 2024-04-24T23:14:07.855 INFO:tasks.workunit.client.0.smithi031.stdout:5/958: write d0/d6/d3b/d4b/dc/d25/f60 [663211,7054] 0 2024-04-24T23:14:07.860 INFO:tasks.workunit.client.1.smithi040.stdout:5/718: dwrite d2/d7/d8/d1f/d57/f91 [0,4194304] 0 2024-04-24T23:14:07.860 INFO:tasks.workunit.client.1.smithi040.stdout:3/853: symlink d1/d2d/d2b/l11f 0 2024-04-24T23:14:07.862 INFO:tasks.workunit.client.0.smithi031.stdout:1/955: creat d1/da/d12/d65/dcf/d11/d21/d5a/f131 x:0 0 0 2024-04-24T23:14:07.863 INFO:tasks.workunit.client.1.smithi040.stdout:5/719: read d2/d7/d8/f13 [3547669,86718] 0 2024-04-24T23:14:07.865 INFO:tasks.workunit.client.1.smithi040.stdout:5/720: dread - d2/d7/d8/d1f/d90/dcb/fd8 zero size 2024-04-24T23:14:07.866 INFO:tasks.workunit.client.0.smithi031.stdout:5/959: symlink d0/d7/d74/dbd/d112/l13d 0 2024-04-24T23:14:07.868 INFO:tasks.workunit.client.1.smithi040.stdout:3/854: dwrite d1/d9/d18/d36/d10d/d45/daa/ff5 [4194304,4194304] 0 2024-04-24T23:14:07.870 INFO:tasks.workunit.client.0.smithi031.stdout:2/931: dread d5/d8/d45/d5b/d49/d2f/d135/d29/d4f/f77 [0,4194304] 0 2024-04-24T23:14:07.871 INFO:tasks.workunit.client.0.smithi031.stdout:3/900: unlink d6/df/d16/le2 0 2024-04-24T23:14:07.874 INFO:tasks.workunit.client.0.smithi031.stdout:2/932: read d5/d8/d15/d56/f8f [240332,130899] 0 2024-04-24T23:14:07.875 INFO:tasks.workunit.client.0.smithi031.stdout:7/984: sync 2024-04-24T23:14:07.878 INFO:tasks.workunit.client.0.smithi031.stdout:2/933: read d5/d8/d45/d5b/d49/d2f/d135/d29/d26/f63 [1598589,51024] 0 2024-04-24T23:14:07.881 INFO:tasks.workunit.client.1.smithi040.stdout:7/863: link db/d12/d20/d52/d4b/f57 db/d2f/f13a 0 2024-04-24T23:14:07.881 INFO:tasks.workunit.client.0.smithi031.stdout:3/901: read d6/fa1 [2605559,99373] 0 2024-04-24T23:14:07.882 INFO:tasks.workunit.client.0.smithi031.stdout:2/934: chown d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/fd0 3 1 2024-04-24T23:14:07.887 INFO:tasks.workunit.client.0.smithi031.stdout:7/985: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d10e/f136 [0,4194304] 0 2024-04-24T23:14:07.887 INFO:tasks.workunit.client.1.smithi040.stdout:7/864: dwrite db/d12/d7d/fcd [0,4194304] 0 2024-04-24T23:14:07.889 INFO:tasks.workunit.client.1.smithi040.stdout:7/865: write db/d12/d20/d56/ff3 [845803,77266] 0 2024-04-24T23:14:07.895 INFO:tasks.workunit.client.0.smithi031.stdout:5/960: dread d0/d6/d3b/d4b/dc/f44 [0,4194304] 0 2024-04-24T23:14:07.895 INFO:tasks.workunit.client.1.smithi040.stdout:6/775: write d7/d1a/f67 [13156,3139] 0 2024-04-24T23:14:07.896 INFO:tasks.workunit.client.0.smithi031.stdout:3/902: unlink d6/d14/d71/l10e 0 2024-04-24T23:14:07.897 INFO:tasks.workunit.client.0.smithi031.stdout:2/935: dwrite d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/d117/d123/f130 [0,4194304] 0 2024-04-24T23:14:07.901 INFO:tasks.workunit.client.0.smithi031.stdout:7/986: rmdir d1/d2 39 2024-04-24T23:14:07.901 INFO:tasks.workunit.client.1.smithi040.stdout:2/870: write da/d29/f69 [312767,41539] 0 2024-04-24T23:14:07.903 INFO:tasks.workunit.client.1.smithi040.stdout:3/855: mkdir d1/d9/d18/da7/d120 0 2024-04-24T23:14:07.904 INFO:tasks.workunit.client.0.smithi031.stdout:1/956: getdents d1/d33 0 2024-04-24T23:14:07.909 INFO:tasks.workunit.client.1.smithi040.stdout:1/778: dwrite da/d12/d18/d2a/d41/fb5 [0,4194304] 0 2024-04-24T23:14:07.910 INFO:tasks.workunit.client.1.smithi040.stdout:9/811: dwrite d7/f37 [4194304,4194304] 0 2024-04-24T23:14:07.912 INFO:tasks.workunit.client.0.smithi031.stdout:2/936: chown d5/d8/d45/d5b/d49/d2f/d135/d29/c107 0 1 2024-04-24T23:14:07.912 INFO:tasks.workunit.client.1.smithi040.stdout:5/721: dread d2/d7/d8/d1f/f9d [0,4194304] 0 2024-04-24T23:14:07.913 INFO:tasks.workunit.client.0.smithi031.stdout:2/937: fsync d5/d28/d43/f67 0 2024-04-24T23:14:07.916 INFO:tasks.workunit.client.1.smithi040.stdout:8/855: dwrite d0/d8/d55/f64 [0,4194304] 0 2024-04-24T23:14:07.917 INFO:tasks.workunit.client.0.smithi031.stdout:3/903: mkdir d6/df/d16/d2a/d11d 0 2024-04-24T23:14:07.919 INFO:tasks.workunit.client.1.smithi040.stdout:9/812: dread - d7/da/d17/d40/f106 zero size 2024-04-24T23:14:07.919 INFO:tasks.workunit.client.1.smithi040.stdout:5/722: fsync d2/d7/d8/d1f/d57/d27/d3d/fd4 0 2024-04-24T23:14:07.922 INFO:tasks.workunit.client.1.smithi040.stdout:9/813: chown d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f49 7864114 1 2024-04-24T23:14:07.924 INFO:tasks.workunit.client.1.smithi040.stdout:2/871: fdatasync da/d10/f34 0 2024-04-24T23:14:07.930 INFO:tasks.workunit.client.0.smithi031.stdout:7/987: read d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/f94 [144716,98843] 0 2024-04-24T23:14:07.935 INFO:tasks.workunit.client.0.smithi031.stdout:1/957: getdents d1/d1d 0 2024-04-24T23:14:07.935 INFO:tasks.workunit.client.0.smithi031.stdout:5/961: dread d0/d6/d3b/d4b/d1d/d2e/f52 [0,4194304] 0 2024-04-24T23:14:07.936 INFO:tasks.workunit.client.0.smithi031.stdout:5/962: dread - d0/d6/d3b/d4b/dc/d25/d55/f13b zero size 2024-04-24T23:14:07.937 INFO:tasks.workunit.client.1.smithi040.stdout:0/880: write df/d12/d58/d5b/ff1 [1552079,64385] 0 2024-04-24T23:14:07.938 INFO:tasks.workunit.client.1.smithi040.stdout:6/776: truncate d7/f9 3907543 0 2024-04-24T23:14:07.943 INFO:tasks.workunit.client.1.smithi040.stdout:3/856: mknod d1/d2a/c121 0 2024-04-24T23:14:07.943 INFO:tasks.workunit.client.0.smithi031.stdout:5/963: dwrite d0/d6/f40 [0,4194304] 0 2024-04-24T23:14:07.949 INFO:tasks.workunit.client.0.smithi031.stdout:7/988: read d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/dd2/fef [2022683,21317] 0 2024-04-24T23:14:07.950 INFO:tasks.workunit.client.0.smithi031.stdout:7/989: fdatasync d1/dbe/fe0 0 2024-04-24T23:14:07.951 INFO:tasks.workunit.client.1.smithi040.stdout:1/779: rename da/d12/d18/d2a/d4d/d7d/la0 to da/d12/d18/d2a/d4d/dfd/l10a 0 2024-04-24T23:14:07.953 INFO:tasks.workunit.client.0.smithi031.stdout:2/938: dread d5/d8/d15/f2b [0,4194304] 0 2024-04-24T23:14:07.956 INFO:tasks.workunit.client.1.smithi040.stdout:1/780: dwrite da/d12/d14/d58/ff0 [0,4194304] 0 2024-04-24T23:14:07.957 INFO:tasks.workunit.client.0.smithi031.stdout:3/904: write d6/df/d16/d2a/d95/d9b/d1a/f97 [136109,101127] 0 2024-04-24T23:14:07.961 INFO:tasks.workunit.client.0.smithi031.stdout:2/939: fdatasync d5/d8/d45/d5b/d49/d2f/d135/f16 0 2024-04-24T23:14:07.962 INFO:tasks.workunit.client.0.smithi031.stdout:1/958: write d1/f66 [2200750,92379] 0 2024-04-24T23:14:07.965 INFO:tasks.workunit.client.0.smithi031.stdout:5/964: link d0/d6/d3b/d4b/dc/d25/ccc d0/d7/d74/dbd/d112/c13e 0 2024-04-24T23:14:07.965 INFO:tasks.workunit.client.1.smithi040.stdout:4/794: sync 2024-04-24T23:14:07.966 INFO:tasks.workunit.client.1.smithi040.stdout:5/723: write d2/d7/f36 [1214777,69916] 0 2024-04-24T23:14:07.966 INFO:tasks.workunit.client.1.smithi040.stdout:9/814: creat d7/dbb/f11a x:0 0 0 2024-04-24T23:14:07.968 INFO:tasks.workunit.client.1.smithi040.stdout:3/857: fsync d1/d9/d18/d36/d10d/fdd 0 2024-04-24T23:14:07.968 INFO:tasks.workunit.client.0.smithi031.stdout:5/965: truncate d0/d6/d3b/d4b/dc/f80 790772 0 2024-04-24T23:14:07.968 INFO:tasks.workunit.client.0.smithi031.stdout:1/959: dread d1/f2 [0,4194304] 0 2024-04-24T23:14:07.969 INFO:tasks.workunit.client.0.smithi031.stdout:5/966: chown d0/d6/d3b/d4b/dc/d25/d55/f13b 511 1 2024-04-24T23:14:07.970 INFO:tasks.workunit.client.0.smithi031.stdout:3/905: creat d6/d14/f11e x:0 0 0 2024-04-24T23:14:07.971 INFO:tasks.workunit.client.1.smithi040.stdout:8/856: truncate d0/d8/dc/d34/f3c 791976 0 2024-04-24T23:14:07.972 INFO:tasks.workunit.client.1.smithi040.stdout:3/858: write d1/d9/d2e/f113 [746806,71906] 0 2024-04-24T23:14:07.974 INFO:tasks.workunit.client.1.smithi040.stdout:1/781: mknod da/d12/d23/dac/c10b 0 2024-04-24T23:14:07.978 INFO:tasks.workunit.client.0.smithi031.stdout:7/990: creat d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/f150 x:0 0 0 2024-04-24T23:14:07.978 INFO:tasks.workunit.client.1.smithi040.stdout:2/872: dwrite da/d1c/d37/d39/d3f/d52/d78/dc7/dcc/ff0 [0,4194304] 0 2024-04-24T23:14:07.978 INFO:tasks.workunit.client.0.smithi031.stdout:7/991: write d1/f13 [8126770,19338] 0 2024-04-24T23:14:07.979 INFO:tasks.workunit.client.1.smithi040.stdout:4/795: dwrite da/d45/d48/d6e/fbd [0,4194304] 0 2024-04-24T23:14:07.980 INFO:tasks.workunit.client.1.smithi040.stdout:4/796: chown da/d3f/d40/d47/d75/daa 851605453 1 2024-04-24T23:14:07.986 INFO:tasks.workunit.client.0.smithi031.stdout:3/906: fsync d6/d14/d1d/d59/f9a 0 2024-04-24T23:14:07.988 INFO:tasks.workunit.client.1.smithi040.stdout:4/797: dwrite da/d3f/d40/d47/d75/d104/f105 [0,4194304] 0 2024-04-24T23:14:07.993 INFO:tasks.workunit.client.0.smithi031.stdout:2/940: dwrite d5/d99/ff3 [0,4194304] 0 2024-04-24T23:14:07.994 INFO:tasks.workunit.client.0.smithi031.stdout:5/967: truncate d0/d6/d3b/d4f/fce 160311 0 2024-04-24T23:14:07.995 INFO:tasks.workunit.client.1.smithi040.stdout:5/724: symlink d2/d7/d8/d1f/d57/d27/d3c/da4/le6 0 2024-04-24T23:14:07.996 INFO:tasks.workunit.client.1.smithi040.stdout:5/725: stat d2/d7/d8/db/d1e/lb7 0 2024-04-24T23:14:08.000 INFO:tasks.workunit.client.0.smithi031.stdout:1/960: write d1/d33/fdc [3130845,28156] 0 2024-04-24T23:14:08.001 INFO:tasks.workunit.client.1.smithi040.stdout:8/857: symlink d0/d8/d66/l11b 0 2024-04-24T23:14:08.003 INFO:tasks.workunit.client.1.smithi040.stdout:3/859: dread - d1/d9/d18/d36/d10d/d45/fbe zero size 2024-04-24T23:14:08.003 INFO:tasks.workunit.client.1.smithi040.stdout:8/858: truncate d0/d8/fc3 1502604 0 2024-04-24T23:14:08.005 INFO:tasks.workunit.client.1.smithi040.stdout:2/873: chown da/f40 31241 1 2024-04-24T23:14:08.006 INFO:tasks.workunit.client.1.smithi040.stdout:0/881: link df/d12/d4b/c53 df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dbe/c11d 0 2024-04-24T23:14:08.010 INFO:tasks.workunit.client.1.smithi040.stdout:1/782: creat da/d12/d18/d2a/dfc/f10c x:0 0 0 2024-04-24T23:14:08.011 INFO:tasks.workunit.client.1.smithi040.stdout:1/783: write da/ff [1394338,30386] 0 2024-04-24T23:14:08.013 INFO:tasks.workunit.client.1.smithi040.stdout:2/874: rename da/d1c/d37/d39/d3f/d52/d78/df5 to da/d1c/d37/d39/d79/d13d 0 2024-04-24T23:14:08.016 INFO:tasks.workunit.client.1.smithi040.stdout:1/784: rmdir da/d12/d18/d2a/d4d/d107 39 2024-04-24T23:14:08.016 INFO:tasks.workunit.client.1.smithi040.stdout:5/726: link d2/d7/d8/db/d1e/lb7 d2/d7/d8/d1f/d57/d27/d3d/dd6/le7 0 2024-04-24T23:14:08.019 INFO:tasks.workunit.client.1.smithi040.stdout:0/882: getdents df/d12/d58/d5b 0 2024-04-24T23:14:08.020 INFO:tasks.workunit.client.1.smithi040.stdout:0/883: fsync df/d12/d4b/db8/fef 0 2024-04-24T23:14:08.020 INFO:tasks.workunit.client.1.smithi040.stdout:2/875: truncate da/d29/d55/f131 2220201 0 2024-04-24T23:14:08.021 INFO:tasks.workunit.client.1.smithi040.stdout:0/884: readlink df/d14/la1 0 2024-04-24T23:14:08.024 INFO:tasks.workunit.client.1.smithi040.stdout:9/815: write d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 [1887771,80354] 0 2024-04-24T23:14:08.025 INFO:tasks.workunit.client.1.smithi040.stdout:5/727: dwrite d2/d62/fb2 [0,4194304] 0 2024-04-24T23:14:08.027 INFO:tasks.workunit.client.1.smithi040.stdout:9/816: readlink d7/da/d17/d40/d55/d2f/d1f/d4e/lce 0 2024-04-24T23:14:08.030 INFO:tasks.workunit.client.1.smithi040.stdout:0/885: mkdir df/d1e/d20/d11e 0 2024-04-24T23:14:08.032 INFO:tasks.workunit.client.1.smithi040.stdout:9/817: symlink d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/l11b 0 2024-04-24T23:14:08.034 INFO:tasks.workunit.client.1.smithi040.stdout:0/886: write df/d12/f4c [6211977,97433] 0 2024-04-24T23:14:08.038 INFO:tasks.workunit.client.1.smithi040.stdout:8/859: write d0/f82 [1797612,111889] 0 2024-04-24T23:14:08.043 INFO:tasks.workunit.client.1.smithi040.stdout:2/876: write da/d1c/d37/d39/d3f/d52/f87 [3603491,52035] 0 2024-04-24T23:14:08.074 INFO:tasks.workunit.client.1.smithi040.stdout:7/866: sync 2024-04-24T23:14:08.077 INFO:tasks.workunit.client.1.smithi040.stdout:3/860: dwrite d1/f88 [0,4194304] 0 2024-04-24T23:14:08.081 INFO:tasks.workunit.client.1.smithi040.stdout:7/867: write db/d12/d20/d52/f70 [1447302,105335] 0 2024-04-24T23:14:08.082 INFO:tasks.workunit.client.1.smithi040.stdout:8/860: dwrite d0/d16/d19/f108 [0,4194304] 0 2024-04-24T23:14:08.083 INFO:tasks.workunit.client.1.smithi040.stdout:1/785: dwrite da/d12/f54 [0,4194304] 0 2024-04-24T23:14:08.084 INFO:tasks.workunit.client.1.smithi040.stdout:7/868: write db/d2f/da8/f12a [227783,57542] 0 2024-04-24T23:14:08.085 INFO:tasks.workunit.client.1.smithi040.stdout:2/877: symlink da/d1c/d37/d39/d3f/d52/l13e 0 2024-04-24T23:14:08.085 INFO:tasks.workunit.client.1.smithi040.stdout:1/786: fdatasync da/d12/d14/d58/f104 0 2024-04-24T23:14:08.085 INFO:tasks.workunit.client.1.smithi040.stdout:3/861: mkdir d1/d2d/d2b/d122 0 2024-04-24T23:14:08.088 INFO:tasks.workunit.client.1.smithi040.stdout:7/869: fdatasync db/d12/d73/fbe 0 2024-04-24T23:14:08.089 INFO:tasks.workunit.client.1.smithi040.stdout:8/861: truncate d0/d38/d4c/ddd/ff1 1319357 0 2024-04-24T23:14:08.095 INFO:tasks.workunit.client.1.smithi040.stdout:5/728: dread d2/d2f/fa0 [0,4194304] 0 2024-04-24T23:14:08.095 INFO:tasks.workunit.client.1.smithi040.stdout:7/870: chown db/d12/d20/d52/d4b/d127/d12b/db9/fe3 13910184 1 2024-04-24T23:14:08.097 INFO:tasks.workunit.client.1.smithi040.stdout:5/729: stat d2/d2f/d14/fbc 0 2024-04-24T23:14:08.099 INFO:tasks.workunit.client.1.smithi040.stdout:1/787: dwrite da/d12/d14/f2d [4194304,4194304] 0 2024-04-24T23:14:08.102 INFO:tasks.workunit.client.1.smithi040.stdout:8/862: dwrite d0/d38/d4c/da3/fe8 [0,4194304] 0 2024-04-24T23:14:08.105 INFO:tasks.workunit.client.1.smithi040.stdout:7/871: dwrite db/d2f/d37/f11f [0,4194304] 0 2024-04-24T23:14:08.135 INFO:tasks.workunit.client.1.smithi040.stdout:6/777: sync 2024-04-24T23:14:08.136 INFO:tasks.workunit.client.1.smithi040.stdout:6/778: chown d7/d1a/d20/d51/d2f/c33 1 1 2024-04-24T23:14:08.137 INFO:tasks.workunit.client.1.smithi040.stdout:6/779: write d7/d1a/d20/d51/d1e/d46/f98 [1109908,115738] 0 2024-04-24T23:14:08.177 INFO:tasks.workunit.client.1.smithi040.stdout:4/798: sync 2024-04-24T23:14:08.177 INFO:tasks.workunit.client.1.smithi040.stdout:0/887: sync 2024-04-24T23:14:08.204 INFO:tasks.workunit.client.0.smithi031.stdout:3/907: truncate d6/d14/d1d/d59/fc0 416950 0 2024-04-24T23:14:08.204 INFO:tasks.workunit.client.0.smithi031.stdout:5/968: mknod d0/d6/d3b/c13f 0 2024-04-24T23:14:08.212 INFO:tasks.workunit.client.1.smithi040.stdout:9/818: dread d7/da/d17/d40/f42 [0,4194304] 0 2024-04-24T23:14:08.212 INFO:tasks.workunit.client.1.smithi040.stdout:8/863: creat d0/d38/d4c/ddd/f11c x:0 0 0 2024-04-24T23:14:08.216 INFO:tasks.workunit.client.1.smithi040.stdout:6/780: rmdir d7/d1a/d20/d8c 39 2024-04-24T23:14:08.223 INFO:tasks.workunit.client.1.smithi040.stdout:1/788: fsync da/d12/d18/d2a/d8b/d97/f98 0 2024-04-24T23:14:08.227 INFO:tasks.workunit.client.1.smithi040.stdout:8/864: creat d0/d8/d22/db8/dc2/f11d x:0 0 0 2024-04-24T23:14:08.228 INFO:tasks.workunit.client.0.smithi031.stdout:2/941: link d5/d8/d45/d5b/d49/f9f d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/f14a 0 2024-04-24T23:14:08.229 INFO:tasks.workunit.client.1.smithi040.stdout:1/789: dwrite da/d12/d18/d2a/d4d/fd2 [4194304,4194304] 0 2024-04-24T23:14:08.229 INFO:tasks.workunit.client.1.smithi040.stdout:7/872: creat db/d12/d20/d52/d4b/d127/d12b/db9/dd4/d101/d121/f13b x:0 0 0 2024-04-24T23:14:08.230 INFO:tasks.workunit.client.1.smithi040.stdout:1/790: read - da/d12/d18/d2a/dfc/f10c zero size 2024-04-24T23:14:08.231 INFO:tasks.workunit.client.1.smithi040.stdout:6/781: mkdir d7/d1a/d20/d51/d2a/dfd 0 2024-04-24T23:14:08.232 INFO:tasks.workunit.client.1.smithi040.stdout:6/782: chown d7/d1a/d20/d51/d1e/d46 314991 1 2024-04-24T23:14:08.233 INFO:tasks.workunit.client.1.smithi040.stdout:0/888: creat df/d14/d11c/f11f x:0 0 0 2024-04-24T23:14:08.239 INFO:tasks.workunit.client.1.smithi040.stdout:3/862: creat d1/d2d/f123 x:0 0 0 2024-04-24T23:14:08.240 INFO:tasks.workunit.client.0.smithi031.stdout:2/942: fdatasync d5/d8/d45/d5b/d49/d2f/d135/d29/f5a 0 2024-04-24T23:14:08.241 INFO:tasks.workunit.client.1.smithi040.stdout:2/878: link da/d1c/d4f/ld4 da/d10/d4d/dd7/d109/l13f 0 2024-04-24T23:14:08.242 INFO:tasks.workunit.client.1.smithi040.stdout:2/879: chown da/d1c/d37/d39/c3c 3 1 2024-04-24T23:14:08.243 INFO:tasks.workunit.client.1.smithi040.stdout:5/730: creat d2/d7/d8/d1f/fe8 x:0 0 0 2024-04-24T23:14:08.244 INFO:tasks.workunit.client.0.smithi031.stdout:1/961: creat d1/da/d12/d7e/dfc/d10e/f132 x:0 0 0 2024-04-24T23:14:08.255 INFO:tasks.workunit.client.1.smithi040.stdout:1/791: creat da/d12/d18/d2a/d8b/f10d x:0 0 0 2024-04-24T23:14:08.256 INFO:tasks.workunit.client.0.smithi031.stdout:2/943: link d5/d28/d4e/d5e/l71 d5/d8/d45/d5b/d49/d2f/d135/d29/d26/l14b 0 2024-04-24T23:14:08.260 INFO:tasks.workunit.client.1.smithi040.stdout:4/799: rename da/d3f/d6b/d2b/f46 to da/f117 0 2024-04-24T23:14:08.261 INFO:tasks.workunit.client.0.smithi031.stdout:2/944: mkdir d5/d8/d45/d5b/d49/d2f/d135/d68/d14c 0 2024-04-24T23:14:08.262 INFO:tasks.workunit.client.1.smithi040.stdout:3/863: truncate d1/d2d/d2b/f4d 208927 0 2024-04-24T23:14:08.262 INFO:tasks.workunit.client.1.smithi040.stdout:5/731: dread d2/d7/d8/d1f/d57/d27/d3c/d43/f6a [0,4194304] 0 2024-04-24T23:14:08.263 INFO:tasks.workunit.client.1.smithi040.stdout:5/732: fdatasync d2/d2f/fd1 0 2024-04-24T23:14:08.265 INFO:tasks.workunit.client.0.smithi031.stdout:2/945: fdatasync d5/d28/d43/f55 0 2024-04-24T23:14:08.267 INFO:tasks.workunit.client.1.smithi040.stdout:8/865: mkdir d0/d8/d11e 0 2024-04-24T23:14:08.268 INFO:tasks.workunit.client.1.smithi040.stdout:9/819: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/d3b/fe3 [0,4194304] 0 2024-04-24T23:14:08.275 INFO:tasks.workunit.client.1.smithi040.stdout:1/792: rename da/d35/d38/c92 to da/d12/d18/d2a/d8b/d97/c10e 0 2024-04-24T23:14:08.277 INFO:tasks.workunit.client.1.smithi040.stdout:4/800: fdatasync da/d3f/d40/d41/d60/f24 0 2024-04-24T23:14:08.277 INFO:tasks.workunit.client.1.smithi040.stdout:5/733: dread d2/d7/fe [0,4194304] 0 2024-04-24T23:14:08.277 INFO:tasks.workunit.client.1.smithi040.stdout:4/801: chown da/d1f/lca 569 1 2024-04-24T23:14:08.280 INFO:tasks.workunit.client.1.smithi040.stdout:5/734: read d2/d7/d8/d6d/fb3 [693159,52168] 0 2024-04-24T23:14:08.281 INFO:tasks.workunit.client.0.smithi031.stdout:3/908: dwrite d6/df/d16/d2a/d95/d9b/d1a/d25/f74 [4194304,4194304] 0 2024-04-24T23:14:08.282 INFO:tasks.workunit.client.0.smithi031.stdout:3/909: chown d6/f8 2253 1 2024-04-24T23:14:08.286 INFO:tasks.workunit.client.0.smithi031.stdout:1/962: write d1/d33/d36/f44 [741550,48812] 0 2024-04-24T23:14:08.290 INFO:tasks.workunit.client.1.smithi040.stdout:8/866: mkdir d0/d16/d19/d102/d11f 0 2024-04-24T23:14:08.292 INFO:tasks.workunit.client.0.smithi031.stdout:1/963: creat d1/d3e/da3/f133 x:0 0 0 2024-04-24T23:14:08.294 INFO:tasks.workunit.client.0.smithi031.stdout:1/964: read d1/d33/f47 [566325,115854] 0 2024-04-24T23:14:08.295 INFO:tasks.workunit.client.0.smithi031.stdout:3/910: creat d6/d14/d1d/d59/dec/f11f x:0 0 0 2024-04-24T23:14:08.297 INFO:tasks.workunit.client.1.smithi040.stdout:0/889: creat df/d1e/d20/f120 x:0 0 0 2024-04-24T23:14:08.297 INFO:tasks.workunit.client.0.smithi031.stdout:3/911: fsync d6/df/d16/d57/de1/fe5 0 2024-04-24T23:14:08.298 INFO:tasks.workunit.client.0.smithi031.stdout:3/912: chown d6/d14/d71/dad/c104 4 1 2024-04-24T23:14:08.299 INFO:tasks.workunit.client.1.smithi040.stdout:7/873: write db/f111 [2430304,110223] 0 2024-04-24T23:14:08.299 INFO:tasks.workunit.client.0.smithi031.stdout:1/965: fdatasync d1/d3e/da2/f101 0 2024-04-24T23:14:08.299 INFO:tasks.workunit.client.1.smithi040.stdout:2/880: dwrite da/d1c/d37/d39/d79/fe0 [0,4194304] 0 2024-04-24T23:14:08.302 INFO:tasks.workunit.client.1.smithi040.stdout:0/890: read df/d1e/d20/d61/f86 [1571821,119881] 0 2024-04-24T23:14:08.302 INFO:tasks.workunit.client.1.smithi040.stdout:2/881: chown da/d10/d7a/f99 74 1 2024-04-24T23:14:08.303 INFO:tasks.workunit.client.1.smithi040.stdout:3/864: write d1/d2a/f6d [3586,9462] 0 2024-04-24T23:14:08.305 INFO:tasks.workunit.client.1.smithi040.stdout:3/865: write d1/d9/d18/d36/d55/f89 [2406628,86088] 0 2024-04-24T23:14:08.308 INFO:tasks.workunit.client.0.smithi031.stdout:1/966: dread - d1/d3e/da2/fb0 zero size 2024-04-24T23:14:08.312 INFO:tasks.workunit.client.1.smithi040.stdout:5/735: creat d2/d7/d8/d1f/d57/d27/d3c/da4/fe9 x:0 0 0 2024-04-24T23:14:08.314 INFO:tasks.workunit.client.1.smithi040.stdout:8/867: mkdir d0/d52/d9b/d101/d113/d120 0 2024-04-24T23:14:08.315 INFO:tasks.workunit.client.0.smithi031.stdout:3/913: dread d6/d55/fdb [0,4194304] 0 2024-04-24T23:14:08.315 INFO:tasks.workunit.client.1.smithi040.stdout:8/868: chown d0/d16/d1b/d47/ffa 0 1 2024-04-24T23:14:08.316 INFO:tasks.workunit.client.0.smithi031.stdout:1/967: truncate d1/d1d/f35 2808968 0 2024-04-24T23:14:08.317 INFO:tasks.workunit.client.1.smithi040.stdout:4/802: write da/d3f/d6b/d3c/f90 [396679,116016] 0 2024-04-24T23:14:08.320 INFO:tasks.workunit.client.1.smithi040.stdout:6/783: link d7/d1a/d20/d51/d1e/l57 d7/d1a/d20/d8c/db0/lfe 0 2024-04-24T23:14:08.326 INFO:tasks.workunit.client.1.smithi040.stdout:7/874: truncate db/d12/f8c 444039 0 2024-04-24T23:14:08.329 INFO:tasks.workunit.client.1.smithi040.stdout:9/820: truncate d7/da/d17/d40/d55/d2f/df7/d6c/d8c/fde 166968 0 2024-04-24T23:14:08.330 INFO:tasks.workunit.client.1.smithi040.stdout:9/821: readlink d7/da/d17/d40/d55/d2f/d1f/d4e/lce 0 2024-04-24T23:14:08.330 INFO:tasks.workunit.client.1.smithi040.stdout:2/882: mknod da/d29/d55/c140 0 2024-04-24T23:14:08.332 INFO:tasks.workunit.client.1.smithi040.stdout:1/793: truncate f2 12398406 0 2024-04-24T23:14:08.334 INFO:tasks.workunit.client.0.smithi031.stdout:1/968: write d1/f8 [1977120,54519] 0 2024-04-24T23:14:08.335 INFO:tasks.workunit.client.1.smithi040.stdout:8/869: chown d0/d38/d4c/lea 1407024 1 2024-04-24T23:14:08.336 INFO:tasks.workunit.client.0.smithi031.stdout:3/914: dwrite d6/d14/fd3 [0,4194304] 0 2024-04-24T23:14:08.338 INFO:tasks.workunit.client.1.smithi040.stdout:1/794: dwrite da/d12/d14/f2d [4194304,4194304] 0 2024-04-24T23:14:08.342 INFO:tasks.workunit.client.1.smithi040.stdout:6/784: creat d7/d1a/d56/fff x:0 0 0 2024-04-24T23:14:08.343 INFO:tasks.workunit.client.0.smithi031.stdout:1/969: creat d1/da/d12/d65/f134 x:0 0 0 2024-04-24T23:14:08.346 INFO:tasks.workunit.client.0.smithi031.stdout:3/915: creat d6/df/d16/d2a/d95/d9b/d29/f120 x:0 0 0 2024-04-24T23:14:08.354 INFO:tasks.workunit.client.1.smithi040.stdout:2/883: dread da/d10/f31 [4194304,4194304] 0 2024-04-24T23:14:08.356 INFO:tasks.workunit.client.1.smithi040.stdout:9/822: mkdir d7/da/d17/d40/d11c 0 2024-04-24T23:14:08.357 INFO:tasks.workunit.client.1.smithi040.stdout:7/875: read db/d12/d73/f9c [44316,113754] 0 2024-04-24T23:14:08.357 INFO:tasks.workunit.client.1.smithi040.stdout:7/876: stat db/d2f/c4a 0 2024-04-24T23:14:08.357 INFO:tasks.workunit.client.1.smithi040.stdout:3/866: creat d1/d9/d18/da7/d120/f124 x:0 0 0 2024-04-24T23:14:08.359 INFO:tasks.workunit.client.1.smithi040.stdout:5/736: creat d2/d7/d8/d1f/d57/d27/d3d/dd6/fea x:0 0 0 2024-04-24T23:14:08.361 INFO:tasks.workunit.client.0.smithi031.stdout:1/970: rename d1/da/df0/fb7 to d1/da/d12/f135 0 2024-04-24T23:14:08.362 INFO:tasks.workunit.client.1.smithi040.stdout:1/795: symlink da/d12/d14/l10f 0 2024-04-24T23:14:08.364 INFO:tasks.workunit.client.0.smithi031.stdout:3/916: rename d6/d14/d1d/d69/f94 to d6/df/d16/d2a/d95/d9b/d29/db7/d106/f121 0 2024-04-24T23:14:08.365 INFO:tasks.workunit.client.1.smithi040.stdout:6/785: symlink d7/d1a/d20/d51/d2f/dba/dde/l100 0 2024-04-24T23:14:08.366 INFO:tasks.workunit.client.1.smithi040.stdout:6/786: readlink d7/d1a/d20/d51/d1e/d46/d85/lc2 0 2024-04-24T23:14:08.367 INFO:tasks.workunit.client.0.smithi031.stdout:1/971: mkdir d1/d3e/d136 0 2024-04-24T23:14:08.367 INFO:tasks.workunit.client.1.smithi040.stdout:6/787: dread - d7/d1a/d20/d51/d1e/d46/fa0 zero size 2024-04-24T23:14:08.368 INFO:tasks.workunit.client.1.smithi040.stdout:2/884: creat da/d1c/d37/d39/d79/f141 x:0 0 0 2024-04-24T23:14:08.369 INFO:tasks.workunit.client.0.smithi031.stdout:1/972: chown d1/da/d12/d7e/d7b/d4e/lae 3795827 1 2024-04-24T23:14:08.371 INFO:tasks.workunit.client.1.smithi040.stdout:9/823: unlink d7/da/d17/d40/d55/d2f/f7c 0 2024-04-24T23:14:08.374 INFO:tasks.workunit.client.1.smithi040.stdout:2/885: dwrite da/d1c/d37/d39/d3f/f7b [4194304,4194304] 0 2024-04-24T23:14:08.375 INFO:tasks.workunit.client.1.smithi040.stdout:2/886: fsync da/d10/fef 0 2024-04-24T23:14:08.378 INFO:tasks.workunit.client.1.smithi040.stdout:5/737: rename d2/d7/d8/d64/d6b/f8b to d2/d7/d8/d1f/d57/d27/d3d/dd6/feb 0 2024-04-24T23:14:08.379 INFO:tasks.workunit.client.0.smithi031.stdout:3/917: dread d6/fa1 [0,4194304] 0 2024-04-24T23:14:08.380 INFO:tasks.workunit.client.1.smithi040.stdout:4/803: creat da/f118 x:0 0 0 2024-04-24T23:14:08.382 INFO:tasks.workunit.client.1.smithi040.stdout:8/870: dread d0/d16/d1b/f40 [0,4194304] 0 2024-04-24T23:14:08.382 INFO:tasks.workunit.client.1.smithi040.stdout:0/891: creat df/d12/d1a/f121 x:0 0 0 2024-04-24T23:14:08.383 INFO:tasks.workunit.client.1.smithi040.stdout:2/887: dread da/d1c/d37/f11e [0,4194304] 0 2024-04-24T23:14:08.384 INFO:tasks.workunit.client.1.smithi040.stdout:2/888: write da/d81/fe7 [4499568,6395] 0 2024-04-24T23:14:08.387 INFO:tasks.workunit.client.1.smithi040.stdout:3/867: write d1/d2a/d6a/f8c [2519700,17404] 0 2024-04-24T23:14:08.389 INFO:tasks.workunit.client.1.smithi040.stdout:3/868: fsync d1/d9/d2e/dcf/d110/f116 0 2024-04-24T23:14:08.395 INFO:tasks.workunit.client.1.smithi040.stdout:1/796: symlink da/d12/d18/d2a/dd4/l110 0 2024-04-24T23:14:08.396 INFO:tasks.workunit.client.1.smithi040.stdout:9/824: dwrite d7/da/d17/d40/d55/d2f/df7/d6c/fa8 [4194304,4194304] 0 2024-04-24T23:14:08.401 INFO:tasks.workunit.client.1.smithi040.stdout:3/869: dwrite d1/d2d/d72/ffa [0,4194304] 0 2024-04-24T23:14:08.402 INFO:tasks.workunit.client.1.smithi040.stdout:3/870: chown d1/d2d/f60 2 1 2024-04-24T23:14:08.403 INFO:tasks.workunit.client.1.smithi040.stdout:4/804: dread da/d3f/d40/d41/f67 [0,4194304] 0 2024-04-24T23:14:08.403 INFO:tasks.workunit.client.1.smithi040.stdout:3/871: readlink d1/d2d/de/d30/d37/l7a 0 2024-04-24T23:14:08.431 INFO:tasks.workunit.client.0.smithi031.stdout:1/973: creat d1/da/df0/f137 x:0 0 0 2024-04-24T23:14:08.431 INFO:tasks.workunit.client.0.smithi031.stdout:1/974: chown d1/d3e/da2 6554322 1 2024-04-24T23:14:08.432 INFO:tasks.workunit.client.1.smithi040.stdout:7/877: write db/d12/d20/d52/d6c/fae [1406029,51532] 0 2024-04-24T23:14:08.433 INFO:tasks.workunit.client.1.smithi040.stdout:7/878: chown db/d12/d20/d52/d6c/d8f/fd0 1699 1 2024-04-24T23:14:08.437 INFO:tasks.workunit.client.1.smithi040.stdout:2/889: creat da/d81/da8/f142 x:0 0 0 2024-04-24T23:14:08.444 INFO:tasks.workunit.client.0.smithi031.stdout:3/918: dwrite d6/df/d16/d2a/f2e [0,4194304] 0 2024-04-24T23:14:08.445 INFO:tasks.workunit.client.0.smithi031.stdout:1/975: dwrite d1/da/d12/d65/dcf/d11/f20 [0,4194304] 0 2024-04-24T23:14:08.445 INFO:tasks.workunit.client.1.smithi040.stdout:5/738: dwrite d2/d7/d8/d6d/fc7 [0,4194304] 0 2024-04-24T23:14:08.452 INFO:tasks.workunit.client.0.smithi031.stdout:1/976: creat d1/d33/d41/d9b/da8/f138 x:0 0 0 2024-04-24T23:14:08.454 INFO:tasks.workunit.client.1.smithi040.stdout:4/805: readlink da/d3f/d6b/l83 0 2024-04-24T23:14:08.455 INFO:tasks.workunit.client.1.smithi040.stdout:0/892: symlink df/d12/d58/d5b/d119/l122 0 2024-04-24T23:14:08.456 INFO:tasks.workunit.client.1.smithi040.stdout:7/879: rename db/d12/d20/d52/d4b/d127/d12b/db9/l12c to db/d2f/d37/d7e/l13c 0 2024-04-24T23:14:08.456 INFO:tasks.workunit.client.1.smithi040.stdout:0/893: read - df/d12/d58/d5b/f104 zero size 2024-04-24T23:14:08.461 INFO:tasks.workunit.client.1.smithi040.stdout:6/788: creat d7/d1a/d20/d51/d1e/d46/d69/f101 x:0 0 0 2024-04-24T23:14:08.462 INFO:tasks.workunit.client.1.smithi040.stdout:7/880: dwrite db/d2f/da8/f12a [0,4194304] 0 2024-04-24T23:14:08.466 INFO:tasks.workunit.client.1.smithi040.stdout:1/797: write da/d12/d55/f6b [3879037,98127] 0 2024-04-24T23:14:08.468 INFO:tasks.workunit.client.1.smithi040.stdout:1/798: fsync da/d12/d23/fc9 0 2024-04-24T23:14:08.472 INFO:tasks.workunit.client.0.smithi031.stdout:1/977: rename d1/d3e/d46/d74/fb5 to d1/d3e/da3/f139 0 2024-04-24T23:14:08.474 INFO:tasks.workunit.client.1.smithi040.stdout:9/825: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f60 [0,4194304] 0 2024-04-24T23:14:08.474 INFO:tasks.workunit.client.1.smithi040.stdout:3/872: dwrite d1/d2d/d44/f6c [0,4194304] 0 2024-04-24T23:14:08.475 INFO:tasks.workunit.client.0.smithi031.stdout:1/978: stat d1/da/df0/l68 0 2024-04-24T23:14:08.475 INFO:tasks.workunit.client.1.smithi040.stdout:8/871: write d0/d16/f7c [1600369,15135] 0 2024-04-24T23:14:08.476 INFO:tasks.workunit.client.1.smithi040.stdout:6/789: creat d7/de/d84/f102 x:0 0 0 2024-04-24T23:14:08.477 INFO:tasks.workunit.client.1.smithi040.stdout:7/881: symlink db/d12/d20/d52/d6c/d8f/db8/l13d 0 2024-04-24T23:14:08.478 INFO:tasks.workunit.client.1.smithi040.stdout:5/739: mkdir d2/d2f/d14/dbe/dec 0 2024-04-24T23:14:08.480 INFO:tasks.workunit.client.1.smithi040.stdout:4/806: mknod da/d45/d48/d6e/ddd/dea/c119 0 2024-04-24T23:14:08.482 INFO:tasks.workunit.client.1.smithi040.stdout:4/807: fsync da/d3f/d40/d47/d75/d104/f105 0 2024-04-24T23:14:08.486 INFO:tasks.workunit.client.0.smithi031.stdout:1/979: link d1/d3e/d46/d74/lf1 d1/d33/d36/d81/l13a 0 2024-04-24T23:14:08.489 INFO:tasks.workunit.client.1.smithi040.stdout:0/894: mkdir df/d12/d123 0 2024-04-24T23:14:08.489 INFO:tasks.workunit.client.1.smithi040.stdout:6/790: dread d7/de/d4e/f8e [0,4194304] 0 2024-04-24T23:14:08.489 INFO:tasks.workunit.client.1.smithi040.stdout:4/808: chown da/d3f/d6b/d2b/f31 6 1 2024-04-24T23:14:08.490 INFO:tasks.workunit.client.1.smithi040.stdout:3/873: mknod d1/d9/d18/d36/d61/c125 0 2024-04-24T23:14:08.492 INFO:tasks.workunit.client.1.smithi040.stdout:2/890: truncate da/d1c/d37/f126 476472 0 2024-04-24T23:14:08.494 INFO:tasks.workunit.client.0.smithi031.stdout:3/919: dwrite d6/df/d16/d53/f67 [4194304,4194304] 0 2024-04-24T23:14:08.495 INFO:tasks.workunit.client.1.smithi040.stdout:8/872: rename d0/d38/d4c/ddd/f11c to d0/d16/d19/f121 0 2024-04-24T23:14:08.502 INFO:tasks.workunit.client.0.smithi031.stdout:3/920: creat d6/df/d16/d2a/d95/d9b/d1a/d25/d83/f122 x:0 0 0 2024-04-24T23:14:08.505 INFO:tasks.workunit.client.0.smithi031.stdout:3/921: fdatasync d6/f48 0 2024-04-24T23:14:08.506 INFO:tasks.workunit.client.0.smithi031.stdout:3/922: dread - d6/d55/ff2 zero size 2024-04-24T23:14:08.509 INFO:tasks.workunit.client.0.smithi031.stdout:1/980: write d1/d3e/d46/dbd/fdd [988448,114052] 0 2024-04-24T23:14:08.512 INFO:tasks.workunit.client.0.smithi031.stdout:3/923: mkdir d6/df/d16/d2a/d123 0 2024-04-24T23:14:08.513 INFO:tasks.workunit.client.0.smithi031.stdout:3/924: truncate d6/d55/ffc 141708 0 2024-04-24T23:14:08.516 INFO:tasks.workunit.client.0.smithi031.stdout:1/981: write d1/d3e/da3/d69/f95 [256123,35612] 0 2024-04-24T23:14:08.518 INFO:tasks.workunit.client.1.smithi040.stdout:4/809: dread da/d3f/d6b/d3c/d69/df3/f10a [0,4194304] 0 2024-04-24T23:14:08.519 INFO:tasks.workunit.client.1.smithi040.stdout:4/810: chown da/d45/d58 32 1 2024-04-24T23:14:08.521 INFO:tasks.workunit.client.1.smithi040.stdout:4/811: write da/d3f/d6b/d3c/fc5 [479083,42701] 0 2024-04-24T23:14:08.523 INFO:tasks.workunit.client.1.smithi040.stdout:7/882: dwrite db/d12/d20/d52/d4b/f57 [0,4194304] 0 2024-04-24T23:14:08.523 INFO:tasks.workunit.client.0.smithi031.stdout:3/925: fsync d6/df/d16/fa6 0 2024-04-24T23:14:08.526 INFO:tasks.workunit.client.0.smithi031.stdout:1/982: mknod d1/da/d12/d65/dcf/d11/deb/c13b 0 2024-04-24T23:14:08.529 INFO:tasks.workunit.client.1.smithi040.stdout:7/883: dwrite db/d2f/d37/fa6 [0,4194304] 0 2024-04-24T23:14:08.532 INFO:tasks.workunit.client.0.smithi031.stdout:1/983: creat d1/da/d12/d65/dcf/d11/d21/d5a/dbb/dff/f13c x:0 0 0 2024-04-24T23:14:08.534 INFO:tasks.workunit.client.0.smithi031.stdout:1/984: write d1/d3e/d46/dbd/fce [2166402,7045] 0 2024-04-24T23:14:08.535 INFO:tasks.workunit.client.0.smithi031.stdout:3/926: creat d6/df/d16/d57/df8/f124 x:0 0 0 2024-04-24T23:14:08.536 INFO:tasks.workunit.client.0.smithi031.stdout:3/927: stat d6/d14/db2/fc3 0 2024-04-24T23:14:08.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/928: stat d6/c93 0 2024-04-24T23:14:08.538 INFO:tasks.workunit.client.0.smithi031.stdout:3/929: write d6/d14/d1d/f9d [174139,101378] 0 2024-04-24T23:14:08.546 INFO:tasks.workunit.client.0.smithi031.stdout:3/930: read d6/fbe [423552,96544] 0 2024-04-24T23:14:08.547 INFO:tasks.workunit.client.0.smithi031.stdout:1/985: getdents d1/d3e/d46/dbd 0 2024-04-24T23:14:08.550 INFO:tasks.workunit.client.0.smithi031.stdout:1/986: creat d1/d33/f13d x:0 0 0 2024-04-24T23:14:08.555 INFO:tasks.workunit.client.0.smithi031.stdout:1/987: rmdir d1/da/d12/d7e/d7b 39 2024-04-24T23:14:08.564 INFO:tasks.workunit.client.0.smithi031.stdout:3/931: link d6/df/d16/c70 d6/d14/d1d/d59/dec/c125 0 2024-04-24T23:14:08.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/932: mknod d6/d14/d91/c126 0 2024-04-24T23:14:08.576 INFO:tasks.workunit.client.1.smithi040.stdout:1/799: dread - da/d12/d18/d2a/d4d/d107/d5e/fd7 zero size 2024-04-24T23:14:08.577 INFO:tasks.workunit.client.1.smithi040.stdout:1/800: stat da/d35/dd6 0 2024-04-24T23:14:08.578 INFO:tasks.workunit.client.1.smithi040.stdout:0/895: chown df/d12/d49/l56 2 1 2024-04-24T23:14:08.581 INFO:tasks.workunit.client.1.smithi040.stdout:2/891: mkdir da/d81/da8/d143 0 2024-04-24T23:14:08.582 INFO:tasks.workunit.client.1.smithi040.stdout:8/873: symlink d0/d16/d19/da2/l122 0 2024-04-24T23:14:08.583 INFO:tasks.workunit.client.0.smithi031.stdout:1/988: write d1/da/d10/f51 [1847512,19925] 0 2024-04-24T23:14:08.583 INFO:tasks.workunit.client.1.smithi040.stdout:8/874: stat d0/d16/d19/d102 0 2024-04-24T23:14:08.583 INFO:tasks.workunit.client.1.smithi040.stdout:8/875: read - d0/d8/d22/db8/dc2/f11d zero size 2024-04-24T23:14:08.584 INFO:tasks.workunit.client.0.smithi031.stdout:1/989: fsync d1/da/d12/d65/fb6 0 2024-04-24T23:14:08.585 INFO:tasks.workunit.client.1.smithi040.stdout:8/876: chown d0/d16/d1b/d58/d8f/db1/f119 15511921 1 2024-04-24T23:14:08.585 INFO:tasks.workunit.client.1.smithi040.stdout:4/812: mknod da/d3f/d8d/c11a 0 2024-04-24T23:14:08.587 INFO:tasks.workunit.client.1.smithi040.stdout:7/884: unlink db/d12/d20/d52/d6c/d8f/cbc 0 2024-04-24T23:14:08.591 INFO:tasks.workunit.client.0.smithi031.stdout:3/933: dread d6/df/d16/d2a/d95/d9b/d1a/f62 [0,4194304] 0 2024-04-24T23:14:08.595 INFO:tasks.workunit.client.0.smithi031.stdout:1/990: symlink d1/da/d12/d7e/d7b/d4e/l13e 0 2024-04-24T23:14:08.596 INFO:tasks.workunit.client.1.smithi040.stdout:9/826: creat d7/da/d17/f11d x:0 0 0 2024-04-24T23:14:08.596 INFO:tasks.workunit.client.1.smithi040.stdout:6/791: mknod d7/d1a/d20/d51/d2f/dba/c103 0 2024-04-24T23:14:08.597 INFO:tasks.workunit.client.0.smithi031.stdout:3/934: chown d6/df/d16/d2a/d95/d9b/d1a/f22 83772257 1 2024-04-24T23:14:08.597 INFO:tasks.workunit.client.1.smithi040.stdout:1/801: creat da/d12/d18/d2a/d8b/f111 x:0 0 0 2024-04-24T23:14:08.599 INFO:tasks.workunit.client.1.smithi040.stdout:8/877: dwrite d0/d8/dc/ff8 [0,4194304] 0 2024-04-24T23:14:08.600 INFO:tasks.workunit.client.0.smithi031.stdout:3/935: creat d6/df/d16/d57/df8/f127 x:0 0 0 2024-04-24T23:14:08.601 INFO:tasks.workunit.client.1.smithi040.stdout:8/878: stat d0/d8/d66/ff5 0 2024-04-24T23:14:08.602 INFO:tasks.workunit.client.1.smithi040.stdout:2/892: dread da/d1c/d37/d39/d77/f68 [0,4194304] 0 2024-04-24T23:14:08.603 INFO:tasks.workunit.client.1.smithi040.stdout:5/740: creat d2/d7/fed x:0 0 0 2024-04-24T23:14:08.604 INFO:tasks.workunit.client.0.smithi031.stdout:3/936: creat d6/df/d16/d2a/d95/d9b/d29/db7/f128 x:0 0 0 2024-04-24T23:14:08.605 INFO:tasks.workunit.client.1.smithi040.stdout:4/813: chown da/d3f/d6b/f7a 6074 1 2024-04-24T23:14:08.606 INFO:tasks.workunit.client.0.smithi031.stdout:1/991: creat d1/da/d12/d7e/f13f x:0 0 0 2024-04-24T23:14:08.608 INFO:tasks.workunit.client.0.smithi031.stdout:1/992: write d1/da/d12/d65/dcf/d11/d21/fdb [2861320,99138] 0 2024-04-24T23:14:08.610 INFO:tasks.workunit.client.1.smithi040.stdout:2/893: dwrite da/f9b [0,4194304] 0 2024-04-24T23:14:08.610 INFO:tasks.workunit.client.1.smithi040.stdout:7/885: dread db/d12/d3f/d7a/fca [0,4194304] 0 2024-04-24T23:14:08.612 INFO:tasks.workunit.client.1.smithi040.stdout:7/886: chown db/d12/d20/l5d 159213 1 2024-04-24T23:14:08.617 INFO:tasks.workunit.client.1.smithi040.stdout:2/894: dwrite da/d1c/d37/d39/d3f/d52/d78/dc7/fee [0,4194304] 0 2024-04-24T23:14:08.620 INFO:tasks.workunit.client.1.smithi040.stdout:9/827: unlink d7/da/d17/d40/d55/d2f/d1f/d31/la9 0 2024-04-24T23:14:08.632 INFO:tasks.workunit.client.1.smithi040.stdout:3/874: dwrite d1/d2a/d6a/fa5 [4194304,4194304] 0 2024-04-24T23:14:08.633 INFO:tasks.workunit.client.1.smithi040.stdout:6/792: mkdir d7/d1a/d9f/df6/d104 0 2024-04-24T23:14:08.637 INFO:tasks.workunit.client.1.smithi040.stdout:9/828: fsync d7/da/d17/d40/d55/d2f/d1f/d31/f98 0 2024-04-24T23:14:08.640 INFO:tasks.workunit.client.1.smithi040.stdout:5/741: mknod d2/d7/d8/d1f/d57/dca/cee 0 2024-04-24T23:14:08.640 INFO:tasks.workunit.client.1.smithi040.stdout:4/814: creat da/d78/d115/f11b x:0 0 0 2024-04-24T23:14:08.641 INFO:tasks.workunit.client.1.smithi040.stdout:9/829: read d7/da/d17/d40/d55/d2f/df7/f8f [1253162,68054] 0 2024-04-24T23:14:08.641 INFO:tasks.workunit.client.0.smithi031.stdout:3/937: write d6/fc4 [1303107,122562] 0 2024-04-24T23:14:08.645 INFO:tasks.workunit.client.0.smithi031.stdout:3/938: symlink d6/d55/l129 0 2024-04-24T23:14:08.646 INFO:tasks.workunit.client.0.smithi031.stdout:1/993: dwrite d1/da/fad [4194304,4194304] 0 2024-04-24T23:14:08.652 INFO:tasks.workunit.client.1.smithi040.stdout:7/887: dwrite db/f61 [0,4194304] 0 2024-04-24T23:14:08.653 INFO:tasks.workunit.client.1.smithi040.stdout:8/879: rmdir d0/d8/d22/db8/de5 0 2024-04-24T23:14:08.654 INFO:tasks.workunit.client.1.smithi040.stdout:1/802: dwrite f2 [0,4194304] 0 2024-04-24T23:14:08.655 INFO:tasks.workunit.client.0.smithi031.stdout:3/939: dwrite d6/df/d16/d2a/d95/d9b/d89/f105 [0,4194304] 0 2024-04-24T23:14:08.657 INFO:tasks.workunit.client.1.smithi040.stdout:1/803: truncate da/d12/d18/d2a/d41/f70 1191645 0 2024-04-24T23:14:08.659 INFO:tasks.workunit.client.1.smithi040.stdout:4/815: rmdir da/d45/d48/d6e/de0 39 2024-04-24T23:14:08.662 INFO:tasks.workunit.client.1.smithi040.stdout:9/830: chown d7/da/d17/d40/d55/ccf 11652516 1 2024-04-24T23:14:08.669 INFO:tasks.workunit.client.0.smithi031.stdout:3/940: read d6/df/d16/d2a/d95/d9b/d29/fa3 [133635,122256] 0 2024-04-24T23:14:08.670 INFO:tasks.workunit.client.1.smithi040.stdout:2/895: dwrite da/d29/d55/f131 [0,4194304] 0 2024-04-24T23:14:08.672 INFO:tasks.workunit.client.0.smithi031.stdout:1/994: rename d1/d33/d41/fc8 to d1/da/d12/d7e/d7b/d4e/d98/f140 0 2024-04-24T23:14:08.674 INFO:tasks.workunit.client.1.smithi040.stdout:8/880: mknod d0/d16/d1b/d58/d8f/c123 0 2024-04-24T23:14:08.675 INFO:tasks.workunit.client.0.smithi031.stdout:1/995: creat d1/da/d12/d7e/d7b/d4e/f141 x:0 0 0 2024-04-24T23:14:08.675 INFO:tasks.workunit.client.1.smithi040.stdout:8/881: stat d0/d16/d1b/f95 0 2024-04-24T23:14:08.676 INFO:tasks.workunit.client.1.smithi040.stdout:8/882: chown d0/d38/d4c/da3/fe8 1590 1 2024-04-24T23:14:08.676 INFO:tasks.workunit.client.1.smithi040.stdout:1/804: rename da/d12/d18/d2a/d41/ff3 to da/d12/d18/d2a/d8b/d97/f112 0 2024-04-24T23:14:08.680 INFO:tasks.workunit.client.1.smithi040.stdout:6/793: creat d7/d1a/d20/d51/d1e/d46/d69/f105 x:0 0 0 2024-04-24T23:14:08.681 INFO:tasks.workunit.client.1.smithi040.stdout:6/794: truncate d7/d1a/d56/fff 72107 0 2024-04-24T23:14:08.683 INFO:tasks.workunit.client.0.smithi031.stdout:3/941: getdents d6/d55 0 2024-04-24T23:14:08.684 INFO:tasks.workunit.client.1.smithi040.stdout:4/816: mkdir da/d3f/d40/d41/d8e/d116/d11c 0 2024-04-24T23:14:08.685 INFO:tasks.workunit.client.1.smithi040.stdout:2/896: rename da/d1c/d37/d102/c12b to da/d1c/d37/d39/d79/d13d/c144 0 2024-04-24T23:14:08.687 INFO:tasks.workunit.client.1.smithi040.stdout:9/831: dread d7/da/d17/d40/d55/d2f/df7/d6c/d8c/fde [0,4194304] 0 2024-04-24T23:14:08.687 INFO:tasks.workunit.client.1.smithi040.stdout:7/888: creat db/f13e x:0 0 0 2024-04-24T23:14:08.688 INFO:tasks.workunit.client.1.smithi040.stdout:6/795: mknod d7/d1a/d20/d51/d1e/d5d/c106 0 2024-04-24T23:14:08.688 INFO:tasks.workunit.client.1.smithi040.stdout:9/832: readlink d7/da/d17/d40/d55/lb2 0 2024-04-24T23:14:08.689 INFO:tasks.workunit.client.1.smithi040.stdout:5/742: getdents d2/d7/d8/db/d1e 0 2024-04-24T23:14:08.690 INFO:tasks.workunit.client.1.smithi040.stdout:9/833: readlink d7/da/d17/d40/d55/d2f/d1f/d31/l7b 0 2024-04-24T23:14:08.690 INFO:tasks.workunit.client.1.smithi040.stdout:1/805: creat da/d12/d18/d2a/d41/d4f/de9/f113 x:0 0 0 2024-04-24T23:14:08.691 INFO:tasks.workunit.client.1.smithi040.stdout:2/897: symlink da/d1c/d37/d39/d79/l145 0 2024-04-24T23:14:08.692 INFO:tasks.workunit.client.1.smithi040.stdout:8/883: link d0/d16/d1b/f40 d0/d7e/f124 0 2024-04-24T23:14:08.693 INFO:tasks.workunit.client.1.smithi040.stdout:9/834: write d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 [7899198,100046] 0 2024-04-24T23:14:08.693 INFO:tasks.workunit.client.1.smithi040.stdout:5/743: symlink d2/d7/d8/db/lef 0 2024-04-24T23:14:08.695 INFO:tasks.workunit.client.1.smithi040.stdout:7/889: mknod db/d12/d62/d8a/c13f 0 2024-04-24T23:14:08.696 INFO:tasks.workunit.client.1.smithi040.stdout:5/744: fdatasync d2/d7/d8/d1f/d57/d27/d3c/d43/f6f 0 2024-04-24T23:14:08.696 INFO:tasks.workunit.client.1.smithi040.stdout:1/806: creat da/d12/d18/d2a/d4d/d107/f114 x:0 0 0 2024-04-24T23:14:08.697 INFO:tasks.workunit.client.1.smithi040.stdout:7/890: read db/d2f/f13a [3495290,88064] 0 2024-04-24T23:14:08.697 INFO:tasks.workunit.client.1.smithi040.stdout:2/898: rmdir da/d81/da8 39 2024-04-24T23:14:08.698 INFO:tasks.workunit.client.1.smithi040.stdout:1/807: chown da/d12/d18/d2a/d41/f6a 121 1 2024-04-24T23:14:08.699 INFO:tasks.workunit.client.1.smithi040.stdout:2/899: chown da/d10/d4d/dd7/ce2 2 1 2024-04-24T23:14:08.700 INFO:tasks.workunit.client.1.smithi040.stdout:8/884: write d0/d8/f4d [1829132,113202] 0 2024-04-24T23:14:08.700 INFO:tasks.workunit.client.0.smithi031.stdout:7/992: sync 2024-04-24T23:14:08.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/969: sync 2024-04-24T23:14:08.702 INFO:tasks.workunit.client.0.smithi031.stdout:2/946: sync 2024-04-24T23:14:08.706 INFO:tasks.workunit.client.0.smithi031.stdout:1/996: write d1/d1d/f35 [3127711,107932] 0 2024-04-24T23:14:08.709 INFO:tasks.workunit.client.0.smithi031.stdout:3/942: dread d6/d55/fd0 [0,4194304] 0 2024-04-24T23:14:08.710 INFO:tasks.workunit.client.1.smithi040.stdout:4/817: dread da/d45/d48/d6e/d70/fa8 [4194304,4194304] 0 2024-04-24T23:14:08.712 INFO:tasks.workunit.client.1.smithi040.stdout:4/818: read da/d45/d48/d6e/d70/fcb [3778113,5724] 0 2024-04-24T23:14:08.712 INFO:tasks.workunit.client.1.smithi040.stdout:9/835: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/d11e 0 2024-04-24T23:14:08.713 INFO:tasks.workunit.client.0.smithi031.stdout:5/970: dwrite d0/d6/d3b/d4b/dc/d86/dc3/f13a [0,4194304] 0 2024-04-24T23:14:08.713 INFO:tasks.workunit.client.1.smithi040.stdout:4/819: write da/d3f/d40/d41/d8e/db8/fc4 [594197,39027] 0 2024-04-24T23:14:08.714 INFO:tasks.workunit.client.0.smithi031.stdout:5/971: chown d0/d6/d3b/d4b/d1d/d2e/dc6/c9f 0 1 2024-04-24T23:14:08.715 INFO:tasks.workunit.client.1.smithi040.stdout:5/745: symlink d2/d7/d8/d1f/d90/lf0 0 2024-04-24T23:14:08.715 INFO:tasks.workunit.client.0.smithi031.stdout:5/972: chown d0/d6/d32/la9 11396 1 2024-04-24T23:14:08.721 INFO:tasks.workunit.client.1.smithi040.stdout:2/900: mknod da/d10/d7a/c146 0 2024-04-24T23:14:08.722 INFO:tasks.workunit.client.0.smithi031.stdout:2/947: truncate d5/d8/d45/d5b/d49/d2f/d135/f4d 791847 0 2024-04-24T23:14:08.723 INFO:tasks.workunit.client.1.smithi040.stdout:1/808: unlink da/d12/d18/d2a/d41/l89 0 2024-04-24T23:14:08.724 INFO:tasks.workunit.client.1.smithi040.stdout:6/796: truncate d7/fae 620963 0 2024-04-24T23:14:08.729 INFO:tasks.workunit.client.0.smithi031.stdout:2/948: fdatasync d5/d8/d45/d5b/d49/f88 0 2024-04-24T23:14:08.733 INFO:tasks.workunit.client.0.smithi031.stdout:1/997: creat d1/d3e/d46/f142 x:0 0 0 2024-04-24T23:14:08.736 INFO:tasks.workunit.client.1.smithi040.stdout:0/896: sync 2024-04-24T23:14:08.738 INFO:tasks.workunit.client.0.smithi031.stdout:5/973: rename d0/d6/d3b/d4b/d1d/d2e/d28/c59 to d0/d6/d3b/d4b/dc/d25/d55/c140 0 2024-04-24T23:14:08.739 INFO:tasks.workunit.client.0.smithi031.stdout:3/943: dread d6/df/f34 [0,4194304] 0 2024-04-24T23:14:08.740 INFO:tasks.workunit.client.0.smithi031.stdout:3/944: chown d6/d14/d1d/f9d 11 1 2024-04-24T23:14:08.742 INFO:tasks.workunit.client.1.smithi040.stdout:4/820: write da/d3f/d6b/d2b/f55 [72804,53185] 0 2024-04-24T23:14:08.748 INFO:tasks.workunit.client.1.smithi040.stdout:5/746: rename d2/d7/d8/d1f/d57/d27/d3d/f7a to d2/d2f/d14/dbe/ff1 0 2024-04-24T23:14:08.748 INFO:tasks.workunit.client.1.smithi040.stdout:8/885: write d0/d38/fe4 [647827,33004] 0 2024-04-24T23:14:08.749 INFO:tasks.workunit.client.1.smithi040.stdout:8/886: fdatasync d0/d8/d66/f78 0 2024-04-24T23:14:08.751 INFO:tasks.workunit.client.1.smithi040.stdout:7/891: link db/d12/d73/fbe db/d2f/d11a/f140 0 2024-04-24T23:14:08.753 INFO:tasks.workunit.client.0.smithi031.stdout:3/945: truncate d6/df/d16/d2a/f31 69498 0 2024-04-24T23:14:08.754 INFO:tasks.workunit.client.0.smithi031.stdout:3/946: stat d6/d14/d1d/d69/fd5 0 2024-04-24T23:14:08.754 INFO:tasks.workunit.client.1.smithi040.stdout:1/809: creat da/d12/d18/d2a/d32/f115 x:0 0 0 2024-04-24T23:14:08.755 INFO:tasks.workunit.client.0.smithi031.stdout:3/947: fdatasync d6/d14/d1d/d59/fe6 0 2024-04-24T23:14:08.756 INFO:tasks.workunit.client.0.smithi031.stdout:3/948: dread - d6/df/d16/d57/df8/f124 zero size 2024-04-24T23:14:08.756 INFO:tasks.workunit.client.0.smithi031.stdout:2/949: mknod d5/d8/d45/d5b/d49/d2f/d135/c14d 0 2024-04-24T23:14:08.757 INFO:tasks.workunit.client.0.smithi031.stdout:3/949: dread - d6/df/d16/d2a/d95/d9b/d29/db7/f128 zero size 2024-04-24T23:14:08.758 INFO:tasks.workunit.client.1.smithi040.stdout:6/797: symlink d7/d1a/d20/d8c/l107 0 2024-04-24T23:14:08.758 INFO:tasks.workunit.client.0.smithi031.stdout:3/950: fsync d6/fc4 0 2024-04-24T23:14:08.758 INFO:tasks.workunit.client.1.smithi040.stdout:6/798: chown d7/d1a/db5 34 1 2024-04-24T23:14:08.760 INFO:tasks.workunit.client.1.smithi040.stdout:9/836: dwrite d7/f82 [0,4194304] 0 2024-04-24T23:14:08.761 INFO:tasks.workunit.client.0.smithi031.stdout:5/974: mknod d0/d6/d3b/d4b/dc/c141 0 2024-04-24T23:14:08.761 INFO:tasks.workunit.client.1.smithi040.stdout:9/837: chown d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/d10b 56163614 1 2024-04-24T23:14:08.766 INFO:tasks.workunit.client.0.smithi031.stdout:2/950: creat d5/d8/d45/d5b/d49/d6f/d119/d11d/f14e x:0 0 0 2024-04-24T23:14:08.769 INFO:tasks.workunit.client.1.smithi040.stdout:5/747: mkdir d2/d2f/d14/dbe/df2 0 2024-04-24T23:14:08.770 INFO:tasks.workunit.client.1.smithi040.stdout:5/748: read - d2/d7/d8/db/fde zero size 2024-04-24T23:14:08.770 INFO:tasks.workunit.client.0.smithi031.stdout:1/998: dwrite d1/da/d61/f109 [0,4194304] 0 2024-04-24T23:14:08.772 INFO:tasks.workunit.client.0.smithi031.stdout:1/999: fdatasync d1/da/fe0 0 2024-04-24T23:14:08.772 INFO:tasks.workunit.client.0.smithi031.stdout:5/975: unlink d0/d6/f40 0 2024-04-24T23:14:08.773 INFO:tasks.workunit.client.1.smithi040.stdout:8/887: creat d0/d38/d4c/da3/f125 x:0 0 0 2024-04-24T23:14:08.774 INFO:tasks.workunit.client.1.smithi040.stdout:8/888: read - d0/d16/d19/f121 zero size 2024-04-24T23:14:08.775 INFO:tasks.workunit.client.1.smithi040.stdout:8/889: write d0/d16/d19/f35 [3094153,12080] 0 2024-04-24T23:14:08.775 INFO:tasks.workunit.client.1.smithi040.stdout:7/892: mknod db/d12/d73/c141 0 2024-04-24T23:14:08.783 INFO:tasks.workunit.client.0.smithi031.stdout:5/976: dread - d0/d6/d3b/d4b/d1d/d2e/ffb zero size 2024-04-24T23:14:08.784 INFO:tasks.workunit.client.1.smithi040.stdout:8/890: dwrite d0/d16/d1b/d58/d8f/db1/fe2 [0,4194304] 0 2024-04-24T23:14:08.785 INFO:tasks.workunit.client.1.smithi040.stdout:8/891: dread - d0/d52/d9b/fe1 zero size 2024-04-24T23:14:08.786 INFO:tasks.workunit.client.1.smithi040.stdout:7/893: dread db/d2f/f35 [0,4194304] 0 2024-04-24T23:14:08.786 INFO:tasks.workunit.client.1.smithi040.stdout:6/799: mknod d7/de/d84/c108 0 2024-04-24T23:14:08.789 INFO:tasks.workunit.client.1.smithi040.stdout:9/838: readlink d7/da/d17/d40/d55/d2f/df7/d6c/lfb 0 2024-04-24T23:14:08.790 INFO:tasks.workunit.client.1.smithi040.stdout:6/800: dread d7/de/d89/fb7 [0,4194304] 0 2024-04-24T23:14:08.791 INFO:tasks.workunit.client.0.smithi031.stdout:5/977: mknod d0/d117/c142 0 2024-04-24T23:14:08.792 INFO:tasks.workunit.client.0.smithi031.stdout:5/978: chown d0/d7/c6c 1798000 1 2024-04-24T23:14:08.793 INFO:tasks.workunit.client.1.smithi040.stdout:8/892: dwrite d0/d8/d66/f78 [0,4194304] 0 2024-04-24T23:14:08.793 INFO:tasks.workunit.client.1.smithi040.stdout:3/875: sync 2024-04-24T23:14:08.794 INFO:tasks.workunit.client.1.smithi040.stdout:4/821: mkdir da/d45/d48/d11d 0 2024-04-24T23:14:08.795 INFO:tasks.workunit.client.1.smithi040.stdout:9/839: dread d7/da/d17/d40/d55/d2f/d1f/d31/f8d [0,4194304] 0 2024-04-24T23:14:08.799 INFO:tasks.workunit.client.0.smithi031.stdout:5/979: rmdir d0/d6/d3b/d4b/dc/d86/dc3 39 2024-04-24T23:14:08.799 INFO:tasks.workunit.client.0.smithi031.stdout:2/951: dread d5/d28/d4e/d5e/fd9 [0,4194304] 0 2024-04-24T23:14:08.800 INFO:tasks.workunit.client.0.smithi031.stdout:5/980: chown d0/d6/d32/cf3 4 1 2024-04-24T23:14:08.803 INFO:tasks.workunit.client.1.smithi040.stdout:2/901: link da/d1c/d37/l12e da/d1c/d37/d39/d79/l147 0 2024-04-24T23:14:08.804 INFO:tasks.workunit.client.1.smithi040.stdout:2/902: chown da/d1c/d37/d39/d3f/d52/d78/dc7/dcc 6691699 1 2024-04-24T23:14:08.804 INFO:tasks.workunit.client.1.smithi040.stdout:2/903: chown da/d1c/d37/d39/d79/d13d 3354 1 2024-04-24T23:14:08.807 INFO:tasks.workunit.client.0.smithi031.stdout:5/981: mknod d0/d7/d74/dbd/d112/c143 0 2024-04-24T23:14:08.811 INFO:tasks.workunit.client.0.smithi031.stdout:5/982: unlink d0/d6/d32/c15 0 2024-04-24T23:14:08.812 INFO:tasks.workunit.client.1.smithi040.stdout:7/894: chown db/d12/c6f 47 1 2024-04-24T23:14:08.815 INFO:tasks.workunit.client.1.smithi040.stdout:0/897: mknod df/d12/d58/dc3/dcd/de6/c124 0 2024-04-24T23:14:08.818 INFO:tasks.workunit.client.1.smithi040.stdout:6/801: creat d7/d1a/d20/d51/d2f/dba/dde/f109 x:0 0 0 2024-04-24T23:14:08.821 INFO:tasks.workunit.client.1.smithi040.stdout:0/898: dread df/d1e/da7/f69 [0,4194304] 0 2024-04-24T23:14:08.822 INFO:tasks.workunit.client.0.smithi031.stdout:5/983: dwrite d0/d6/d3b/d4b/d1d/d2e/dc6/f21 [0,4194304] 0 2024-04-24T23:14:08.824 INFO:tasks.workunit.client.1.smithi040.stdout:7/895: dread db/d12/f16 [4194304,4194304] 0 2024-04-24T23:14:08.824 INFO:tasks.workunit.client.1.smithi040.stdout:0/899: dread df/de1/db6/fbc [0,4194304] 0 2024-04-24T23:14:08.825 INFO:tasks.workunit.client.0.smithi031.stdout:2/952: dread d5/d28/d43/f53 [0,4194304] 0 2024-04-24T23:14:08.828 INFO:tasks.workunit.client.1.smithi040.stdout:1/810: rename da/d12/d18/d2a/d4d/d7d/fa9 to da/d12/f116 0 2024-04-24T23:14:08.828 INFO:tasks.workunit.client.1.smithi040.stdout:7/896: write db/d12/d20/d52/d4b/fa1 [3710901,26622] 0 2024-04-24T23:14:08.828 INFO:tasks.workunit.client.1.smithi040.stdout:0/900: truncate df/d1e/d89/ddf/f113 159326 0 2024-04-24T23:14:08.828 INFO:tasks.workunit.client.1.smithi040.stdout:1/811: fsync f2 0 2024-04-24T23:14:08.829 INFO:tasks.workunit.client.1.smithi040.stdout:2/904: truncate da/d10/fa1 537925 0 2024-04-24T23:14:08.830 INFO:tasks.workunit.client.0.smithi031.stdout:2/953: mkdir d5/d8/d45/d5b/d49/d6f/d14f 0 2024-04-24T23:14:08.832 INFO:tasks.workunit.client.0.smithi031.stdout:5/984: mknod d0/d6/d3b/d4b/d1d/c144 0 2024-04-24T23:14:08.833 INFO:tasks.workunit.client.1.smithi040.stdout:6/802: mknod d7/d1a/d20/d8c/c10a 0 2024-04-24T23:14:08.835 INFO:tasks.workunit.client.0.smithi031.stdout:5/985: creat d0/d7/d74/dbd/f145 x:0 0 0 2024-04-24T23:14:08.836 INFO:tasks.workunit.client.0.smithi031.stdout:2/954: dread d5/d8/d15/fbc [0,4194304] 0 2024-04-24T23:14:08.837 INFO:tasks.workunit.client.1.smithi040.stdout:8/893: dwrite d0/d8/d22/f73 [0,4194304] 0 2024-04-24T23:14:08.838 INFO:tasks.workunit.client.1.smithi040.stdout:9/840: getdents d7/da/d17/d40/d55/d2f/ded 0 2024-04-24T23:14:08.847 INFO:tasks.workunit.client.1.smithi040.stdout:0/901: creat df/d12/d58/d5b/f125 x:0 0 0 2024-04-24T23:14:08.848 INFO:tasks.workunit.client.0.smithi031.stdout:5/986: link d0/d6/d3b/f8d d0/d6/d3b/d4b/d1d/d2e/f146 0 2024-04-24T23:14:08.851 INFO:tasks.workunit.client.0.smithi031.stdout:5/987: symlink d0/d6/d3b/d4b/d1d/d2e/d28/l147 0 2024-04-24T23:14:08.852 INFO:tasks.workunit.client.1.smithi040.stdout:1/812: creat da/d12/d18/d2a/d8b/dd3/f117 x:0 0 0 2024-04-24T23:14:08.853 INFO:tasks.workunit.client.1.smithi040.stdout:5/749: getdents d2/d7/d8/d6d/d89 0 2024-04-24T23:14:08.854 INFO:tasks.workunit.client.1.smithi040.stdout:1/813: stat da/d12/d18/d2a/d4d/l93 0 2024-04-24T23:14:08.857 INFO:tasks.workunit.client.1.smithi040.stdout:8/894: creat d0/d7e/dc1/d9e/d90/d91/f126 x:0 0 0 2024-04-24T23:14:08.857 INFO:tasks.workunit.client.1.smithi040.stdout:2/905: mkdir da/d10/d148 0 2024-04-24T23:14:08.858 INFO:tasks.workunit.client.1.smithi040.stdout:2/906: readlink da/d1c/d37/d8a/lf3 0 2024-04-24T23:14:08.859 INFO:tasks.workunit.client.1.smithi040.stdout:6/803: rename d7/c11 to d7/d1a/d20/d51/d2a/dfd/c10b 0 2024-04-24T23:14:08.860 INFO:tasks.workunit.client.1.smithi040.stdout:3/876: link d1/d9/d18/d36/d10d/f1d d1/d2d/f126 0 2024-04-24T23:14:08.861 INFO:tasks.workunit.client.1.smithi040.stdout:8/895: rmdir d0/d8 39 2024-04-24T23:14:08.863 INFO:tasks.workunit.client.1.smithi040.stdout:2/907: creat da/d1c/d37/d102/f149 x:0 0 0 2024-04-24T23:14:08.865 INFO:tasks.workunit.client.1.smithi040.stdout:0/902: rename df/d14/l27 to df/d12/d4b/db8/l126 0 2024-04-24T23:14:08.865 INFO:tasks.workunit.client.1.smithi040.stdout:1/814: dread da/d12/d23/f30 [0,4194304] 0 2024-04-24T23:14:08.866 INFO:tasks.workunit.client.1.smithi040.stdout:6/804: mkdir d7/d1a/d20/d51/d2f/dba/dde/d10c 0 2024-04-24T23:14:08.870 INFO:tasks.workunit.client.1.smithi040.stdout:9/841: dread d7/da/d17/d40/d55/d2f/df7/f8f [0,4194304] 0 2024-04-24T23:14:08.870 INFO:tasks.workunit.client.1.smithi040.stdout:8/896: fsync d0/d16/d1b/f40 0 2024-04-24T23:14:08.871 INFO:tasks.workunit.client.1.smithi040.stdout:7/897: dwrite db/d12/d20/d52/d6c/fb2 [0,4194304] 0 2024-04-24T23:14:08.873 INFO:tasks.workunit.client.0.smithi031.stdout:5/988: dwrite d0/d6/d3b/d4b/dc/d86/dfd/da8/f12a [0,4194304] 0 2024-04-24T23:14:08.873 INFO:tasks.workunit.client.1.smithi040.stdout:1/815: truncate da/d35/d38/f3e 1025674 0 2024-04-24T23:14:08.876 INFO:tasks.workunit.client.0.smithi031.stdout:5/989: chown d0/d6/d3b/d4b/dc/d8b/lb0 33112943 1 2024-04-24T23:14:08.877 INFO:tasks.workunit.client.1.smithi040.stdout:8/897: symlink d0/d52/d9b/d101/d113/l127 0 2024-04-24T23:14:08.878 INFO:tasks.workunit.client.1.smithi040.stdout:9/842: rename d7/da/d17/d40/d55/cda to d7/da/d17/d40/c11f 0 2024-04-24T23:14:08.881 INFO:tasks.workunit.client.1.smithi040.stdout:9/843: fdatasync d7/d5e/da1/dfa/f10c 0 2024-04-24T23:14:08.881 INFO:tasks.workunit.client.1.smithi040.stdout:1/816: dwrite da/d12/d14/d58/fc2 [0,4194304] 0 2024-04-24T23:14:08.881 INFO:tasks.workunit.client.1.smithi040.stdout:8/898: fsync d0/d8/dc/ff8 0 2024-04-24T23:14:08.883 INFO:tasks.workunit.client.1.smithi040.stdout:8/899: chown d0/d8/dc/d34/lb6 482229 1 2024-04-24T23:14:08.884 INFO:tasks.workunit.client.1.smithi040.stdout:8/900: chown d0/d16/d19/c26 14284480 1 2024-04-24T23:14:08.885 INFO:tasks.workunit.client.1.smithi040.stdout:9/844: mkdir d7/da/d17/d120 0 2024-04-24T23:14:08.887 INFO:tasks.workunit.client.1.smithi040.stdout:8/901: chown d0/d16/d19/d71/d76/l93 1335601 1 2024-04-24T23:14:08.889 INFO:tasks.workunit.client.1.smithi040.stdout:8/902: creat d0/d8/d22/f128 x:0 0 0 2024-04-24T23:14:08.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:08 smithi040 ceph-mon[33128]: pgmap v37: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 92 MiB/s rd, 238 MiB/s wr, 528 op/s 2024-04-24T23:14:08.920 INFO:tasks.workunit.client.1.smithi040.stdout:4/822: sync 2024-04-24T23:14:08.921 INFO:tasks.workunit.client.1.smithi040.stdout:4/823: fdatasync da/d3f/d8d/ffd 0 2024-04-24T23:14:08.926 INFO:tasks.workunit.client.0.smithi031.stdout:5/990: dwrite d0/d6/d3b/d4b/d1d/d2e/dc6/f124 [0,4194304] 0 2024-04-24T23:14:08.927 INFO:tasks.workunit.client.1.smithi040.stdout:4/824: dwrite da/d3f/d40/d41/fd2 [0,4194304] 0 2024-04-24T23:14:08.933 INFO:tasks.workunit.client.1.smithi040.stdout:4/825: symlink da/d3f/d40/d47/d75/d104/l11e 0 2024-04-24T23:14:08.934 INFO:tasks.workunit.client.0.smithi031.stdout:5/991: mkdir d0/d6/d3b/d4b/dc/de3/d125/d148 0 2024-04-24T23:14:08.937 INFO:tasks.workunit.client.1.smithi040.stdout:0/903: write df/de1/db6/fbd [28362,90565] 0 2024-04-24T23:14:08.938 INFO:tasks.workunit.client.1.smithi040.stdout:2/908: dread da/f40 [0,4194304] 0 2024-04-24T23:14:08.939 INFO:tasks.workunit.client.0.smithi031.stdout:5/992: mknod d0/d6/d3b/d4b/d1d/c149 0 2024-04-24T23:14:08.940 INFO:tasks.workunit.client.1.smithi040.stdout:6/805: dread d7/d1a/d9f/fb3 [0,4194304] 0 2024-04-24T23:14:08.940 INFO:tasks.workunit.client.1.smithi040.stdout:2/909: chown da/d1c/d37/d39/c6a 7360 1 2024-04-24T23:14:08.941 INFO:tasks.workunit.client.1.smithi040.stdout:6/806: chown d7/d1a/c2b 79172 1 2024-04-24T23:14:08.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/993: write d0/d6/d3b/d4b/d1d/d2e/dc6/f21 [6570763,116827] 0 2024-04-24T23:14:08.942 INFO:tasks.workunit.client.1.smithi040.stdout:0/904: creat df/d1e/d20/d21/d41/d8a/df3/f127 x:0 0 0 2024-04-24T23:14:08.946 INFO:tasks.workunit.client.1.smithi040.stdout:7/898: dread db/d2f/ff8 [0,4194304] 0 2024-04-24T23:14:08.947 INFO:tasks.workunit.client.1.smithi040.stdout:4/826: getdents da/d3f/d40/d41/d8e/de9 0 2024-04-24T23:14:08.948 INFO:tasks.workunit.client.0.smithi031.stdout:5/994: rename d0/d6/d3b/d4b/d1d/d2e/dc6/l137 to d0/d6/d32/d6b/d11a/l14a 0 2024-04-24T23:14:08.949 INFO:tasks.workunit.client.1.smithi040.stdout:6/807: dwrite d7/d1a/d20/d51/d1e/d5d/f31 [4194304,4194304] 0 2024-04-24T23:14:08.951 INFO:tasks.workunit.client.1.smithi040.stdout:7/899: mknod db/d12/d20/d52/d4b/d127/d12b/db7/c142 0 2024-04-24T23:14:08.951 INFO:tasks.workunit.client.0.smithi031.stdout:5/995: mknod d0/d6/d3b/d4b/d1d/d12d/c14b 0 2024-04-24T23:14:08.954 INFO:tasks.workunit.client.1.smithi040.stdout:6/808: symlink d7/d1a/d20/d51/d1e/d46/d85/l10d 0 2024-04-24T23:14:08.956 INFO:tasks.workunit.client.1.smithi040.stdout:6/809: chown d7/d1a/l32 2626293 1 2024-04-24T23:14:08.956 INFO:tasks.workunit.client.1.smithi040.stdout:4/827: dwrite da/d3f/d6b/f32 [4194304,4194304] 0 2024-04-24T23:14:08.958 INFO:tasks.workunit.client.0.smithi031.stdout:5/996: dwrite d0/d6/d3b/d4b/dc/d25/f60 [0,4194304] 0 2024-04-24T23:14:08.960 INFO:tasks.workunit.client.1.smithi040.stdout:6/810: rmdir d7/d1a/d20/d8c/dd5 39 2024-04-24T23:14:08.960 INFO:tasks.workunit.client.1.smithi040.stdout:4/828: fsync da/f18 0 2024-04-24T23:14:08.970 INFO:tasks.workunit.client.1.smithi040.stdout:4/829: symlink da/d45/d48/d6e/de0/l11f 0 2024-04-24T23:14:08.971 INFO:tasks.workunit.client.1.smithi040.stdout:4/830: fdatasync da/d3f/d40/d41/fd9 0 2024-04-24T23:14:08.971 INFO:tasks.workunit.client.0.smithi031.stdout:5/997: getdents d0/d7/d56 0 2024-04-24T23:14:08.975 INFO:tasks.workunit.client.1.smithi040.stdout:4/831: creat da/d3f/d40/d41/d8e/db8/f120 x:0 0 0 2024-04-24T23:14:08.975 INFO:tasks.workunit.client.1.smithi040.stdout:1/817: write da/d12/d18/f78 [3464203,60214] 0 2024-04-24T23:14:08.976 INFO:tasks.workunit.client.1.smithi040.stdout:9/845: write d7/da/d17/d40/d55/d2f/d1f/fc5 [609714,46447] 0 2024-04-24T23:14:08.978 INFO:tasks.workunit.client.1.smithi040.stdout:1/818: chown da/d12/d18/d2a/d4d/d107/db9/fef 1 1 2024-04-24T23:14:08.978 INFO:tasks.workunit.client.1.smithi040.stdout:4/832: creat da/d3f/d40/d47/d75/d104/f121 x:0 0 0 2024-04-24T23:14:08.978 INFO:tasks.workunit.client.1.smithi040.stdout:0/905: dread f6 [0,4194304] 0 2024-04-24T23:14:08.979 INFO:tasks.workunit.client.1.smithi040.stdout:4/833: readlink da/d45/d58/lcf 0 2024-04-24T23:14:08.981 INFO:tasks.workunit.client.1.smithi040.stdout:9/846: truncate d7/da/d17/d40/d55/d2f/d28/fe2 933695 0 2024-04-24T23:14:08.986 INFO:tasks.workunit.client.1.smithi040.stdout:5/750: sync 2024-04-24T23:14:08.986 INFO:tasks.workunit.client.1.smithi040.stdout:9/847: unlink d7/da/d17/d40/d55/d2f/d28/l62 0 2024-04-24T23:14:08.989 INFO:tasks.workunit.client.1.smithi040.stdout:5/751: symlink d2/d2f/d14/dbe/df2/lf3 0 2024-04-24T23:14:08.989 INFO:tasks.workunit.client.1.smithi040.stdout:9/848: truncate d7/da/d17/d40/d55/fc6 1007107 0 2024-04-24T23:14:08.993 INFO:tasks.workunit.client.1.smithi040.stdout:9/849: creat d7/da/d17/d40/d55/d2f/d28/f121 x:0 0 0 2024-04-24T23:14:08.994 INFO:tasks.workunit.client.1.smithi040.stdout:2/910: write da/d1c/d37/d39/d3f/d52/f95 [7619413,60425] 0 2024-04-24T23:14:08.996 INFO:tasks.workunit.client.1.smithi040.stdout:1/819: dread da/ff [0,4194304] 0 2024-04-24T23:14:08.996 INFO:tasks.workunit.client.1.smithi040.stdout:9/850: write d7/d92/ff1 [1513615,103794] 0 2024-04-24T23:14:08.996 INFO:tasks.workunit.client.1.smithi040.stdout:5/752: rename d2/d7/d8/d1f/d57/d3e/c72 to d2/d7/d8/d1f/d57/d27/d3c/d43/cf4 0 2024-04-24T23:14:08.998 INFO:tasks.workunit.client.1.smithi040.stdout:0/906: dread df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dbe/f105 [0,4194304] 0 2024-04-24T23:14:08.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:08 smithi031 ceph-mon[26411]: pgmap v37: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 92 MiB/s rd, 238 MiB/s wr, 528 op/s 2024-04-24T23:14:08.999 INFO:tasks.workunit.client.1.smithi040.stdout:9/851: write d7/dbb/f11a [14856,81469] 0 2024-04-24T23:14:09.000 INFO:tasks.workunit.client.1.smithi040.stdout:0/907: write df/de1/db6/ff9 [956669,69465] 0 2024-04-24T23:14:09.001 INFO:tasks.workunit.client.1.smithi040.stdout:1/820: creat da/d12/d23/f118 x:0 0 0 2024-04-24T23:14:09.003 INFO:tasks.workunit.client.1.smithi040.stdout:5/753: mkdir d2/d7/d8/d1f/d90/df5 0 2024-04-24T23:14:09.003 INFO:tasks.workunit.client.1.smithi040.stdout:7/900: truncate db/d2f/f9f 934230 0 2024-04-24T23:14:09.004 INFO:tasks.workunit.client.1.smithi040.stdout:6/811: write d7/d1a/d20/d51/d1e/fc3 [1077635,122451] 0 2024-04-24T23:14:09.006 INFO:tasks.workunit.client.1.smithi040.stdout:1/821: creat da/d12/d18/d2a/d4d/dfd/f119 x:0 0 0 2024-04-24T23:14:09.007 INFO:tasks.workunit.client.1.smithi040.stdout:9/852: dwrite d7/da/d17/d40/d55/d2f/f6f [4194304,4194304] 0 2024-04-24T23:14:09.009 INFO:tasks.workunit.client.1.smithi040.stdout:0/908: dread df/d1e/d20/d61/fb0 [0,4194304] 0 2024-04-24T23:14:09.010 INFO:tasks.workunit.client.1.smithi040.stdout:7/901: write db/d12/d3f/d7a/f131 [788049,50793] 0 2024-04-24T23:14:09.013 INFO:tasks.workunit.client.1.smithi040.stdout:0/909: readlink df/d1e/d20/d21/d41/la4 0 2024-04-24T23:14:09.014 INFO:tasks.workunit.client.1.smithi040.stdout:0/910: chown df/d12/d49 1 1 2024-04-24T23:14:09.015 INFO:tasks.workunit.client.1.smithi040.stdout:4/834: truncate da/f2f 829986 0 2024-04-24T23:14:09.016 INFO:tasks.workunit.client.1.smithi040.stdout:4/835: dread - da/d3f/d40/d41/d8e/ffb zero size 2024-04-24T23:14:09.018 INFO:tasks.workunit.client.1.smithi040.stdout:1/822: rename da/d35/d38/d47/f80 to da/d12/d18/d2a/d4d/d107/db9/f11a 0 2024-04-24T23:14:09.018 INFO:tasks.workunit.client.1.smithi040.stdout:1/823: chown da/d12/d14 6 1 2024-04-24T23:14:09.019 INFO:tasks.workunit.client.1.smithi040.stdout:1/824: dread - da/d35/d38/fda zero size 2024-04-24T23:14:09.019 INFO:tasks.workunit.client.1.smithi040.stdout:1/825: fsync da/f71 0 2024-04-24T23:14:09.021 INFO:tasks.workunit.client.1.smithi040.stdout:8/903: sync 2024-04-24T23:14:09.021 INFO:tasks.workunit.client.1.smithi040.stdout:3/877: sync 2024-04-24T23:14:09.024 INFO:tasks.workunit.client.1.smithi040.stdout:4/836: mknod da/d45/d48/d6e/ddd/c122 0 2024-04-24T23:14:09.024 INFO:tasks.workunit.client.1.smithi040.stdout:8/904: truncate d0/d7e/dc1/d9e/d90/d91/f126 394157 0 2024-04-24T23:14:09.025 INFO:tasks.workunit.client.1.smithi040.stdout:4/837: read - da/d3f/d40/d47/ff2 zero size 2024-04-24T23:14:09.027 INFO:tasks.workunit.client.1.smithi040.stdout:2/911: dwrite da/d29/f4a [0,4194304] 0 2024-04-24T23:14:09.027 INFO:tasks.workunit.client.1.smithi040.stdout:4/838: chown da/d3f/d40/d41/fd2 258573 1 2024-04-24T23:14:09.030 INFO:tasks.workunit.client.1.smithi040.stdout:3/878: symlink d1/d80/dbf/l127 0 2024-04-24T23:14:09.033 INFO:tasks.workunit.client.1.smithi040.stdout:9/853: write d7/da/d17/fe7 [190254,76288] 0 2024-04-24T23:14:09.036 INFO:tasks.workunit.client.1.smithi040.stdout:6/812: dwrite d7/d1a/d20/f47 [0,4194304] 0 2024-04-24T23:14:09.039 INFO:tasks.workunit.client.1.smithi040.stdout:5/754: dwrite d2/d7/d8/d1f/d57/f44 [0,4194304] 0 2024-04-24T23:14:09.040 INFO:tasks.workunit.client.1.smithi040.stdout:9/854: fsync d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/ffe 0 2024-04-24T23:14:09.042 INFO:tasks.workunit.client.1.smithi040.stdout:6/813: chown d7/d1a/d20/d51/d1e/d46/l8f 2230444 1 2024-04-24T23:14:09.042 INFO:tasks.workunit.client.1.smithi040.stdout:2/912: rename da/d1c/d37/d39/d3f/d52/l125 to da/d1c/d4f/l14a 0 2024-04-24T23:14:09.042 INFO:tasks.workunit.client.1.smithi040.stdout:6/814: chown d7/de/d84 1 1 2024-04-24T23:14:09.043 INFO:tasks.workunit.client.1.smithi040.stdout:3/879: symlink d1/d9/d18/d36/d61/l128 0 2024-04-24T23:14:09.044 INFO:tasks.workunit.client.1.smithi040.stdout:8/905: mkdir d0/d16/d19/d71/d129 0 2024-04-24T23:14:09.050 INFO:tasks.workunit.client.1.smithi040.stdout:3/880: mkdir d1/d9/d2e/dcf/d129 0 2024-04-24T23:14:09.051 INFO:tasks.workunit.client.1.smithi040.stdout:8/906: mknod d0/d7e/dc1/c12a 0 2024-04-24T23:14:09.051 INFO:tasks.workunit.client.1.smithi040.stdout:2/913: write da/d1c/d37/d39/d3f/f67 [736563,28631] 0 2024-04-24T23:14:09.055 INFO:tasks.workunit.client.1.smithi040.stdout:7/902: dwrite db/d12/d20/d52/d44/f8d [0,4194304] 0 2024-04-24T23:14:09.055 INFO:tasks.workunit.client.1.smithi040.stdout:4/839: getdents da/d78/d115 0 2024-04-24T23:14:09.058 INFO:tasks.workunit.client.1.smithi040.stdout:8/907: chown d0/d7e/dc1/d9e/d90/de7 45 1 2024-04-24T23:14:09.059 INFO:tasks.workunit.client.1.smithi040.stdout:2/914: write da/d29/d55/da7/fd6 [444649,103993] 0 2024-04-24T23:14:09.060 INFO:tasks.workunit.client.1.smithi040.stdout:9/855: getdents d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8 0 2024-04-24T23:14:09.062 INFO:tasks.workunit.client.1.smithi040.stdout:1/826: dwrite da/d35/d38/d47/f51 [0,4194304] 0 2024-04-24T23:14:09.062 INFO:tasks.workunit.client.1.smithi040.stdout:4/840: write da/d1f/f9c [1418352,106365] 0 2024-04-24T23:14:09.063 INFO:tasks.workunit.client.1.smithi040.stdout:6/815: dwrite d7/d1a/d20/d51/d1e/d46/fac [0,4194304] 0 2024-04-24T23:14:09.063 INFO:tasks.workunit.client.1.smithi040.stdout:7/903: stat db/d12/d20/d52/d6c/d8f/db8 0 2024-04-24T23:14:09.069 INFO:tasks.workunit.client.1.smithi040.stdout:8/908: creat d0/d7e/dc1/d87/f12b x:0 0 0 2024-04-24T23:14:09.069 INFO:tasks.workunit.client.1.smithi040.stdout:2/915: fsync da/d1c/d37/d39/d3f/d52/d78/dc7/fee 0 2024-04-24T23:14:09.072 INFO:tasks.workunit.client.1.smithi040.stdout:3/881: link d1/d2d/de/d30/d37/cb5 d1/d2d/d44/c12a 0 2024-04-24T23:14:09.072 INFO:tasks.workunit.client.1.smithi040.stdout:9/856: mkdir d7/da/d17/d40/d122 0 2024-04-24T23:14:09.073 INFO:tasks.workunit.client.1.smithi040.stdout:1/827: fsync da/d12/d18/d2a/d8b/d97/f98 0 2024-04-24T23:14:09.074 INFO:tasks.workunit.client.1.smithi040.stdout:2/916: write da/d29/d55/da7/fd6 [1534261,29675] 0 2024-04-24T23:14:09.077 INFO:tasks.workunit.client.1.smithi040.stdout:9/857: fsync d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/ffe 0 2024-04-24T23:14:09.078 INFO:tasks.workunit.client.1.smithi040.stdout:6/816: truncate d7/d1a/d20/d51/d1e/d46/f7d 259145 0 2024-04-24T23:14:09.078 INFO:tasks.workunit.client.1.smithi040.stdout:7/904: dwrite db/d12/d20/d52/d4b/d127/d12b/db9/dd4/d101/f119 [4194304,4194304] 0 2024-04-24T23:14:09.080 INFO:tasks.workunit.client.1.smithi040.stdout:5/755: write d2/d7/d8/f1b [1965610,105238] 0 2024-04-24T23:14:09.082 INFO:tasks.workunit.client.1.smithi040.stdout:1/828: creat da/d12/d18/d2a/d4d/d107/db9/f11b x:0 0 0 2024-04-24T23:14:09.085 INFO:tasks.workunit.client.1.smithi040.stdout:3/882: mkdir d1/d9/d2e/dcf/d129/d12b 0 2024-04-24T23:14:09.086 INFO:tasks.workunit.client.1.smithi040.stdout:7/905: truncate db/d12/d20/d52/d44/f53 1483493 0 2024-04-24T23:14:09.087 INFO:tasks.workunit.client.1.smithi040.stdout:7/906: chown db/d2f/ff8 954 1 2024-04-24T23:14:09.088 INFO:tasks.workunit.client.1.smithi040.stdout:9/858: link d7/da/d17/d40/d55/d2f/d28/d67/f112 d7/dbb/f123 0 2024-04-24T23:14:09.089 INFO:tasks.workunit.client.1.smithi040.stdout:5/756: symlink d2/d7/d8/d1f/d57/lf6 0 2024-04-24T23:14:09.090 INFO:tasks.workunit.client.1.smithi040.stdout:5/757: chown d2/d2f/d14/dbe 53 1 2024-04-24T23:14:09.097 INFO:tasks.workunit.client.1.smithi040.stdout:0/911: sync 2024-04-24T23:14:09.106 INFO:tasks.workunit.client.1.smithi040.stdout:3/883: creat d1/d9/d18/d36/d61/f12c x:0 0 0 2024-04-24T23:14:09.111 INFO:tasks.workunit.client.1.smithi040.stdout:8/909: dread d0/d8/f29 [0,4194304] 0 2024-04-24T23:14:09.114 INFO:tasks.workunit.client.1.smithi040.stdout:9/859: creat d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f124 x:0 0 0 2024-04-24T23:14:09.115 INFO:tasks.workunit.client.1.smithi040.stdout:5/758: dread d2/d2f/d14/f61 [0,4194304] 0 2024-04-24T23:14:09.115 INFO:tasks.workunit.client.1.smithi040.stdout:2/917: link da/d1c/d37/l12e da/d1c/d37/d39/d3f/db2/l14b 0 2024-04-24T23:14:09.116 INFO:tasks.workunit.client.1.smithi040.stdout:3/884: truncate d1/d2a/fd2 647432 0 2024-04-24T23:14:09.116 INFO:tasks.workunit.client.1.smithi040.stdout:9/860: truncate d7/d5e/da1/dfa/f10c 99091 0 2024-04-24T23:14:09.117 INFO:tasks.workunit.client.1.smithi040.stdout:8/910: unlink d0/f82 0 2024-04-24T23:14:09.117 INFO:tasks.workunit.client.1.smithi040.stdout:3/885: chown d1/d9/d18/d36/ffd 18279 1 2024-04-24T23:14:09.121 INFO:tasks.workunit.client.1.smithi040.stdout:4/841: write da/d78/dbb/fc2 [3211814,57211] 0 2024-04-24T23:14:09.122 INFO:tasks.workunit.client.1.smithi040.stdout:2/918: mkdir da/d1c/d37/d39/d77/d108/d10c/d14c 0 2024-04-24T23:14:09.122 INFO:tasks.workunit.client.1.smithi040.stdout:4/842: write da/d1f/fe7 [4894668,96575] 0 2024-04-24T23:14:09.123 INFO:tasks.workunit.client.1.smithi040.stdout:5/759: truncate d2/d7/d8/d1f/d57/d27/f32 5028736 0 2024-04-24T23:14:09.124 INFO:tasks.workunit.client.1.smithi040.stdout:5/760: chown d2/d2f/l29 1977 1 2024-04-24T23:14:09.127 INFO:tasks.workunit.client.1.smithi040.stdout:9/861: creat d7/da/d17/d40/d55/d2f/df7/d6c/f125 x:0 0 0 2024-04-24T23:14:09.131 INFO:tasks.workunit.client.1.smithi040.stdout:2/919: rmdir da/d1c/d37/d39/d79/da3 39 2024-04-24T23:14:09.132 INFO:tasks.workunit.client.1.smithi040.stdout:4/843: symlink da/d3f/d40/d41/d60/l123 0 2024-04-24T23:14:09.133 INFO:tasks.workunit.client.1.smithi040.stdout:4/844: fsync da/d3f/d40/fb2 0 2024-04-24T23:14:09.137 INFO:tasks.workunit.client.1.smithi040.stdout:4/845: fsync da/d3f/d40/d41/d60/f24 0 2024-04-24T23:14:09.138 INFO:tasks.workunit.client.1.smithi040.stdout:3/886: read d1/d2a/d6a/d94/f95 [1451222,74159] 0 2024-04-24T23:14:09.139 INFO:tasks.workunit.client.1.smithi040.stdout:1/829: truncate da/d12/d18/d2a/d4d/d107/fee 2783324 0 2024-04-24T23:14:09.140 INFO:tasks.workunit.client.1.smithi040.stdout:6/817: dwrite d7/d1a/d9f/fb4 [0,4194304] 0 2024-04-24T23:14:09.140 INFO:tasks.workunit.client.1.smithi040.stdout:2/920: fdatasync da/d10/f71 0 2024-04-24T23:14:09.143 INFO:tasks.workunit.client.1.smithi040.stdout:1/830: chown da/d12/d18/d2a/d4d/d107 0 1 2024-04-24T23:14:09.144 INFO:tasks.workunit.client.1.smithi040.stdout:9/862: dread d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f43 [4194304,4194304] 0 2024-04-24T23:14:09.145 INFO:tasks.workunit.client.1.smithi040.stdout:4/846: read da/d3f/d40/d47/d75/d104/f105 [2203024,123303] 0 2024-04-24T23:14:09.148 INFO:tasks.workunit.client.1.smithi040.stdout:2/921: symlink da/d1c/d37/d39/d79/d13d/l14d 0 2024-04-24T23:14:09.151 INFO:tasks.workunit.client.1.smithi040.stdout:6/818: mknod d7/d1a/d20/d51/d1e/c10e 0 2024-04-24T23:14:09.152 INFO:tasks.workunit.client.1.smithi040.stdout:1/831: unlink da/d12/d18/fa6 0 2024-04-24T23:14:09.155 INFO:tasks.workunit.client.1.smithi040.stdout:7/907: dwrite db/d12/d20/d52/d4b/d127/d12b/db9/fe3 [0,4194304] 0 2024-04-24T23:14:09.156 INFO:tasks.workunit.client.1.smithi040.stdout:5/761: dread d2/d7/d8/f34 [0,4194304] 0 2024-04-24T23:14:09.157 INFO:tasks.workunit.client.1.smithi040.stdout:4/847: creat da/d3f/d40/d41/f124 x:0 0 0 2024-04-24T23:14:09.159 INFO:tasks.workunit.client.1.smithi040.stdout:9/863: creat d7/da/d17/d40/d55/d2f/d28/f126 x:0 0 0 2024-04-24T23:14:09.160 INFO:tasks.workunit.client.1.smithi040.stdout:6/819: truncate d7/d1a/f42 1878010 0 2024-04-24T23:14:09.161 INFO:tasks.workunit.client.1.smithi040.stdout:0/912: dwrite df/d1e/f40 [0,4194304] 0 2024-04-24T23:14:09.166 INFO:tasks.workunit.client.1.smithi040.stdout:7/908: write db/d12/d20/d52/d44/f8d [5136582,117392] 0 2024-04-24T23:14:09.168 INFO:tasks.workunit.client.1.smithi040.stdout:8/911: dwrite d0/d8/dc/f96 [0,4194304] 0 2024-04-24T23:14:09.171 INFO:tasks.workunit.client.1.smithi040.stdout:0/913: mkdir df/d1e/d89/d128 0 2024-04-24T23:14:09.174 INFO:tasks.workunit.client.1.smithi040.stdout:7/909: write db/d12/d62/de7/ffb [317310,122029] 0 2024-04-24T23:14:09.177 INFO:tasks.workunit.client.1.smithi040.stdout:2/922: creat da/d1c/d37/d39/d3f/f14e x:0 0 0 2024-04-24T23:14:09.178 INFO:tasks.workunit.client.1.smithi040.stdout:3/887: dwrite d1/d9/d18/d36/d10d/f53 [0,4194304] 0 2024-04-24T23:14:09.178 INFO:tasks.workunit.client.1.smithi040.stdout:5/762: truncate d2/d7/d8/db/da2/dcc/fdc 498172 0 2024-04-24T23:14:09.179 INFO:tasks.workunit.client.1.smithi040.stdout:2/923: fsync da/d29/d55/da7/fd6 0 2024-04-24T23:14:09.182 INFO:tasks.workunit.client.1.smithi040.stdout:6/820: rename d7/d1a/d20/c48 to d7/d1a/d20/d51/d1e/d46/d69/c10f 0 2024-04-24T23:14:09.183 INFO:tasks.workunit.client.1.smithi040.stdout:7/910: symlink db/d12/d3f/d7a/l143 0 2024-04-24T23:14:09.183 INFO:tasks.workunit.client.1.smithi040.stdout:3/888: stat d1/d9/d18/d36/d10d/fdd 0 2024-04-24T23:14:09.184 INFO:tasks.workunit.client.1.smithi040.stdout:2/924: write da/d10/d15/f2e [1711433,43913] 0 2024-04-24T23:14:09.187 INFO:tasks.workunit.client.1.smithi040.stdout:9/864: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f124 [0,4194304] 0 2024-04-24T23:14:09.188 INFO:tasks.workunit.client.1.smithi040.stdout:6/821: fsync d7/d1a/d20/d51/d2f/f35 0 2024-04-24T23:14:09.189 INFO:tasks.workunit.client.1.smithi040.stdout:9/865: chown d7/dbb 1727987 1 2024-04-24T23:14:09.190 INFO:tasks.workunit.client.1.smithi040.stdout:5/763: symlink d2/d7/d8/lf7 0 2024-04-24T23:14:09.191 INFO:tasks.workunit.client.1.smithi040.stdout:5/764: write d2/d7/f36 [1064018,76665] 0 2024-04-24T23:14:09.194 INFO:tasks.workunit.client.1.smithi040.stdout:6/822: unlink d7/d1a/d56/l6a 0 2024-04-24T23:14:09.194 INFO:tasks.workunit.client.1.smithi040.stdout:3/889: chown d1/d2d/de/d30/d37/d107/c11b 644 1 2024-04-24T23:14:09.200 INFO:tasks.workunit.client.1.smithi040.stdout:2/925: dwrite da/d10/d4d/dd7/f110 [0,4194304] 0 2024-04-24T23:14:09.202 INFO:tasks.workunit.client.1.smithi040.stdout:1/832: dread da/d12/d18/d2a/fc7 [0,4194304] 0 2024-04-24T23:14:09.210 INFO:tasks.workunit.client.1.smithi040.stdout:2/926: rename da/d10/d15/f2e to da/d10/d15/f14f 0 2024-04-24T23:14:09.210 INFO:tasks.workunit.client.1.smithi040.stdout:0/914: dread df/de1/f9e [0,4194304] 0 2024-04-24T23:14:09.211 INFO:tasks.workunit.client.1.smithi040.stdout:4/848: write da/d45/d48/fa2 [460512,14777] 0 2024-04-24T23:14:09.213 INFO:tasks.workunit.client.1.smithi040.stdout:1/833: dwrite da/d12/d18/d2a/d4d/d107/db9/fef [0,4194304] 0 2024-04-24T23:14:09.215 INFO:tasks.workunit.client.1.smithi040.stdout:4/849: dread - da/d3f/d40/d47/d75/daa/ff1 zero size 2024-04-24T23:14:09.219 INFO:tasks.workunit.client.1.smithi040.stdout:2/927: dwrite da/d29/d55/f131 [0,4194304] 0 2024-04-24T23:14:09.221 INFO:tasks.workunit.client.1.smithi040.stdout:2/928: rename da/d1c to da/d1c/d37/d102/d150 22 2024-04-24T23:14:09.221 INFO:tasks.workunit.client.1.smithi040.stdout:1/834: mknod da/d12/d18/d2a/d41/d8c/c11c 0 2024-04-24T23:14:09.230 INFO:tasks.workunit.client.1.smithi040.stdout:2/929: fsync da/d81/da8/f116 0 2024-04-24T23:14:09.230 INFO:tasks.workunit.client.1.smithi040.stdout:7/911: truncate db/f1e 191072 0 2024-04-24T23:14:09.233 INFO:tasks.workunit.client.1.smithi040.stdout:6/823: write d7/fca [1069525,59541] 0 2024-04-24T23:14:09.236 INFO:tasks.workunit.client.1.smithi040.stdout:6/824: truncate d7/d1a/d20/d51/d1e/f49 5536964 0 2024-04-24T23:14:09.242 INFO:tasks.workunit.client.1.smithi040.stdout:5/765: dwrite d2/d7/d8/db/d1e/f46 [0,4194304] 0 2024-04-24T23:14:09.245 INFO:tasks.workunit.client.1.smithi040.stdout:3/890: dwrite d1/d2a/d6a/fc2 [0,4194304] 0 2024-04-24T23:14:09.246 INFO:tasks.workunit.client.1.smithi040.stdout:0/915: dread df/d12/d58/dc3/dcd/ffa [8388608,4194304] 0 2024-04-24T23:14:09.249 INFO:tasks.workunit.client.1.smithi040.stdout:4/850: link da/d45/d58/lcf da/d3f/d40/d41/d8e/db8/l125 0 2024-04-24T23:14:09.251 INFO:tasks.workunit.client.1.smithi040.stdout:7/912: mkdir db/d12/d62/d8a/db3/dbf/d144 0 2024-04-24T23:14:09.259 INFO:tasks.workunit.client.1.smithi040.stdout:5/766: creat d2/d2f/d14/ddf/ff8 x:0 0 0 2024-04-24T23:14:09.260 INFO:tasks.workunit.client.1.smithi040.stdout:6/825: dread d7/d1a/d20/d51/d1e/fe8 [4194304,4194304] 0 2024-04-24T23:14:09.262 INFO:tasks.workunit.client.1.smithi040.stdout:3/891: creat d1/f12d x:0 0 0 2024-04-24T23:14:09.263 INFO:tasks.workunit.client.1.smithi040.stdout:0/916: mknod df/c129 0 2024-04-24T23:14:09.265 INFO:tasks.workunit.client.1.smithi040.stdout:4/851: symlink da/d45/d48/d6e/ddd/dea/l126 0 2024-04-24T23:14:09.267 INFO:tasks.workunit.client.1.smithi040.stdout:1/835: dwrite da/d12/d14/f96 [0,4194304] 0 2024-04-24T23:14:09.268 INFO:tasks.workunit.client.1.smithi040.stdout:3/892: read d1/d9/f21 [6914338,36767] 0 2024-04-24T23:14:09.269 INFO:tasks.workunit.client.1.smithi040.stdout:2/930: rename da/d1c/d37/d39/d79/l145 to da/d81/l151 0 2024-04-24T23:14:09.271 INFO:tasks.workunit.client.1.smithi040.stdout:3/893: dread - d1/d9/d18/d36/d61/f12c zero size 2024-04-24T23:14:09.280 INFO:tasks.workunit.client.1.smithi040.stdout:4/852: unlink l1 0 2024-04-24T23:14:09.285 INFO:tasks.workunit.client.1.smithi040.stdout:1/836: rename da/f71 to da/d12/d18/d2a/d4d/dfd/f11d 0 2024-04-24T23:14:09.286 INFO:tasks.workunit.client.1.smithi040.stdout:1/837: dread - da/d12/d18/d2a/d8b/f111 zero size 2024-04-24T23:14:09.287 INFO:tasks.workunit.client.1.smithi040.stdout:2/931: symlink da/d10/d4d/da4/l152 0 2024-04-24T23:14:09.287 INFO:tasks.workunit.client.1.smithi040.stdout:8/912: sync 2024-04-24T23:14:09.288 INFO:tasks.workunit.client.1.smithi040.stdout:9/866: sync 2024-04-24T23:14:09.290 INFO:tasks.workunit.client.1.smithi040.stdout:3/894: mkdir d1/d2d/d72/d106/d12e 0 2024-04-24T23:14:09.290 INFO:tasks.workunit.client.1.smithi040.stdout:6/826: dread d7/d1a/d20/d51/d1e/f22 [0,4194304] 0 2024-04-24T23:14:09.290 INFO:tasks.workunit.client.1.smithi040.stdout:2/932: write da/d81/da8/f116 [984393,82673] 0 2024-04-24T23:14:09.293 INFO:tasks.workunit.client.1.smithi040.stdout:5/767: truncate d2/d7/d8/d1f/d57/d27/d3d/dd6/feb 5182611 0 2024-04-24T23:14:09.294 INFO:tasks.workunit.client.1.smithi040.stdout:8/913: truncate d0/d8/d22/db8/dc2/f11d 1017268 0 2024-04-24T23:14:09.297 INFO:tasks.workunit.client.1.smithi040.stdout:0/917: dread df/d12/d58/d5b/fa0 [0,4194304] 0 2024-04-24T23:14:09.297 INFO:tasks.workunit.client.1.smithi040.stdout:4/853: fsync da/d3f/d40/d41/f67 0 2024-04-24T23:14:09.298 INFO:tasks.workunit.client.1.smithi040.stdout:1/838: dwrite da/d12/d14/fdd [0,4194304] 0 2024-04-24T23:14:09.302 INFO:tasks.workunit.client.1.smithi040.stdout:2/933: dwrite da/f19 [4194304,4194304] 0 2024-04-24T23:14:09.304 INFO:tasks.workunit.client.1.smithi040.stdout:1/839: write da/d12/d14/d58/ff0 [2394982,37447] 0 2024-04-24T23:14:09.305 INFO:tasks.workunit.client.1.smithi040.stdout:3/895: symlink d1/d80/dbf/l12f 0 2024-04-24T23:14:09.308 INFO:tasks.workunit.client.1.smithi040.stdout:6/827: mkdir d7/d1a/d9f/df6/d110 0 2024-04-24T23:14:09.310 INFO:tasks.workunit.client.1.smithi040.stdout:7/913: dwrite db/f24 [0,4194304] 0 2024-04-24T23:14:09.311 INFO:tasks.workunit.client.1.smithi040.stdout:1/840: dread - da/d12/d18/d2a/d8b/d97/f98 zero size 2024-04-24T23:14:09.313 INFO:tasks.workunit.client.1.smithi040.stdout:5/768: symlink d2/d7/d8/db/da2/lf9 0 2024-04-24T23:14:09.313 INFO:tasks.workunit.client.1.smithi040.stdout:6/828: readlink d7/d1a/d20/d51/d1e/d46/d85/lc2 0 2024-04-24T23:14:09.315 INFO:tasks.workunit.client.1.smithi040.stdout:5/769: stat d2/d7/d8/f39 0 2024-04-24T23:14:09.315 INFO:tasks.workunit.client.1.smithi040.stdout:1/841: dread - da/d12/d18/d2a/d8b/f10d zero size 2024-04-24T23:14:09.315 INFO:tasks.workunit.client.1.smithi040.stdout:8/914: rename d0/d16/d19/da2 to d0/d38/d4c/da3/d12c 0 2024-04-24T23:14:09.315 INFO:tasks.workunit.client.1.smithi040.stdout:6/829: chown d7/d1a/d56/l95 0 1 2024-04-24T23:14:09.318 INFO:tasks.workunit.client.1.smithi040.stdout:3/896: dwrite d1/f12d [0,4194304] 0 2024-04-24T23:14:09.319 INFO:tasks.workunit.client.1.smithi040.stdout:6/830: fdatasync d7/d1a/d20/f47 0 2024-04-24T23:14:09.319 INFO:tasks.workunit.client.1.smithi040.stdout:2/934: readlink da/d1c/d37/d39/d77/d108/d10c/l137 0 2024-04-24T23:14:09.319 INFO:tasks.workunit.client.1.smithi040.stdout:5/770: write d2/d7/d8/db/f96 [3703731,29359] 0 2024-04-24T23:14:09.321 INFO:tasks.workunit.client.1.smithi040.stdout:3/897: chown d1/d9/d18/d36/d10d/f53 1 1 2024-04-24T23:14:09.322 INFO:tasks.workunit.client.1.smithi040.stdout:4/854: rename da/d3f/d40/d47/d75/daa to da/d78/dbb/d127 0 2024-04-24T23:14:09.323 INFO:tasks.workunit.client.1.smithi040.stdout:5/771: write d2/d7/d8/d1f/d57/f44 [183125,69705] 0 2024-04-24T23:14:09.328 INFO:tasks.workunit.client.1.smithi040.stdout:9/867: link d7/d5e/da1/lac d7/da/d17/d40/d55/d2f/d1f/d4e/d83/l127 0 2024-04-24T23:14:09.329 INFO:tasks.workunit.client.1.smithi040.stdout:9/868: fsync d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/ffe 0 2024-04-24T23:14:09.331 INFO:tasks.workunit.client.1.smithi040.stdout:1/842: rmdir da/d12/d18/d2a/d41/d8c 39 2024-04-24T23:14:09.333 INFO:tasks.workunit.client.1.smithi040.stdout:6/831: creat d7/d1a/f111 x:0 0 0 2024-04-24T23:14:09.334 INFO:tasks.workunit.client.1.smithi040.stdout:2/935: mkdir da/d1c/d37/d39/d79/d13d/d153 0 2024-04-24T23:14:09.334 INFO:tasks.workunit.client.1.smithi040.stdout:6/832: dread - d7/d1a/d20/d51/d1e/fec zero size 2024-04-24T23:14:09.336 INFO:tasks.workunit.client.1.smithi040.stdout:3/898: fdatasync d1/d2d/de/d30/f41 0 2024-04-24T23:14:09.338 INFO:tasks.workunit.client.1.smithi040.stdout:7/914: rename db/d2f/d11a to db/d2f/d37/d7e/d118/d145 0 2024-04-24T23:14:09.341 INFO:tasks.workunit.client.1.smithi040.stdout:9/869: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/d128 0 2024-04-24T23:14:09.341 INFO:tasks.workunit.client.1.smithi040.stdout:5/772: dread d2/d7/d8/db/f56 [0,4194304] 0 2024-04-24T23:14:09.343 INFO:tasks.workunit.client.1.smithi040.stdout:2/936: creat da/d10/d4d/d6c/f154 x:0 0 0 2024-04-24T23:14:09.345 INFO:tasks.workunit.client.1.smithi040.stdout:3/899: mknod d1/d80/c130 0 2024-04-24T23:14:09.349 INFO:tasks.workunit.client.1.smithi040.stdout:5/773: fsync d2/d7/d8/db/d1e/f2e 0 2024-04-24T23:14:09.356 INFO:tasks.workunit.client.1.smithi040.stdout:9/870: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/d128/c129 0 2024-04-24T23:14:09.357 INFO:tasks.workunit.client.1.smithi040.stdout:2/937: truncate da/d29/f42 768244 0 2024-04-24T23:14:09.359 INFO:tasks.workunit.client.1.smithi040.stdout:6/833: dread d7/d1a/d20/d51/d1e/d5d/f31 [0,4194304] 0 2024-04-24T23:14:09.360 INFO:tasks.workunit.client.1.smithi040.stdout:8/915: write d0/d7e/dc1/f72 [1019120,75489] 0 2024-04-24T23:14:09.360 INFO:tasks.workunit.client.1.smithi040.stdout:2/938: read da/d10/d7a/f8d [1930618,121602] 0 2024-04-24T23:14:09.360 INFO:tasks.workunit.client.1.smithi040.stdout:0/918: dwrite df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dbe/f105 [0,4194304] 0 2024-04-24T23:14:09.362 INFO:tasks.workunit.client.1.smithi040.stdout:1/843: getdents da/d12 0 2024-04-24T23:14:09.363 INFO:tasks.workunit.client.0.smithi031.stdout:7/993: sync 2024-04-24T23:14:09.363 INFO:tasks.workunit.client.0.smithi031.stdout:3/951: sync 2024-04-24T23:14:09.363 INFO:tasks.workunit.client.0.smithi031.stdout:2/955: sync 2024-04-24T23:14:09.365 INFO:tasks.workunit.client.0.smithi031.stdout:3/952: chown d6/df/d16/d57/df8 69517 1 2024-04-24T23:14:09.365 INFO:tasks.workunit.client.1.smithi040.stdout:3/900: link d1/d9/d18/d36/d61/lb6 d1/d9/d2e/dcf/l131 0 2024-04-24T23:14:09.365 INFO:tasks.workunit.client.1.smithi040.stdout:7/915: link db/d2f/d37/d7e/l13c db/d12/d62/d103/l146 0 2024-04-24T23:14:09.367 INFO:tasks.workunit.client.1.smithi040.stdout:2/939: write da/d1c/d37/d39/d3f/d52/d78/f133 [162625,29158] 0 2024-04-24T23:14:09.368 INFO:tasks.workunit.client.1.smithi040.stdout:0/919: rmdir df/d1e/d20/d21/d41 39 2024-04-24T23:14:09.369 INFO:tasks.workunit.client.1.smithi040.stdout:7/916: dread - db/d12/d20/d52/d4b/d127/d12b/db9/dd4/d101/f115 zero size 2024-04-24T23:14:09.370 INFO:tasks.workunit.client.1.smithi040.stdout:2/940: truncate da/d1c/d37/d102/f149 724705 0 2024-04-24T23:14:09.370 INFO:tasks.workunit.client.1.smithi040.stdout:2/941: write da/d10/f31 [6391555,57923] 0 2024-04-24T23:14:09.371 INFO:tasks.workunit.client.0.smithi031.stdout:2/956: creat d5/d8/d45/d5b/d49/d2f/d135/d68/f150 x:0 0 0 2024-04-24T23:14:09.371 INFO:tasks.workunit.client.1.smithi040.stdout:2/942: chown da/d81/de6/fd3 0 1 2024-04-24T23:14:09.372 INFO:tasks.workunit.client.1.smithi040.stdout:8/916: mknod d0/d16/d1b/d58/c12d 0 2024-04-24T23:14:09.372 INFO:tasks.workunit.client.1.smithi040.stdout:2/943: truncate da/d1c/d37/d102/f149 1165822 0 2024-04-24T23:14:09.373 INFO:tasks.workunit.client.0.smithi031.stdout:2/957: readlink d5/d8/d45/d5b/d49/d2f/d135/d29/d26/l74 0 2024-04-24T23:14:09.374 INFO:tasks.workunit.client.0.smithi031.stdout:2/958: chown d5/d8/d45/d5b/d49/d2f/d135/d68/lec 6077 1 2024-04-24T23:14:09.375 INFO:tasks.workunit.client.1.smithi040.stdout:2/944: dread da/d1c/d37/d102/f149 [0,4194304] 0 2024-04-24T23:14:09.376 INFO:tasks.workunit.client.1.smithi040.stdout:5/774: write d2/d7/d8/d1f/d57/d27/d65/fa5 [601672,10964] 0 2024-04-24T23:14:09.377 INFO:tasks.workunit.client.1.smithi040.stdout:1/844: truncate da/d12/d14/f3f 2122616 0 2024-04-24T23:14:09.377 INFO:tasks.workunit.client.1.smithi040.stdout:5/775: stat d2/d7/d8/d64/d6b/c83 0 2024-04-24T23:14:09.378 INFO:tasks.workunit.client.1.smithi040.stdout:1/845: write da/d12/d14/fe0 [1137561,107302] 0 2024-04-24T23:14:09.379 INFO:tasks.workunit.client.0.smithi031.stdout:3/953: creat d6/df/d16/d2a/d95/d9b/d29/db7/de9/d101/f12a x:0 0 0 2024-04-24T23:14:09.379 INFO:tasks.workunit.client.1.smithi040.stdout:1/846: stat da/ce7 0 2024-04-24T23:14:09.379 INFO:tasks.workunit.client.0.smithi031.stdout:3/954: read d6/df/f34 [634481,63540] 0 2024-04-24T23:14:09.382 INFO:tasks.workunit.client.0.smithi031.stdout:2/959: creat d5/d8/d15/d56/d84/d75/f151 x:0 0 0 2024-04-24T23:14:09.383 INFO:tasks.workunit.client.1.smithi040.stdout:9/871: write d7/fe [654361,110154] 0 2024-04-24T23:14:09.384 INFO:tasks.workunit.client.0.smithi031.stdout:5/998: sync 2024-04-24T23:14:09.386 INFO:tasks.workunit.client.0.smithi031.stdout:5/999: dread - d0/d6/d3b/d4b/dc/d86/dfd/fe7 zero size 2024-04-24T23:14:09.387 INFO:tasks.workunit.client.1.smithi040.stdout:7/917: chown f6 3538930 1 2024-04-24T23:14:09.387 INFO:tasks.workunit.client.0.smithi031.stdout:3/955: read d6/f8 [2963806,12128] 0 2024-04-24T23:14:09.392 INFO:tasks.workunit.client.0.smithi031.stdout:3/956: creat d6/d14/d1d/d59/f12b x:0 0 0 2024-04-24T23:14:09.396 INFO:tasks.workunit.client.0.smithi031.stdout:7/994: dwrite d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/d66/fd6 [0,4194304] 0 2024-04-24T23:14:09.397 INFO:tasks.workunit.client.1.smithi040.stdout:5/776: fsync d2/d7/d8/d1f/f6e 0 2024-04-24T23:14:09.402 INFO:tasks.workunit.client.0.smithi031.stdout:2/960: dread d5/d8/d15/d56/d84/d75/f91 [0,4194304] 0 2024-04-24T23:14:09.402 INFO:tasks.workunit.client.0.smithi031.stdout:7/995: chown d1/d2/dd/d33/d5e/d64/d6d/de1/d75/d9f/l27 30 1 2024-04-24T23:14:09.404 INFO:tasks.workunit.client.1.smithi040.stdout:9/872: rmdir d7/d92/d97 39 2024-04-24T23:14:09.405 INFO:tasks.workunit.client.1.smithi040.stdout:4/855: sync 2024-04-24T23:14:09.406 INFO:tasks.workunit.client.1.smithi040.stdout:3/901: dread d1/d9/d18/d36/d55/f67 [0,4194304] 0 2024-04-24T23:14:09.407 INFO:tasks.workunit.client.0.smithi031.stdout:7/996: creat d1/d121/f151 x:0 0 0 2024-04-24T23:14:09.408 INFO:tasks.workunit.client.0.smithi031.stdout:7/997: stat d1/d2/dd/d33/d5e/d64/d6d/de1/d84/f96 0 2024-04-24T23:14:09.409 INFO:tasks.workunit.client.1.smithi040.stdout:5/777: dwrite d2/d7/d8/d1f/d57/d27/d3c/da4/fe9 [0,4194304] 0 2024-04-24T23:14:09.411 INFO:tasks.workunit.client.1.smithi040.stdout:3/902: chown d1/d2d/de/d30/cf2 4511338 1 2024-04-24T23:14:09.414 INFO:tasks.workunit.client.0.smithi031.stdout:7/998: symlink d1/d2/dd/d33/d5e/d64/d6d/de1/d84/de2/df2/l152 0 2024-04-24T23:14:09.414 INFO:tasks.workunit.client.0.smithi031.stdout:2/961: dwrite d5/d8/d15/d56/d84/d75/fed [0,4194304] 0 2024-04-24T23:14:09.415 INFO:tasks.workunit.client.1.smithi040.stdout:2/945: rename da/d1c/d37/d39/d79/da3 to da/d1c/d37/d39/d79/d13d/d153/d155 0 2024-04-24T23:14:09.415 INFO:tasks.workunit.client.1.smithi040.stdout:8/917: dwrite d0/d52/d9b/fe1 [0,4194304] 0 2024-04-24T23:14:09.417 INFO:tasks.workunit.client.1.smithi040.stdout:5/778: truncate d2/d7/d8/d1f/d57/fdd 1910373 0 2024-04-24T23:14:09.417 INFO:tasks.workunit.client.0.smithi031.stdout:7/999: fdatasync d1/d2/dd/d33/d5e/d64/f99 0 2024-04-24T23:14:09.418 INFO:tasks.workunit.client.1.smithi040.stdout:0/920: mknod df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dbe/c12a 0 2024-04-24T23:14:09.420 INFO:tasks.workunit.client.0.smithi031.stdout:2/962: creat d5/d3d/f152 x:0 0 0 2024-04-24T23:14:09.421 INFO:tasks.workunit.client.1.smithi040.stdout:8/918: stat d0/d16/d19/d71/d76/l93 0 2024-04-24T23:14:09.423 INFO:tasks.workunit.client.0.smithi031.stdout:2/963: mkdir d5/d8/d45/d5b/d153 0 2024-04-24T23:14:09.425 INFO:tasks.workunit.client.0.smithi031.stdout:2/964: dread - d5/d28/d4e/d5e/f136 zero size 2024-04-24T23:14:09.426 INFO:tasks.workunit.client.1.smithi040.stdout:9/873: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/d83/fb3 [0,4194304] 0 2024-04-24T23:14:09.427 INFO:tasks.workunit.client.1.smithi040.stdout:8/919: chown d0/d16/d1b/d3f/d44/lcc 86 1 2024-04-24T23:14:09.428 INFO:tasks.workunit.client.1.smithi040.stdout:3/903: creat d1/d9/d18/d36/d10d/f132 x:0 0 0 2024-04-24T23:14:09.429 INFO:tasks.workunit.client.1.smithi040.stdout:6/834: getdents d7/d1a/d20/d8c/dd5 0 2024-04-24T23:14:09.430 INFO:tasks.workunit.client.1.smithi040.stdout:1/847: dwrite da/d12/d18/d2a/d4d/f59 [0,4194304] 0 2024-04-24T23:14:09.431 INFO:tasks.workunit.client.1.smithi040.stdout:2/946: fdatasync da/f11 0 2024-04-24T23:14:09.437 INFO:tasks.workunit.client.1.smithi040.stdout:0/921: mknod df/d12/d1a/c12b 0 2024-04-24T23:14:09.438 INFO:tasks.workunit.client.1.smithi040.stdout:7/918: dwrite db/d12/d20/d52/d4b/f11e [4194304,4194304] 0 2024-04-24T23:14:09.439 INFO:tasks.workunit.client.1.smithi040.stdout:0/922: fdatasync df/d12/d4b/f6a 0 2024-04-24T23:14:09.440 INFO:tasks.workunit.client.1.smithi040.stdout:1/848: fdatasync da/d12/d23/f3c 0 2024-04-24T23:14:09.442 INFO:tasks.workunit.client.1.smithi040.stdout:0/923: dread - df/d1e/d20/d61/f10e zero size 2024-04-24T23:14:09.443 INFO:tasks.workunit.client.1.smithi040.stdout:2/947: unlink da/d29/d55/da7/f112 0 2024-04-24T23:14:09.443 INFO:tasks.workunit.client.1.smithi040.stdout:1/849: write da/d35/d38/d47/f51 [3949675,105142] 0 2024-04-24T23:14:09.446 INFO:tasks.workunit.client.1.smithi040.stdout:6/835: dwrite d7/d1a/d20/d8c/fcf [0,4194304] 0 2024-04-24T23:14:09.456 INFO:tasks.workunit.client.1.smithi040.stdout:6/836: dwrite d7/d1a/f67 [0,4194304] 0 2024-04-24T23:14:09.459 INFO:tasks.workunit.client.1.smithi040.stdout:1/850: mkdir da/d12/d18/d2a/d4d/dfd/d11e 0 2024-04-24T23:14:09.462 INFO:tasks.workunit.client.1.smithi040.stdout:6/837: rmdir d7/d1a/d56 39 2024-04-24T23:14:09.462 INFO:tasks.workunit.client.1.smithi040.stdout:4/856: write da/d3f/d6b/d3c/f5c [1464601,92833] 0 2024-04-24T23:14:09.464 INFO:tasks.workunit.client.1.smithi040.stdout:1/851: symlink da/d35/d38/d47/dbc/de5/l11f 0 2024-04-24T23:14:09.464 INFO:tasks.workunit.client.1.smithi040.stdout:6/838: fsync d7/d1a/d20/d51/d1e/d46/d69/f101 0 2024-04-24T23:14:09.465 INFO:tasks.workunit.client.1.smithi040.stdout:6/839: fdatasync d7/d1a/d20/d51/d2f/f6f 0 2024-04-24T23:14:09.465 INFO:tasks.workunit.client.1.smithi040.stdout:2/948: rmdir da/d1c/d37/d39/d77/d108/d10c/d14c 0 2024-04-24T23:14:09.466 INFO:tasks.workunit.client.0.smithi031.stdout:2/965: truncate d5/d3d/f10f 472606 0 2024-04-24T23:14:09.467 INFO:tasks.workunit.client.1.smithi040.stdout:2/949: dread - da/d81/fbc zero size 2024-04-24T23:14:09.467 INFO:tasks.workunit.client.1.smithi040.stdout:2/950: write da/d1c/f57 [4571446,48521] 0 2024-04-24T23:14:09.468 INFO:tasks.workunit.client.1.smithi040.stdout:4/857: creat da/d3f/d8d/db5/f128 x:0 0 0 2024-04-24T23:14:09.468 INFO:tasks.workunit.client.1.smithi040.stdout:5/779: truncate d2/d7/d8/db/f96 3565649 0 2024-04-24T23:14:09.472 INFO:tasks.workunit.client.1.smithi040.stdout:9/874: dread d7/da/d17/d40/d55/d2f/df7/fe9 [0,4194304] 0 2024-04-24T23:14:09.472 INFO:tasks.workunit.client.1.smithi040.stdout:1/852: read da/d12/d18/d2a/f79 [2957046,116047] 0 2024-04-24T23:14:09.473 INFO:tasks.workunit.client.0.smithi031.stdout:2/966: mkdir d5/d8/d45/d5b/d49/dc4/d13b/d154 0 2024-04-24T23:14:09.475 INFO:tasks.workunit.client.1.smithi040.stdout:4/858: creat da/d78/dbb/f129 x:0 0 0 2024-04-24T23:14:09.478 INFO:tasks.workunit.client.1.smithi040.stdout:5/780: mkdir d2/d7/d8/d1f/d57/d27/d65/dfa 0 2024-04-24T23:14:09.479 INFO:tasks.workunit.client.1.smithi040.stdout:9/875: symlink d7/da/d17/d40/d55/d2f/df7/l12a 0 2024-04-24T23:14:09.481 INFO:tasks.workunit.client.1.smithi040.stdout:2/951: mkdir da/d81/da8/d143/d156 0 2024-04-24T23:14:09.481 INFO:tasks.workunit.client.1.smithi040.stdout:7/919: dwrite db/d2f/d37/fa5 [0,4194304] 0 2024-04-24T23:14:09.482 INFO:tasks.workunit.client.1.smithi040.stdout:4/859: truncate da/d3f/d6b/d2b/f6d 871489 0 2024-04-24T23:14:09.482 INFO:tasks.workunit.client.1.smithi040.stdout:7/920: fdatasync db/d12/d20/f113 0 2024-04-24T23:14:09.483 INFO:tasks.workunit.client.1.smithi040.stdout:2/952: chown da/d1c/d37/d39/d3f/db2/l12d 141 1 2024-04-24T23:14:09.484 INFO:tasks.workunit.client.1.smithi040.stdout:0/924: dread df/f63 [0,4194304] 0 2024-04-24T23:14:09.484 INFO:tasks.workunit.client.1.smithi040.stdout:4/860: read da/f59 [98016,8759] 0 2024-04-24T23:14:09.487 INFO:tasks.workunit.client.1.smithi040.stdout:9/876: mknod d7/d5e/de5/c12b 0 2024-04-24T23:14:09.495 INFO:tasks.workunit.client.1.smithi040.stdout:7/921: creat db/d12/d62/f147 x:0 0 0 2024-04-24T23:14:09.496 INFO:tasks.workunit.client.1.smithi040.stdout:2/953: fsync da/d81/de6/fd3 0 2024-04-24T23:14:09.496 INFO:tasks.workunit.client.1.smithi040.stdout:1/853: rename da/d12/d18/d2a/d4d/d107/cf6 to da/d12/d18/d2a/d41/d8c/c120 0 2024-04-24T23:14:09.496 INFO:tasks.workunit.client.0.smithi031.stdout:2/967: dwrite d5/f25 [0,4194304] 0 2024-04-24T23:14:09.498 INFO:tasks.workunit.client.1.smithi040.stdout:9/877: write d7/da/d17/f11d [200956,41067] 0 2024-04-24T23:14:09.500 INFO:tasks.workunit.client.1.smithi040.stdout:5/781: rmdir d2/d2f/d14/dbe/dec 0 2024-04-24T23:14:09.502 INFO:tasks.workunit.client.0.smithi031.stdout:2/968: creat d5/d8/d45/d5b/d49/d2f/d135/d68/f155 x:0 0 0 2024-04-24T23:14:09.502 INFO:tasks.workunit.client.1.smithi040.stdout:6/840: dwrite d7/de/d4e/f64 [0,4194304] 0 2024-04-24T23:14:09.507 INFO:tasks.workunit.client.0.smithi031.stdout:2/969: mknod d5/d8/d45/d5b/d49/d2f/d135/d29/c156 0 2024-04-24T23:14:09.514 INFO:tasks.workunit.client.1.smithi040.stdout:0/925: dwrite df/d12/d58/d5b/fee [0,4194304] 0 2024-04-24T23:14:09.514 INFO:tasks.workunit.client.0.smithi031.stdout:2/970: dwrite d5/d8/f10 [0,4194304] 0 2024-04-24T23:14:09.517 INFO:tasks.workunit.client.0.smithi031.stdout:2/971: write d5/d8/d45/d5b/d49/d2f/d135/d68/f13e [278850,80885] 0 2024-04-24T23:14:09.524 INFO:tasks.workunit.client.0.smithi031.stdout:2/972: truncate d5/d8/d45/d5b/d49/d2f/d135/d29/f12 728460 0 2024-04-24T23:14:09.530 INFO:tasks.workunit.client.0.smithi031.stdout:2/973: unlink d5/d8/d45/d5b/d49/d2f/d135/d68/cf9 0 2024-04-24T23:14:09.530 INFO:tasks.workunit.client.0.smithi031.stdout:2/974: truncate d5/d8/d45/d5b/d49/d2f/d135/d68/f13e 1183882 0 2024-04-24T23:14:09.531 INFO:tasks.workunit.client.1.smithi040.stdout:8/920: sync 2024-04-24T23:14:09.535 INFO:tasks.workunit.client.1.smithi040.stdout:2/954: mknod da/d81/da8/d143/c157 0 2024-04-24T23:14:09.536 INFO:tasks.workunit.client.1.smithi040.stdout:1/854: creat da/d12/d18/d2a/d32/f121 x:0 0 0 2024-04-24T23:14:09.541 INFO:tasks.workunit.client.1.smithi040.stdout:6/841: creat d7/d1a/d20/d51/d1e/d46/f112 x:0 0 0 2024-04-24T23:14:09.544 INFO:tasks.workunit.client.1.smithi040.stdout:9/878: dwrite d7/da/d17/d40/d55/d2f/d28/d67/faf [0,4194304] 0 2024-04-24T23:14:09.549 INFO:tasks.workunit.client.1.smithi040.stdout:8/921: dread d0/d52/f84 [0,4194304] 0 2024-04-24T23:14:09.549 INFO:tasks.workunit.client.1.smithi040.stdout:1/855: creat da/d12/d18/d2a/d32/f122 x:0 0 0 2024-04-24T23:14:09.551 INFO:tasks.workunit.client.1.smithi040.stdout:1/856: write da/d12/d18/d2a/d4d/d107/d5e/fa4 [4459241,60271] 0 2024-04-24T23:14:09.556 INFO:tasks.workunit.client.1.smithi040.stdout:9/879: readlink d7/da/l10a 0 2024-04-24T23:14:09.558 INFO:tasks.workunit.client.1.smithi040.stdout:8/922: mknod d0/d16/d1b/d58/c12e 0 2024-04-24T23:14:09.561 INFO:tasks.workunit.client.1.smithi040.stdout:1/857: mkdir da/d12/d14/d58/d123 0 2024-04-24T23:14:09.562 INFO:tasks.workunit.client.1.smithi040.stdout:9/880: symlink d7/da/d17/d40/d55/d2f/d1f/d4e/l12c 0 2024-04-24T23:14:09.563 INFO:tasks.workunit.client.1.smithi040.stdout:7/922: link db/d12/d20/d52/d4b/d127/c138 db/d12/d3f/d7a/c148 0 2024-04-24T23:14:09.565 INFO:tasks.workunit.client.1.smithi040.stdout:0/926: getdents df/d1e/da7 0 2024-04-24T23:14:09.568 INFO:tasks.workunit.client.1.smithi040.stdout:7/923: dread - db/d2f/da8/f107 zero size 2024-04-24T23:14:09.569 INFO:tasks.workunit.client.1.smithi040.stdout:9/881: unlink d7/da/d17/d40/d55/d2f/d1f/d31/f98 0 2024-04-24T23:14:09.575 INFO:tasks.workunit.client.1.smithi040.stdout:7/924: symlink db/d12/d20/d52/d4b/d127/d12b/l149 0 2024-04-24T23:14:09.576 INFO:tasks.workunit.client.1.smithi040.stdout:0/927: symlink df/d1e/d20/d21/l12c 0 2024-04-24T23:14:09.578 INFO:tasks.workunit.client.1.smithi040.stdout:1/858: rename da/d12/d18/d2a/d4d/d107/f114 to da/d12/d18/d2a/d41/f124 0 2024-04-24T23:14:09.579 INFO:tasks.workunit.client.1.smithi040.stdout:8/923: getdents d0/d8/d22/d81 0 2024-04-24T23:14:09.580 INFO:tasks.workunit.client.1.smithi040.stdout:1/859: chown da/d12/d18/f1d 30 1 2024-04-24T23:14:09.580 INFO:tasks.workunit.client.1.smithi040.stdout:1/860: chown da/d35/dab/f7c 0 1 2024-04-24T23:14:09.582 INFO:tasks.workunit.client.1.smithi040.stdout:0/928: rmdir df/d1e/d20/d21 39 2024-04-24T23:14:09.585 INFO:tasks.workunit.client.1.smithi040.stdout:0/929: rmdir df/d12/d58/dc3/dcd 39 2024-04-24T23:14:09.587 INFO:tasks.workunit.client.1.smithi040.stdout:3/904: sync 2024-04-24T23:14:09.588 INFO:tasks.workunit.client.1.smithi040.stdout:4/861: truncate da/d1f/f9c 1092743 0 2024-04-24T23:14:09.589 INFO:tasks.workunit.client.1.smithi040.stdout:1/861: dread da/ff [0,4194304] 0 2024-04-24T23:14:09.589 INFO:tasks.workunit.client.1.smithi040.stdout:0/930: symlink df/d12/d58/dc3/dde/l12d 0 2024-04-24T23:14:09.589 INFO:tasks.workunit.client.1.smithi040.stdout:2/955: rmdir da/d81 39 2024-04-24T23:14:09.590 INFO:tasks.workunit.client.1.smithi040.stdout:1/862: readlink da/d12/d18/d2a/d4d/dfd/l101 0 2024-04-24T23:14:09.591 INFO:tasks.workunit.client.1.smithi040.stdout:2/956: fdatasync da/d1c/d37/d8a/f104 0 2024-04-24T23:14:09.591 INFO:tasks.workunit.client.1.smithi040.stdout:4/862: rmdir da/d45/d48/d6e/de0 39 2024-04-24T23:14:09.592 INFO:tasks.workunit.client.1.smithi040.stdout:7/925: dread db/d12/d7d/fcd [0,4194304] 0 2024-04-24T23:14:09.592 INFO:tasks.workunit.client.1.smithi040.stdout:7/926: readlink db/lee 0 2024-04-24T23:14:09.592 INFO:tasks.workunit.client.1.smithi040.stdout:0/931: unlink df/l3d 0 2024-04-24T23:14:09.594 INFO:tasks.workunit.client.1.smithi040.stdout:3/905: creat d1/d2d/de/d30/d37/d107/f133 x:0 0 0 2024-04-24T23:14:09.597 INFO:tasks.workunit.client.1.smithi040.stdout:1/863: symlink da/d12/l125 0 2024-04-24T23:14:09.598 INFO:tasks.workunit.client.1.smithi040.stdout:7/927: dwrite db/d12/d20/d52/d6c/fae [0,4194304] 0 2024-04-24T23:14:09.600 INFO:tasks.workunit.client.1.smithi040.stdout:3/906: unlink d1/d2a/d6a/d94/l109 0 2024-04-24T23:14:09.602 INFO:tasks.workunit.client.1.smithi040.stdout:5/782: truncate d2/d2f/d14/f86 2754308 0 2024-04-24T23:14:09.607 INFO:tasks.workunit.client.1.smithi040.stdout:0/932: chown df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dcf/lc2 11323 1 2024-04-24T23:14:09.611 INFO:tasks.workunit.client.1.smithi040.stdout:2/957: creat da/d1c/d37/d39/d79/f158 x:0 0 0 2024-04-24T23:14:09.613 INFO:tasks.workunit.client.1.smithi040.stdout:3/907: creat d1/d9/d2e/dcf/d129/d12b/f134 x:0 0 0 2024-04-24T23:14:09.615 INFO:tasks.workunit.client.1.smithi040.stdout:3/908: dread - d1/d2a/d8d/d7c/f102 zero size 2024-04-24T23:14:09.615 INFO:tasks.workunit.client.1.smithi040.stdout:6/842: rename d7/d1a/d20/d51/d1e/d46/d85 to d7/d1a/d56/d113 0 2024-04-24T23:14:09.616 INFO:tasks.workunit.client.1.smithi040.stdout:5/783: unlink d2/d7/d8/d1f/d57/d27/d3d/dd6/le7 0 2024-04-24T23:14:09.617 INFO:tasks.workunit.client.1.smithi040.stdout:5/784: stat d2/d7/d8/f13 0 2024-04-24T23:14:09.618 INFO:tasks.workunit.client.1.smithi040.stdout:5/785: stat d2/d7/d8/db/caa 0 2024-04-24T23:14:09.622 INFO:tasks.workunit.client.1.smithi040.stdout:2/958: truncate da/d10/d7a/f99 2236786 0 2024-04-24T23:14:09.624 INFO:tasks.workunit.client.1.smithi040.stdout:9/882: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/d12d 0 2024-04-24T23:14:09.624 INFO:tasks.workunit.client.1.smithi040.stdout:2/959: write da/d10/d4d/dd7/f110 [263097,16634] 0 2024-04-24T23:14:09.626 INFO:tasks.workunit.client.1.smithi040.stdout:7/928: dread db/f31 [0,4194304] 0 2024-04-24T23:14:09.627 INFO:tasks.workunit.client.1.smithi040.stdout:8/924: write d0/d16/d1b/d58/d8f/fdf [837822,22545] 0 2024-04-24T23:14:09.629 INFO:tasks.workunit.client.1.smithi040.stdout:5/786: getdents d2/d7/d8/d6d/d89 0 2024-04-24T23:14:09.630 INFO:tasks.workunit.client.1.smithi040.stdout:7/929: write db/d12/d20/d52/d4b/d127/d12b/db9/dd4/d101/f115 [5385,121525] 0 2024-04-24T23:14:09.630 INFO:tasks.workunit.client.1.smithi040.stdout:0/933: dread df/d1e/d20/d21/d41/d8a/df3/df8/d79/fae [0,4194304] 0 2024-04-24T23:14:09.632 INFO:tasks.workunit.client.1.smithi040.stdout:5/787: dread d2/d2f/d14/f61 [0,4194304] 0 2024-04-24T23:14:09.634 INFO:tasks.workunit.client.1.smithi040.stdout:2/960: mknod da/d81/da8/c159 0 2024-04-24T23:14:09.635 INFO:tasks.workunit.client.1.smithi040.stdout:7/930: read db/d12/d20/d52/d44/fab [120662,114365] 0 2024-04-24T23:14:09.636 INFO:tasks.workunit.client.1.smithi040.stdout:9/883: creat d7/da/d17/d40/d55/d2f/d1f/d31/dff/d111/f12e x:0 0 0 2024-04-24T23:14:09.642 INFO:tasks.workunit.client.1.smithi040.stdout:9/884: dread - d7/ff5 zero size 2024-04-24T23:14:09.644 INFO:tasks.workunit.client.1.smithi040.stdout:3/909: write d1/d9/d39/fb0 [924476,65073] 0 2024-04-24T23:14:09.644 INFO:tasks.workunit.client.1.smithi040.stdout:1/864: rmdir da/d12/d18/d2a/d4d 39 2024-04-24T23:14:09.645 INFO:tasks.workunit.client.1.smithi040.stdout:2/961: symlink da/d10/d138/l15a 0 2024-04-24T23:14:09.646 INFO:tasks.workunit.client.1.smithi040.stdout:7/931: mkdir db/d12/d14a 0 2024-04-24T23:14:09.648 INFO:tasks.workunit.client.1.smithi040.stdout:9/885: mkdir d7/da/d17/d40/d55/d2f/df7/d12f 0 2024-04-24T23:14:09.650 INFO:tasks.workunit.client.1.smithi040.stdout:1/865: creat da/d12/d18/d2a/dd4/f126 x:0 0 0 2024-04-24T23:14:09.652 INFO:tasks.workunit.client.1.smithi040.stdout:0/934: dread df/de1/d6e/fba [8388608,4194304] 0 2024-04-24T23:14:09.652 INFO:tasks.workunit.client.1.smithi040.stdout:3/910: mknod d1/d9/d2e/dcf/d110/c135 0 2024-04-24T23:14:09.653 INFO:tasks.workunit.client.1.smithi040.stdout:2/962: symlink da/d1c/d37/d102/l15b 0 2024-04-24T23:14:09.659 INFO:tasks.workunit.client.1.smithi040.stdout:9/886: read d7/da/d17/d40/d55/d2f/d28/fe2 [3111,25247] 0 2024-04-24T23:14:09.659 INFO:tasks.workunit.client.1.smithi040.stdout:8/925: dwrite d0/d16/d1b/f95 [0,4194304] 0 2024-04-24T23:14:09.659 INFO:tasks.workunit.client.1.smithi040.stdout:9/887: readlink d7/da/d17/d40/d55/d2f/d1f/l7f 0 2024-04-24T23:14:09.661 INFO:tasks.workunit.client.1.smithi040.stdout:9/888: write d7/d5e/da1/dfa/f10c [430916,103312] 0 2024-04-24T23:14:09.663 INFO:tasks.workunit.client.1.smithi040.stdout:1/866: truncate da/d12/f1c 5471930 0 2024-04-24T23:14:09.665 INFO:tasks.workunit.client.1.smithi040.stdout:2/963: symlink da/d10/d148/l15c 0 2024-04-24T23:14:09.668 INFO:tasks.workunit.client.1.smithi040.stdout:7/932: write db/d12/d62/d8a/db3/fed [2043590,111408] 0 2024-04-24T23:14:09.670 INFO:tasks.workunit.client.1.smithi040.stdout:7/933: stat db/d12/d20/d52/d6c/d8f/db8/lc7 0 2024-04-24T23:14:09.672 INFO:tasks.workunit.client.1.smithi040.stdout:0/935: creat df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/f12e x:0 0 0 2024-04-24T23:14:09.673 INFO:tasks.workunit.client.1.smithi040.stdout:2/964: dwrite da/d1c/d37/d39/d3f/d52/f87 [0,4194304] 0 2024-04-24T23:14:09.676 INFO:tasks.workunit.client.1.smithi040.stdout:7/934: mkdir db/d12/d3f/d14b 0 2024-04-24T23:14:09.676 INFO:tasks.workunit.client.1.smithi040.stdout:8/926: write d0/d16/d1b/d47/fdc [937366,130021] 0 2024-04-24T23:14:09.678 INFO:tasks.workunit.client.1.smithi040.stdout:2/965: mknod da/d29/d55/da7/c15d 0 2024-04-24T23:14:09.681 INFO:tasks.workunit.client.1.smithi040.stdout:1/867: rename da/d35/dab to da/d12/d18/d2a/d41/d127 0 2024-04-24T23:14:09.684 INFO:tasks.workunit.client.1.smithi040.stdout:0/936: dwrite df/d12/d1a/d71/f36 [0,4194304] 0 2024-04-24T23:14:09.684 INFO:tasks.workunit.client.1.smithi040.stdout:8/927: symlink d0/d7e/dc1/d9e/d90/d91/l12f 0 2024-04-24T23:14:09.685 INFO:tasks.workunit.client.1.smithi040.stdout:1/868: dread - da/d12/d18/d2a/d32/f121 zero size 2024-04-24T23:14:09.687 INFO:tasks.workunit.client.1.smithi040.stdout:9/889: dwrite d7/da/d17/d40/d55/d2f/d1f/d31/f8d [0,4194304] 0 2024-04-24T23:14:09.689 INFO:tasks.workunit.client.1.smithi040.stdout:8/928: write d0/d8/fc3 [2367717,96151] 0 2024-04-24T23:14:09.691 INFO:tasks.workunit.client.1.smithi040.stdout:0/937: mkdir df/d14/d11c/d12f 0 2024-04-24T23:14:09.692 INFO:tasks.workunit.client.1.smithi040.stdout:7/935: write db/f2e [2117423,84276] 0 2024-04-24T23:14:09.694 INFO:tasks.workunit.client.1.smithi040.stdout:1/869: mkdir da/d12/d18/d2a/d41/d4f/de9/d128 0 2024-04-24T23:14:09.694 INFO:tasks.workunit.client.1.smithi040.stdout:9/890: read d7/f59 [3835642,124485] 0 2024-04-24T23:14:09.694 INFO:tasks.workunit.client.1.smithi040.stdout:8/929: truncate d0/d16/d1b/d58/fa5 796958 0 2024-04-24T23:14:09.697 INFO:tasks.workunit.client.1.smithi040.stdout:8/930: write d0/d7e/dc1/d9e/f109 [365041,92461] 0 2024-04-24T23:14:09.700 INFO:tasks.workunit.client.1.smithi040.stdout:7/936: unlink db/d12/d20/d52/d4b/d127/d12b/db9/dd4/d101/f119 0 2024-04-24T23:14:09.701 INFO:tasks.workunit.client.1.smithi040.stdout:1/870: symlink da/d12/d18/d2a/d41/d8c/l129 0 2024-04-24T23:14:09.703 INFO:tasks.workunit.client.1.smithi040.stdout:9/891: dread d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/def/f10e [0,4194304] 0 2024-04-24T23:14:09.703 INFO:tasks.workunit.client.0.smithi031.stdout:3/957: sync 2024-04-24T23:14:09.706 INFO:tasks.workunit.client.1.smithi040.stdout:7/937: write db/d12/d62/de7/f110 [729832,96969] 0 2024-04-24T23:14:09.706 INFO:tasks.workunit.client.0.smithi031.stdout:3/958: rmdir d6/df/d16/d53 39 2024-04-24T23:14:09.707 INFO:tasks.workunit.client.1.smithi040.stdout:2/966: dwrite da/d1c/d37/d39/d79/d13d/d153/d155/fea [0,4194304] 0 2024-04-24T23:14:09.708 INFO:tasks.workunit.client.1.smithi040.stdout:9/892: dread - d7/da/d17/d40/d55/d2f/d28/d67/f112 zero size 2024-04-24T23:14:09.708 INFO:tasks.workunit.client.1.smithi040.stdout:6/843: sync 2024-04-24T23:14:09.708 INFO:tasks.workunit.client.1.smithi040.stdout:4/863: sync 2024-04-24T23:14:09.709 INFO:tasks.workunit.client.0.smithi031.stdout:3/959: fdatasync d6/d14/d1d/fc2 0 2024-04-24T23:14:09.713 INFO:tasks.workunit.client.1.smithi040.stdout:2/967: symlink da/d10/d7a/l15e 0 2024-04-24T23:14:09.714 INFO:tasks.workunit.client.0.smithi031.stdout:3/960: creat d6/df/d16/d2a/f12c x:0 0 0 2024-04-24T23:14:09.717 INFO:tasks.workunit.client.1.smithi040.stdout:6/844: dwrite d7/d1a/d20/d51/d1e/f49 [0,4194304] 0 2024-04-24T23:14:09.717 INFO:tasks.workunit.client.1.smithi040.stdout:1/871: mknod da/d35/dec/c12a 0 2024-04-24T23:14:09.721 INFO:tasks.workunit.client.1.smithi040.stdout:4/864: mknod da/d45/d48/d6e/d70/c12a 0 2024-04-24T23:14:09.726 INFO:tasks.workunit.client.1.smithi040.stdout:0/938: dread df/d12/d58/f7d [0,4194304] 0 2024-04-24T23:14:09.727 INFO:tasks.workunit.client.1.smithi040.stdout:9/893: creat d7/da/d17/d120/f130 x:0 0 0 2024-04-24T23:14:09.729 INFO:tasks.workunit.client.1.smithi040.stdout:0/939: chown df/d1e/d20/d21/c42 4813 1 2024-04-24T23:14:09.731 INFO:tasks.workunit.client.0.smithi031.stdout:3/961: write d6/d14/d1d/d69/fd5 [540722,493] 0 2024-04-24T23:14:09.731 INFO:tasks.workunit.client.1.smithi040.stdout:7/938: link db/c25 db/d12/d20/d52/d4b/d127/d12b/db9/c14c 0 2024-04-24T23:14:09.733 INFO:tasks.workunit.client.1.smithi040.stdout:4/865: dwrite da/d78/dbb/d127/ff1 [0,4194304] 0 2024-04-24T23:14:09.733 INFO:tasks.workunit.client.1.smithi040.stdout:8/931: dread d0/d7e/dc1/d9e/d90/d91/fd3 [0,4194304] 0 2024-04-24T23:14:09.735 INFO:tasks.workunit.client.1.smithi040.stdout:0/940: chown df/d1e/d20/d21/d41/d8a/df3/df8/da3/d8e/f99 25823 1 2024-04-24T23:14:09.736 INFO:tasks.workunit.client.1.smithi040.stdout:6/845: fdatasync d7/d1a/d20/d51/d1e/d46/f7d 0 2024-04-24T23:14:09.736 INFO:tasks.workunit.client.1.smithi040.stdout:2/968: getdents da/d1c/d37/d102 0 2024-04-24T23:14:09.736 INFO:tasks.workunit.client.0.smithi031.stdout:3/962: fsync d6/d14/d91/f99 0 2024-04-24T23:14:09.737 INFO:tasks.workunit.client.1.smithi040.stdout:2/969: stat da/d81/da8/d143/c157 0 2024-04-24T23:14:09.738 INFO:tasks.workunit.client.0.smithi031.stdout:3/963: truncate d6/df/d16/d57/f100 87967 0 2024-04-24T23:14:09.744 INFO:tasks.workunit.client.0.smithi031.stdout:3/964: creat d6/df/d16/d57/f12d x:0 0 0 2024-04-24T23:14:09.749 INFO:tasks.workunit.client.0.smithi031.stdout:3/965: dwrite d6/df/d16/d2a/f2e [0,4194304] 0 2024-04-24T23:14:09.751 INFO:tasks.workunit.client.0.smithi031.stdout:3/966: mknod d6/df/d16/d57/df8/c12e 0 2024-04-24T23:14:09.754 INFO:tasks.workunit.client.0.smithi031.stdout:3/967: mkdir d6/df/d16/d2a/d123/d12f 0 2024-04-24T23:14:09.757 INFO:tasks.workunit.client.0.smithi031.stdout:3/968: chown d6/df/d16/d53/fa8 6642062 1 2024-04-24T23:14:09.774 INFO:tasks.workunit.client.0.smithi031.stdout:3/969: dread d6/d14/d1d/fcd [0,4194304] 0 2024-04-24T23:14:09.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/975: sync 2024-04-24T23:14:09.778 INFO:tasks.workunit.client.0.smithi031.stdout:3/970: truncate d6/d14/d1d/fb6 3061106 0 2024-04-24T23:14:09.779 INFO:tasks.workunit.client.0.smithi031.stdout:2/976: mknod d5/d3d/c157 0 2024-04-24T23:14:09.780 INFO:tasks.workunit.client.0.smithi031.stdout:2/977: stat d5/d8/d45/d5b/d49/d6f/ce6 0 2024-04-24T23:14:09.781 INFO:tasks.workunit.client.0.smithi031.stdout:2/978: chown d5/d8/d15/f93 5628364 1 2024-04-24T23:14:09.782 INFO:tasks.workunit.client.0.smithi031.stdout:3/971: creat d6/df/d16/d2a/d95/d9b/d1a/d25/f130 x:0 0 0 2024-04-24T23:14:09.784 INFO:tasks.workunit.client.0.smithi031.stdout:2/979: creat d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/dad/f158 x:0 0 0 2024-04-24T23:14:09.785 INFO:tasks.workunit.client.0.smithi031.stdout:2/980: chown d5/d8/d15 0 1 2024-04-24T23:14:09.785 INFO:tasks.workunit.client.0.smithi031.stdout:3/972: truncate d6/d14/d1d/fba 462044 0 2024-04-24T23:14:09.788 INFO:tasks.workunit.client.0.smithi031.stdout:3/973: dread d6/d14/d1d/f8b [0,4194304] 0 2024-04-24T23:14:09.789 INFO:tasks.workunit.client.0.smithi031.stdout:2/981: rename d5/d8/d15/l7d to d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/d117/l159 0 2024-04-24T23:14:09.791 INFO:tasks.workunit.client.0.smithi031.stdout:2/982: rmdir d5/d28/d4e/d5e 39 2024-04-24T23:14:09.792 INFO:tasks.workunit.client.0.smithi031.stdout:3/974: mknod d6/df/d16/d2a/d95/d9b/df1/c131 0 2024-04-24T23:14:09.795 INFO:tasks.workunit.client.0.smithi031.stdout:2/983: creat d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/da7/f15a x:0 0 0 2024-04-24T23:14:09.798 INFO:tasks.workunit.client.0.smithi031.stdout:2/984: mkdir d5/d8/d15/d15b 0 2024-04-24T23:14:09.798 INFO:tasks.workunit.client.1.smithi040.stdout:0/941: fdatasync df/d1e/da7/f69 0 2024-04-24T23:14:09.799 INFO:tasks.workunit.client.1.smithi040.stdout:8/932: rename d0/d16/d19/d102/d11f to d0/d7e/dc1/d9e/d90/d130 0 2024-04-24T23:14:09.802 INFO:tasks.workunit.client.1.smithi040.stdout:8/933: write d0/d8/d22/d81/f74 [5711692,56309] 0 2024-04-24T23:14:09.802 INFO:tasks.workunit.client.0.smithi031.stdout:2/985: mknod d5/d8/d45/d5b/d49/dc4/d104/c15c 0 2024-04-24T23:14:09.804 INFO:tasks.workunit.client.0.smithi031.stdout:2/986: write d5/d8/d45/d127/f145 [623331,30837] 0 2024-04-24T23:14:09.805 INFO:tasks.workunit.client.1.smithi040.stdout:4/866: dwrite da/d45/d48/d6e/d70/fcb [0,4194304] 0 2024-04-24T23:14:09.805 INFO:tasks.workunit.client.0.smithi031.stdout:2/987: chown d5/d8/d45/d5b/d49/d2f/d135/d29/d26/f63 10116 1 2024-04-24T23:14:09.808 INFO:tasks.workunit.client.1.smithi040.stdout:1/872: dread da/d12/d18/d2a/d4d/dfd/f11d [0,4194304] 0 2024-04-24T23:14:09.811 INFO:tasks.workunit.client.1.smithi040.stdout:6/846: unlink d7/d1a/d20/d51/d2f/dba/dde/led 0 2024-04-24T23:14:09.815 INFO:tasks.workunit.client.0.smithi031.stdout:2/988: unlink d5/d8/c66 0 2024-04-24T23:14:09.817 INFO:tasks.workunit.client.0.smithi031.stdout:3/975: dwrite d6/df/d16/d2a/d95/d9b/d29/fa3 [0,4194304] 0 2024-04-24T23:14:09.818 INFO:tasks.workunit.client.1.smithi040.stdout:8/934: readlink d0/d52/l7f 0 2024-04-24T23:14:09.823 INFO:tasks.workunit.client.1.smithi040.stdout:0/942: dwrite df/d12/d58/d5b/fa0 [0,4194304] 0 2024-04-24T23:14:09.827 INFO:tasks.workunit.client.0.smithi031.stdout:3/976: rename d6/df/d16/d2a/f31 to d6/d14/d71/dad/f132 0 2024-04-24T23:14:09.828 INFO:tasks.workunit.client.1.smithi040.stdout:1/873: creat da/d12/d18/d2a/d8b/dd3/f12b x:0 0 0 2024-04-24T23:14:09.829 INFO:tasks.workunit.client.1.smithi040.stdout:1/874: stat da/d12/d18/d2a/d32/f121 0 2024-04-24T23:14:09.829 INFO:tasks.workunit.client.1.smithi040.stdout:1/875: fdatasync da/d12/d18/d2a/dd4/f126 0 2024-04-24T23:14:09.831 INFO:tasks.workunit.client.1.smithi040.stdout:6/847: unlink d7/f71 0 2024-04-24T23:14:09.832 INFO:tasks.workunit.client.1.smithi040.stdout:2/970: creat da/d10/f15f x:0 0 0 2024-04-24T23:14:09.833 INFO:tasks.workunit.client.1.smithi040.stdout:8/935: mkdir d0/d52/d9b/d101/d131 0 2024-04-24T23:14:09.836 INFO:tasks.workunit.client.0.smithi031.stdout:3/977: truncate d6/df/d16/d2a/d95/d9b/d1a/d25/faf 4916792 0 2024-04-24T23:14:09.838 INFO:tasks.workunit.client.1.smithi040.stdout:2/971: dwrite da/d81/de6/ff6 [0,4194304] 0 2024-04-24T23:14:09.842 INFO:tasks.workunit.client.1.smithi040.stdout:5/788: sync 2024-04-24T23:14:09.842 INFO:tasks.workunit.client.1.smithi040.stdout:3/911: sync 2024-04-24T23:14:09.843 INFO:tasks.workunit.client.1.smithi040.stdout:9/894: getdents d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8 0 2024-04-24T23:14:09.845 INFO:tasks.workunit.client.1.smithi040.stdout:8/936: mknod d0/d28/c132 0 2024-04-24T23:14:09.845 INFO:tasks.workunit.client.0.smithi031.stdout:3/978: truncate d6/d14/d1d/d59/fc0 523946 0 2024-04-24T23:14:09.847 INFO:tasks.workunit.client.1.smithi040.stdout:1/876: dwrite da/d12/d14/f24 [4194304,4194304] 0 2024-04-24T23:14:09.847 INFO:tasks.workunit.client.1.smithi040.stdout:9/895: write d7/da/d17/d40/d55/f115 [1016952,6223] 0 2024-04-24T23:14:09.848 INFO:tasks.workunit.client.1.smithi040.stdout:0/943: rename df/d12/f22 to df/d14/d11c/f130 0 2024-04-24T23:14:09.849 INFO:tasks.workunit.client.1.smithi040.stdout:1/877: readlink da/d35/d38/d47/dbc/de5/lfa 0 2024-04-24T23:14:09.853 INFO:tasks.workunit.client.0.smithi031.stdout:2/989: dwrite d5/d28/d43/f57 [0,4194304] 0 2024-04-24T23:14:09.853 INFO:tasks.workunit.client.1.smithi040.stdout:2/972: mkdir da/d1c/d37/d39/d77/d108/d10c/d160 0 2024-04-24T23:14:09.855 INFO:tasks.workunit.client.1.smithi040.stdout:0/944: dread df/d12/d58/d5b/fee [0,4194304] 0 2024-04-24T23:14:09.856 INFO:tasks.workunit.client.1.smithi040.stdout:0/945: dread - df/dd4/f118 zero size 2024-04-24T23:14:09.857 INFO:tasks.workunit.client.1.smithi040.stdout:2/973: truncate da/d1c/d37/d39/d79/f141 179497 0 2024-04-24T23:14:09.858 INFO:tasks.workunit.client.1.smithi040.stdout:6/848: mkdir d7/d1a/d114 0 2024-04-24T23:14:09.859 INFO:tasks.workunit.client.1.smithi040.stdout:3/912: mkdir d1/d2d/d72/d106/d136 0 2024-04-24T23:14:09.863 INFO:tasks.workunit.client.0.smithi031.stdout:3/979: dwrite d6/df/d16/f35 [8388608,4194304] 0 2024-04-24T23:14:09.864 INFO:tasks.workunit.client.1.smithi040.stdout:0/946: dwrite df/de1/db6/fbd [0,4194304] 0 2024-04-24T23:14:09.864 INFO:tasks.workunit.client.1.smithi040.stdout:8/937: creat d0/d8/d22/db8/dc2/f133 x:0 0 0 2024-04-24T23:14:09.866 INFO:tasks.workunit.client.0.smithi031.stdout:3/980: chown d6/fbe 25681 1 2024-04-24T23:14:09.872 INFO:tasks.workunit.client.0.smithi031.stdout:2/990: dwrite d5/d8/d45/d5b/d49/d6f/d119/d11d/f14e [0,4194304] 0 2024-04-24T23:14:09.876 INFO:tasks.workunit.client.1.smithi040.stdout:1/878: creat da/d35/d38/d47/dbc/d109/f12c x:0 0 0 2024-04-24T23:14:09.883 INFO:tasks.workunit.client.0.smithi031.stdout:2/991: rename d5/d8/d45/le8 to d5/d3d/l15d 0 2024-04-24T23:14:09.885 INFO:tasks.workunit.client.1.smithi040.stdout:2/974: rmdir da/d1c/d37/d39/d79 39 2024-04-24T23:14:09.887 INFO:tasks.workunit.client.1.smithi040.stdout:6/849: symlink d7/d1a/d20/d51/d2f/df2/l115 0 2024-04-24T23:14:09.888 INFO:tasks.workunit.client.0.smithi031.stdout:3/981: symlink d6/df/d16/d2a/d123/d12f/l133 0 2024-04-24T23:14:09.890 INFO:tasks.workunit.client.0.smithi031.stdout:2/992: truncate d5/d8/d45/d5b/d49/d2f/d135/d29/d4f/fc5 1147294 0 2024-04-24T23:14:09.891 INFO:tasks.workunit.client.1.smithi040.stdout:7/939: sync 2024-04-24T23:14:09.891 INFO:tasks.workunit.client.1.smithi040.stdout:7/940: fdatasync db/d12/d62/d8a/f102 0 2024-04-24T23:14:09.891 INFO:tasks.workunit.client.1.smithi040.stdout:0/947: creat df/d1e/d20/d21/d41/f131 x:0 0 0 2024-04-24T23:14:09.892 INFO:tasks.workunit.client.0.smithi031.stdout:3/982: creat d6/d14/f134 x:0 0 0 2024-04-24T23:14:09.896 INFO:tasks.workunit.client.0.smithi031.stdout:2/993: creat d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/d117/d123/d114/f15e x:0 0 0 2024-04-24T23:14:09.896 INFO:tasks.workunit.client.1.smithi040.stdout:4/867: getdents da/d45/d48/d6e/de0 0 2024-04-24T23:14:09.900 INFO:tasks.workunit.client.1.smithi040.stdout:5/789: write d2/d7/d8/d1f/d57/d27/d3c/d43/f6a [694830,130318] 0 2024-04-24T23:14:09.908 INFO:tasks.workunit.client.0.smithi031.stdout:2/994: dwrite d5/d8/d15/d56/d84/d75/f9d [4194304,4194304] 0 2024-04-24T23:14:09.910 INFO:tasks.workunit.client.1.smithi040.stdout:0/948: symlink df/d1e/d89/ddf/l132 0 2024-04-24T23:14:09.914 INFO:tasks.workunit.client.0.smithi031.stdout:3/983: dread d6/d14/f43 [0,4194304] 0 2024-04-24T23:14:09.914 INFO:tasks.workunit.client.0.smithi031.stdout:2/995: dread d5/f25 [0,4194304] 0 2024-04-24T23:14:09.918 INFO:tasks.workunit.client.1.smithi040.stdout:9/896: rename d7/da/d17/d40/d55/d2f/df7/c72 to d7/da/d17/d40/d55/c131 0 2024-04-24T23:14:09.920 INFO:tasks.workunit.client.1.smithi040.stdout:8/938: write d0/d28/db7/fbf [249833,18636] 0 2024-04-24T23:14:09.924 INFO:tasks.workunit.client.1.smithi040.stdout:2/975: truncate da/d1c/d37/f126 1431964 0 2024-04-24T23:14:09.924 INFO:tasks.workunit.client.1.smithi040.stdout:2/976: chown da/d10/d4d/d6c 1866 1 2024-04-24T23:14:09.930 INFO:tasks.workunit.client.1.smithi040.stdout:4/868: link da/d45/d48/d6e/ddd/f10d da/d3f/d40/d47/d71/f12b 0 2024-04-24T23:14:09.931 INFO:tasks.workunit.client.1.smithi040.stdout:1/879: creat da/d12/d18/d2a/d41/f12d x:0 0 0 2024-04-24T23:14:09.933 INFO:tasks.workunit.client.1.smithi040.stdout:6/850: write d7/d1a/d20/d51/d1e/d46/d69/d87/fcc [951687,44836] 0 2024-04-24T23:14:09.936 INFO:tasks.workunit.client.0.smithi031.stdout:2/996: truncate d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc2/dad/dbe/f116 775523 0 2024-04-24T23:14:09.938 INFO:tasks.workunit.client.1.smithi040.stdout:5/790: unlink d2/d7/d8/d1f/f51 0 2024-04-24T23:14:09.941 INFO:tasks.workunit.client.1.smithi040.stdout:3/913: getdents d1/d2a/d6a/d94 0 2024-04-24T23:14:09.942 INFO:tasks.workunit.client.0.smithi031.stdout:3/984: dread d6/df/d16/d2a/f5a [0,4194304] 0 2024-04-24T23:14:09.944 INFO:tasks.workunit.client.1.smithi040.stdout:0/949: mkdir df/d12/d58/dc3/dcd/d133 0 2024-04-24T23:14:09.946 INFO:tasks.workunit.client.1.smithi040.stdout:4/869: unlink da/d3f/d8d/fb9 0 2024-04-24T23:14:09.948 INFO:tasks.workunit.client.0.smithi031.stdout:3/985: creat d6/df/d16/d2a/f135 x:0 0 0 2024-04-24T23:14:09.949 INFO:tasks.workunit.client.1.smithi040.stdout:6/851: dread d7/d1a/d20/d51/d1e/d46/d69/f8d [0,4194304] 0 2024-04-24T23:14:09.949 INFO:tasks.workunit.client.0.smithi031.stdout:2/997: getdents d5/d8/d45/d5b/d49/dc4/d13b 0 2024-04-24T23:14:09.951 INFO:tasks.workunit.client.1.smithi040.stdout:8/939: truncate d0/d16/d1b/d58/fa5 664608 0 2024-04-24T23:14:09.959 INFO:tasks.workunit.client.1.smithi040.stdout:5/791: mkdir d2/d7/d8/d1f/d57/dca/dfb 0 2024-04-24T23:14:09.962 INFO:tasks.workunit.client.0.smithi031.stdout:2/998: rename d5/d8/d45/d5b/d49/d2f/d135/f35 to d5/d8/d45/d149/f15f 0 2024-04-24T23:14:09.963 INFO:tasks.workunit.client.1.smithi040.stdout:3/914: mknod d1/d9/d39/d112/c137 0 2024-04-24T23:14:09.963 INFO:tasks.workunit.client.0.smithi031.stdout:2/999: dread - d5/d8/d45/d5b/d49/d2f/d135/d29/d26/db0/dc7/da7/f15a zero size 2024-04-24T23:14:09.970 INFO:tasks.workunit.client.1.smithi040.stdout:4/870: mkdir da/d78/dbb/d12c 0 2024-04-24T23:14:09.972 INFO:tasks.workunit.client.1.smithi040.stdout:1/880: mkdir da/d35/d38/d47/dbc/de5/df5/d12e 0 2024-04-24T23:14:09.972 INFO:tasks.workunit.client.1.smithi040.stdout:4/871: read da/d3f/d6b/d2b/f52 [5431088,107660] 0 2024-04-24T23:14:09.973 INFO:tasks.workunit.client.1.smithi040.stdout:0/950: dread df/d1e/d20/d21/d41/d8a/df3/df8/f8f [0,4194304] 0 2024-04-24T23:14:09.975 INFO:tasks.workunit.client.1.smithi040.stdout:2/977: write da/d10/d4d/d6c/f9c [661814,80471] 0 2024-04-24T23:14:09.976 INFO:tasks.workunit.client.1.smithi040.stdout:8/940: creat d0/d8/d22/db8/dc2/f134 x:0 0 0 2024-04-24T23:14:09.977 INFO:tasks.workunit.client.0.smithi031.stdout:3/986: write d6/d14/d1d/fb6 [1198351,118108] 0 2024-04-24T23:14:09.977 INFO:tasks.workunit.client.1.smithi040.stdout:8/941: chown d0/d16/d1b/d58/lcf 22103 1 2024-04-24T23:14:09.981 INFO:tasks.workunit.client.1.smithi040.stdout:3/915: dread - d1/d2d/d72/f90 zero size 2024-04-24T23:14:09.982 INFO:tasks.workunit.client.0.smithi031.stdout:3/987: dread d6/df/d16/d2a/d95/d9b/d1a/f62 [0,4194304] 0 2024-04-24T23:14:09.983 INFO:tasks.workunit.client.1.smithi040.stdout:7/941: link db/d12/d3f/d7a/cc9 db/d12/d20/c14d 0 2024-04-24T23:14:09.983 INFO:tasks.workunit.client.1.smithi040.stdout:7/942: fsync db/f111 0 2024-04-24T23:14:09.984 INFO:tasks.workunit.client.1.smithi040.stdout:7/943: write f2 [3853713,45106] 0 2024-04-24T23:14:09.988 INFO:tasks.workunit.client.0.smithi031.stdout:3/988: creat d6/df/d16/d2a/d95/d9b/d1a/d25/dd1/f136 x:0 0 0 2024-04-24T23:14:09.991 INFO:tasks.workunit.client.1.smithi040.stdout:1/881: truncate da/d12/d18/d2a/d41/d127/f52 69999 0 2024-04-24T23:14:09.992 INFO:tasks.workunit.client.1.smithi040.stdout:1/882: write da/d12/d14/f96 [859001,118897] 0 2024-04-24T23:14:09.992 INFO:tasks.workunit.client.0.smithi031.stdout:3/989: creat d6/df/d16/d2a/d95/d9b/d1a/f137 x:0 0 0 2024-04-24T23:14:09.993 INFO:tasks.workunit.client.1.smithi040.stdout:1/883: chown da/d12/d18/d2a/d41 14 1 2024-04-24T23:14:09.994 INFO:tasks.workunit.client.1.smithi040.stdout:6/852: link d7/d1a/d20/d51/d2f/f6f d7/d1a/d9f/df6/d110/f116 0 2024-04-24T23:14:09.998 INFO:tasks.workunit.client.1.smithi040.stdout:5/792: dwrite d2/d7/d8/d1f/d90/fd3 [0,4194304] 0 2024-04-24T23:14:10.000 INFO:tasks.workunit.client.1.smithi040.stdout:6/853: dread d7/d1a/d20/d51/d2f/f6f [0,4194304] 0 2024-04-24T23:14:10.000 INFO:tasks.workunit.client.1.smithi040.stdout:5/793: fsync d2/d7/d8/db/d1e/f46 0 2024-04-24T23:14:10.000 INFO:tasks.workunit.client.1.smithi040.stdout:9/897: getdents d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/d128 0 2024-04-24T23:14:10.002 INFO:tasks.workunit.client.0.smithi031.stdout:3/990: rename d6/df/d16/d2a/d95/d9b/d29/db7/d106 to d6/df/d16/d53/d138 0 2024-04-24T23:14:10.007 INFO:tasks.workunit.client.1.smithi040.stdout:9/898: dread d7/da/d17/d40/d55/fc6 [0,4194304] 0 2024-04-24T23:14:10.007 INFO:tasks.workunit.client.1.smithi040.stdout:5/794: dwrite d2/d7/fe [0,4194304] 0 2024-04-24T23:14:10.008 INFO:tasks.workunit.client.1.smithi040.stdout:9/899: readlink d7/da/l21 0 2024-04-24T23:14:10.008 INFO:tasks.workunit.client.0.smithi031.stdout:3/991: mkdir d6/df/d16/d57/d139 0 2024-04-24T23:14:10.008 INFO:tasks.workunit.client.1.smithi040.stdout:2/978: write da/d10/f84 [2985166,118028] 0 2024-04-24T23:14:10.013 INFO:tasks.workunit.client.0.smithi031.stdout:3/992: readlink d6/df/d16/l3a 0 2024-04-24T23:14:10.032 INFO:tasks.workunit.client.0.smithi031.stdout:3/993: dwrite d6/d14/d71/dad/f132 [0,4194304] 0 2024-04-24T23:14:10.034 INFO:tasks.workunit.client.0.smithi031.stdout:3/994: mknod d6/d14/db2/c13a 0 2024-04-24T23:14:10.034 INFO:tasks.workunit.client.1.smithi040.stdout:2/979: fsync da/d1c/d37/d39/f4e 0 2024-04-24T23:14:10.035 INFO:tasks.workunit.client.1.smithi040.stdout:4/872: mknod da/d3f/c12d 0 2024-04-24T23:14:10.036 INFO:tasks.workunit.client.1.smithi040.stdout:8/942: mkdir d0/d8/d135 0 2024-04-24T23:14:10.036 INFO:tasks.workunit.client.1.smithi040.stdout:8/943: readlink d0/lb 0 2024-04-24T23:14:10.037 INFO:tasks.workunit.client.1.smithi040.stdout:2/980: dread - da/d1c/d37/d39/d3f/d52/f120 zero size 2024-04-24T23:14:10.038 INFO:tasks.workunit.client.1.smithi040.stdout:9/900: creat d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/f132 x:0 0 0 2024-04-24T23:14:10.039 INFO:tasks.workunit.client.0.smithi031.stdout:3/995: rename d6/la to d6/d55/d113/l13b 0 2024-04-24T23:14:10.041 INFO:tasks.workunit.client.1.smithi040.stdout:0/951: rename df/lf7 to df/d12/l134 0 2024-04-24T23:14:10.043 INFO:tasks.workunit.client.1.smithi040.stdout:0/952: chown df/d1e/d89/ddf/f113 335266151 1 2024-04-24T23:14:10.044 INFO:tasks.workunit.client.1.smithi040.stdout:4/873: readlink da/l16 0 2024-04-24T23:14:10.045 INFO:tasks.workunit.client.0.smithi031.stdout:3/996: rename d6/df/d16/d2a/d95/cab to d6/d14/d1d/d59/dec/c13c 0 2024-04-24T23:14:10.046 INFO:tasks.workunit.client.1.smithi040.stdout:5/795: symlink d2/d7/d8/lfc 0 2024-04-24T23:14:10.047 INFO:tasks.workunit.client.1.smithi040.stdout:7/944: dwrite db/d12/f8c [0,4194304] 0 2024-04-24T23:14:10.048 INFO:tasks.workunit.client.0.smithi031.stdout:3/997: mknod d6/d14/d1d/d59/c13d 0 2024-04-24T23:14:10.049 INFO:tasks.workunit.client.1.smithi040.stdout:5/796: write d2/d7/d8/d1f/d57/d27/d65/f5a [3994880,75410] 0 2024-04-24T23:14:10.052 INFO:tasks.workunit.client.1.smithi040.stdout:3/916: rename d1/d9/d39/fb0 to d1/d9/d2e/dcf/d110/f138 0 2024-04-24T23:14:10.052 INFO:tasks.workunit.client.1.smithi040.stdout:5/797: read d2/d7/d8/d1f/d57/d27/d3c/f95 [3892431,116789] 0 2024-04-24T23:14:10.052 INFO:tasks.workunit.client.0.smithi031.stdout:3/998: truncate d6/df/d16/d2a/d95/d9b/d1a/d25/d83/f10f 153973 0 2024-04-24T23:14:10.053 INFO:tasks.workunit.client.1.smithi040.stdout:5/798: write d2/d7/d8/d64/f68 [2953932,116539] 0 2024-04-24T23:14:10.056 INFO:tasks.workunit.client.1.smithi040.stdout:1/884: creat da/d35/d38/d47/dbc/f12f x:0 0 0 2024-04-24T23:14:10.061 INFO:tasks.workunit.client.1.smithi040.stdout:2/981: symlink da/d1c/d37/d39/d3f/d52/d78/l161 0 2024-04-24T23:14:10.062 INFO:tasks.workunit.client.0.smithi031.stdout:3/999: dread d6/df/d16/d2a/d95/d9b/d1a/d25/f87 [0,4194304] 0 2024-04-24T23:14:10.063 INFO:tasks.workunit.client.1.smithi040.stdout:8/944: write d0/d52/d9b/d101/d113/fcd [44529,113207] 0 2024-04-24T23:14:10.064 INFO:tasks.workunit.client.1.smithi040.stdout:2/982: read da/d1c/d37/d39/d3f/f67 [90725,63259] 0 2024-04-24T23:14:10.064 INFO:tasks.workunit.client.1.smithi040.stdout:9/901: truncate d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d 2938467 0 2024-04-24T23:14:10.066 INFO:tasks.workunit.client.1.smithi040.stdout:7/945: creat db/d2f/d37/d7e/d118/d145/f14e x:0 0 0 2024-04-24T23:14:10.067 INFO:tasks.workunit.client.0.smithi031.stderr:+ rm -rf -- ./tmp.77cF6t9Mox 2024-04-24T23:14:10.071 INFO:tasks.workunit.client.1.smithi040.stdout:5/799: dread d2/d7/d8/d1f/d57/d27/d3d/fd4 [0,4194304] 0 2024-04-24T23:14:10.073 INFO:tasks.workunit.client.1.smithi040.stdout:1/885: rmdir da/d12/d18/d2a/d41/d4f/de9 39 2024-04-24T23:14:10.075 INFO:tasks.workunit.client.1.smithi040.stdout:0/953: dwrite df/d1e/d20/d21/d41/d8a/df3/df8/f8c [0,4194304] 0 2024-04-24T23:14:10.077 INFO:tasks.workunit.client.1.smithi040.stdout:1/886: truncate da/d12/d18/d2a/d32/f115 687433 0 2024-04-24T23:14:10.078 INFO:tasks.workunit.client.1.smithi040.stdout:8/945: rmdir d0/d8 39 2024-04-24T23:14:10.080 INFO:tasks.workunit.client.1.smithi040.stdout:2/983: chown da/d10/c83 87 1 2024-04-24T23:14:10.080 INFO:tasks.workunit.client.1.smithi040.stdout:1/887: chown da/d12/d18/c57 868914916 1 2024-04-24T23:14:10.080 INFO:tasks.workunit.client.1.smithi040.stdout:1/888: fdatasync da/d12/d14/fdd 0 2024-04-24T23:14:10.081 INFO:tasks.workunit.client.1.smithi040.stdout:1/889: stat da/d35/d38/d47/f84 0 2024-04-24T23:14:10.081 INFO:tasks.workunit.client.1.smithi040.stdout:9/902: creat d7/d5e/da1/f133 x:0 0 0 2024-04-24T23:14:10.084 INFO:tasks.workunit.client.1.smithi040.stdout:7/946: rmdir db/d2f/d37/d7e/d118/d145 39 2024-04-24T23:14:10.087 INFO:tasks.workunit.client.1.smithi040.stdout:4/874: rename da/c15 to da/d3f/d40/d47/d75/d104/c12e 0 2024-04-24T23:14:10.088 INFO:tasks.workunit.client.1.smithi040.stdout:3/917: creat d1/d2d/d2b/d122/f139 x:0 0 0 2024-04-24T23:14:10.090 INFO:tasks.workunit.client.1.smithi040.stdout:9/903: dwrite d7/da/d17/d40/d55/d2f/d1f/f5a [0,4194304] 0 2024-04-24T23:14:10.092 INFO:tasks.workunit.client.1.smithi040.stdout:3/918: chown d1/d80/dbf/l12f 165 1 2024-04-24T23:14:10.094 INFO:tasks.workunit.client.1.smithi040.stdout:7/947: dwrite db/d12/d20/d52/d6c/fae [0,4194304] 0 2024-04-24T23:14:10.100 INFO:tasks.workunit.client.1.smithi040.stdout:4/875: chown da/d3f/d40/d47/c5b 30618 1 2024-04-24T23:14:10.101 INFO:tasks.workunit.client.1.smithi040.stdout:0/954: unlink df/d1e/da7/c93 0 2024-04-24T23:14:10.102 INFO:tasks.workunit.client.1.smithi040.stdout:9/904: dwrite d7/dbb/f11a [0,4194304] 0 2024-04-24T23:14:10.105 INFO:tasks.workunit.client.1.smithi040.stdout:2/984: mkdir da/d10/d4d/d162 0 2024-04-24T23:14:10.105 INFO:tasks.workunit.client.1.smithi040.stdout:5/800: dwrite d2/d7/d8/d1f/d31/f4e [4194304,4194304] 0 2024-04-24T23:14:10.107 INFO:tasks.workunit.client.1.smithi040.stdout:3/919: rename d1/d2d/d72/d106/d136 to d1/d9/d18/d36/d10d/d13a 0 2024-04-24T23:14:10.108 INFO:tasks.workunit.client.1.smithi040.stdout:7/948: truncate db/d12/d20/d52/d4b/f57 896136 0 2024-04-24T23:14:10.108 INFO:tasks.workunit.client.1.smithi040.stdout:5/801: chown d2/d7/d8/f34 38 1 2024-04-24T23:14:10.109 INFO:tasks.workunit.client.1.smithi040.stdout:4/876: mknod da/d45/d48/d6e/ddd/dea/c12f 0 2024-04-24T23:14:10.112 INFO:tasks.workunit.client.1.smithi040.stdout:1/890: dread da/d12/d23/f30 [0,4194304] 0 2024-04-24T23:14:10.112 INFO:tasks.workunit.client.1.smithi040.stdout:2/985: mkdir da/d1c/d37/d39/d3f/db2/d163 0 2024-04-24T23:14:10.113 INFO:tasks.workunit.client.1.smithi040.stdout:7/949: fdatasync db/d2f/da8/f107 0 2024-04-24T23:14:10.115 INFO:tasks.workunit.client.1.smithi040.stdout:4/877: mkdir da/d3f/d40/d47/d75/de4/d130 0 2024-04-24T23:14:10.116 INFO:tasks.workunit.client.1.smithi040.stdout:4/878: fdatasync da/d78/d115/f11b 0 2024-04-24T23:14:10.119 INFO:tasks.workunit.client.1.smithi040.stdout:2/986: creat da/d1c/d37/d39/d77/d108/f164 x:0 0 0 2024-04-24T23:14:10.122 INFO:tasks.workunit.client.1.smithi040.stdout:2/987: truncate da/d1c/d37/d39/d3f/f14e 656478 0 2024-04-24T23:14:10.122 INFO:tasks.workunit.client.1.smithi040.stdout:4/879: mknod da/d3f/d40/d41/d8e/d116/c131 0 2024-04-24T23:14:10.130 INFO:tasks.workunit.client.1.smithi040.stdout:3/920: rename d1/d9/d18/d36/d10d/d45/daa/ff5 to d1/d9/d18/f13b 0 2024-04-24T23:14:10.131 INFO:tasks.workunit.client.1.smithi040.stdout:3/921: fdatasync d1/d2d/d44/f108 0 2024-04-24T23:14:10.134 INFO:tasks.workunit.client.1.smithi040.stdout:2/988: rename da/d10/d15/c82 to da/d1c/d37/d102/c165 0 2024-04-24T23:14:10.136 INFO:tasks.workunit.client.1.smithi040.stdout:3/922: creat d1/d80/f13c x:0 0 0 2024-04-24T23:14:10.142 INFO:tasks.workunit.client.1.smithi040.stdout:1/891: dread da/d12/d14/f3f [0,4194304] 0 2024-04-24T23:14:10.142 INFO:tasks.workunit.client.1.smithi040.stdout:1/892: chown da/d12/d18/d2a/d32/db3 13623286 1 2024-04-24T23:14:10.144 INFO:tasks.workunit.client.1.smithi040.stdout:5/802: dwrite d2/d2f/d14/fbc [0,4194304] 0 2024-04-24T23:14:10.145 INFO:tasks.workunit.client.1.smithi040.stdout:7/950: write db/d12/d20/d52/d6c/d8f/fe2 [806441,40471] 0 2024-04-24T23:14:10.145 INFO:tasks.workunit.client.1.smithi040.stdout:3/923: dwrite d1/d9/d18/da7/f118 [0,4194304] 0 2024-04-24T23:14:10.149 INFO:tasks.workunit.client.1.smithi040.stdout:6/854: sync 2024-04-24T23:14:10.150 INFO:tasks.workunit.client.1.smithi040.stdout:2/989: link da/d1c/c2f da/d10/d7a/c166 0 2024-04-24T23:14:10.154 INFO:tasks.workunit.client.1.smithi040.stdout:5/803: unlink d2/d2f/f84 0 2024-04-24T23:14:10.154 INFO:tasks.workunit.client.1.smithi040.stdout:5/804: stat d2/d7/d8/d6d/l7b 0 2024-04-24T23:14:10.155 INFO:tasks.workunit.client.1.smithi040.stdout:4/880: dwrite da/d78/dbb/d127/fc8 [0,4194304] 0 2024-04-24T23:14:10.172 INFO:tasks.workunit.client.1.smithi040.stdout:6/855: mknod d7/d1a/d9f/df6/c117 0 2024-04-24T23:14:10.176 INFO:tasks.workunit.client.1.smithi040.stdout:2/990: mknod da/d1c/d37/d39/d3f/d52/d78/dc7/dcc/c167 0 2024-04-24T23:14:10.178 INFO:tasks.workunit.client.1.smithi040.stdout:4/881: rmdir da/d45/d48/d6e/ddd/dea 39 2024-04-24T23:14:10.182 INFO:tasks.workunit.client.1.smithi040.stdout:3/924: dwrite d1/d9/d18/d36/d61/fa6 [0,4194304] 0 2024-04-24T23:14:10.193 INFO:tasks.workunit.client.1.smithi040.stdout:2/991: dread da/d29/d55/f58 [0,4194304] 0 2024-04-24T23:14:10.195 INFO:tasks.workunit.client.1.smithi040.stdout:3/925: mknod d1/d9/d2e/dcf/d129/c13d 0 2024-04-24T23:14:10.195 INFO:tasks.workunit.client.1.smithi040.stdout:3/926: fsync d1/d9/d18/d36/d10d/f132 0 2024-04-24T23:14:10.198 INFO:tasks.workunit.client.1.smithi040.stdout:7/951: getdents db/d12/d62/d8a/db3 0 2024-04-24T23:14:10.200 INFO:tasks.workunit.client.1.smithi040.stdout:5/805: truncate d2/d7/d8/d1f/d31/f4e 5820571 0 2024-04-24T23:14:10.201 INFO:tasks.workunit.client.1.smithi040.stdout:3/927: creat d1/d2d/de/f13e x:0 0 0 2024-04-24T23:14:10.203 INFO:tasks.workunit.client.1.smithi040.stdout:6/856: link d7/d1a/d56/l4f d7/de/l118 0 2024-04-24T23:14:10.204 INFO:tasks.workunit.client.1.smithi040.stdout:4/882: dwrite da/d45/d58/f98 [0,4194304] 0 2024-04-24T23:14:10.206 INFO:tasks.workunit.client.1.smithi040.stdout:7/952: dwrite db/d12/f8c [0,4194304] 0 2024-04-24T23:14:10.206 INFO:tasks.workunit.client.1.smithi040.stdout:2/992: creat da/d1c/d37/d39/d79/d13d/d11b/f168 x:0 0 0 2024-04-24T23:14:10.208 INFO:tasks.workunit.client.1.smithi040.stdout:3/928: rmdir d1/d80 39 2024-04-24T23:14:10.212 INFO:tasks.workunit.client.1.smithi040.stdout:5/806: fdatasync d2/d2f/d14/f86 0 2024-04-24T23:14:10.215 INFO:tasks.workunit.client.1.smithi040.stdout:6/857: mkdir d7/d1a/d56/d119 0 2024-04-24T23:14:10.220 INFO:tasks.workunit.client.1.smithi040.stdout:4/883: rmdir da/d45/d58 39 2024-04-24T23:14:10.222 INFO:tasks.workunit.client.1.smithi040.stdout:4/884: write da/d78/dbb/d127/fc8 [262320,73792] 0 2024-04-24T23:14:10.223 INFO:tasks.workunit.client.1.smithi040.stdout:7/953: rmdir db/d12/d3f/d7a 39 2024-04-24T23:14:10.223 INFO:tasks.workunit.client.1.smithi040.stdout:4/885: fsync da/d3f/d6b/d3c/f90 0 2024-04-24T23:14:10.224 INFO:tasks.workunit.client.1.smithi040.stdout:8/946: sync 2024-04-24T23:14:10.231 INFO:tasks.workunit.client.1.smithi040.stdout:5/807: write d2/d2f/d14/f86 [2800452,34479] 0 2024-04-24T23:14:10.233 INFO:tasks.workunit.client.1.smithi040.stdout:6/858: dread - d7/de/d84/fd1 zero size 2024-04-24T23:14:10.234 INFO:tasks.workunit.client.1.smithi040.stdout:2/993: dread da/d1c/d37/d39/f76 [4194304,4194304] 0 2024-04-24T23:14:10.235 INFO:tasks.workunit.client.1.smithi040.stdout:2/994: dread - da/d1c/d37/d8a/f90 zero size 2024-04-24T23:14:10.241 INFO:tasks.workunit.client.1.smithi040.stdout:4/886: mkdir da/d3f/d40/d47/d75/d132 0 2024-04-24T23:14:10.242 INFO:tasks.workunit.client.1.smithi040.stdout:4/887: chown da/d3f/d40/d47/d75/c8f 636331 1 2024-04-24T23:14:10.242 INFO:tasks.workunit.client.1.smithi040.stdout:7/954: truncate db/d2f/d37/d7e/fc3 2766352 0 2024-04-24T23:14:10.243 INFO:tasks.workunit.client.1.smithi040.stdout:4/888: truncate da/d3f/d8d/ffd 874121 0 2024-04-24T23:14:10.244 INFO:tasks.workunit.client.1.smithi040.stdout:7/955: write db/d2f/d37/fa5 [125371,38391] 0 2024-04-24T23:14:10.244 INFO:tasks.workunit.client.1.smithi040.stdout:3/929: creat d1/d80/dbf/f13f x:0 0 0 2024-04-24T23:14:10.248 INFO:tasks.workunit.client.1.smithi040.stdout:2/995: creat da/d1c/d37/d39/d79/d13d/d153/d155/f169 x:0 0 0 2024-04-24T23:14:10.248 INFO:tasks.workunit.client.1.smithi040.stdout:8/947: rmdir d0/d8/d22/d81 39 2024-04-24T23:14:10.252 INFO:tasks.workunit.client.1.smithi040.stdout:7/956: chown db/d12/d20/d52/d44/f53 11023224 1 2024-04-24T23:14:10.253 INFO:tasks.workunit.client.1.smithi040.stdout:5/808: chown d2/d7/d8/d1f/d57/dca/dfb 956 1 2024-04-24T23:14:10.254 INFO:tasks.workunit.client.1.smithi040.stdout:6/859: symlink d7/d1a/d20/d51/d1e/d46/l11a 0 2024-04-24T23:14:10.254 INFO:tasks.workunit.client.1.smithi040.stdout:6/860: fsync d7/de/d84/f102 0 2024-04-24T23:14:10.255 INFO:tasks.workunit.client.1.smithi040.stdout:8/948: fdatasync d0/d7e/dc1/f6c 0 2024-04-24T23:14:10.257 INFO:tasks.workunit.client.1.smithi040.stdout:2/996: truncate da/d1c/d37/d39/d3f/f88 1736970 0 2024-04-24T23:14:10.259 INFO:tasks.workunit.client.1.smithi040.stdout:4/889: fdatasync da/d45/d58/f73 0 2024-04-24T23:14:10.259 INFO:tasks.workunit.client.1.smithi040.stdout:2/997: chown da/d1c/d37/d39/c6a 0 1 2024-04-24T23:14:10.260 INFO:tasks.workunit.client.1.smithi040.stdout:7/957: creat db/d12/d20/d52/d4b/d127/d12b/db9/f14f x:0 0 0 2024-04-24T23:14:10.262 INFO:tasks.workunit.client.1.smithi040.stdout:5/809: creat d2/d7/d8/d1f/d90/dcb/ffd x:0 0 0 2024-04-24T23:14:10.264 INFO:tasks.workunit.client.1.smithi040.stdout:8/949: dwrite d0/d16/d1b/d47/fdc [0,4194304] 0 2024-04-24T23:14:10.264 INFO:tasks.workunit.client.1.smithi040.stdout:3/930: dread d1/d9/d2e/f5f [0,4194304] 0 2024-04-24T23:14:10.266 INFO:tasks.workunit.client.1.smithi040.stdout:2/998: dread da/d1c/d37/d39/d79/d13d/d153/d155/fea [0,4194304] 0 2024-04-24T23:14:10.267 INFO:tasks.workunit.client.1.smithi040.stdout:9/905: sync 2024-04-24T23:14:10.267 INFO:tasks.workunit.client.1.smithi040.stdout:0/955: sync 2024-04-24T23:14:10.268 INFO:tasks.workunit.client.1.smithi040.stdout:1/893: sync 2024-04-24T23:14:10.269 INFO:tasks.workunit.client.1.smithi040.stdout:9/906: dread - d7/da/f107 zero size 2024-04-24T23:14:10.281 INFO:tasks.workunit.client.1.smithi040.stdout:4/890: symlink da/d3f/d40/d47/l133 0 2024-04-24T23:14:10.282 INFO:tasks.workunit.client.1.smithi040.stdout:7/958: mknod db/d12/d62/d8a/db3/dbf/c150 0 2024-04-24T23:14:10.283 INFO:tasks.workunit.client.1.smithi040.stdout:8/950: rename d0/d16/d1b/d58/lcf to d0/d16/d1b/d47/def/l136 0 2024-04-24T23:14:10.286 INFO:tasks.workunit.client.1.smithi040.stdout:0/956: symlink df/d1e/da7/l135 0 2024-04-24T23:14:10.287 INFO:tasks.workunit.client.1.smithi040.stdout:9/907: mknod d7/d5e/de5/c134 0 2024-04-24T23:14:10.290 INFO:tasks.workunit.client.1.smithi040.stdout:1/894: mknod da/d12/d18/d2a/d4d/d7d/c130 0 2024-04-24T23:14:10.290 INFO:tasks.workunit.client.1.smithi040.stdout:6/861: write d7/d1a/d20/d51/d1e/d46/d69/f6b [4013810,10823] 0 2024-04-24T23:14:10.291 INFO:tasks.workunit.client.1.smithi040.stdout:1/895: write da/d12/d18/f78 [989536,56139] 0 2024-04-24T23:14:10.376 INFO:tasks.workunit.client.1.smithi040.stdout:7/959: read db/d12/d20/d52/d44/f88 [1294605,124719] 0 2024-04-24T23:14:10.381 INFO:tasks.workunit.client.1.smithi040.stdout:5/810: sync 2024-04-24T23:14:10.425 INFO:tasks.workunit.client.1.smithi040.stdout:6/862: sync 2024-04-24T23:14:10.426 INFO:tasks.workunit.client.1.smithi040.stdout:6/863: chown d7/d1a/d20/d51/d2f/dba/fa4 626825 1 2024-04-24T23:14:10.446 INFO:tasks.workunit.client.1.smithi040.stdout:0/957: truncate df/f39 1502379 0 2024-04-24T23:14:10.451 INFO:tasks.workunit.client.1.smithi040.stdout:2/999: getdents da/d1c/d37/d39/d3f/db2/d163 0 2024-04-24T23:14:10.452 INFO:tasks.workunit.client.1.smithi040.stdout:1/896: rmdir da/d35/d38/d47 39 2024-04-24T23:14:10.461 INFO:tasks.workunit.client.1.smithi040.stdout:6/864: mknod d7/d1a/d20/d51/d1e/d46/d69/d87/c11b 0 2024-04-24T23:14:10.466 INFO:tasks.workunit.client.1.smithi040.stdout:3/931: write d1/d2d/fc1 [5252614,121225] 0 2024-04-24T23:14:10.468 INFO:tasks.workunit.client.1.smithi040.stdout:0/958: creat df/d14/d11c/f136 x:0 0 0 2024-04-24T23:14:10.488 INFO:tasks.workunit.client.1.smithi040.stdout:1/897: symlink da/d12/d14/d58/ded/l131 0 2024-04-24T23:14:10.490 INFO:tasks.workunit.client.1.smithi040.stdout:6/865: dread d7/d1a/d20/d51/d1e/f86 [0,4194304] 0 2024-04-24T23:14:10.494 INFO:tasks.workunit.client.1.smithi040.stdout:1/898: fsync da/d12/d23/f30 0 2024-04-24T23:14:10.585 INFO:tasks.workunit.client.1.smithi040.stdout:6/866: rmdir d7/d1a/d56/d119 0 2024-04-24T23:14:10.590 INFO:tasks.workunit.client.1.smithi040.stdout:6/867: rename d7/d1a/db5/fe2 to d7/d1a/d9f/df6/f11c 0 2024-04-24T23:14:10.591 INFO:tasks.workunit.client.1.smithi040.stdout:6/868: read d7/d1a/d9f/fb3 [2510826,48773] 0 2024-04-24T23:14:10.591 INFO:tasks.workunit.client.1.smithi040.stdout:1/899: creat da/d12/d18/d2a/d41/d4f/f132 x:0 0 0 2024-04-24T23:14:10.594 INFO:tasks.workunit.client.1.smithi040.stdout:6/869: mkdir d7/de/d4e/d11d 0 2024-04-24T23:14:10.598 INFO:tasks.workunit.client.1.smithi040.stdout:1/900: symlink da/d35/d38/d47/dbc/l133 0 2024-04-24T23:14:10.600 INFO:tasks.workunit.client.1.smithi040.stdout:1/901: chown da/d12/d18/d2a/d32/f121 9810 1 2024-04-24T23:14:10.600 INFO:tasks.workunit.client.1.smithi040.stdout:6/870: dwrite d7/d1a/d20/d51/d2f/df2/ff7 [0,4194304] 0 2024-04-24T23:14:10.605 INFO:tasks.workunit.client.1.smithi040.stdout:1/902: mknod da/d12/d18/d2a/d4d/d107/c134 0 2024-04-24T23:14:10.606 INFO:tasks.workunit.client.1.smithi040.stdout:6/871: symlink d7/d1a/d9f/df6/d110/l11e 0 2024-04-24T23:14:10.609 INFO:tasks.workunit.client.1.smithi040.stdout:6/872: mkdir d7/d1a/d20/d51/d2a/dfd/d11f 0 2024-04-24T23:14:10.611 INFO:tasks.workunit.client.1.smithi040.stdout:1/903: dwrite da/d35/d38/d47/dbc/f12f [0,4194304] 0 2024-04-24T23:14:10.613 INFO:tasks.workunit.client.1.smithi040.stdout:1/904: chown da/d12/d18/d2a/d4d/d107/d5e/fd7 259347670 1 2024-04-24T23:14:10.617 INFO:tasks.workunit.client.1.smithi040.stdout:6/873: write d7/d1a/d20/d51/d2f/dba/dde/fd4 [823077,125987] 0 2024-04-24T23:14:10.619 INFO:tasks.workunit.client.1.smithi040.stdout:6/874: write d7/d1a/d20/d51/d1e/d46/d69/f101 [975331,115237] 0 2024-04-24T23:14:10.620 INFO:tasks.workunit.client.1.smithi040.stdout:6/875: write d7/d1a/d20/d51/d1e/f3d [3124350,76931] 0 2024-04-24T23:14:10.623 INFO:tasks.workunit.client.1.smithi040.stdout:1/905: link da/d12/d18/d2a/d32/c5c da/d12/d18/d2a/d4d/dfd/d11e/c135 0 2024-04-24T23:14:10.628 INFO:tasks.workunit.client.1.smithi040.stdout:6/876: dwrite d7/d1a/d56/faf [0,4194304] 0 2024-04-24T23:14:10.629 INFO:tasks.workunit.client.1.smithi040.stdout:6/877: fdatasync d7/de/d84/f102 0 2024-04-24T23:14:10.632 INFO:tasks.workunit.client.1.smithi040.stdout:1/906: mknod da/d12/d18/d2a/d4d/d107/db9/c136 0 2024-04-24T23:14:10.638 INFO:tasks.workunit.client.1.smithi040.stdout:6/878: dwrite d7/d1a/d20/d51/d1e/d46/fac [0,4194304] 0 2024-04-24T23:14:10.649 INFO:tasks.workunit.client.1.smithi040.stdout:6/879: symlink d7/d1a/l120 0 2024-04-24T23:14:10.692 INFO:tasks.workunit.client.1.smithi040.stdout:8/951: truncate d0/d16/d19/fb5 3981151 0 2024-04-24T23:14:10.695 INFO:tasks.workunit.client.1.smithi040.stdout:9/908: write d7/da/d17/d40/d55/d2f/d1f/d4e/f68 [2282836,43990] 0 2024-04-24T23:14:10.703 INFO:tasks.workunit.client.1.smithi040.stdout:5/811: dwrite d2/d7/d8/d1f/d57/f91 [0,4194304] 0 2024-04-24T23:14:10.712 INFO:tasks.workunit.client.1.smithi040.stdout:8/952: dread d0/d52/fb3 [0,4194304] 0 2024-04-24T23:14:10.712 INFO:tasks.workunit.client.1.smithi040.stdout:3/932: dwrite d1/d9/d18/f66 [0,4194304] 0 2024-04-24T23:14:10.712 INFO:tasks.workunit.client.1.smithi040.stdout:0/959: dwrite df/de1/db6/fdc [0,4194304] 0 2024-04-24T23:14:10.715 INFO:tasks.workunit.client.1.smithi040.stdout:8/953: fsync d0/d8/d22/db8/dc2/f11d 0 2024-04-24T23:14:10.716 INFO:tasks.workunit.client.1.smithi040.stdout:8/954: write d0/d16/d1b/d3f/f114 [245704,38794] 0 2024-04-24T23:14:10.718 INFO:tasks.workunit.client.1.smithi040.stdout:3/933: mknod d1/d2d/d2b/d122/c140 0 2024-04-24T23:14:10.723 INFO:tasks.workunit.client.1.smithi040.stdout:5/812: getdents d2/d7/d8/d1f 0 2024-04-24T23:14:10.725 INFO:tasks.workunit.client.1.smithi040.stdout:3/934: dwrite d1/d9/d2e/dcf/d110/f116 [0,4194304] 0 2024-04-24T23:14:10.727 INFO:tasks.workunit.client.1.smithi040.stdout:3/935: truncate d1/d2d/d2b/d122/f139 951016 0 2024-04-24T23:14:10.730 INFO:tasks.workunit.client.1.smithi040.stdout:5/813: readlink d2/d7/d8/d1f/d57/d27/d3c/l8f 0 2024-04-24T23:14:10.730 INFO:tasks.workunit.client.1.smithi040.stdout:0/960: link df/c44 df/d12/d58/dc3/c137 0 2024-04-24T23:14:10.739 INFO:tasks.workunit.client.1.smithi040.stdout:5/814: symlink d2/d7/d8/d64/lfe 0 2024-04-24T23:14:10.745 INFO:tasks.workunit.client.1.smithi040.stdout:5/815: dwrite d2/d7/fed [0,4194304] 0 2024-04-24T23:14:10.747 INFO:tasks.workunit.client.1.smithi040.stdout:5/816: chown d2/d7/d8/d1f/d57/d27/d3c/f47 0 1 2024-04-24T23:14:10.749 INFO:tasks.workunit.client.1.smithi040.stdout:8/955: dread d0/d7e/f124 [0,4194304] 0 2024-04-24T23:14:10.765 INFO:tasks.workunit.client.1.smithi040.stdout:9/909: rmdir d7/da 39 2024-04-24T23:14:10.771 INFO:tasks.workunit.client.1.smithi040.stdout:6/880: rmdir d7/de/d89 39 2024-04-24T23:14:10.773 INFO:tasks.workunit.client.1.smithi040.stdout:6/881: creat d7/d1a/d20/d8c/df5/f121 x:0 0 0 2024-04-24T23:14:10.774 INFO:tasks.workunit.client.1.smithi040.stdout:6/882: readlink d7/de/l118 0 2024-04-24T23:14:10.776 INFO:tasks.workunit.client.1.smithi040.stdout:4/891: mkdir da/d3f/d6b/d134 0 2024-04-24T23:14:10.777 INFO:tasks.workunit.client.1.smithi040.stdout:4/892: truncate da/f118 888871 0 2024-04-24T23:14:10.779 INFO:tasks.workunit.client.1.smithi040.stdout:6/883: creat d7/de/f122 x:0 0 0 2024-04-24T23:14:10.780 INFO:tasks.workunit.client.1.smithi040.stdout:4/893: chown da/d3f/d40/fe5 15247958 1 2024-04-24T23:14:10.787 INFO:tasks.workunit.client.1.smithi040.stdout:4/894: dwrite da/d45/d48/d6e/d70/fa8 [4194304,4194304] 0 2024-04-24T23:14:10.788 INFO:tasks.workunit.client.1.smithi040.stdout:4/895: chown da/d3f/d8d/db5/l111 651 1 2024-04-24T23:14:10.790 INFO:tasks.workunit.client.1.smithi040.stdout:4/896: truncate da/d45/d48/d6e/d70/ffc 796991 0 2024-04-24T23:14:10.790 INFO:tasks.workunit.client.1.smithi040.stdout:1/907: truncate da/d35/dec/ff1 825811 0 2024-04-24T23:14:10.791 INFO:tasks.workunit.client.1.smithi040.stdout:4/897: chown da/d3f/lae 8 1 2024-04-24T23:14:10.792 INFO:tasks.workunit.client.1.smithi040.stdout:6/884: dread d7/de/d4e/f64 [0,4194304] 0 2024-04-24T23:14:10.797 INFO:tasks.workunit.client.1.smithi040.stdout:4/898: rename da/d3f/d40/d47/l133 to da/d3f/d40/d47/d75/d7c/l135 0 2024-04-24T23:14:10.798 INFO:tasks.workunit.client.1.smithi040.stdout:1/908: getdents da/d35/d38/d47/dbc/de5/df5/d12e 0 2024-04-24T23:14:10.799 INFO:tasks.workunit.client.1.smithi040.stdout:6/885: getdents d7/d1a/d20/d8c/dd5 0 2024-04-24T23:14:10.801 INFO:tasks.workunit.client.1.smithi040.stdout:1/909: write da/d12/d18/d2a/d41/f124 [223437,29648] 0 2024-04-24T23:14:10.801 INFO:tasks.workunit.client.1.smithi040.stdout:4/899: creat da/d3f/d40/d47/d75/d132/f136 x:0 0 0 2024-04-24T23:14:10.802 INFO:tasks.workunit.client.1.smithi040.stdout:1/910: read da/d12/d14/f24 [4652814,109252] 0 2024-04-24T23:14:10.802 INFO:tasks.workunit.client.1.smithi040.stdout:6/886: creat d7/d1a/d20/d51/d2a/dfd/f123 x:0 0 0 2024-04-24T23:14:10.804 INFO:tasks.workunit.client.1.smithi040.stdout:1/911: read - da/d12/d18/d2a/d4d/d107/d5e/fd7 zero size 2024-04-24T23:14:10.805 INFO:tasks.workunit.client.1.smithi040.stdout:3/936: symlink d1/d2d/l141 0 2024-04-24T23:14:10.805 INFO:tasks.workunit.client.1.smithi040.stdout:1/912: dread - da/d12/d18/d2a/d8b/d97/f112 zero size 2024-04-24T23:14:10.806 INFO:tasks.workunit.client.1.smithi040.stdout:7/960: creat db/d12/d20/d52/d4b/d127/d12b/db9/f151 x:0 0 0 2024-04-24T23:14:10.810 INFO:tasks.workunit.client.1.smithi040.stdout:1/913: rmdir da/d35/d38/d47/dbc 39 2024-04-24T23:14:10.811 INFO:tasks.workunit.client.1.smithi040.stdout:1/914: chown da/d12/d18/d2a/d4d/d107/d5e/fd7 49102 1 2024-04-24T23:14:10.812 INFO:tasks.workunit.client.1.smithi040.stdout:6/887: symlink d7/d1a/d20/d8c/l124 0 2024-04-24T23:14:10.812 INFO:tasks.workunit.client.1.smithi040.stdout:1/915: write da/d12/d18/d2a/d4d/d107/db9/f11b [232317,28866] 0 2024-04-24T23:14:10.812 INFO:tasks.workunit.client.1.smithi040.stdout:9/910: symlink d7/da/l135 0 2024-04-24T23:14:10.814 INFO:tasks.workunit.client.1.smithi040.stdout:6/888: read d7/d1a/d20/f47 [223393,65986] 0 2024-04-24T23:14:10.815 INFO:tasks.workunit.client.1.smithi040.stdout:9/911: truncate d7/da/d17/d120/f130 460554 0 2024-04-24T23:14:10.815 INFO:tasks.workunit.client.1.smithi040.stdout:4/900: creat da/d3f/d6b/f137 x:0 0 0 2024-04-24T23:14:10.819 INFO:tasks.workunit.client.1.smithi040.stdout:9/912: unlink d7/da/d17/d40/d55/d2f/d28/c108 0 2024-04-24T23:14:10.822 INFO:tasks.workunit.client.1.smithi040.stdout:4/901: creat da/d3f/d40/d41/d8e/d116/d11c/f138 x:0 0 0 2024-04-24T23:14:10.824 INFO:tasks.workunit.client.1.smithi040.stdout:9/913: mknod d7/da/d17/c136 0 2024-04-24T23:14:10.827 INFO:tasks.workunit.client.1.smithi040.stdout:9/914: creat d7/da/d17/d40/d55/d2f/d28/d67/f137 x:0 0 0 2024-04-24T23:14:10.828 INFO:tasks.workunit.client.1.smithi040.stdout:4/902: creat da/d45/d48/d6e/ddd/f139 x:0 0 0 2024-04-24T23:14:10.829 INFO:tasks.workunit.client.1.smithi040.stdout:9/915: fsync d7/da/d17/d40/fb4 0 2024-04-24T23:14:10.868 INFO:tasks.workunit.client.1.smithi040.stdout:1/916: dread da/d12/d18/d2a/fc7 [0,4194304] 0 2024-04-24T23:14:10.868 INFO:tasks.workunit.client.1.smithi040.stdout:0/961: write df/de1/f29 [5240620,98883] 0 2024-04-24T23:14:10.869 INFO:tasks.workunit.client.1.smithi040.stdout:9/916: stat d7/da/d17/d40/d55/lb2 0 2024-04-24T23:14:10.873 INFO:tasks.workunit.client.1.smithi040.stdout:4/903: mknod da/d3f/d40/d47/d75/d7c/c13a 0 2024-04-24T23:14:10.874 INFO:tasks.workunit.client.1.smithi040.stdout:9/917: fdatasync d7/da/d17/d40/d55/d2f/f6f 0 2024-04-24T23:14:10.874 INFO:tasks.workunit.client.1.smithi040.stdout:7/961: write db/d12/d20/ff5 [458834,100761] 0 2024-04-24T23:14:10.876 INFO:tasks.workunit.client.1.smithi040.stdout:3/937: write d1/d2d/de/d30/d37/f63 [804606,51969] 0 2024-04-24T23:14:10.878 INFO:tasks.workunit.client.1.smithi040.stdout:5/817: dwrite d2/fb6 [0,4194304] 0 2024-04-24T23:14:10.878 INFO:tasks.workunit.client.1.smithi040.stdout:8/956: dwrite d0/d8/dc/d34/da7/fd4 [0,4194304] 0 2024-04-24T23:14:10.878 INFO:tasks.workunit.client.1.smithi040.stdout:7/962: chown db/d12/d62/d8a/f124 25 1 2024-04-24T23:14:10.881 INFO:tasks.workunit.client.1.smithi040.stdout:7/963: stat db/d12/d3f/c122 0 2024-04-24T23:14:10.883 INFO:tasks.workunit.client.1.smithi040.stdout:7/964: dread - db/d2f/da8/f107 zero size 2024-04-24T23:14:10.893 INFO:tasks.workunit.client.1.smithi040.stdout:0/962: creat df/d12/d58/dc3/f138 x:0 0 0 2024-04-24T23:14:10.893 INFO:tasks.workunit.client.1.smithi040.stdout:0/963: chown df/d1e/d20 113 1 2024-04-24T23:14:10.895 INFO:tasks.workunit.client.1.smithi040.stdout:9/918: dread d7/d5e/da1/dfa/f10c [0,4194304] 0 2024-04-24T23:14:10.899 INFO:tasks.workunit.client.1.smithi040.stdout:3/938: rename d1/d9/d2e/dcf/d129 to d1/d9/d18/d36/d10d/d45/d96/d142 0 2024-04-24T23:14:10.900 INFO:tasks.workunit.client.1.smithi040.stdout:3/939: stat d1/d2d/d72/d106/cdb 0 2024-04-24T23:14:10.908 INFO:tasks.workunit.client.1.smithi040.stdout:9/919: dread d7/da/d17/d40/d55/d2f/d1f/fc5 [0,4194304] 0 2024-04-24T23:14:10.909 INFO:tasks.workunit.client.1.smithi040.stdout:9/920: chown d7/da/d17/l1d 0 1 2024-04-24T23:14:10.909 INFO:tasks.workunit.client.1.smithi040.stdout:8/957: truncate d0/d8/dc/f86 1475746 0 2024-04-24T23:14:10.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:10 smithi040 ceph-mon[33128]: pgmap v38: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 67 MiB/s rd, 166 MiB/s wr, 365 op/s 2024-04-24T23:14:10.915 INFO:tasks.workunit.client.1.smithi040.stdout:0/964: rmdir df/d1e/d89/ddf 39 2024-04-24T23:14:10.932 INFO:tasks.workunit.client.1.smithi040.stdout:9/921: creat d7/da/d17/d40/d55/d2f/df7/f138 x:0 0 0 2024-04-24T23:14:10.933 INFO:tasks.workunit.client.1.smithi040.stdout:8/958: mknod d0/d8/dc/c137 0 2024-04-24T23:14:10.938 INFO:tasks.workunit.client.1.smithi040.stdout:7/965: write db/d12/d7d/dd2/f100 [935256,13250] 0 2024-04-24T23:14:10.941 INFO:tasks.workunit.client.1.smithi040.stdout:8/959: dread d0/d16/d19/f9a [0,4194304] 0 2024-04-24T23:14:10.947 INFO:tasks.workunit.client.1.smithi040.stdout:5/818: truncate d2/d7/d8/f34 1549740 0 2024-04-24T23:14:10.956 INFO:tasks.workunit.client.1.smithi040.stdout:8/960: dread - d0/d8/d66/fc0 zero size 2024-04-24T23:14:10.956 INFO:tasks.workunit.client.1.smithi040.stdout:8/961: readlink d0/d8/d22/db8/lfb 0 2024-04-24T23:14:10.958 INFO:tasks.workunit.client.1.smithi040.stdout:1/917: link da/d12/d18/d2a/d4d/d107/db9/lcd da/d35/d38/d47/dbc/l137 0 2024-04-24T23:14:10.959 INFO:tasks.workunit.client.1.smithi040.stdout:0/965: write df/d1e/d20/d61/f86 [4864417,89486] 0 2024-04-24T23:14:10.962 INFO:tasks.workunit.client.1.smithi040.stdout:9/922: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/c139 0 2024-04-24T23:14:10.962 INFO:tasks.workunit.client.1.smithi040.stdout:9/923: chown d7 3676 1 2024-04-24T23:14:10.963 INFO:tasks.workunit.client.1.smithi040.stdout:3/940: write d1/d2a/d6a/fe0 [3159942,125651] 0 2024-04-24T23:14:10.969 INFO:tasks.workunit.client.1.smithi040.stdout:1/918: write da/d12/f54 [4159432,59487] 0 2024-04-24T23:14:10.970 INFO:tasks.workunit.client.1.smithi040.stdout:3/941: dwrite d1/d9/d18/d36/d55/d62/f114 [0,4194304] 0 2024-04-24T23:14:10.971 INFO:tasks.workunit.client.1.smithi040.stdout:1/919: read - da/d12/d18/d2a/d8b/f111 zero size 2024-04-24T23:14:10.977 INFO:tasks.workunit.client.1.smithi040.stdout:5/819: fsync d2/d7/d8/d1f/d90/fd3 0 2024-04-24T23:14:10.978 INFO:tasks.workunit.client.1.smithi040.stdout:5/820: write d2/d7/fed [2417490,22629] 0 2024-04-24T23:14:10.979 INFO:tasks.workunit.client.1.smithi040.stdout:9/924: dread - d7/d5e/da1/fe1 zero size 2024-04-24T23:14:10.985 INFO:tasks.workunit.client.1.smithi040.stdout:1/920: unlink da/d12/d18/d2a/d4d/d107/d5e/fa4 0 2024-04-24T23:14:10.989 INFO:tasks.workunit.client.1.smithi040.stdout:9/925: unlink d7/d5e/da1/f133 0 2024-04-24T23:14:10.991 INFO:tasks.workunit.client.1.smithi040.stdout:5/821: dwrite d2/d7/d8/f34 [0,4194304] 0 2024-04-24T23:14:10.992 INFO:tasks.workunit.client.1.smithi040.stdout:5/822: stat d2/d7/d8/d1f 0 2024-04-24T23:14:10.993 INFO:tasks.workunit.client.1.smithi040.stdout:0/966: creat df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dcf/f139 x:0 0 0 2024-04-24T23:14:10.994 INFO:tasks.workunit.client.1.smithi040.stdout:0/967: readlink df/d12/d58/dc3/dde/l12d 0 2024-04-24T23:14:10.997 INFO:tasks.workunit.client.1.smithi040.stdout:3/942: rename d1/d80/f13c to d1/d2a/d6a/f143 0 2024-04-24T23:14:10.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:10 smithi031 ceph-mon[26411]: pgmap v38: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 67 MiB/s rd, 166 MiB/s wr, 365 op/s 2024-04-24T23:14:11.001 INFO:tasks.workunit.client.1.smithi040.stdout:5/823: creat d2/d7/d8/d1f/d90/fff x:0 0 0 2024-04-24T23:14:11.002 INFO:tasks.workunit.client.1.smithi040.stdout:5/824: write d2/d7/d8/d64/f68 [830018,102752] 0 2024-04-24T23:14:11.003 INFO:tasks.workunit.client.1.smithi040.stdout:6/889: sync 2024-04-24T23:14:11.003 INFO:tasks.workunit.client.1.smithi040.stdout:0/968: mknod df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/dcf/c13a 0 2024-04-24T23:14:11.010 INFO:tasks.workunit.client.1.smithi040.stdout:1/921: link da/d12/d55/f8e da/d35/f138 0 2024-04-24T23:14:11.014 INFO:tasks.workunit.client.1.smithi040.stdout:6/890: mkdir d7/d1a/d20/d8c/db0/d125 0 2024-04-24T23:14:11.025 INFO:tasks.workunit.client.1.smithi040.stdout:9/926: truncate d7/da/d17/d40/d55/d2f/d28/f29 6609849 0 2024-04-24T23:14:11.028 INFO:tasks.workunit.client.1.smithi040.stdout:5/825: getdents d2/d7/d8/d1f/d57/d27/d3c/d9b 0 2024-04-24T23:14:11.028 INFO:tasks.workunit.client.1.smithi040.stdout:0/969: write df/d14/ffe [4831498,59212] 0 2024-04-24T23:14:11.030 INFO:tasks.workunit.client.1.smithi040.stdout:1/922: write da/d12/d18/d2a/f56 [801766,15404] 0 2024-04-24T23:14:11.031 INFO:tasks.workunit.client.1.smithi040.stdout:6/891: write d7/de/d84/f91 [610156,491] 0 2024-04-24T23:14:11.032 INFO:tasks.workunit.client.1.smithi040.stdout:9/927: truncate d7/f57 4432085 0 2024-04-24T23:14:11.035 INFO:tasks.workunit.client.1.smithi040.stdout:0/970: truncate df/d12/f2a 5878074 0 2024-04-24T23:14:11.036 INFO:tasks.workunit.client.1.smithi040.stdout:5/826: dwrite d2/d7/d8/db/fde [0,4194304] 0 2024-04-24T23:14:11.042 INFO:tasks.workunit.client.1.smithi040.stdout:1/923: write da/fd9 [1478833,3365] 0 2024-04-24T23:14:11.043 INFO:tasks.workunit.client.1.smithi040.stdout:9/928: chown d7/da/d17/d40/d55/d2f/d1f/d4e/lb9 29372160 1 2024-04-24T23:14:11.043 INFO:tasks.workunit.client.1.smithi040.stdout:1/924: chown da/d12/d18/d2a/d32/cbe 139 1 2024-04-24T23:14:11.050 INFO:tasks.workunit.client.1.smithi040.stdout:6/892: creat d7/d1a/d20/d8c/db0/d125/f126 x:0 0 0 2024-04-24T23:14:11.051 INFO:tasks.workunit.client.1.smithi040.stdout:1/925: creat da/d35/d38/d47/dbc/d109/f139 x:0 0 0 2024-04-24T23:14:11.052 INFO:tasks.workunit.client.1.smithi040.stdout:1/926: readlink da/d12/d18/d2a/d41/d8c/l129 0 2024-04-24T23:14:11.053 INFO:tasks.workunit.client.1.smithi040.stdout:5/827: unlink d2/d7/d8/d1f/d57/ce2 0 2024-04-24T23:14:11.054 INFO:tasks.workunit.client.1.smithi040.stdout:5/828: chown d2/d7/d8/d1f/d90/dcb/c77 260 1 2024-04-24T23:14:11.054 INFO:tasks.workunit.client.1.smithi040.stdout:6/893: mkdir d7/d1a/db5/d127 0 2024-04-24T23:14:11.056 INFO:tasks.workunit.client.1.smithi040.stdout:0/971: rename df/d1e/d20/f120 to df/d1e/d20/d21/f13b 0 2024-04-24T23:14:11.063 INFO:tasks.workunit.client.1.smithi040.stdout:6/894: symlink d7/de/d84/l128 0 2024-04-24T23:14:11.066 INFO:tasks.workunit.client.1.smithi040.stdout:5/829: rmdir d2/d7 39 2024-04-24T23:14:11.068 INFO:tasks.workunit.client.1.smithi040.stdout:9/929: rename d7/da/d17/d40/d55/d2f/df7/f8f to d7/da/d17/d40/d122/f13a 0 2024-04-24T23:14:11.068 INFO:tasks.workunit.client.1.smithi040.stdout:4/904: sync 2024-04-24T23:14:11.069 INFO:tasks.workunit.client.1.smithi040.stdout:9/930: fsync d7/da/d17/d40/d55/d2f/df7/d6c/f125 0 2024-04-24T23:14:11.069 INFO:tasks.workunit.client.1.smithi040.stdout:9/931: stat d7/da/d17/c5f 0 2024-04-24T23:14:11.076 INFO:tasks.workunit.client.1.smithi040.stdout:0/972: symlink df/d1e/d20/d11e/l13c 0 2024-04-24T23:14:11.077 INFO:tasks.workunit.client.1.smithi040.stdout:0/973: dread - df/d14/d11c/f136 zero size 2024-04-24T23:14:11.077 INFO:tasks.workunit.client.1.smithi040.stdout:0/974: readlink df/d1e/d20/d21/l12c 0 2024-04-24T23:14:11.078 INFO:tasks.workunit.client.1.smithi040.stdout:1/927: truncate da/fd9 3923935 0 2024-04-24T23:14:11.079 INFO:tasks.workunit.client.1.smithi040.stdout:1/928: readlink da/d12/d14/d58/l63 0 2024-04-24T23:14:11.080 INFO:tasks.workunit.client.1.smithi040.stdout:5/830: rename d2/d7/d8/d6d/d89/lc1 to d2/d7/d8/d1f/d57/d27/d3c/d43/l100 0 2024-04-24T23:14:11.082 INFO:tasks.workunit.client.1.smithi040.stdout:6/895: creat d7/f129 x:0 0 0 2024-04-24T23:14:11.083 INFO:tasks.workunit.client.1.smithi040.stdout:0/975: creat df/d12/d4b/f13d x:0 0 0 2024-04-24T23:14:11.084 INFO:tasks.workunit.client.1.smithi040.stdout:6/896: dread - d7/d1a/d20/d51/d2f/dba/dde/f109 zero size 2024-04-24T23:14:11.084 INFO:tasks.workunit.client.1.smithi040.stdout:6/897: fsync d7/d1a/d56/fff 0 2024-04-24T23:14:11.085 INFO:tasks.workunit.client.1.smithi040.stdout:6/898: chown d7/d1a/l83 30280255 1 2024-04-24T23:14:11.086 INFO:tasks.workunit.client.1.smithi040.stdout:9/932: dread d7/d5e/f7d [0,4194304] 0 2024-04-24T23:14:11.089 INFO:tasks.workunit.client.1.smithi040.stdout:1/929: unlink da/d12/d18/d2a/d41/d8c/c120 0 2024-04-24T23:14:11.089 INFO:tasks.workunit.client.1.smithi040.stdout:0/976: dwrite df/d1e/d20/d61/f86 [0,4194304] 0 2024-04-24T23:14:11.090 INFO:tasks.workunit.client.1.smithi040.stdout:4/905: getdents da/d3f/d40 0 2024-04-24T23:14:11.092 INFO:tasks.workunit.client.1.smithi040.stdout:0/977: fsync df/d1e/d20/d21/fd9 0 2024-04-24T23:14:11.092 INFO:tasks.workunit.client.1.smithi040.stdout:6/899: mkdir d7/d1a/db5/d12a 0 2024-04-24T23:14:11.093 INFO:tasks.workunit.client.1.smithi040.stdout:1/930: dread - da/d12/d55/fc4 zero size 2024-04-24T23:14:11.094 INFO:tasks.workunit.client.1.smithi040.stdout:9/933: chown d7/d92/d9c/f119 45949 1 2024-04-24T23:14:11.098 INFO:tasks.workunit.client.1.smithi040.stdout:0/978: truncate df/d12/d1a/f121 347272 0 2024-04-24T23:14:11.099 INFO:tasks.workunit.client.1.smithi040.stdout:0/979: readlink df/d1e/d20/d61/le4 0 2024-04-24T23:14:11.099 INFO:tasks.workunit.client.1.smithi040.stdout:6/900: truncate d7/d1a/d20/d51/f26 1229274 0 2024-04-24T23:14:11.102 INFO:tasks.workunit.client.1.smithi040.stdout:1/931: rmdir da/d12/d23 39 2024-04-24T23:14:11.102 INFO:tasks.workunit.client.1.smithi040.stdout:5/831: dread d2/d7/d8/d1f/d31/f4e [0,4194304] 0 2024-04-24T23:14:11.111 INFO:tasks.workunit.client.1.smithi040.stdout:6/901: chown d7/d1a/d20/d51/d1e/d46/fa0 5264 1 2024-04-24T23:14:11.112 INFO:tasks.workunit.client.1.smithi040.stdout:6/902: chown d7/d1a/d20/d8c/fcf 20 1 2024-04-24T23:14:11.115 INFO:tasks.workunit.client.1.smithi040.stdout:4/906: link da/d78/dbb/f129 da/d3f/d6b/d3c/d69/df3/d10b/f13b 0 2024-04-24T23:14:11.119 INFO:tasks.workunit.client.1.smithi040.stdout:6/903: chown d7/d1a/d20/d51/d2f/dba/dde/ldd 12528 1 2024-04-24T23:14:11.120 INFO:tasks.workunit.client.1.smithi040.stdout:9/934: dwrite d7/da/d17/d40/d55/d2f/d1f/fc5 [0,4194304] 0 2024-04-24T23:14:11.124 INFO:tasks.workunit.client.1.smithi040.stdout:3/943: sync 2024-04-24T23:14:11.125 INFO:tasks.workunit.client.1.smithi040.stdout:8/962: sync 2024-04-24T23:14:11.125 INFO:tasks.workunit.client.1.smithi040.stdout:7/966: sync 2024-04-24T23:14:11.125 INFO:tasks.workunit.client.1.smithi040.stdout:0/980: write df/d12/f1f [6950767,29936] 0 2024-04-24T23:14:11.129 INFO:tasks.workunit.client.1.smithi040.stdout:0/981: chown df/d1e/d20/d21/d41/d8a/df3/df8/da3/d8e/ca8 493868 1 2024-04-24T23:14:11.132 INFO:tasks.workunit.client.1.smithi040.stdout:3/944: mkdir d1/d2a/d8d/d7c/d144 0 2024-04-24T23:14:11.132 INFO:tasks.workunit.client.1.smithi040.stdout:0/982: write df/d1e/d20/d21/d41/d8a/f10f [1797926,31802] 0 2024-04-24T23:14:11.135 INFO:tasks.workunit.client.1.smithi040.stdout:9/935: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f9d [0,4194304] 0 2024-04-24T23:14:11.139 INFO:tasks.workunit.client.1.smithi040.stdout:4/907: truncate da/d78/dbb/fc2 3067612 0 2024-04-24T23:14:11.140 INFO:tasks.workunit.client.1.smithi040.stdout:8/963: truncate d0/d16/d1b/f40 1774276 0 2024-04-24T23:14:11.140 INFO:tasks.workunit.client.1.smithi040.stdout:4/908: chown da/d3f/d40/d47/d75/de4/d102 13301787 1 2024-04-24T23:14:11.142 INFO:tasks.workunit.client.1.smithi040.stdout:4/909: write da/d45/d58/f98 [4879913,6791] 0 2024-04-24T23:14:11.147 INFO:tasks.workunit.client.1.smithi040.stdout:1/932: rename da/d12/d18/d2a/d4d/dfd to da/d12/d18/d13a 0 2024-04-24T23:14:11.151 INFO:tasks.workunit.client.1.smithi040.stdout:7/967: dread db/d12/d3f/fc1 [0,4194304] 0 2024-04-24T23:14:11.152 INFO:tasks.workunit.client.1.smithi040.stdout:7/968: write db/d12/ff2 [201155,19603] 0 2024-04-24T23:14:11.157 INFO:tasks.workunit.client.1.smithi040.stdout:6/904: dread d7/d1a/d20/f28 [0,4194304] 0 2024-04-24T23:14:11.164 INFO:tasks.workunit.client.1.smithi040.stdout:8/964: creat d0/d7e/dc1/d9e/d90/de7/f138 x:0 0 0 2024-04-24T23:14:11.164 INFO:tasks.workunit.client.1.smithi040.stdout:8/965: readlink d0/d52/l92 0 2024-04-24T23:14:11.165 INFO:tasks.workunit.client.1.smithi040.stdout:4/910: creat da/d3f/d40/d41/d62/f13c x:0 0 0 2024-04-24T23:14:11.167 INFO:tasks.workunit.client.1.smithi040.stdout:1/933: mknod da/d12/d18/d2a/d4d/d107/c13b 0 2024-04-24T23:14:11.172 INFO:tasks.workunit.client.1.smithi040.stdout:7/969: read db/d2f/d37/d7e/fc3 [1858679,1299] 0 2024-04-24T23:14:11.172 INFO:tasks.workunit.client.1.smithi040.stdout:5/832: getdents d2/d7/d8/d1f 0 2024-04-24T23:14:11.175 INFO:tasks.workunit.client.1.smithi040.stdout:0/983: symlink df/d1e/d20/d21/d41/d8a/df3/df8/d79/daa/l13e 0 2024-04-24T23:14:11.178 INFO:tasks.workunit.client.1.smithi040.stdout:9/936: fsync d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/ff0 0 2024-04-24T23:14:11.184 INFO:tasks.workunit.client.1.smithi040.stdout:8/966: symlink d0/d7e/dc1/d9e/l139 0 2024-04-24T23:14:11.184 INFO:tasks.workunit.client.1.smithi040.stdout:6/905: write d7/d1a/d20/f61 [1654247,119135] 0 2024-04-24T23:14:11.189 INFO:tasks.workunit.client.1.smithi040.stdout:1/934: stat da/d12/d14/d58/fc6 0 2024-04-24T23:14:11.189 INFO:tasks.workunit.client.1.smithi040.stdout:3/945: rmdir d1/d9/d18/d36/d10d/d45/deb 0 2024-04-24T23:14:11.190 INFO:tasks.workunit.client.1.smithi040.stdout:5/833: fsync d2/d2f/d14/f61 0 2024-04-24T23:14:11.194 INFO:tasks.workunit.client.1.smithi040.stdout:7/970: rename db/d12/f3e to db/d12/d62/d8a/db3/dbf/d144/f152 0 2024-04-24T23:14:11.195 INFO:tasks.workunit.client.1.smithi040.stdout:0/984: symlink df/d12/d58/d5b/l13f 0 2024-04-24T23:14:11.200 INFO:tasks.workunit.client.1.smithi040.stdout:9/937: fdatasync d7/da/d17/d40/d55/d2f/f46 0 2024-04-24T23:14:11.201 INFO:tasks.workunit.client.1.smithi040.stdout:7/971: dwrite db/f13e [0,4194304] 0 2024-04-24T23:14:11.208 INFO:tasks.workunit.client.1.smithi040.stdout:1/935: creat da/d35/d38/d47/dbc/de5/f13c x:0 0 0 2024-04-24T23:14:11.209 INFO:tasks.workunit.client.1.smithi040.stdout:1/936: readlink da/d35/d38/d47/dbc/de5/l11f 0 2024-04-24T23:14:11.210 INFO:tasks.workunit.client.1.smithi040.stdout:5/834: symlink d2/d7/d8/d1f/d90/dcb/l101 0 2024-04-24T23:14:11.215 INFO:tasks.workunit.client.1.smithi040.stdout:3/946: rename d1/d2a/d6a/d94/l101 to d1/d9/d18/d36/d55/l145 0 2024-04-24T23:14:11.216 INFO:tasks.workunit.client.1.smithi040.stdout:0/985: dread df/d12/d58/dc3/dcd/f108 [0,4194304] 0 2024-04-24T23:14:11.216 INFO:tasks.workunit.client.1.smithi040.stdout:0/986: dread - df/d1e/d20/d61/f10e zero size 2024-04-24T23:14:11.218 INFO:tasks.workunit.client.1.smithi040.stdout:0/987: write df/d1e/d20/d21/d41/d8a/df3/f127 [647027,82470] 0 2024-04-24T23:14:11.221 INFO:tasks.workunit.client.1.smithi040.stdout:8/967: dread d0/d16/d19/fac [0,4194304] 0 2024-04-24T23:14:11.231 INFO:tasks.workunit.client.1.smithi040.stdout:7/972: unlink db/d2f/d37/fa5 0 2024-04-24T23:14:11.232 INFO:tasks.workunit.client.1.smithi040.stdout:6/906: fdatasync d7/d1a/fee 0 2024-04-24T23:14:11.233 INFO:tasks.workunit.client.1.smithi040.stdout:6/907: chown f2 426 1 2024-04-24T23:14:11.238 INFO:tasks.workunit.client.1.smithi040.stdout:9/938: write d7/d5e/f7d [1158020,19522] 0 2024-04-24T23:14:11.244 INFO:tasks.workunit.client.1.smithi040.stdout:3/947: dread - d1/d9/d18/d36/ffd zero size 2024-04-24T23:14:11.248 INFO:tasks.workunit.client.1.smithi040.stdout:8/968: mknod d0/d8/d22/db8/dc2/c13a 0 2024-04-24T23:14:11.252 INFO:tasks.workunit.client.1.smithi040.stdout:7/973: rename db/d12/d20/d52/d4b/d127/d12b/c12d to db/d12/d62/d103/c153 0 2024-04-24T23:14:11.254 INFO:tasks.workunit.client.1.smithi040.stdout:4/911: getdents da/d3f/d8d/db5 0 2024-04-24T23:14:11.258 INFO:tasks.workunit.client.1.smithi040.stdout:3/948: creat d1/d9/d18/d36/d10d/d45/daa/f146 x:0 0 0 2024-04-24T23:14:11.258 INFO:tasks.workunit.client.1.smithi040.stdout:3/949: chown d1/d9/d18 3610084 1 2024-04-24T23:14:11.259 INFO:tasks.workunit.client.1.smithi040.stdout:0/988: fsync df/f34 0 2024-04-24T23:14:11.260 INFO:tasks.workunit.client.1.smithi040.stdout:0/989: fsync df/d14/d11c/f11f 0 2024-04-24T23:14:11.263 INFO:tasks.workunit.client.1.smithi040.stdout:8/969: rename d0/d16/d19 to d0/d16/d1b/d58/d107/d13b 0 2024-04-24T23:14:11.264 INFO:tasks.workunit.client.1.smithi040.stdout:6/908: getdents d7/d1a/db5/d127 0 2024-04-24T23:14:11.264 INFO:tasks.workunit.client.1.smithi040.stdout:8/970: dread - d0/d8/dc/f8c zero size 2024-04-24T23:14:11.265 INFO:tasks.workunit.client.1.smithi040.stdout:8/971: fsync d0/d8/d22/f73 0 2024-04-24T23:14:11.265 INFO:tasks.workunit.client.1.smithi040.stdout:7/974: symlink db/d12/d20/d52/d4b/d127/d12b/db9/dd4/d101/d121/l154 0 2024-04-24T23:14:11.266 INFO:tasks.workunit.client.1.smithi040.stdout:8/972: write d0/d8/d66/ff5 [3630221,31526] 0 2024-04-24T23:14:11.266 INFO:tasks.workunit.client.1.smithi040.stdout:9/939: symlink d7/da/d17/l13b 0 2024-04-24T23:14:11.269 INFO:tasks.workunit.client.1.smithi040.stdout:1/937: link da/d12/d18/d2a/d41/d8c/l129 da/d35/l13d 0 2024-04-24T23:14:11.270 INFO:tasks.workunit.client.1.smithi040.stdout:9/940: chown d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/ff0 1449 1 2024-04-24T23:14:11.274 INFO:tasks.workunit.client.1.smithi040.stdout:6/909: write d7/d1a/d20/d51/d1e/d46/f98 [5136104,91078] 0 2024-04-24T23:14:11.277 INFO:tasks.workunit.client.1.smithi040.stdout:0/990: dread df/d12/f4c [0,4194304] 0 2024-04-24T23:14:11.279 INFO:tasks.workunit.client.1.smithi040.stdout:0/991: write df/d12/d58/d5b/f125 [173442,92545] 0 2024-04-24T23:14:11.281 INFO:tasks.workunit.client.1.smithi040.stdout:5/835: link d2/d7/d8/d64/d6b/f92 d2/d7/d8/d1f/d57/d27/d3d/f102 0 2024-04-24T23:14:11.282 INFO:tasks.workunit.client.1.smithi040.stdout:8/973: dread d0/d38/faa [0,4194304] 0 2024-04-24T23:14:11.286 INFO:tasks.workunit.client.1.smithi040.stdout:1/938: write da/d12/d23/f30 [2118243,106752] 0 2024-04-24T23:14:11.288 INFO:tasks.workunit.client.1.smithi040.stdout:9/941: rename d7/da/d17/d40/d55/d2f/df7/cd7 to d7/d5e/da1/c13c 0 2024-04-24T23:14:11.288 INFO:tasks.workunit.client.1.smithi040.stdout:6/910: dwrite d7/d1a/d20/d8c/df5/f121 [0,4194304] 0 2024-04-24T23:14:11.290 INFO:tasks.workunit.client.1.smithi040.stdout:4/912: dread da/f37 [0,4194304] 0 2024-04-24T23:14:11.292 INFO:tasks.workunit.client.1.smithi040.stdout:6/911: chown d7/d1a/d20/d8c/l124 42 1 2024-04-24T23:14:11.292 INFO:tasks.workunit.client.1.smithi040.stdout:0/992: dwrite df/d1e/d20/d61/f86 [4194304,4194304] 0 2024-04-24T23:14:11.303 INFO:tasks.workunit.client.1.smithi040.stdout:7/975: write db/d12/d20/d52/d44/f53 [333959,26828] 0 2024-04-24T23:14:11.307 INFO:tasks.workunit.client.1.smithi040.stdout:8/974: symlink d0/d8/da0/l13c 0 2024-04-24T23:14:11.308 INFO:tasks.workunit.client.1.smithi040.stdout:1/939: mknod da/d12/d18/d13a/c13e 0 2024-04-24T23:14:11.308 INFO:tasks.workunit.client.1.smithi040.stdout:1/940: stat da/d12/d23/f118 0 2024-04-24T23:14:11.310 INFO:tasks.workunit.client.1.smithi040.stdout:9/942: creat d7/da/d17/d40/d55/d2f/f13d x:0 0 0 2024-04-24T23:14:11.317 INFO:tasks.workunit.client.1.smithi040.stdout:4/913: dread da/d1f/f9c [0,4194304] 0 2024-04-24T23:14:11.318 INFO:tasks.workunit.client.1.smithi040.stdout:3/950: link d1/d9/d18/d36/d55/l145 d1/d2d/d72/d106/d12e/l147 0 2024-04-24T23:14:11.320 INFO:tasks.workunit.client.1.smithi040.stdout:8/975: dread - d0/d16/d1b/d58/d107/d13b/fc7 zero size 2024-04-24T23:14:11.321 INFO:tasks.workunit.client.1.smithi040.stdout:9/943: rmdir d7/da/d17/d40/d55/d2f/d1f/d31/dff 39 2024-04-24T23:14:11.323 INFO:tasks.workunit.client.1.smithi040.stdout:6/912: symlink d7/d1a/db5/d12a/l12b 0 2024-04-24T23:14:11.325 INFO:tasks.workunit.client.1.smithi040.stdout:7/976: rmdir db/d12/d3f/d7a 39 2024-04-24T23:14:11.326 INFO:tasks.workunit.client.1.smithi040.stdout:4/914: dread - da/d78/f91 zero size 2024-04-24T23:14:11.330 INFO:tasks.workunit.client.1.smithi040.stdout:8/976: mkdir d0/d38/d13d 0 2024-04-24T23:14:11.332 INFO:tasks.workunit.client.1.smithi040.stdout:1/941: dread da/f1f [0,4194304] 0 2024-04-24T23:14:11.333 INFO:tasks.workunit.client.1.smithi040.stdout:0/993: write df/f39 [746151,64062] 0 2024-04-24T23:14:11.343 INFO:tasks.workunit.client.1.smithi040.stdout:8/977: mknod d0/d7e/dc1/d9e/d90/c13e 0 2024-04-24T23:14:11.344 INFO:tasks.workunit.client.1.smithi040.stdout:9/944: symlink d7/d5e/l13e 0 2024-04-24T23:14:11.348 INFO:tasks.workunit.client.1.smithi040.stdout:0/994: fdatasync df/d12/d1a/fad 0 2024-04-24T23:14:11.349 INFO:tasks.workunit.client.1.smithi040.stdout:0/995: write df/dd4/f118 [520419,75738] 0 2024-04-24T23:14:11.350 INFO:tasks.workunit.client.1.smithi040.stdout:7/977: creat db/d12/d14a/f155 x:0 0 0 2024-04-24T23:14:11.351 INFO:tasks.workunit.client.1.smithi040.stdout:4/915: mknod da/d45/d48/d6e/c13d 0 2024-04-24T23:14:11.352 INFO:tasks.workunit.client.1.smithi040.stdout:4/916: write da/d3f/d40/d47/d71/fdf [1781475,70589] 0 2024-04-24T23:14:11.352 INFO:tasks.workunit.client.1.smithi040.stdout:3/951: link d1/d2d/d44/l5b d1/d9/d18/d36/d10d/d13a/l148 0 2024-04-24T23:14:11.353 INFO:tasks.workunit.client.1.smithi040.stdout:6/913: truncate d7/d1a/f42 1642318 0 2024-04-24T23:14:11.354 INFO:tasks.workunit.client.1.smithi040.stdout:5/836: link d2/d7/d8/d1f/d57/c26 d2/d7/d8/d1f/d57/c103 0 2024-04-24T23:14:11.355 INFO:tasks.workunit.client.1.smithi040.stdout:1/942: write da/d12/d18/d2a/d41/f62 [2770579,29665] 0 2024-04-24T23:14:11.358 INFO:tasks.workunit.client.1.smithi040.stdout:0/996: fsync df/d1e/d89/ddf/f113 0 2024-04-24T23:14:11.361 INFO:tasks.workunit.client.1.smithi040.stdout:7/978: rename db/d12/d20/d52/d4b/d127/d12b/db7 to db/d12/d20/d56/d156 0 2024-04-24T23:14:11.369 INFO:tasks.workunit.client.1.smithi040.stdout:8/978: dwrite d0/d8/d66/fc0 [0,4194304] 0 2024-04-24T23:14:11.373 INFO:tasks.workunit.client.1.smithi040.stdout:1/943: mknod da/d12/d14/d58/c13f 0 2024-04-24T23:14:11.378 INFO:tasks.workunit.client.1.smithi040.stdout:5/837: write d2/d7/d8/db/fc9 [141950,66664] 0 2024-04-24T23:14:11.382 INFO:tasks.workunit.client.1.smithi040.stdout:3/952: mkdir d1/d2a/d6a/d149 0 2024-04-24T23:14:11.384 INFO:tasks.workunit.client.1.smithi040.stdout:7/979: dread db/d12/d62/fbd [0,4194304] 0 2024-04-24T23:14:11.384 INFO:tasks.workunit.client.1.smithi040.stdout:4/917: truncate da/f117 3156185 0 2024-04-24T23:14:11.385 INFO:tasks.workunit.client.1.smithi040.stdout:5/838: dwrite d2/d7/d8/f34 [0,4194304] 0 2024-04-24T23:14:11.386 INFO:tasks.workunit.client.1.smithi040.stdout:6/914: mkdir d7/de/d89/d12c 0 2024-04-24T23:14:11.388 INFO:tasks.workunit.client.1.smithi040.stdout:6/915: stat d7/d1a/d20/d51/d2f/cbe 0 2024-04-24T23:14:11.388 INFO:tasks.workunit.client.1.smithi040.stdout:5/839: stat d2/d7/d8/db/da2/c8e 0 2024-04-24T23:14:11.389 INFO:tasks.workunit.client.1.smithi040.stdout:6/916: read - d7/d1a/d9f/fa9 zero size 2024-04-24T23:14:11.389 INFO:tasks.workunit.client.1.smithi040.stdout:9/945: creat d7/d92/f13f x:0 0 0 2024-04-24T23:14:11.392 INFO:tasks.workunit.client.1.smithi040.stdout:1/944: unlink da/d35/d38/d47/dbc/d109/f12c 0 2024-04-24T23:14:11.397 INFO:tasks.workunit.client.1.smithi040.stdout:9/946: dwrite d7/da/d17/d40/d55/d2f/d28/f126 [0,4194304] 0 2024-04-24T23:14:11.401 INFO:tasks.workunit.client.1.smithi040.stdout:4/918: symlink da/d3f/d40/d47/d75/d104/l13e 0 2024-04-24T23:14:11.402 INFO:tasks.workunit.client.1.smithi040.stdout:4/919: dread - da/d3f/d8d/db5/f128 zero size 2024-04-24T23:14:11.402 INFO:tasks.workunit.client.1.smithi040.stdout:8/979: creat d0/d8/d135/f13f x:0 0 0 2024-04-24T23:14:11.403 INFO:tasks.workunit.client.1.smithi040.stdout:4/920: fsync da/d3f/d40/d47/d75/f100 0 2024-04-24T23:14:11.405 INFO:tasks.workunit.client.1.smithi040.stdout:3/953: rename d1/d2d/de/l105 to d1/d9/d18/d36/d10d/d45/d96/l14a 0 2024-04-24T23:14:11.407 INFO:tasks.workunit.client.1.smithi040.stdout:7/980: truncate db/d12/d20/d52/d44/fab 3634473 0 2024-04-24T23:14:11.409 INFO:tasks.workunit.client.1.smithi040.stdout:6/917: mkdir d7/de/d84/d12d 0 2024-04-24T23:14:11.409 INFO:tasks.workunit.client.1.smithi040.stdout:4/921: dread da/d3f/d40/d47/fa4 [0,4194304] 0 2024-04-24T23:14:11.410 INFO:tasks.workunit.client.1.smithi040.stdout:7/981: write db/d12/d62/de7/ffb [775682,112013] 0 2024-04-24T23:14:11.411 INFO:tasks.workunit.client.1.smithi040.stdout:3/954: dwrite d1/d2a/d6a/f8c [0,4194304] 0 2024-04-24T23:14:11.413 INFO:tasks.workunit.client.1.smithi040.stdout:0/997: creat df/d1e/d89/f140 x:0 0 0 2024-04-24T23:14:11.421 INFO:tasks.workunit.client.1.smithi040.stdout:8/980: dread d0/d8/dc/d34/f3c [0,4194304] 0 2024-04-24T23:14:11.433 INFO:tasks.workunit.client.1.smithi040.stdout:1/945: creat da/d12/d18/d2a/d32/d102/f140 x:0 0 0 2024-04-24T23:14:11.435 INFO:tasks.workunit.client.1.smithi040.stdout:4/922: symlink da/d45/d58/d107/l13f 0 2024-04-24T23:14:11.436 INFO:tasks.workunit.client.1.smithi040.stdout:4/923: readlink da/d3f/d40/l87 0 2024-04-24T23:14:11.436 INFO:tasks.workunit.client.1.smithi040.stdout:4/924: write da/f118 [450069,123072] 0 2024-04-24T23:14:11.441 INFO:tasks.workunit.client.1.smithi040.stdout:3/955: dread d1/d2d/d2b/f42 [4194304,4194304] 0 2024-04-24T23:14:11.442 INFO:tasks.workunit.client.1.smithi040.stdout:0/998: dread - df/d1e/da7/d6b/ffd zero size 2024-04-24T23:14:11.448 INFO:tasks.workunit.client.1.smithi040.stdout:1/946: stat da/d12/f42 0 2024-04-24T23:14:11.449 INFO:tasks.workunit.client.1.smithi040.stdout:9/947: write d7/fbe [1035773,53480] 0 2024-04-24T23:14:11.449 INFO:tasks.workunit.client.1.smithi040.stdout:0/999: dwrite df/d14/d11c/f136 [0,4194304] 0 2024-04-24T23:14:11.451 INFO:tasks.workunit.client.1.smithi040.stdout:4/925: dread da/d3f/d40/d41/f67 [0,4194304] 0 2024-04-24T23:14:11.456 INFO:tasks.workunit.client.1.smithi040.stdout:5/840: dwrite d2/d2f/d14/dbe/ff1 [0,4194304] 0 2024-04-24T23:14:11.457 INFO:tasks.workunit.client.1.smithi040.stdout:6/918: symlink d7/d1a/d20/d51/d1e/d46/d69/l12e 0 2024-04-24T23:14:11.458 INFO:tasks.workunit.client.1.smithi040.stdout:5/841: readlink d2/d7/d8/d1f/d90/lf0 0 2024-04-24T23:14:11.459 INFO:tasks.workunit.client.1.smithi040.stdout:3/956: read - d1/d80/ff1 zero size 2024-04-24T23:14:11.464 INFO:tasks.workunit.client.1.smithi040.stdout:7/982: dwrite db/d2f/d37/d7e/d118/d145/f140 [0,4194304] 0 2024-04-24T23:14:11.466 INFO:tasks.workunit.client.1.smithi040.stdout:4/926: mknod da/d45/d58/d107/c140 0 2024-04-24T23:14:11.466 INFO:tasks.workunit.client.1.smithi040.stdout:4/927: fsync da/f18 0 2024-04-24T23:14:11.473 INFO:tasks.workunit.client.1.smithi040.stdout:6/919: rename d7/d1a/d20/d51/d2f/cbe to d7/d1a/dbf/c12f 0 2024-04-24T23:14:11.474 INFO:tasks.workunit.client.1.smithi040.stdout:8/981: truncate d0/d8/dc/f3a 1416647 0 2024-04-24T23:14:11.475 INFO:tasks.workunit.client.1.smithi040.stdout:5/842: mkdir d2/d7/d8/db/da2/dcc/d104 0 2024-04-24T23:14:11.476 INFO:tasks.workunit.client.1.smithi040.stdout:5/843: write d2/d7/d8/d1f/d57/d27/d65/f5a [5675158,27031] 0 2024-04-24T23:14:11.479 INFO:tasks.workunit.client.1.smithi040.stdout:8/982: mkdir d0/d7e/dc1/d140 0 2024-04-24T23:14:11.482 INFO:tasks.workunit.client.1.smithi040.stdout:3/957: truncate d1/d2a/d6a/f3a 428355 0 2024-04-24T23:14:11.485 INFO:tasks.workunit.client.1.smithi040.stdout:7/983: dwrite db/d12/d62/d8a/db3/fed [0,4194304] 0 2024-04-24T23:14:11.486 INFO:tasks.workunit.client.1.smithi040.stdout:8/983: readlink d0/d16/d1b/d47/def/l136 0 2024-04-24T23:14:11.487 INFO:tasks.workunit.client.1.smithi040.stdout:3/958: chown d1/d2d/d72/d106/cfb 1591066 1 2024-04-24T23:14:11.487 INFO:tasks.workunit.client.1.smithi040.stdout:9/948: dwrite d7/d92/fb5 [0,4194304] 0 2024-04-24T23:14:11.487 INFO:tasks.workunit.client.1.smithi040.stdout:7/984: stat db/d12/d20/ff5 0 2024-04-24T23:14:11.489 INFO:tasks.workunit.client.1.smithi040.stdout:8/984: chown d0/d8/d135/f13f 2518074 1 2024-04-24T23:14:11.491 INFO:tasks.workunit.client.1.smithi040.stdout:3/959: write d1/d9/d2e/fac [894859,53534] 0 2024-04-24T23:14:11.492 INFO:tasks.workunit.client.1.smithi040.stdout:6/920: link d7/d1a/d20/d8c/l124 d7/d1a/dbf/l130 0 2024-04-24T23:14:11.494 INFO:tasks.workunit.client.1.smithi040.stdout:1/947: getdents da/d12/d18/d2a/d41/d4f/de9 0 2024-04-24T23:14:11.496 INFO:tasks.workunit.client.1.smithi040.stdout:1/948: stat da/d35/d38/d47 0 2024-04-24T23:14:11.497 INFO:tasks.workunit.client.1.smithi040.stdout:4/928: dread da/d45/d58/f73 [0,4194304] 0 2024-04-24T23:14:11.499 INFO:tasks.workunit.client.1.smithi040.stdout:5/844: dwrite d2/d7/d8/d1f/d57/d27/d3c/da4/fce [0,4194304] 0 2024-04-24T23:14:11.500 INFO:tasks.workunit.client.1.smithi040.stdout:1/949: truncate da/d12/d18/d2a/d8b/f10d 646473 0 2024-04-24T23:14:11.507 INFO:tasks.workunit.client.1.smithi040.stdout:7/985: symlink db/d12/d20/d52/d6c/d8f/l157 0 2024-04-24T23:14:11.509 INFO:tasks.workunit.client.1.smithi040.stdout:9/949: symlink d7/da/d17/d40/d55/d2f/d1f/d31/l140 0 2024-04-24T23:14:11.513 INFO:tasks.workunit.client.1.smithi040.stdout:4/929: rename da/d3f/d40/d41/d8e/dd3/cf7 to da/d3f/d6b/d2b/c141 0 2024-04-24T23:14:11.513 INFO:tasks.workunit.client.1.smithi040.stdout:8/985: symlink d0/d7e/dc1/d140/l141 0 2024-04-24T23:14:11.514 INFO:tasks.workunit.client.1.smithi040.stdout:4/930: chown da/d3f/d40/d41/lb6 79 1 2024-04-24T23:14:11.514 INFO:tasks.workunit.client.1.smithi040.stdout:8/986: dread - d0/d8/d22/db8/dc2/f133 zero size 2024-04-24T23:14:11.515 INFO:tasks.workunit.client.1.smithi040.stdout:1/950: creat da/d12/d18/d2a/d32/d102/f141 x:0 0 0 2024-04-24T23:14:11.515 INFO:tasks.workunit.client.1.smithi040.stdout:4/931: chown da/d78/dbb/d127/cc1 14118 1 2024-04-24T23:14:11.516 INFO:tasks.workunit.client.1.smithi040.stdout:5/845: mknod d2/d7/d8/d1f/d57/d27/d3d/c105 0 2024-04-24T23:14:11.521 INFO:tasks.workunit.client.1.smithi040.stdout:5/846: dwrite d2/d7/d8/d1f/d57/d27/d3c/d43/f6a [0,4194304] 0 2024-04-24T23:14:11.554 INFO:tasks.workunit.client.1.smithi040.stdout:6/921: symlink d7/de/d84/d12d/l131 0 2024-04-24T23:14:11.555 INFO:tasks.workunit.client.1.smithi040.stdout:7/986: dread - db/d12/d20/d56/f11d zero size 2024-04-24T23:14:11.558 INFO:tasks.workunit.client.1.smithi040.stdout:9/950: mknod d7/d5e/da1/dfa/c141 0 2024-04-24T23:14:11.559 INFO:tasks.workunit.client.1.smithi040.stdout:1/951: rmdir da/d35 39 2024-04-24T23:14:11.562 INFO:tasks.workunit.client.1.smithi040.stdout:4/932: chown da/d78/dbb/fc2 5391177 1 2024-04-24T23:14:11.562 INFO:tasks.workunit.client.1.smithi040.stdout:4/933: chown da/d3f/d6b/f137 142044859 1 2024-04-24T23:14:11.562 INFO:tasks.workunit.client.1.smithi040.stdout:4/934: chown da/d3f/d40/d47/d75/d7c/cda 4366 1 2024-04-24T23:14:11.564 INFO:tasks.workunit.client.1.smithi040.stdout:5/847: creat d2/d7/d8/db/d1e/f106 x:0 0 0 2024-04-24T23:14:11.572 INFO:tasks.workunit.client.1.smithi040.stdout:9/951: mknod d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/c142 0 2024-04-24T23:14:11.580 INFO:tasks.workunit.client.1.smithi040.stdout:5/848: creat d2/d7/d8/d1f/da1/f107 x:0 0 0 2024-04-24T23:14:11.584 INFO:tasks.workunit.client.1.smithi040.stdout:6/922: link d7/de/d84/f91 d7/d1a/d9f/df6/f132 0 2024-04-24T23:14:11.585 INFO:tasks.workunit.client.1.smithi040.stdout:6/923: fsync d7/d1a/d20/d51/d1e/d46/d69/f101 0 2024-04-24T23:14:11.587 INFO:tasks.workunit.client.1.smithi040.stdout:9/952: mkdir d7/d5e/de5/d143 0 2024-04-24T23:14:11.588 INFO:tasks.workunit.client.1.smithi040.stdout:1/952: creat da/d35/d38/d47/dbc/d109/f142 x:0 0 0 2024-04-24T23:14:11.589 INFO:tasks.workunit.client.1.smithi040.stdout:1/953: readlink da/d12/d18/d2a/d41/d4f/l82 0 2024-04-24T23:14:11.590 INFO:tasks.workunit.client.1.smithi040.stdout:8/987: link d0/d38/c46 d0/d7e/dc1/d9e/d90/d130/c142 0 2024-04-24T23:14:11.598 INFO:tasks.workunit.client.1.smithi040.stdout:1/954: creat da/d12/d18/d2a/d8b/dd3/f143 x:0 0 0 2024-04-24T23:14:11.600 INFO:tasks.workunit.client.1.smithi040.stdout:6/924: dread d7/fca [0,4194304] 0 2024-04-24T23:14:11.601 INFO:tasks.workunit.client.1.smithi040.stdout:9/953: rename d7/da/d17/d40/d55/d2f/d1f/d31/d3b/f6d to d7/da/d17/d40/f144 0 2024-04-24T23:14:11.601 INFO:tasks.workunit.client.1.smithi040.stdout:3/960: dwrite d1/d2d/f126 [0,4194304] 0 2024-04-24T23:14:11.606 INFO:tasks.workunit.client.1.smithi040.stdout:6/925: rmdir d7/d1a/d20/d51/d1e 39 2024-04-24T23:14:11.615 INFO:tasks.workunit.client.1.smithi040.stdout:6/926: creat d7/d1a/db5/d127/f133 x:0 0 0 2024-04-24T23:14:11.616 INFO:tasks.workunit.client.1.smithi040.stdout:4/935: write da/d3f/d6b/d2b/faf [220914,109388] 0 2024-04-24T23:14:11.618 INFO:tasks.workunit.client.1.smithi040.stdout:7/987: dwrite db/d12/d20/d52/d4b/d127/d12b/ff9 [0,4194304] 0 2024-04-24T23:14:11.620 INFO:tasks.workunit.client.1.smithi040.stdout:4/936: write da/d3f/d40/d41/d62/f13c [116529,106350] 0 2024-04-24T23:14:11.621 INFO:tasks.workunit.client.1.smithi040.stdout:9/954: write d7/d92/d9c/fe6 [627666,145] 0 2024-04-24T23:14:11.625 INFO:tasks.workunit.client.1.smithi040.stdout:4/937: chown da/d78/dbb/fc2 283149 1 2024-04-24T23:14:11.625 INFO:tasks.workunit.client.1.smithi040.stdout:9/955: fdatasync d7/da/d17/d40/d55/d2f/df7/f138 0 2024-04-24T23:14:11.626 INFO:tasks.workunit.client.1.smithi040.stdout:6/927: truncate d7/d1a/d20/d8c/db0/d125/f126 221144 0 2024-04-24T23:14:11.626 INFO:tasks.workunit.client.1.smithi040.stdout:4/938: read - da/d3f/d40/d47/d75/d104/f121 zero size 2024-04-24T23:14:11.627 INFO:tasks.workunit.client.1.smithi040.stdout:9/956: fsync d7/da/d17/d40/d55/d2f/df7/d6c/fa8 0 2024-04-24T23:14:11.633 INFO:tasks.workunit.client.1.smithi040.stdout:1/955: dread da/d12/d18/d2a/d41/f62 [0,4194304] 0 2024-04-24T23:14:11.634 INFO:tasks.workunit.client.1.smithi040.stdout:7/988: dwrite db/d12/d3f/fa4 [0,4194304] 0 2024-04-24T23:14:11.635 INFO:tasks.workunit.client.1.smithi040.stdout:7/989: stat db/d12/d20/d52/d44/f47 0 2024-04-24T23:14:11.640 INFO:tasks.workunit.client.1.smithi040.stdout:7/990: dread db/d12/d62/de7/f110 [0,4194304] 0 2024-04-24T23:14:11.645 INFO:tasks.workunit.client.1.smithi040.stdout:4/939: dwrite da/d3f/d6b/d2b/f52 [4194304,4194304] 0 2024-04-24T23:14:11.652 INFO:tasks.workunit.client.1.smithi040.stdout:7/991: mknod db/d12/d62/c158 0 2024-04-24T23:14:11.654 INFO:tasks.workunit.client.1.smithi040.stdout:7/992: fdatasync db/d12/f8c 0 2024-04-24T23:14:11.655 INFO:tasks.workunit.client.1.smithi040.stdout:6/928: dwrite d7/fae [0,4194304] 0 2024-04-24T23:14:11.660 INFO:tasks.workunit.client.1.smithi040.stdout:7/993: rename db/d12/d3f/d7a/l143 to db/d12/d14a/l159 0 2024-04-24T23:14:11.663 INFO:tasks.workunit.client.1.smithi040.stdout:7/994: creat db/d12/d20/d52/dfc/f15a x:0 0 0 2024-04-24T23:14:11.667 INFO:tasks.workunit.client.1.smithi040.stdout:9/957: dread d7/da/d17/d40/d55/d2f/fd2 [0,4194304] 0 2024-04-24T23:14:11.672 INFO:tasks.workunit.client.1.smithi040.stdout:4/940: write da/d45/d48/d6e/d70/fee [1001155,47523] 0 2024-04-24T23:14:11.674 INFO:tasks.workunit.client.1.smithi040.stdout:1/956: dread da/fd9 [0,4194304] 0 2024-04-24T23:14:11.674 INFO:tasks.workunit.client.1.smithi040.stdout:7/995: link db/d12/d20/d52/d4b/d127/d12b/db9/dd4/f106 db/d12/d62/d8a/db3/dbf/d144/f15b 0 2024-04-24T23:14:11.675 INFO:tasks.workunit.client.1.smithi040.stdout:1/957: stat da/d35/d38/d47/dbc 0 2024-04-24T23:14:11.675 INFO:tasks.workunit.client.1.smithi040.stdout:1/958: stat da/d12/d18/d2a/d41/d8c 0 2024-04-24T23:14:11.676 INFO:tasks.workunit.client.1.smithi040.stdout:6/929: write d7/d1a/d20/d51/d1e/d46/d69/d87/fad [310002,26460] 0 2024-04-24T23:14:11.678 INFO:tasks.workunit.client.1.smithi040.stdout:4/941: creat da/d3f/d6b/d3c/d69/df3/f142 x:0 0 0 2024-04-24T23:14:11.679 INFO:tasks.workunit.client.1.smithi040.stdout:7/996: rmdir db/d2f/d37 39 2024-04-24T23:14:11.684 INFO:tasks.workunit.client.1.smithi040.stdout:1/959: creat da/d35/d38/d47/dbc/de5/f144 x:0 0 0 2024-04-24T23:14:11.685 INFO:tasks.workunit.client.1.smithi040.stdout:4/942: dwrite da/d45/d48/d6e/d70/fa8 [8388608,4194304] 0 2024-04-24T23:14:11.687 INFO:tasks.workunit.client.1.smithi040.stdout:6/930: symlink d7/d1a/d20/d8c/l134 0 2024-04-24T23:14:11.689 INFO:tasks.workunit.client.1.smithi040.stdout:7/997: dwrite db/d12/d3f/d7a/f134 [0,4194304] 0 2024-04-24T23:14:11.691 INFO:tasks.workunit.client.1.smithi040.stdout:4/943: stat da/d45/d48/d6e/d70/f72 0 2024-04-24T23:14:11.692 INFO:tasks.workunit.client.1.smithi040.stdout:9/958: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/f78 [4194304,4194304] 0 2024-04-24T23:14:11.695 INFO:tasks.workunit.client.1.smithi040.stdout:7/998: stat db/d12/d20/d56/l67 0 2024-04-24T23:14:11.697 INFO:tasks.workunit.client.1.smithi040.stdout:7/999: write db/d12/d20/d52/d4b/d127/d12b/db9/f14f [980909,44848] 0 2024-04-24T23:14:11.705 INFO:tasks.workunit.client.1.smithi040.stdout:4/944: creat da/d3f/d6b/d3c/f143 x:0 0 0 2024-04-24T23:14:11.705 INFO:tasks.workunit.client.1.smithi040.stdout:4/945: readlink da/d3f/d40/d41/d60/l123 0 2024-04-24T23:14:11.707 INFO:tasks.workunit.client.1.smithi040.stdout:4/946: write da/d45/d48/d6e/ddd/f139 [411019,39904] 0 2024-04-24T23:14:11.707 INFO:tasks.workunit.client.1.smithi040.stdout:1/960: getdents da/d35/d38/d47/dbc/de5/df5 0 2024-04-24T23:14:11.710 INFO:tasks.workunit.client.1.smithi040.stdout:4/947: creat da/d78/dbb/f144 x:0 0 0 2024-04-24T23:14:11.712 INFO:tasks.workunit.client.1.smithi040.stdout:9/959: link d7/da/l53 d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/l145 0 2024-04-24T23:14:11.717 INFO:tasks.workunit.client.1.smithi040.stdout:9/960: stat d7/da/d17/d40/c58 0 2024-04-24T23:14:11.718 INFO:tasks.workunit.client.1.smithi040.stdout:4/948: dwrite da/d3f/d40/d41/d8e/ffb [0,4194304] 0 2024-04-24T23:14:11.721 INFO:tasks.workunit.client.1.smithi040.stdout:1/961: dwrite da/d12/d18/d13a/f11d [8388608,4194304] 0 2024-04-24T23:14:11.723 INFO:tasks.workunit.client.1.smithi040.stdout:6/931: dread d7/d1a/d20/d51/d1e/d46/fac [0,4194304] 0 2024-04-24T23:14:11.724 INFO:tasks.workunit.client.1.smithi040.stdout:4/949: symlink da/d3f/d40/d41/d8e/db8/l145 0 2024-04-24T23:14:11.725 INFO:tasks.workunit.client.1.smithi040.stdout:4/950: read da/d3f/d40/d41/d62/f13c [187131,55750] 0 2024-04-24T23:14:11.727 INFO:tasks.workunit.client.1.smithi040.stdout:4/951: truncate da/d3f/d6b/d3c/f90 1084443 0 2024-04-24T23:14:11.728 INFO:tasks.workunit.client.1.smithi040.stdout:1/962: mkdir da/d35/dec/d145 0 2024-04-24T23:14:11.729 INFO:tasks.workunit.client.1.smithi040.stdout:4/952: chown da/d78/dbb/fe8 18 1 2024-04-24T23:14:11.735 INFO:tasks.workunit.client.1.smithi040.stdout:5/849: sync 2024-04-24T23:14:11.736 INFO:tasks.workunit.client.1.smithi040.stdout:4/953: creat da/d3f/d40/d41/d8e/d116/f146 x:0 0 0 2024-04-24T23:14:11.737 INFO:tasks.workunit.client.1.smithi040.stdout:9/961: mkdir d7/da/d17/d40/d55/d2f/ded/d146 0 2024-04-24T23:14:11.740 INFO:tasks.workunit.client.1.smithi040.stdout:5/850: write d2/d7/d8/d1f/d57/f44 [2349820,36524] 0 2024-04-24T23:14:11.741 INFO:tasks.workunit.client.1.smithi040.stdout:4/954: rmdir da/d3f/d40/d47/d75/de4 39 2024-04-24T23:14:11.742 INFO:tasks.workunit.client.1.smithi040.stdout:9/962: getdents d7/d5e/de5/d143 0 2024-04-24T23:14:11.743 INFO:tasks.workunit.client.1.smithi040.stdout:9/963: chown d7/da/d17/d120 27848881 1 2024-04-24T23:14:11.743 INFO:tasks.workunit.client.1.smithi040.stdout:4/955: symlink da/d3f/d8d/db5/l147 0 2024-04-24T23:14:11.746 INFO:tasks.workunit.client.1.smithi040.stdout:6/932: dread d7/d1a/d9f/fb4 [0,4194304] 0 2024-04-24T23:14:11.746 INFO:tasks.workunit.client.1.smithi040.stdout:5/851: getdents d2/d7/d8 0 2024-04-24T23:14:11.749 INFO:tasks.workunit.client.1.smithi040.stdout:4/956: symlink da/d3f/d40/d47/d75/d132/l148 0 2024-04-24T23:14:11.751 INFO:tasks.workunit.client.1.smithi040.stdout:9/964: mkdir d7/da/d17/d40/d55/d2f/df7/d12f/d147 0 2024-04-24T23:14:11.751 INFO:tasks.workunit.client.1.smithi040.stdout:9/965: stat d7/d5e/cb1 0 2024-04-24T23:14:11.761 INFO:tasks.workunit.client.1.smithi040.stdout:4/957: creat da/d45/d48/d6e/ddd/dea/f149 x:0 0 0 2024-04-24T23:14:11.762 INFO:tasks.workunit.client.1.smithi040.stdout:1/963: write da/d12/d55/fc4 [38189,5185] 0 2024-04-24T23:14:11.771 INFO:tasks.workunit.client.1.smithi040.stdout:1/964: mkdir da/d12/d14/d58/ded/d146 0 2024-04-24T23:14:11.772 INFO:tasks.workunit.client.1.smithi040.stdout:9/966: truncate d7/da/d17/d40/d55/f115 806807 0 2024-04-24T23:14:11.773 INFO:tasks.workunit.client.1.smithi040.stdout:6/933: dwrite d7/d1a/d9f/fa9 [0,4194304] 0 2024-04-24T23:14:11.773 INFO:tasks.workunit.client.1.smithi040.stdout:5/852: getdents d2/d7/d8/d1f/d57/d27 0 2024-04-24T23:14:11.778 INFO:tasks.workunit.client.1.smithi040.stdout:6/934: read - d7/d1a/d20/d51/d1e/d46/f112 zero size 2024-04-24T23:14:11.778 INFO:tasks.workunit.client.1.smithi040.stdout:4/958: write da/d3f/d40/d41/d62/f6f [210560,24638] 0 2024-04-24T23:14:11.778 INFO:tasks.workunit.client.1.smithi040.stdout:5/853: read d2/d7/d8/d1f/d57/d27/d3c/d43/f6f [1027556,1251] 0 2024-04-24T23:14:11.782 INFO:tasks.workunit.client.1.smithi040.stdout:1/965: rename da/d12/d18/d2a/f9c to da/d12/d18/d2a/d41/d4f/de9/d128/f147 0 2024-04-24T23:14:11.782 INFO:tasks.workunit.client.1.smithi040.stdout:6/935: dread - d7/d1a/fd8 zero size 2024-04-24T23:14:11.786 INFO:tasks.workunit.client.1.smithi040.stdout:3/961: sync 2024-04-24T23:14:11.786 INFO:tasks.workunit.client.1.smithi040.stdout:8/988: sync 2024-04-24T23:14:11.787 INFO:tasks.workunit.client.1.smithi040.stdout:1/966: symlink da/d12/d18/d2a/d32/db3/l148 0 2024-04-24T23:14:11.787 INFO:tasks.workunit.client.1.smithi040.stdout:6/936: fsync d7/d1a/d9f/df6/d110/f116 0 2024-04-24T23:14:11.789 INFO:tasks.workunit.client.1.smithi040.stdout:8/989: write d0/d52/d9b/d101/d113/fcd [690369,114903] 0 2024-04-24T23:14:11.790 INFO:tasks.workunit.client.1.smithi040.stdout:6/937: readlink d7/d1a/d20/d8c/db0/lfe 0 2024-04-24T23:14:11.792 INFO:tasks.workunit.client.1.smithi040.stdout:3/962: mknod d1/d2d/de/d30/d37/d107/c14b 0 2024-04-24T23:14:11.797 INFO:tasks.workunit.client.1.smithi040.stdout:9/967: dread d7/da/d17/d40/d55/d2f/fca [0,4194304] 0 2024-04-24T23:14:11.798 INFO:tasks.workunit.client.1.smithi040.stdout:1/967: mkdir da/d35/d38/d47/dbc/de5/df5/d12e/d149 0 2024-04-24T23:14:11.800 INFO:tasks.workunit.client.1.smithi040.stdout:9/968: read d7/fbe [1001411,36805] 0 2024-04-24T23:14:11.801 INFO:tasks.workunit.client.1.smithi040.stdout:1/968: write da/d12/d18/d2a/d32/d102/f141 [1034581,104545] 0 2024-04-24T23:14:11.802 INFO:tasks.workunit.client.1.smithi040.stdout:6/938: mknod d7/d1a/d20/d51/d1e/d46/d69/c135 0 2024-04-24T23:14:11.804 INFO:tasks.workunit.client.1.smithi040.stdout:4/959: dwrite da/d45/fac [0,4194304] 0 2024-04-24T23:14:11.806 INFO:tasks.workunit.client.1.smithi040.stdout:3/963: mkdir d1/d2d/de/d30/dde/d14c 0 2024-04-24T23:14:11.806 INFO:tasks.workunit.client.1.smithi040.stdout:1/969: chown da/d12/d18/d2a/d41/d127/fe3 4 1 2024-04-24T23:14:11.806 INFO:tasks.workunit.client.1.smithi040.stdout:5/854: dread d2/d2f/d14/f81 [0,4194304] 0 2024-04-24T23:14:11.806 INFO:tasks.workunit.client.1.smithi040.stdout:4/960: chown da/d45 0 1 2024-04-24T23:14:11.806 INFO:tasks.workunit.client.1.smithi040.stdout:6/939: chown d7/d1a/d20/d51/d1e/d46/d69/f8d 251 1 2024-04-24T23:14:11.808 INFO:tasks.workunit.client.1.smithi040.stdout:5/855: write d2/d7/d8/db/fc9 [1170070,93694] 0 2024-04-24T23:14:11.809 INFO:tasks.workunit.client.1.smithi040.stdout:8/990: rename d0/d16/d1b/d58/d107/d13b/feb to d0/f143 0 2024-04-24T23:14:11.813 INFO:tasks.workunit.client.1.smithi040.stdout:9/969: creat d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/f148 x:0 0 0 2024-04-24T23:14:11.813 INFO:tasks.workunit.client.1.smithi040.stdout:9/970: read d7/d92/fb5 [3953237,130092] 0 2024-04-24T23:14:11.830 INFO:tasks.workunit.client.1.smithi040.stdout:8/991: symlink d0/d8/dc/l144 0 2024-04-24T23:14:11.831 INFO:tasks.workunit.client.1.smithi040.stdout:6/940: read d7/d1a/d20/d51/d2f/f5c [553520,120495] 0 2024-04-24T23:14:11.832 INFO:tasks.workunit.client.1.smithi040.stdout:1/970: creat da/d35/d38/d47/dbc/de5/df5/d12e/d149/f14a x:0 0 0 2024-04-24T23:14:11.833 INFO:tasks.workunit.client.1.smithi040.stdout:4/961: mknod da/d3f/d40/d47/d75/de4/c14a 0 2024-04-24T23:14:11.834 INFO:tasks.workunit.client.1.smithi040.stdout:4/962: fsync da/f18 0 2024-04-24T23:14:11.843 INFO:tasks.workunit.client.1.smithi040.stdout:9/971: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/d12d/d149 0 2024-04-24T23:14:11.845 INFO:tasks.workunit.client.1.smithi040.stdout:9/972: write d7/da/d17/d40/f106 [180987,81508] 0 2024-04-24T23:14:11.847 INFO:tasks.workunit.client.1.smithi040.stdout:1/971: mkdir da/d12/d18/d13a/d11e/d14b 0 2024-04-24T23:14:11.847 INFO:tasks.workunit.client.1.smithi040.stdout:6/941: dread d7/d1a/d20/d8c/fc6 [0,4194304] 0 2024-04-24T23:14:11.848 INFO:tasks.workunit.client.1.smithi040.stdout:1/972: write da/d12/d14/fe0 [523856,11857] 0 2024-04-24T23:14:11.851 INFO:tasks.workunit.client.1.smithi040.stdout:5/856: truncate d2/d7/d8/d1f/d57/d27/d65/f5a 2894810 0 2024-04-24T23:14:11.852 INFO:tasks.workunit.client.1.smithi040.stdout:5/857: write d2/d7/d8/d1f/d90/dcb/ffd [641226,104619] 0 2024-04-24T23:14:11.852 INFO:tasks.workunit.client.1.smithi040.stdout:9/973: symlink d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/d128/l14a 0 2024-04-24T23:14:11.854 INFO:tasks.workunit.client.1.smithi040.stdout:6/942: truncate d7/d1a/fc0 86458 0 2024-04-24T23:14:11.856 INFO:tasks.workunit.client.1.smithi040.stdout:1/973: unlink da/d12/d18/d13a/c13e 0 2024-04-24T23:14:11.857 INFO:tasks.workunit.client.1.smithi040.stdout:1/974: dread - da/d12/d18/d2a/d8b/d97/f98 zero size 2024-04-24T23:14:11.861 INFO:tasks.workunit.client.1.smithi040.stdout:6/943: truncate d7/de/d4e/fa6 3035240 0 2024-04-24T23:14:11.864 INFO:tasks.workunit.client.1.smithi040.stdout:1/975: dwrite da/d12/d18/d2a/d32/f115 [0,4194304] 0 2024-04-24T23:14:11.869 INFO:tasks.workunit.client.1.smithi040.stdout:5/858: dread d2/d7/d8/db/fde [0,4194304] 0 2024-04-24T23:14:11.872 INFO:tasks.workunit.client.1.smithi040.stdout:6/944: rmdir d7/d1a/d20/d51/d1e/d46/d69 39 2024-04-24T23:14:11.876 INFO:tasks.workunit.client.1.smithi040.stdout:5/859: mknod d2/d7/d8/d1f/d31/c108 0 2024-04-24T23:14:11.877 INFO:tasks.workunit.client.1.smithi040.stdout:6/945: creat d7/d1a/db5/f136 x:0 0 0 2024-04-24T23:14:11.881 INFO:tasks.workunit.client.1.smithi040.stdout:1/976: dwrite da/d12/d18/d2a/fc7 [4194304,4194304] 0 2024-04-24T23:14:11.883 INFO:tasks.workunit.client.1.smithi040.stdout:9/974: dwrite d7/d5e/fd5 [4194304,4194304] 0 2024-04-24T23:14:11.885 INFO:tasks.workunit.client.1.smithi040.stdout:6/946: readlink d7/d1a/d20/d8c/l124 0 2024-04-24T23:14:11.885 INFO:tasks.workunit.client.1.smithi040.stdout:5/860: dread d2/d2f/d14/f61 [0,4194304] 0 2024-04-24T23:14:11.887 INFO:tasks.workunit.client.1.smithi040.stdout:5/861: write d2/d62/fb2 [2582149,114304] 0 2024-04-24T23:14:11.892 INFO:tasks.workunit.client.1.smithi040.stdout:1/977: mknod da/d12/d18/d2a/d41/d4f/de9/c14c 0 2024-04-24T23:14:11.903 INFO:tasks.workunit.client.1.smithi040.stdout:9/975: symlink d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/ddf/l14b 0 2024-04-24T23:14:11.905 INFO:tasks.workunit.client.1.smithi040.stdout:5/862: creat d2/d7/d8/db/da2/dcc/d104/f109 x:0 0 0 2024-04-24T23:14:11.906 INFO:tasks.workunit.client.1.smithi040.stdout:9/976: rmdir d7/da/d17/d120 39 2024-04-24T23:14:11.906 INFO:tasks.workunit.client.1.smithi040.stdout:5/863: read d2/d7/d8/d1f/d57/d27/d3d/d99/fa6 [34888,103442] 0 2024-04-24T23:14:11.908 INFO:tasks.workunit.client.1.smithi040.stdout:6/947: creat d7/d1a/d20/d51/d1e/d46/d69/f137 x:0 0 0 2024-04-24T23:14:11.909 INFO:tasks.workunit.client.1.smithi040.stdout:9/977: mknod d7/da/d17/d40/d55/d2f/d28/d67/c14c 0 2024-04-24T23:14:11.910 INFO:tasks.workunit.client.1.smithi040.stdout:9/978: fsync d7/d92/d9c/fe6 0 2024-04-24T23:14:11.910 INFO:tasks.workunit.client.1.smithi040.stdout:5/864: creat d2/d2f/d14/ddf/f10a x:0 0 0 2024-04-24T23:14:11.917 INFO:tasks.workunit.client.1.smithi040.stdout:5/865: unlink d2/d7/d8/db/l7e 0 2024-04-24T23:14:11.917 INFO:tasks.workunit.client.1.smithi040.stdout:1/978: write da/d12/d14/f3f [1832464,19015] 0 2024-04-24T23:14:11.921 INFO:tasks.workunit.client.1.smithi040.stdout:6/948: creat d7/d1a/d20/d51/d2f/dba/dde/d10c/f138 x:0 0 0 2024-04-24T23:14:11.923 INFO:tasks.workunit.client.1.smithi040.stdout:5/866: mkdir d2/d7/d8/d1f/d57/d27/d65/d10b 0 2024-04-24T23:14:11.927 INFO:tasks.workunit.client.1.smithi040.stdout:9/979: creat d7/da/d17/d40/d55/d2f/d1f/d4e/f14d x:0 0 0 2024-04-24T23:14:11.927 INFO:tasks.workunit.client.1.smithi040.stdout:6/949: symlink d7/d1a/d20/d8c/db0/l139 0 2024-04-24T23:14:11.928 INFO:tasks.workunit.client.1.smithi040.stdout:3/964: sync 2024-04-24T23:14:11.928 INFO:tasks.workunit.client.1.smithi040.stdout:5/867: rmdir d2/d7/d8/db/da2 39 2024-04-24T23:14:11.931 INFO:tasks.workunit.client.1.smithi040.stdout:3/965: dread d1/d9/d18/da7/fad [0,4194304] 0 2024-04-24T23:14:11.938 INFO:tasks.workunit.client.1.smithi040.stdout:5/868: mkdir d2/d7/d8/db/d1e/d10c 0 2024-04-24T23:14:11.942 INFO:tasks.workunit.client.1.smithi040.stdout:9/980: mkdir d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/d10b/d14e 0 2024-04-24T23:14:11.943 INFO:tasks.workunit.client.1.smithi040.stdout:1/979: dwrite da/d12/d18/d2a/d4d/d107/d5e/ffb [0,4194304] 0 2024-04-24T23:14:11.946 INFO:tasks.workunit.client.1.smithi040.stdout:9/981: readlink d7/d5e/da1/lac 0 2024-04-24T23:14:11.947 INFO:tasks.workunit.client.1.smithi040.stdout:6/950: dwrite f2 [0,4194304] 0 2024-04-24T23:14:11.950 INFO:tasks.workunit.client.1.smithi040.stdout:5/869: mkdir d2/d7/d8/db/da2/d10d 0 2024-04-24T23:14:11.951 INFO:tasks.workunit.client.1.smithi040.stdout:3/966: dwrite d1/fd [4194304,4194304] 0 2024-04-24T23:14:11.953 INFO:tasks.workunit.client.1.smithi040.stdout:8/992: sync 2024-04-24T23:14:11.955 INFO:tasks.workunit.client.1.smithi040.stdout:1/980: mknod da/d12/d14/d58/d123/c14d 0 2024-04-24T23:14:11.958 INFO:tasks.workunit.client.1.smithi040.stdout:5/870: dwrite d2/d7/d8/d1f/d57/d27/d3d/fd4 [0,4194304] 0 2024-04-24T23:14:11.962 INFO:tasks.workunit.client.1.smithi040.stdout:9/982: rename d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4/ddf to d7/da/d17/d40/d14f 0 2024-04-24T23:14:11.963 INFO:tasks.workunit.client.1.smithi040.stdout:6/951: creat d7/de/d89/d12c/f13a x:0 0 0 2024-04-24T23:14:11.965 INFO:tasks.workunit.client.1.smithi040.stdout:8/993: mknod d0/d16/d1b/d3f/d44/c145 0 2024-04-24T23:14:11.966 INFO:tasks.workunit.client.1.smithi040.stdout:5/871: symlink d2/d7/d8/d64/d6b/l10e 0 2024-04-24T23:14:11.967 INFO:tasks.workunit.client.1.smithi040.stdout:5/872: readlink d2/d7/d8/d1f/da1/lad 0 2024-04-24T23:14:11.973 INFO:tasks.workunit.client.1.smithi040.stdout:9/983: readlink d7/da/d17/d40/d55/d2f/d1f/d4e/la6 0 2024-04-24T23:14:11.974 INFO:tasks.workunit.client.1.smithi040.stdout:8/994: truncate d0/d16/d1b/d58/d107/d13b/f9a 377035 0 2024-04-24T23:14:11.976 INFO:tasks.workunit.client.1.smithi040.stdout:1/981: rename da/d12/d23/c108 to da/d35/d38/d47/dbc/de5/df5/d12e/c14e 0 2024-04-24T23:14:11.977 INFO:tasks.workunit.client.1.smithi040.stdout:3/967: dwrite d1/d80/fcc [0,4194304] 0 2024-04-24T23:14:11.982 INFO:tasks.workunit.client.1.smithi040.stdout:9/984: readlink d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/l145 0 2024-04-24T23:14:11.986 INFO:tasks.workunit.client.1.smithi040.stdout:9/985: dwrite d7/da/d17/d40/d55/d2f/d1f/fc5 [0,4194304] 0 2024-04-24T23:14:11.986 INFO:tasks.workunit.client.1.smithi040.stdout:6/952: creat d7/d1a/d20/d51/f13b x:0 0 0 2024-04-24T23:14:11.988 INFO:tasks.workunit.client.1.smithi040.stdout:9/986: stat d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/db8/dd4 0 2024-04-24T23:14:11.989 INFO:tasks.workunit.client.1.smithi040.stdout:1/982: mknod da/d12/d18/d2a/d4d/d7d/c14f 0 2024-04-24T23:14:11.992 INFO:tasks.workunit.client.1.smithi040.stdout:8/995: mkdir d0/d146 0 2024-04-24T23:14:11.993 INFO:tasks.workunit.client.1.smithi040.stdout:6/953: creat d7/d1a/d20/d51/d2f/dba/dde/f13c x:0 0 0 2024-04-24T23:14:11.995 INFO:tasks.workunit.client.1.smithi040.stdout:1/983: creat da/d12/d18/d2a/d41/d4f/de9/f150 x:0 0 0 2024-04-24T23:14:11.996 INFO:tasks.workunit.client.1.smithi040.stdout:1/984: write da/d12/d18/d2a/d32/d102/f141 [463351,103234] 0 2024-04-24T23:14:11.997 INFO:tasks.workunit.client.1.smithi040.stdout:1/985: readlink da/d35/d38/d47/dbc/l133 0 2024-04-24T23:14:11.997 INFO:tasks.workunit.client.1.smithi040.stdout:9/987: symlink d7/da/d17/d40/d55/d2f/d1f/d4e/d83/dbc/d12d/l150 0 2024-04-24T23:14:12.011 INFO:tasks.workunit.client.1.smithi040.stdout:5/873: dwrite d2/d7/d8/d1f/d57/d27/d3c/d9b/fd5 [0,4194304] 0 2024-04-24T23:14:12.016 INFO:tasks.workunit.client.1.smithi040.stdout:8/996: link d0/d8/d22/f6a d0/d8/d22/f147 0 2024-04-24T23:14:12.021 INFO:tasks.workunit.client.1.smithi040.stdout:5/874: creat d2/d7/d8/d1f/d57/d27/f10f x:0 0 0 2024-04-24T23:14:12.024 INFO:tasks.workunit.client.1.smithi040.stdout:6/954: getdents d7/d1a/d20/d8c/db0 0 2024-04-24T23:14:12.032 INFO:tasks.workunit.client.1.smithi040.stdout:6/955: truncate d7/d1a/d20/d51/d2f/f35 2590114 0 2024-04-24T23:14:12.033 INFO:tasks.workunit.client.1.smithi040.stdout:8/997: fsync d0/d8/fc3 0 2024-04-24T23:14:12.036 INFO:tasks.workunit.client.1.smithi040.stdout:6/956: creat d7/de/d89/d12c/f13d x:0 0 0 2024-04-24T23:14:12.037 INFO:tasks.workunit.client.1.smithi040.stdout:9/988: dwrite d7/da/d17/d40/d55/d2f/df7/d6c/d8c/fd6 [0,4194304] 0 2024-04-24T23:14:12.047 INFO:tasks.workunit.client.1.smithi040.stdout:6/957: creat d7/d1a/dbf/f13e x:0 0 0 2024-04-24T23:14:12.049 INFO:tasks.workunit.client.1.smithi040.stdout:9/989: mknod d7/d5e/de5/d143/c151 0 2024-04-24T23:14:12.051 INFO:tasks.workunit.client.1.smithi040.stdout:9/990: write d7/da/d17/d40/d55/d2f/f13d [387896,70974] 0 2024-04-24T23:14:12.051 INFO:tasks.workunit.client.1.smithi040.stdout:6/958: creat d7/d1a/d20/d51/d1e/d46/d69/d87/f13f x:0 0 0 2024-04-24T23:14:12.053 INFO:tasks.workunit.client.1.smithi040.stdout:4/963: sync 2024-04-24T23:14:12.054 INFO:tasks.workunit.client.1.smithi040.stdout:4/964: stat da/d3f/d40/d47/d75/d132/l148 0 2024-04-24T23:14:12.068 INFO:tasks.workunit.client.1.smithi040.stdout:8/998: truncate d0/d8/d22/fec 3003476 0 2024-04-24T23:14:12.072 INFO:tasks.workunit.client.1.smithi040.stdout:8/999: getdents d0/d7e/dc1/d9e/d90/d130 0 2024-04-24T23:14:12.079 INFO:tasks.workunit.client.1.smithi040.stdout:5/875: dwrite d2/d7/d8/d1f/d57/d27/f32 [0,4194304] 0 2024-04-24T23:14:12.080 INFO:tasks.workunit.client.1.smithi040.stdout:6/959: dwrite d7/d1a/d20/d51/daa/fb6 [0,4194304] 0 2024-04-24T23:14:12.083 INFO:tasks.workunit.client.1.smithi040.stdout:9/991: dwrite d7/da/d17/d40/d55/d2f/d1f/d4e/d83/f48 [0,4194304] 0 2024-04-24T23:14:12.083 INFO:tasks.workunit.client.1.smithi040.stdout:4/965: dwrite da/d3f/d6b/d2b/f31 [0,4194304] 0 2024-04-24T23:14:12.086 INFO:tasks.workunit.client.1.smithi040.stdout:4/966: stat da/d3f/d40/d41/d60/f4d 0 2024-04-24T23:14:12.092 INFO:tasks.workunit.client.1.smithi040.stdout:4/967: creat da/d3f/d40/d47/d71/f14b x:0 0 0 2024-04-24T23:14:12.094 INFO:tasks.workunit.client.1.smithi040.stdout:3/968: sync 2024-04-24T23:14:12.095 INFO:tasks.workunit.client.1.smithi040.stdout:5/876: link d2/d7/d8/d6d/l7b d2/d7/d8/db/da2/l110 0 2024-04-24T23:14:12.096 INFO:tasks.workunit.client.1.smithi040.stdout:6/960: getdents d7/d1a/db5/d127 0 2024-04-24T23:14:12.098 INFO:tasks.workunit.client.1.smithi040.stdout:9/992: getdents d7/da/d17/d40/d55/d2f/d1f/d4e 0 2024-04-24T23:14:12.098 INFO:tasks.workunit.client.1.smithi040.stdout:4/968: creat da/d3f/d40/d47/d75/d7c/f14c x:0 0 0 2024-04-24T23:14:12.100 INFO:tasks.workunit.client.1.smithi040.stdout:9/993: write d7/da/d17/d40/f106 [43079,86279] 0 2024-04-24T23:14:12.101 INFO:tasks.workunit.client.1.smithi040.stdout:3/969: creat d1/d9/d39/d112/f14d x:0 0 0 2024-04-24T23:14:12.102 INFO:tasks.workunit.client.1.smithi040.stdout:5/877: unlink d2/d2f/d14/f81 0 2024-04-24T23:14:12.104 INFO:tasks.workunit.client.1.smithi040.stdout:4/969: dread da/d3f/d40/d47/d75/d7c/fef [0,4194304] 0 2024-04-24T23:14:12.104 INFO:tasks.workunit.client.1.smithi040.stdout:9/994: mknod d7/d5e/da1/c152 0 2024-04-24T23:14:12.105 INFO:tasks.workunit.client.1.smithi040.stdout:3/970: stat d1/d2d/d44/c12a 0 2024-04-24T23:14:12.107 INFO:tasks.workunit.client.1.smithi040.stdout:6/961: dwrite d7/d1a/d20/d51/d1e/d46/d69/f6b [4194304,4194304] 0 2024-04-24T23:14:12.107 INFO:tasks.workunit.client.1.smithi040.stdout:9/995: dread - d7/da/d17/d40/d55/d2f/df7/f118 zero size 2024-04-24T23:14:12.108 INFO:tasks.workunit.client.1.smithi040.stdout:3/971: symlink d1/d80/dbf/l14e 0 2024-04-24T23:14:12.109 INFO:tasks.workunit.client.1.smithi040.stdout:4/970: truncate da/d3f/d6b/fb1 2317032 0 2024-04-24T23:14:12.110 INFO:tasks.workunit.client.1.smithi040.stdout:6/962: mknod d7/d1a/dbf/c140 0 2024-04-24T23:14:12.113 INFO:tasks.workunit.client.1.smithi040.stdout:6/963: write d7/d1a/d20/d51/d1e/f22 [2047778,35819] 0 2024-04-24T23:14:12.113 INFO:tasks.workunit.client.1.smithi040.stdout:4/971: truncate da/d3f/d6b/d2b/f39 3181080 0 2024-04-24T23:14:12.116 INFO:tasks.workunit.client.1.smithi040.stdout:5/878: dread d2/d7/d8/d1f/d57/d27/f32 [4194304,4194304] 0 2024-04-24T23:14:12.117 INFO:tasks.workunit.client.1.smithi040.stdout:6/964: mknod d7/de/d84/d12d/c141 0 2024-04-24T23:14:12.121 INFO:tasks.workunit.client.1.smithi040.stdout:4/972: mkdir da/d3f/d40/d47/d75/de4/d130/d14d 0 2024-04-24T23:14:12.124 INFO:tasks.workunit.client.1.smithi040.stdout:6/965: mknod d7/d1a/d20/d8c/dd5/c142 0 2024-04-24T23:14:12.125 INFO:tasks.workunit.client.1.smithi040.stdout:3/972: link d1/d9/d18/d36/d55/lba d1/d9/d39/d112/l14f 0 2024-04-24T23:14:12.126 INFO:tasks.workunit.client.1.smithi040.stdout:3/973: stat d1/d9/d18/d36/d10d/d45/daa/f146 0 2024-04-24T23:14:12.130 INFO:tasks.workunit.client.1.smithi040.stdout:4/973: rmdir da/d3f/d6b/d3c/d69 39 2024-04-24T23:14:12.137 INFO:tasks.workunit.client.1.smithi040.stdout:3/974: creat d1/d2a/d8d/d7c/f150 x:0 0 0 2024-04-24T23:14:12.139 INFO:tasks.workunit.client.1.smithi040.stdout:6/966: dread d7/d1a/d20/d8c/fcf [0,4194304] 0 2024-04-24T23:14:12.140 INFO:tasks.workunit.client.1.smithi040.stdout:6/967: readlink d7/d1a/dbf/l130 0 2024-04-24T23:14:12.144 INFO:tasks.workunit.client.1.smithi040.stdout:6/968: truncate d7/d1a/d20/d51/d1e/fec 69098 0 2024-04-24T23:14:12.144 INFO:tasks.workunit.client.1.smithi040.stdout:4/974: dread da/d3f/d6b/d2b/f55 [0,4194304] 0 2024-04-24T23:14:12.148 INFO:tasks.workunit.client.1.smithi040.stdout:4/975: write da/d45/fac [3694244,91471] 0 2024-04-24T23:14:12.148 INFO:tasks.workunit.client.1.smithi040.stdout:5/879: getdents d2/d7/dab 0 2024-04-24T23:14:12.148 INFO:tasks.workunit.client.1.smithi040.stdout:9/996: dwrite d7/f3a [0,4194304] 0 2024-04-24T23:14:12.151 INFO:tasks.workunit.client.1.smithi040.stdout:3/975: rename d1/d2d/de/d30/l50 to d1/d2d/de/d30/d10f/l151 0 2024-04-24T23:14:12.151 INFO:tasks.workunit.client.1.smithi040.stdout:4/976: chown da/d45/d48/d6e/ddd/dea 226 1 2024-04-24T23:14:12.152 INFO:tasks.workunit.client.1.smithi040.stdout:1/986: sync 2024-04-24T23:14:12.152 INFO:tasks.workunit.client.1.smithi040.stdout:5/880: chown d2/d7/d8/d1f/fe8 362440675 1 2024-04-24T23:14:12.153 INFO:tasks.workunit.client.1.smithi040.stdout:6/969: dread d7/fae [0,4194304] 0 2024-04-24T23:14:12.158 INFO:tasks.workunit.client.1.smithi040.stdout:1/987: dread da/d12/d23/fb8 [0,4194304] 0 2024-04-24T23:14:12.159 INFO:tasks.workunit.client.1.smithi040.stdout:4/977: mknod da/d45/d48/d6e/ddd/dea/c14e 0 2024-04-24T23:14:12.161 INFO:tasks.workunit.client.1.smithi040.stdout:1/988: chown da/d35/d38/d47/dbc/de5/df5/d12e/d149 79 1 2024-04-24T23:14:12.161 INFO:tasks.workunit.client.1.smithi040.stdout:4/978: read - da/d78/dbb/f144 zero size 2024-04-24T23:14:12.161 INFO:tasks.workunit.client.1.smithi040.stdout:9/997: creat d7/da/d17/d40/d55/d2f/ded/f153 x:0 0 0 2024-04-24T23:14:12.162 INFO:tasks.workunit.client.1.smithi040.stdout:5/881: dwrite d2/d7/d8/d1f/d90/dcb/fd8 [0,4194304] 0 2024-04-24T23:14:12.163 INFO:tasks.workunit.client.1.smithi040.stdout:3/976: mkdir d1/d9/d18/d36/d10d/d13a/d152 0 2024-04-24T23:14:12.164 INFO:tasks.workunit.client.1.smithi040.stdout:3/977: chown d1/d2d/de 1422 1 2024-04-24T23:14:12.168 INFO:tasks.workunit.client.1.smithi040.stdout:1/989: mkdir da/d12/d23/dac/d151 0 2024-04-24T23:14:12.171 INFO:tasks.workunit.client.1.smithi040.stdout:4/979: chown da/d3f/d6b/d3c/d69/df3 429852 1 2024-04-24T23:14:12.180 INFO:tasks.workunit.client.1.smithi040.stdout:5/882: dwrite d2/d7/d8/d1f/d57/d27/d3d/dd6/fea [0,4194304] 0 2024-04-24T23:14:12.180 INFO:tasks.workunit.client.1.smithi040.stdout:4/980: symlink da/d45/d48/d6e/ddd/dea/l14f 0 2024-04-24T23:14:12.182 INFO:tasks.workunit.client.1.smithi040.stdout:4/981: write da/d3f/d40/d41/fd2 [4810380,127097] 0 2024-04-24T23:14:12.183 INFO:tasks.workunit.client.1.smithi040.stdout:4/982: stat da/d3f/d6b/l83 0 2024-04-24T23:14:12.183 INFO:tasks.workunit.client.1.smithi040.stdout:4/983: stat da/d3f/d40/d47/d75/d104/f105 0 2024-04-24T23:14:12.185 INFO:tasks.workunit.client.1.smithi040.stdout:6/970: creat d7/d1a/d20/d51/d1e/d46/f143 x:0 0 0 2024-04-24T23:14:12.190 INFO:tasks.workunit.client.1.smithi040.stdout:3/978: rename d1/d2a/d6a/f3a to d1/d9/d18/d36/d10d/f153 0 2024-04-24T23:14:12.191 INFO:tasks.workunit.client.1.smithi040.stdout:9/998: write d7/d5e/da1/feb [4408282,60374] 0 2024-04-24T23:14:12.194 INFO:tasks.workunit.client.1.smithi040.stdout:9/999: chown d7/da/d17/d40/d55/d2f/d1f/d4e/dbf/c142 7209234 1 2024-04-24T23:14:12.194 INFO:tasks.workunit.client.1.smithi040.stdout:4/984: symlink da/d3f/d40/d47/l150 0 2024-04-24T23:14:12.196 INFO:tasks.workunit.client.1.smithi040.stdout:4/985: write da/d3f/d40/d41/d8e/db8/f120 [123165,85449] 0 2024-04-24T23:14:12.196 INFO:tasks.workunit.client.1.smithi040.stdout:6/971: creat d7/d1a/d20/d51/d1e/d5d/f144 x:0 0 0 2024-04-24T23:14:12.198 INFO:tasks.workunit.client.1.smithi040.stdout:1/990: rename da/d12/d18/d2a/d8b/d97/f112 to da/d12/d18/d2a/d4d/d107/db9/f152 0 2024-04-24T23:14:12.201 INFO:tasks.workunit.client.1.smithi040.stdout:4/986: chown da/cd 80 1 2024-04-24T23:14:12.205 INFO:tasks.workunit.client.1.smithi040.stdout:3/979: getdents d1/d2d/d2b/d11e 0 2024-04-24T23:14:12.206 INFO:tasks.workunit.client.1.smithi040.stdout:4/987: rmdir da/d45/d48 39 2024-04-24T23:14:12.206 INFO:tasks.workunit.client.1.smithi040.stdout:3/980: dread - d1/d2d/f123 zero size 2024-04-24T23:14:12.208 INFO:tasks.workunit.client.1.smithi040.stdout:1/991: write da/d12/d18/d2a/d41/fa8 [298436,45992] 0 2024-04-24T23:14:12.210 INFO:tasks.workunit.client.1.smithi040.stdout:4/988: creat da/d3f/d40/d47/d71/f151 x:0 0 0 2024-04-24T23:14:12.211 INFO:tasks.workunit.client.1.smithi040.stdout:6/972: rename d7/de/d84/l128 to d7/d1a/d20/d51/d1e/d46/l145 0 2024-04-24T23:14:12.212 INFO:tasks.workunit.client.1.smithi040.stdout:3/981: truncate d1/d80/dbf/fd5 800660 0 2024-04-24T23:14:12.214 INFO:tasks.workunit.client.1.smithi040.stdout:1/992: read da/d35/dec/ff1 [742777,129046] 0 2024-04-24T23:14:12.214 INFO:tasks.workunit.client.1.smithi040.stdout:4/989: mkdir da/d3f/d40/d41/d60/d152 0 2024-04-24T23:14:12.215 INFO:tasks.workunit.client.1.smithi040.stdout:6/973: rmdir d7/de/d89 39 2024-04-24T23:14:12.215 INFO:tasks.workunit.client.1.smithi040.stdout:1/993: dread - da/d12/d18/d2a/d41/d4f/de9/f113 zero size 2024-04-24T23:14:12.216 INFO:tasks.workunit.client.1.smithi040.stdout:1/994: chown da/d35/d38/d47/dbc/l105 49108304 1 2024-04-24T23:14:12.220 INFO:tasks.workunit.client.1.smithi040.stdout:1/995: dread da/d12/d18/d2a/d41/f62 [0,4194304] 0 2024-04-24T23:14:12.231 INFO:tasks.workunit.client.1.smithi040.stdout:3/982: dread d1/f79 [0,4194304] 0 2024-04-24T23:14:12.236 INFO:tasks.workunit.client.1.smithi040.stdout:3/983: symlink d1/d9/d2e/l154 0 2024-04-24T23:14:12.239 INFO:tasks.workunit.client.1.smithi040.stdout:1/996: dread da/d12/d23/fc9 [0,4194304] 0 2024-04-24T23:14:12.240 INFO:tasks.workunit.client.1.smithi040.stdout:6/974: truncate d7/d1a/d20/d51/d1e/d46/d69/f6b 6270038 0 2024-04-24T23:14:12.240 INFO:tasks.workunit.client.1.smithi040.stdout:4/990: dwrite da/f117 [0,4194304] 0 2024-04-24T23:14:12.248 INFO:tasks.workunit.client.1.smithi040.stdout:4/991: dread da/d45/d58/f73 [0,4194304] 0 2024-04-24T23:14:12.251 INFO:tasks.workunit.client.1.smithi040.stdout:6/975: dread d7/d1a/d20/f61 [0,4194304] 0 2024-04-24T23:14:12.255 INFO:tasks.workunit.client.1.smithi040.stdout:6/976: dread d7/d1a/d9f/fb4 [0,4194304] 0 2024-04-24T23:14:12.257 INFO:tasks.workunit.client.1.smithi040.stdout:6/977: dread d7/d1a/d20/d8c/db0/fd2 [0,4194304] 0 2024-04-24T23:14:12.258 INFO:tasks.workunit.client.1.smithi040.stdout:4/992: symlink da/d3f/d40/d41/d62/l153 0 2024-04-24T23:14:12.264 INFO:tasks.workunit.client.1.smithi040.stdout:6/978: creat d7/d1a/db5/d127/f146 x:0 0 0 2024-04-24T23:14:12.264 INFO:tasks.workunit.client.1.smithi040.stdout:3/984: dread d1/d9/d18/d36/d55/f67 [0,4194304] 0 2024-04-24T23:14:12.265 INFO:tasks.workunit.client.1.smithi040.stdout:3/985: chown d1/d9/d2e/f113 27828 1 2024-04-24T23:14:12.266 INFO:tasks.workunit.client.1.smithi040.stdout:4/993: mknod da/d3f/d6b/d3c/d69/df3/c154 0 2024-04-24T23:14:12.267 INFO:tasks.workunit.client.1.smithi040.stdout:1/997: getdents da/d12/d18/d2a/d32/d102 0 2024-04-24T23:14:12.273 INFO:tasks.workunit.client.1.smithi040.stdout:3/986: creat d1/d2d/de/d30/d37/f155 x:0 0 0 2024-04-24T23:14:12.281 INFO:tasks.workunit.client.1.smithi040.stdout:3/987: mknod d1/d9/d18/d36/d10d/d45/d96/c156 0 2024-04-24T23:14:12.282 INFO:tasks.workunit.client.1.smithi040.stdout:3/988: dread - d1/d2a/d8d/d7c/f150 zero size 2024-04-24T23:14:12.285 INFO:tasks.workunit.client.1.smithi040.stdout:3/989: dread d1/f16 [4194304,4194304] 0 2024-04-24T23:14:12.288 INFO:tasks.workunit.client.1.smithi040.stdout:6/979: creat d7/d1a/d20/d51/d1e/d46/d69/f147 x:0 0 0 2024-04-24T23:14:12.289 INFO:tasks.workunit.client.1.smithi040.stdout:4/994: rename da/d3f/c12d to da/d45/c155 0 2024-04-24T23:14:12.295 INFO:tasks.workunit.client.1.smithi040.stdout:3/990: symlink d1/d2d/d72/d106/l157 0 2024-04-24T23:14:12.295 INFO:tasks.workunit.client.1.smithi040.stdout:4/995: dwrite da/d3f/d40/d41/d8e/db8/fc4 [4194304,4194304] 0 2024-04-24T23:14:12.302 INFO:tasks.workunit.client.1.smithi040.stdout:1/998: write da/d35/d38/f3e [190670,31298] 0 2024-04-24T23:14:12.305 INFO:tasks.workunit.client.1.smithi040.stdout:3/991: symlink d1/d2d/de/d30/d37/d107/l158 0 2024-04-24T23:14:12.308 INFO:tasks.workunit.client.1.smithi040.stdout:4/996: mkdir da/d3f/d40/d47/d75/d104/d156 0 2024-04-24T23:14:12.309 INFO:tasks.workunit.client.1.smithi040.stdout:4/997: write da/d3f/d40/d47/d75/d7c/f14c [555477,120298] 0 2024-04-24T23:14:12.311 INFO:tasks.workunit.client.1.smithi040.stdout:4/998: chown da/d3f/d40/d41/d8e/d116/c131 1057 1 2024-04-24T23:14:12.312 INFO:tasks.workunit.client.1.smithi040.stdout:1/999: chown da/d12/d18/d2a/d4d/d107/db9/lcd 428158440 1 2024-04-24T23:14:12.314 INFO:tasks.workunit.client.1.smithi040.stdout:3/992: rmdir d1/d2d/d2b 39 2024-04-24T23:14:12.316 INFO:tasks.workunit.client.1.smithi040.stdout:4/999: dread - da/d3f/d40/d47/d71/fa9 zero size 2024-04-24T23:14:12.318 INFO:tasks.workunit.client.1.smithi040.stdout:3/993: creat d1/d2a/d6a/d94/f159 x:0 0 0 2024-04-24T23:14:12.319 INFO:tasks.workunit.client.1.smithi040.stdout:5/883: sync 2024-04-24T23:14:12.321 INFO:tasks.workunit.client.1.smithi040.stdout:6/980: dwrite d7/d1a/fee [0,4194304] 0 2024-04-24T23:14:12.323 INFO:tasks.workunit.client.1.smithi040.stdout:5/884: mknod d2/d7/d8/db/d1e/c111 0 2024-04-24T23:14:12.323 INFO:tasks.workunit.client.1.smithi040.stdout:3/994: truncate d1/d9/d18/d36/d10d/d45/daa/f10b 1039698 0 2024-04-24T23:14:12.325 INFO:tasks.workunit.client.1.smithi040.stdout:6/981: creat d7/d1a/d20/d51/d2f/dba/dde/dd3/f148 x:0 0 0 2024-04-24T23:14:12.326 INFO:tasks.workunit.client.1.smithi040.stdout:5/885: fsync d2/d7/d8/d1f/d57/fdd 0 2024-04-24T23:14:12.327 INFO:tasks.workunit.client.1.smithi040.stdout:3/995: mknod d1/d9/d18/d36/d10d/d45/d96/d142/d12b/c15a 0 2024-04-24T23:14:12.329 INFO:tasks.workunit.client.1.smithi040.stdout:6/982: rmdir d7/d1a/d20/d8c/dd5 39 2024-04-24T23:14:12.334 INFO:tasks.workunit.client.1.smithi040.stdout:5/886: creat d2/d7/d8/d1f/d57/dca/dfb/f112 x:0 0 0 2024-04-24T23:14:12.335 INFO:tasks.workunit.client.1.smithi040.stdout:5/887: readlink d2/d2f/l5b 0 2024-04-24T23:14:12.336 INFO:tasks.workunit.client.1.smithi040.stdout:6/983: creat d7/de/d89/d12c/f149 x:0 0 0 2024-04-24T23:14:12.337 INFO:tasks.workunit.client.1.smithi040.stdout:5/888: fsync d2/d7/d8/db/f75 0 2024-04-24T23:14:12.342 INFO:tasks.workunit.client.1.smithi040.stdout:5/889: mkdir d2/d7/d8/d64/d6b/d113 0 2024-04-24T23:14:12.346 INFO:tasks.workunit.client.1.smithi040.stdout:6/984: getdents d7/d1a/d20/d51/d1e/d46/d69 0 2024-04-24T23:14:12.348 INFO:tasks.workunit.client.1.smithi040.stdout:5/890: dwrite d2/d7/d8/db/d1e/f46 [0,4194304] 0 2024-04-24T23:14:12.351 INFO:tasks.workunit.client.1.smithi040.stdout:5/891: rmdir d2/d7/d8/d1f 39 2024-04-24T23:14:12.351 INFO:tasks.workunit.client.1.smithi040.stdout:6/985: truncate d7/d1a/d20/d8c/dd5/ff4 1033881 0 2024-04-24T23:14:12.352 INFO:tasks.workunit.client.1.smithi040.stdout:6/986: chown d7/d1a/l83 18764832 1 2024-04-24T23:14:12.352 INFO:tasks.workunit.client.1.smithi040.stdout:3/996: write d1/d2a/d8d/fca [665638,39640] 0 2024-04-24T23:14:12.359 INFO:tasks.workunit.client.1.smithi040.stdout:3/997: dwrite d1/d9/d18/d36/d10d/f53 [0,4194304] 0 2024-04-24T23:14:12.363 INFO:tasks.workunit.client.1.smithi040.stdout:6/987: write d7/d1a/d9f/fc1 [71436,94080] 0 2024-04-24T23:14:12.368 INFO:tasks.workunit.client.1.smithi040.stdout:3/998: dread d1/fd [4194304,4194304] 0 2024-04-24T23:14:12.372 INFO:tasks.workunit.client.1.smithi040.stdout:3/999: mkdir d1/d9/d15b 0 2024-04-24T23:14:12.373 INFO:tasks.workunit.client.1.smithi040.stdout:6/988: dwrite d7/d1a/d20/d51/d2a/dfd/f123 [0,4194304] 0 2024-04-24T23:14:12.375 INFO:tasks.workunit.client.1.smithi040.stdout:5/892: dwrite d2/d7/d8/d1f/d90/dcb/fda [0,4194304] 0 2024-04-24T23:14:12.376 INFO:tasks.workunit.client.1.smithi040.stdout:6/989: chown d7/d1a/lbc 2557 1 2024-04-24T23:14:12.383 INFO:tasks.workunit.client.1.smithi040.stdout:5/893: dwrite d2/d2f/d14/f76 [0,4194304] 0 2024-04-24T23:14:12.388 INFO:tasks.workunit.client.1.smithi040.stdout:5/894: mknod d2/d7/dab/c114 0 2024-04-24T23:14:12.392 INFO:tasks.workunit.client.1.smithi040.stdout:5/895: write d2/d7/d8/d64/f93 [3527939,23575] 0 2024-04-24T23:14:12.393 INFO:tasks.workunit.client.1.smithi040.stdout:6/990: truncate d7/d1a/f67 1201191 0 2024-04-24T23:14:12.393 INFO:tasks.workunit.client.1.smithi040.stdout:5/896: write d2/d2f/d14/f86 [3271939,91832] 0 2024-04-24T23:14:12.401 INFO:tasks.workunit.client.1.smithi040.stdout:5/897: creat d2/d7/d8/d1f/d57/d27/d3d/dd6/f115 x:0 0 0 2024-04-24T23:14:12.406 INFO:tasks.workunit.client.1.smithi040.stdout:5/898: rename d2/d7/d8/d1f/d31 to d2/d2f/d14/dbe/df2/d116 0 2024-04-24T23:14:12.411 INFO:tasks.workunit.client.1.smithi040.stdout:5/899: symlink d2/d7/d8/d1f/d57/d3e/l117 0 2024-04-24T23:14:12.418 INFO:tasks.workunit.client.1.smithi040.stdout:5/900: dwrite d2/d7/d8/d1f/d57/d27/d3c/d9b/fd5 [0,4194304] 0 2024-04-24T23:14:12.426 INFO:tasks.workunit.client.1.smithi040.stdout:6/991: dwrite d7/d1a/d20/d51/d1e/d5d/fc4 [0,4194304] 0 2024-04-24T23:14:12.427 INFO:tasks.workunit.client.1.smithi040.stdout:6/992: chown d7/d1a/d20/d51/d1e/d46/f98 3419116 1 2024-04-24T23:14:12.430 INFO:tasks.workunit.client.1.smithi040.stdout:5/901: getdents d2/d7/d8/d1f/da1 0 2024-04-24T23:14:12.433 INFO:tasks.workunit.client.1.smithi040.stdout:6/993: fdatasync d7/d1a/d56/f7f 0 2024-04-24T23:14:12.440 INFO:tasks.workunit.client.1.smithi040.stdout:6/994: mkdir d7/d1a/d20/d8c/dd5/d14a 0 2024-04-24T23:14:12.442 INFO:tasks.workunit.client.1.smithi040.stdout:5/902: mknod d2/d7/d8/db/da2/d10d/c118 0 2024-04-24T23:14:12.442 INFO:tasks.workunit.client.1.smithi040.stdout:5/903: dread - d2/d7/d8/d1f/da1/f107 zero size 2024-04-24T23:14:12.452 INFO:tasks.workunit.client.1.smithi040.stdout:5/904: link d2/d7/d8/d1f/d57/d27/fd7 d2/d7/d8/db/f119 0 2024-04-24T23:14:12.458 INFO:tasks.workunit.client.1.smithi040.stdout:5/905: getdents d2/d7/d8/d1f/d57/d27/d3c 0 2024-04-24T23:14:12.461 INFO:tasks.workunit.client.1.smithi040.stdout:5/906: getdents d2/d7/d8/d1f/d57/d27/d3c/d9b 0 2024-04-24T23:14:12.461 INFO:tasks.workunit.client.1.smithi040.stdout:5/907: chown d2/d2f/d14/ce1 1757 1 2024-04-24T23:14:12.467 INFO:tasks.workunit.client.1.smithi040.stdout:5/908: creat d2/d7/d8/d1f/d57/d27/f11a x:0 0 0 2024-04-24T23:14:12.477 INFO:tasks.workunit.client.1.smithi040.stdout:5/909: truncate d2/d7/d8/d1f/d57/d27/d3c/d9b/fd5 1389590 0 2024-04-24T23:14:12.492 INFO:tasks.workunit.client.1.smithi040.stdout:5/910: dread d2/d7/fe [0,4194304] 0 2024-04-24T23:14:12.496 INFO:tasks.workunit.client.1.smithi040.stdout:5/911: creat d2/d2f/d14/ddf/f11b x:0 0 0 2024-04-24T23:14:12.496 INFO:tasks.workunit.client.1.smithi040.stdout:5/912: truncate d2/d2f/d14/fbc 5089166 0 2024-04-24T23:14:12.499 INFO:tasks.workunit.client.1.smithi040.stdout:5/913: mkdir d2/d7/d8/db/da2/d11c 0 2024-04-24T23:14:12.500 INFO:tasks.workunit.client.1.smithi040.stdout:5/914: fsync d2/d7/d8/d1f/d57/d27/d3c/da4/fe9 0 2024-04-24T23:14:12.501 INFO:tasks.workunit.client.1.smithi040.stdout:5/915: write d2/d7/f36 [2149752,54914] 0 2024-04-24T23:14:12.527 INFO:tasks.workunit.client.1.smithi040.stdout:6/995: sync 2024-04-24T23:14:12.547 INFO:tasks.workunit.client.1.smithi040.stdout:5/916: getdents d2/d7/d8 0 2024-04-24T23:14:12.550 INFO:tasks.workunit.client.1.smithi040.stdout:6/996: mknod d7/c14b 0 2024-04-24T23:14:12.553 INFO:tasks.workunit.client.1.smithi040.stdout:6/997: dread d7/d1a/d20/d51/d1e/d46/fac [0,4194304] 0 2024-04-24T23:14:12.555 INFO:tasks.workunit.client.1.smithi040.stdout:6/998: rmdir d7/de/d89 39 2024-04-24T23:14:12.557 INFO:tasks.workunit.client.1.smithi040.stdout:6/999: dread - d7/d1a/d56/d113/ff0 zero size 2024-04-24T23:14:12.584 INFO:tasks.workunit.client.1.smithi040.stdout:5/917: dwrite d2/d7/d8/d64/d6b/f92 [0,4194304] 0 2024-04-24T23:14:12.590 INFO:tasks.workunit.client.1.smithi040.stdout:5/918: mknod d2/d7/d8/d1f/c11d 0 2024-04-24T23:14:12.611 INFO:tasks.workunit.client.1.smithi040.stdout:5/919: dwrite d2/d7/d8/db/d1e/fd0 [0,4194304] 0 2024-04-24T23:14:12.623 INFO:tasks.workunit.client.1.smithi040.stdout:5/920: dread d2/d7/d8/d1f/d57/d27/d3d/fd4 [0,4194304] 0 2024-04-24T23:14:12.645 INFO:tasks.workunit.client.1.smithi040.stdout:5/921: fdatasync d2/d7/d8/db/f96 0 2024-04-24T23:14:12.649 INFO:tasks.workunit.client.1.smithi040.stdout:5/922: mknod d2/d7/d8/d1f/d57/d27/d3c/da4/c11e 0 2024-04-24T23:14:12.650 INFO:tasks.workunit.client.1.smithi040.stdout:5/923: write d2/d7/d8/db/d1e/fd0 [365928,44110] 0 2024-04-24T23:14:12.650 INFO:tasks.workunit.client.1.smithi040.stdout:5/924: stat d2/d7/d8/d64/d6b/f92 0 2024-04-24T23:14:12.662 INFO:tasks.workunit.client.1.smithi040.stdout:5/925: write d2/d7/d8/db/f96 [252174,70426] 0 2024-04-24T23:14:12.669 INFO:tasks.workunit.client.1.smithi040.stdout:5/926: mknod d2/d7/d8/db/da2/d10d/c11f 0 2024-04-24T23:14:12.674 INFO:tasks.workunit.client.1.smithi040.stdout:5/927: dwrite d2/fb6 [0,4194304] 0 2024-04-24T23:14:12.680 INFO:tasks.workunit.client.1.smithi040.stdout:5/928: link d2/d7/d8/d64/lcf d2/d7/d8/db/l120 0 2024-04-24T23:14:12.681 INFO:tasks.workunit.client.1.smithi040.stdout:5/929: stat d2/d7/d8/d6d/fc7 0 2024-04-24T23:14:12.684 INFO:tasks.workunit.client.1.smithi040.stdout:5/930: symlink d2/d2f/d14/ddf/de5/l121 0 2024-04-24T23:14:12.699 INFO:tasks.workunit.client.1.smithi040.stdout:5/931: truncate d2/d7/d8/db/d1e/fd0 653508 0 2024-04-24T23:14:12.699 INFO:tasks.workunit.client.1.smithi040.stdout:5/932: readlink d2/d7/d8/d64/l7d 0 2024-04-24T23:14:12.700 INFO:tasks.workunit.client.1.smithi040.stdout:5/933: chown d2/d7/d8/d1f/da1/f107 67 1 2024-04-24T23:14:12.713 INFO:tasks.workunit.client.1.smithi040.stdout:5/934: write d2/d7/d8/d1f/d57/d27/f54 [4042535,118051] 0 2024-04-24T23:14:12.717 INFO:tasks.workunit.client.1.smithi040.stdout:5/935: write d2/d7/d8/d6d/fc7 [4267213,57656] 0 2024-04-24T23:14:12.813 INFO:tasks.workunit.client.1.smithi040.stdout:5/936: sync 2024-04-24T23:14:12.814 INFO:tasks.workunit.client.1.smithi040.stdout:5/937: write d2/d7/d8/d64/d6b/f92 [3430901,101599] 0 2024-04-24T23:14:12.814 INFO:tasks.workunit.client.1.smithi040.stdout:5/938: dread - d2/d7/d8/d1f/fe8 zero size 2024-04-24T23:14:12.818 INFO:tasks.workunit.client.1.smithi040.stdout:5/939: creat d2/d7/d8/db/da2/d11c/f122 x:0 0 0 2024-04-24T23:14:12.824 INFO:tasks.workunit.client.1.smithi040.stdout:5/940: dread d2/d7/d8/db/fc9 [0,4194304] 0 2024-04-24T23:14:12.825 INFO:tasks.workunit.client.1.smithi040.stdout:5/941: truncate d2/d7/d8/d1f/fe8 520359 0 2024-04-24T23:14:12.825 INFO:tasks.workunit.client.1.smithi040.stdout:5/942: chown d2/d7/d8/d1f/da1 1 1 2024-04-24T23:14:12.826 INFO:tasks.workunit.client.1.smithi040.stdout:5/943: chown d2/d7/d8/d1f/d57/d27/d3c/da4/le6 15748433 1 2024-04-24T23:14:12.827 INFO:tasks.workunit.client.1.smithi040.stdout:5/944: write d2/d7/d8/d1f/d57/d27/d3c/d43/f6a [5522675,77219] 0 2024-04-24T23:14:12.837 INFO:tasks.workunit.client.1.smithi040.stdout:5/945: dwrite d2/d7/d8/d1f/d57/d27/f32 [0,4194304] 0 2024-04-24T23:14:12.849 INFO:tasks.workunit.client.1.smithi040.stdout:5/946: dwrite d2/f9e [4194304,4194304] 0 2024-04-24T23:14:12.853 INFO:tasks.workunit.client.1.smithi040.stdout:5/947: creat d2/d7/d8/f123 x:0 0 0 2024-04-24T23:14:12.874 INFO:tasks.workunit.client.1.smithi040.stdout:5/948: rmdir d2/d7/d8/db/da2/dcc/d104 39 2024-04-24T23:14:12.875 INFO:tasks.workunit.client.1.smithi040.stdout:5/949: fsync d2/d7/d8/d64/f93 0 2024-04-24T23:14:12.890 INFO:tasks.workunit.client.1.smithi040.stdout:5/950: truncate d2/d7/d8/d1f/d57/d27/d3c/da4/fce 826865 0 2024-04-24T23:14:12.905 INFO:tasks.workunit.client.1.smithi040.stdout:5/951: dwrite d2/d7/d8/d1f/d57/d27/d3c/d9b/fd5 [0,4194304] 0 2024-04-24T23:14:12.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:12 smithi040 ceph-mon[33128]: pgmap v39: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 107 MiB/s rd, 251 MiB/s wr, 573 op/s 2024-04-24T23:14:12.930 INFO:tasks.workunit.client.1.smithi040.stdout:5/952: dread d2/d7/d8/f1b [0,4194304] 0 2024-04-24T23:14:12.934 INFO:tasks.workunit.client.1.smithi040.stdout:5/953: rename d2/d2f/d14/dbe to d2/d7/d8/d1f/d90/d124 0 2024-04-24T23:14:12.935 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:12 smithi031 ceph-mon[26411]: pgmap v39: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 107 MiB/s rd, 251 MiB/s wr, 573 op/s 2024-04-24T23:14:12.936 INFO:tasks.workunit.client.1.smithi040.stdout:5/954: write d2/d7/d8/d1f/d57/d27/d3c/d9b/fd5 [1809290,74661] 0 2024-04-24T23:14:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.932+0000 7fabc6b45700 1 -- 172.21.15.31:0/2324404001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 msgr2=0x7fabc0104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.932+0000 7fabc6b45700 1 --2- 172.21.15.31:0/2324404001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 0x7fabc0104670 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fabac004940 tx=0x7fabac009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:12.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.932+0000 7fabc6b45700 1 -- 172.21.15.31:0/2324404001 shutdown_connections 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.932+0000 7fabc6b45700 1 --2- 172.21.15.31:0/2324404001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 0x7fabc0104670 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.932+0000 7fabc6b45700 1 --2- 172.21.15.31:0/2324404001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc00fd3d0 0x7fabc00fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.932+0000 7fabc6b45700 1 -- 172.21.15.31:0/2324404001 >> 172.21.15.31:0/2324404001 conn(0x7fabc00f6b80 msgr2=0x7fabc00f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.932+0000 7fabc6b45700 1 -- 172.21.15.31:0/2324404001 shutdown_connections 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.932+0000 7fabc6b45700 1 -- 172.21.15.31:0/2324404001 wait complete. 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.933+0000 7fabc6b45700 1 Processor -- start 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.933+0000 7fabc6b45700 1 -- start start 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.933+0000 7fabc6b45700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc00fd3d0 0x7fabc010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabc6b45700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 0x7fabc01100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabc6b45700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc01106e0 con 0x7fabc00fdd20 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabc6b45700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc0110ee0 con 0x7fabc00fd3d0 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabbffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 0x7fabc01100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:12.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabbffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 0x7fabc01100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58858/0 (socket says 172.21.15.31:58858) 2024-04-24T23:14:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabbffff700 1 -- 172.21.15.31:0/1236561229 learned_addr learned my addr 172.21.15.31:0/1236561229 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabc48e1700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc00fd3d0 0x7fabc010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabc48e1700 1 -- 172.21.15.31:0/1236561229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 msgr2=0x7fabc01100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabc48e1700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 0x7fabc01100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabc48e1700 1 -- 172.21.15.31:0/1236561229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac005040 con 0x7fabc00fd3d0 2024-04-24T23:14:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.934+0000 7fabc48e1700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc00fd3d0 0x7fabc010fba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009fd0 tx=0x7fabb400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.935+0000 7fabbd7fa700 1 -- 172.21.15.31:0/1236561229 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb4004e90 con 0x7fabc00fd3d0 2024-04-24T23:14:12.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.935+0000 7fabbd7fa700 1 -- 172.21.15.31:0/1236561229 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabb400f040 con 0x7fabc00fd3d0 2024-04-24T23:14:12.939 INFO:tasks.workunit.client.1.smithi040.stdout:5/955: write d2/d2f/fa0 [8978557,98554] 0 2024-04-24T23:14:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.936+0000 7fabbd7fa700 1 -- 172.21.15.31:0/1236561229 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb400ca50 con 0x7fabc00fd3d0 2024-04-24T23:14:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.936+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc0111160 con 0x7fabc00fd3d0 2024-04-24T23:14:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.936+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc01115e0 con 0x7fabc00fd3d0 2024-04-24T23:14:12.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.937+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc005f5c0 con 0x7fabc00fd3d0 2024-04-24T23:14:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.937+0000 7fabbd7fa700 1 -- 172.21.15.31:0/1236561229 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7fabb4007cc0 con 0x7fabc00fd3d0 2024-04-24T23:14:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.938+0000 7fabbd7fa700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fabb006b790 0x7fabb006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.938+0000 7fabbd7fa700 1 -- 172.21.15.31:0/1236561229 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fabb4089d50 con 0x7fabc00fd3d0 2024-04-24T23:14:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.938+0000 7fabbffff700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fabb006b790 0x7fabb006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:12.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.939+0000 7fabbffff700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fabb006b790 0x7fabb006dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fabac0040d0 tx=0x7fabac009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:12.942 INFO:tasks.workunit.client.1.smithi040.stdout:5/956: mkdir d2/d7/d8/d1f/d90/d124/df2/d125 0 2024-04-24T23:14:12.942 INFO:tasks.workunit.client.1.smithi040.stdout:5/957: readlink d2/d7/d8/d1f/d90/dcb/l5d 0 2024-04-24T23:14:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:12.943+0000 7fabbd7fa700 1 -- 172.21.15.31:0/1236561229 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fabb4057340 con 0x7fabc00fd3d0 2024-04-24T23:14:12.945 INFO:tasks.workunit.client.1.smithi040.stdout:5/958: symlink d2/d7/d8/d1f/d57/d3e/l126 0 2024-04-24T23:14:12.951 INFO:tasks.workunit.client.1.smithi040.stdout:5/959: write d2/d7/d8/d1f/d57/d27/fe4 [1235225,74889] 0 2024-04-24T23:14:12.954 INFO:tasks.workunit.client.1.smithi040.stdout:5/960: mkdir d2/d7/d8/db/d1e/d127 0 2024-04-24T23:14:12.955 INFO:tasks.workunit.client.1.smithi040.stdout:5/961: chown d2/d7/d8/d1f/d57/d27/d3c/da4 4424 1 2024-04-24T23:14:12.955 INFO:tasks.workunit.client.1.smithi040.stdout:5/962: chown d2/d7/d8/d1f/d57/d27/d3c/d9b/ce3 200166081 1 2024-04-24T23:14:12.957 INFO:tasks.workunit.client.1.smithi040.stdout:5/963: mknod d2/d7/d8/d1f/d90/d124/df2/d116/c128 0 2024-04-24T23:14:12.964 INFO:tasks.workunit.client.1.smithi040.stdout:5/964: link d2/d7/d8/d6d/c7c d2/d7/d8/d1f/d57/d27/d3d/c129 0 2024-04-24T23:14:12.981 INFO:tasks.workunit.client.1.smithi040.stdout:5/965: dwrite d2/d7/d8/d1f/d57/d27/d3d/dd6/feb [4194304,4194304] 0 2024-04-24T23:14:12.982 INFO:tasks.workunit.client.1.smithi040.stdout:5/966: write d2/d7/d8/db/d1e/f106 [954305,47292] 0 2024-04-24T23:14:13.079 INFO:tasks.workunit.client.1.smithi040.stdout:5/967: sync 2024-04-24T23:14:13.080 INFO:tasks.workunit.client.1.smithi040.stdout:5/968: chown d2/d7/d8/db/f96 56425013 1 2024-04-24T23:14:13.082 INFO:tasks.workunit.client.1.smithi040.stdout:5/969: dread d2/d7/d8/d1f/d57/fdd [0,4194304] 0 2024-04-24T23:14:13.088 INFO:tasks.workunit.client.1.smithi040.stdout:5/970: dwrite d2/d7/d8/d1f/da1/f107 [0,4194304] 0 2024-04-24T23:14:13.104 INFO:tasks.workunit.client.1.smithi040.stdout:5/971: dwrite d2/d7/d8/db/d1e/fd0 [0,4194304] 0 2024-04-24T23:14:13.108 INFO:tasks.workunit.client.1.smithi040.stdout:5/972: fdatasync d2/d7/d8/d1f/d57/f70 0 2024-04-24T23:14:13.114 INFO:tasks.workunit.client.1.smithi040.stdout:5/973: symlink d2/d7/d8/d1f/d57/d27/l12a 0 2024-04-24T23:14:13.116 INFO:tasks.workunit.client.1.smithi040.stdout:5/974: rmdir d2/d7/d8/d1f/d57/d3e 39 2024-04-24T23:14:13.118 INFO:tasks.workunit.client.1.smithi040.stdout:5/975: creat d2/d7/d8/d1f/d90/f12b x:0 0 0 2024-04-24T23:14:13.120 INFO:tasks.workunit.client.1.smithi040.stdout:5/976: truncate d2/d7/d8/d1f/d57/d27/d3d/fd4 5128665 0 2024-04-24T23:14:13.121 INFO:tasks.workunit.client.1.smithi040.stdout:5/977: readlink d2/d7/d8/d6d/d89/l8c 0 2024-04-24T23:14:13.121 INFO:tasks.workunit.client.1.smithi040.stdout:5/978: chown d2/d7/d8/d64/d6b/c83 286048 1 2024-04-24T23:14:13.123 INFO:tasks.workunit.client.1.smithi040.stdout:5/979: creat d2/d7/d8/d1f/d57/dca/f12c x:0 0 0 2024-04-24T23:14:13.125 INFO:tasks.workunit.client.1.smithi040.stdout:5/980: unlink d2/d7/d8/d1f/d57/d27/d3c/fb9 0 2024-04-24T23:14:13.126 INFO:tasks.workunit.client.1.smithi040.stdout:5/981: write d2/f9e [848691,73238] 0 2024-04-24T23:14:13.132 INFO:tasks.workunit.client.1.smithi040.stdout:5/982: dwrite d2/d7/d8/d1f/d57/d27/d3c/f95 [0,4194304] 0 2024-04-24T23:14:13.136 INFO:tasks.workunit.client.1.smithi040.stdout:5/983: chown d2/d7/d8/d1f/d57/d27/d65/d10b 1 1 2024-04-24T23:14:13.137 INFO:tasks.workunit.client.1.smithi040.stdout:5/984: chown d2/d7/d8/d1f/d57/d27/d3d/f102 45891371 1 2024-04-24T23:14:13.144 INFO:tasks.workunit.client.1.smithi040.stdout:5/985: rmdir d2/d7/dc3 0 2024-04-24T23:14:13.149 INFO:tasks.workunit.client.1.smithi040.stdout:5/986: symlink d2/d7/d8/l12d 0 2024-04-24T23:14:13.155 INFO:tasks.workunit.client.1.smithi040.stdout:5/987: mkdir d2/d7/d8/db/da2/d12e 0 2024-04-24T23:14:13.157 INFO:tasks.workunit.client.1.smithi040.stdout:5/988: creat d2/d7/d8/db/da2/d11c/f12f x:0 0 0 2024-04-24T23:14:13.164 INFO:tasks.workunit.client.1.smithi040.stdout:5/989: rename d2/d7/d8/db/da2/dcc/d104 to d2/d7/d8/d1f/d57/d27/d3d/d130 0 2024-04-24T23:14:13.168 INFO:tasks.workunit.client.1.smithi040.stdout:5/990: unlink d2/d7/d8/d1f/c3a 0 2024-04-24T23:14:13.173 INFO:tasks.workunit.client.1.smithi040.stdout:5/991: rename d2/d7/d8/d1f/d90/dcb/ffd to d2/d7/d8/d64/f131 0 2024-04-24T23:14:13.176 INFO:tasks.workunit.client.1.smithi040.stdout:5/992: creat d2/d7/d8/f132 x:0 0 0 2024-04-24T23:14:13.178 INFO:tasks.workunit.client.1.smithi040.stdout:5/993: readlink d2/d7/d8/d1f/d57/d3e/l126 0 2024-04-24T23:14:13.183 INFO:tasks.workunit.client.1.smithi040.stdout:5/994: dread d2/d7/d8/d6d/fc7 [0,4194304] 0 2024-04-24T23:14:13.184 INFO:tasks.workunit.client.1.smithi040.stdout:5/995: write d2/d7/d8/d1f/d90/fff [935042,63421] 0 2024-04-24T23:14:13.185 INFO:tasks.workunit.client.1.smithi040.stdout:5/996: dread - d2/d2f/d14/ddf/f10a zero size 2024-04-24T23:14:13.192 INFO:tasks.workunit.client.1.smithi040.stdout:5/997: truncate d2/d7/d8/d1f/d57/d27/d3c/f95 2556413 0 2024-04-24T23:14:13.195 INFO:tasks.workunit.client.1.smithi040.stdout:5/998: unlink d2/d7/d8/db/da2/d11c/f12f 0 2024-04-24T23:14:13.198 INFO:tasks.workunit.client.1.smithi040.stdout:5/999: creat d2/d7/d8/d1f/d57/d27/d3c/da4/f133 x:0 0 0 2024-04-24T23:14:13.201 INFO:tasks.workunit.client.1.smithi040.stderr:+ rm -rf -- ./tmp.pF6GRdPU60 2024-04-24T23:14:13.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.234+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabc0111950 con 0x7fabb006b790 2024-04-24T23:14:13.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.235+0000 7fabbd7fa700 1 -- 172.21.15.31:0/1236561229 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fabc0111950 con 0x7fabb006b790 2024-04-24T23:14:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.238+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fabb006b790 msgr2=0x7fabb006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.238+0000 7fabc6b45700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fabb006b790 0x7fabb006dc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fabac0040d0 tx=0x7fabac009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.238+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc00fd3d0 msgr2=0x7fabc010fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.238+0000 7fabc6b45700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc00fd3d0 0x7fabc010fba0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009fd0 tx=0x7fabb400b040 comp rx=0 tx=0).stop 2024-04-24T23:14:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.239+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 shutdown_connections 2024-04-24T23:14:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.239+0000 7fabc6b45700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fabb006b790 0x7fabb006dc40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.239+0000 7fabc6b45700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fabc00fd3d0 0x7fabc010fba0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.239+0000 7fabc6b45700 1 --2- 172.21.15.31:0/1236561229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc00fdd20 0x7fabc01100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.239+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 >> 172.21.15.31:0/1236561229 conn(0x7fabc00f6b80 msgr2=0x7fabc00f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.239+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 shutdown_connections 2024-04-24T23:14:13.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.239+0000 7fabc6b45700 1 -- 172.21.15.31:0/1236561229 wait complete. 2024-04-24T23:14:13.251 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:14:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.360+0000 7f9d97347700 1 -- 172.21.15.31:0/1231790257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 msgr2=0x7f9d900fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.360+0000 7f9d97347700 1 --2- 172.21.15.31:0/1231790257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 0x7f9d900fd270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c004940 tx=0x7f9d8c009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.360+0000 7f9d97347700 1 -- 172.21.15.31:0/1231790257 shutdown_connections 2024-04-24T23:14:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.360+0000 7f9d97347700 1 --2- 172.21.15.31:0/1231790257 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d900fdfd0 0x7f9d900fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.360+0000 7f9d97347700 1 --2- 172.21.15.31:0/1231790257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 0x7f9d900fd270 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.360+0000 7f9d97347700 1 -- 172.21.15.31:0/1231790257 >> 172.21.15.31:0/1231790257 conn(0x7f9d900f8560 msgr2=0x7f9d900fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.360+0000 7f9d97347700 1 -- 172.21.15.31:0/1231790257 shutdown_connections 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.360+0000 7f9d97347700 1 -- 172.21.15.31:0/1231790257 wait complete. 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d97347700 1 Processor -- start 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d97347700 1 -- start start 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d97347700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 0x7f9d9010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d97347700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d900fdfd0 0x7f9d901100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d97347700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d901106e0 con 0x7f9d900fce60 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d97347700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90110f30 con 0x7f9d900fdfd0 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d950e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 0x7f9d9010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d950e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 0x7f9d9010fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58884/0 (socket says 172.21.15.31:58884) 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d950e3700 1 -- 172.21.15.31:0/990278936 learned_addr learned my addr 172.21.15.31:0/990278936 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d948e2700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d900fdfd0 0x7f9d901100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d950e3700 1 -- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d900fdfd0 msgr2=0x7f9d901100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d950e3700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d900fdfd0 0x7f9d901100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.361+0000 7f9d950e3700 1 -- 172.21.15.31:0/990278936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c005040 con 0x7f9d900fce60 2024-04-24T23:14:13.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.362+0000 7f9d950e3700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 0x7f9d9010fba0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c013870 tx=0x7f9d8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.362+0000 7f9d85ffb700 1 -- 172.21.15.31:0/990278936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d8c021070 con 0x7f9d900fce60 2024-04-24T23:14:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.362+0000 7f9d85ffb700 1 -- 172.21.15.31:0/990278936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d8c009430 con 0x7f9d900fce60 2024-04-24T23:14:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.362+0000 7f9d97347700 1 -- 172.21.15.31:0/990278936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d901111b0 con 0x7f9d900fce60 2024-04-24T23:14:13.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.362+0000 7f9d85ffb700 1 -- 172.21.15.31:0/990278936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d8c00e070 con 0x7f9d900fce60 2024-04-24T23:14:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.362+0000 7f9d97347700 1 -- 172.21.15.31:0/990278936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d901116f0 con 0x7f9d900fce60 2024-04-24T23:14:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.363+0000 7f9d7b7fe700 1 -- 172.21.15.31:0/990278936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d700052b0 con 0x7f9d900fce60 2024-04-24T23:14:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.364+0000 7f9d85ffb700 1 -- 172.21.15.31:0/990278936 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f9d8c013af0 con 0x7f9d900fce60 2024-04-24T23:14:13.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.364+0000 7f9d85ffb700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f9d7c06b730 0x7f9d7c06dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.364+0000 7f9d85ffb700 1 -- 172.21.15.31:0/990278936 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f9d8c08b5f0 con 0x7f9d900fce60 2024-04-24T23:14:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.365+0000 7f9d948e2700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f9d7c06b730 0x7f9d7c06dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.365+0000 7f9d948e2700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f9d7c06b730 0x7f9d7c06dbe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009b80 tx=0x7f9d8000b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.370+0000 7f9d85ffb700 1 -- 172.21.15.31:0/990278936 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f9d8c058d80 con 0x7f9d900fce60 2024-04-24T23:14:13.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.706+0000 7f9d7b7fe700 1 -- 172.21.15.31:0/990278936 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d70000bc0 con 0x7f9d7c06b730 2024-04-24T23:14:13.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.707+0000 7f9d85ffb700 1 -- 172.21.15.31:0/990278936 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f9d70000bc0 con 0x7f9d7c06b730 2024-04-24T23:14:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.710+0000 7f9d7b7fe700 1 -- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f9d7c06b730 msgr2=0x7f9d7c06dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.710+0000 7f9d7b7fe700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f9d7c06b730 0x7f9d7c06dbe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009b80 tx=0x7f9d8000b040 comp rx=0 tx=0).stop 2024-04-24T23:14:13.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.710+0000 7f9d7b7fe700 1 -- 172.21.15.31:0/990278936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 msgr2=0x7f9d9010fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.710+0000 7f9d7b7fe700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 0x7f9d9010fba0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c013870 tx=0x7f9d8c009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.711+0000 7f9d7b7fe700 1 -- 172.21.15.31:0/990278936 shutdown_connections 2024-04-24T23:14:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.711+0000 7f9d7b7fe700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f9d7c06b730 0x7f9d7c06dbe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.711+0000 7f9d7b7fe700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d900fce60 0x7f9d9010fba0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.711+0000 7f9d7b7fe700 1 --2- 172.21.15.31:0/990278936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9d900fdfd0 0x7f9d901100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.711+0000 7f9d7b7fe700 1 -- 172.21.15.31:0/990278936 >> 172.21.15.31:0/990278936 conn(0x7f9d900f8560 msgr2=0x7f9d900f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.711+0000 7f9d7b7fe700 1 -- 172.21.15.31:0/990278936 shutdown_connections 2024-04-24T23:14:13.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.711+0000 7f9d7b7fe700 1 -- 172.21.15.31:0/990278936 wait complete. 2024-04-24T23:14:13.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.841+0000 7f8f83c6a700 1 -- 172.21.15.31:0/225695032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fcf60 msgr2=0x7f8f7c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.841+0000 7f8f83c6a700 1 --2- 172.21.15.31:0/225695032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fcf60 0x7f8f7c0fd3d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c004940 tx=0x7f8f6c009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:13.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.841+0000 7f8f83c6a700 1 -- 172.21.15.31:0/225695032 shutdown_connections 2024-04-24T23:14:13.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.841+0000 7f8f83c6a700 1 --2- 172.21.15.31:0/225695032 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fcf60 0x7f8f7c0fd3d0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c004940 tx=0x7f8f6c009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:13.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.841+0000 7f8f83c6a700 1 --2- 172.21.15.31:0/225695032 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c0fbcb0 0x7f8f7c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.841+0000 7f8f83c6a700 1 -- 172.21.15.31:0/225695032 >> 172.21.15.31:0/225695032 conn(0x7f8f7c0f7430 msgr2=0x7f8f7c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:13.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.841+0000 7f8f83c6a700 1 -- 172.21.15.31:0/225695032 shutdown_connections 2024-04-24T23:14:13.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.841+0000 7f8f83c6a700 1 -- 172.21.15.31:0/225695032 wait complete. 2024-04-24T23:14:13.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f83c6a700 1 Processor -- start 2024-04-24T23:14:13.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f83c6a700 1 -- start start 2024-04-24T23:14:13.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f83c6a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fbcb0 0x7f8f7c10ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f83c6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c10cf60 0x7f8f7c10ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f83c6a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c10d420 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f83c6a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c10d560 con 0x7f8f7c10cf60 2024-04-24T23:14:13.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f81a06700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fbcb0 0x7f8f7c10ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f81a06700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fbcb0 0x7f8f7c10ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58898/0 (socket says 172.21.15.31:58898) 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.842+0000 7f8f81a06700 1 -- 172.21.15.31:0/226263972 learned_addr learned my addr 172.21.15.31:0/226263972 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f81205700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c10cf60 0x7f8f7c10ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f81a06700 1 -- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c10cf60 msgr2=0x7f8f7c10ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f81a06700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c10cf60 0x7f8f7c10ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f81a06700 1 -- 172.21.15.31:0/226263972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c005040 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f81a06700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fbcb0 0x7f8f7c10ca20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f78002a80 tx=0x7f8f780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f727fc700 1 -- 172.21.15.31:0/226263972 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f78007cc0 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f727fc700 1 -- 172.21.15.31:0/226263972 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f7800f040 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f727fc700 1 -- 172.21.15.31:0/226263972 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f7800dac0 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c10e410 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.843+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c10e9a0 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.845+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7c104fb0 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.845+0000 7f8f727fc700 1 -- 172.21.15.31:0/226263972 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f8f7800dc20 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.846+0000 7f8f727fc700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f8f6806b790 0x7f8f6806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:13.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.846+0000 7f8f727fc700 1 -- 172.21.15.31:0/226263972 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f8f78089fd0 con 0x7f8f7c0fbcb0 2024-04-24T23:14:13.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.846+0000 7f8f81205700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f8f6806b790 0x7f8f6806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:13.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.846+0000 7f8f81205700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f8f6806b790 0x7f8f6806dc40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00b970 tx=0x7f8f6c009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:13.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:13.851+0000 7f8f727fc700 1 -- 172.21.15.31:0/226263972 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8f78057640 con 0x7f8f7c0fbcb0 2024-04-24T23:14:14.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.142+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f7c1013d0 con 0x7f8f6806b790 2024-04-24T23:14:14.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.153+0000 7f8f727fc700 1 -- 172.21.15.31:0/226263972 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f8f7c1013d0 con 0x7f8f6806b790 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 starting - - - - 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (9m) 58s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (8m) 65s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 58s ago 8m 84.1M - 9.4.7 954c08fa6188 44ecf082adf8 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (3m) 58s ago 3m 135M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (3m) 58s ago 3m 11.6M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (3m) 65s ago 3m 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (3m) 65s ago 3m 13.2M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:9283 running (10m) 58s ago 10m 322M - 17.2.7-902-gc61472c8 0de407c1b3af 7b9a4ca2c955 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (95s) 65s ago 8m 491M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8b888d3a97fe 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (10m) 58s ago 10m 47.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (7m) 65s ago 7m 38.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:14:14.155 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 starting - - - - 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (8m) 65s ago 8m 16.0M - 1.5.0 0da6a335fe13 7bd2678617bc 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 58s ago 7m 95.4M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (6m) 58s ago 6m 99.8M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (6m) 58s ago 6m 87.7M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (5m) 65s ago 5m 104M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (5m) 65s ago 5m 99.5M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (4m) 65s ago 4m 93.9M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:14:14.156 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (7m) 58s ago 8m 58.2M - 2.43.0 a07b618ecd1d aac36626882b 2024-04-24T23:14:14.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f8f6806b790 msgr2=0x7f8f6806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f8f6806b790 0x7f8f6806dc40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00b970 tx=0x7f8f6c009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fbcb0 msgr2=0x7f8f7c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fbcb0 0x7f8f7c10ca20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8f78002a80 tx=0x7f8f780046b0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 shutdown_connections 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f8f6806b790 0x7f8f6806dc40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f7c0fbcb0 0x7f8f7c10ca20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 --2- 172.21.15.31:0/226263972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f7c10cf60 0x7f8f7c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 >> 172.21.15.31:0/226263972 conn(0x7f8f7c0f7430 msgr2=0x7f8f7c0f8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 shutdown_connections 2024-04-24T23:14:14.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.156+0000 7f8f83c6a700 1 -- 172.21.15.31:0/226263972 wait complete. 2024-04-24T23:14:14.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.279+0000 7f26abb43700 1 -- 172.21.15.31:0/1251279750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a4066260 msgr2=0x7f26a40ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.279+0000 7f26abb43700 1 --2- 172.21.15.31:0/1251279750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a4066260 0x7f26a40ffa00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f26a0004940 tx=0x7f26a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:14.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.279+0000 7f26abb43700 1 -- 172.21.15.31:0/1251279750 shutdown_connections 2024-04-24T23:14:14.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.279+0000 7f26abb43700 1 --2- 172.21.15.31:0/1251279750 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26a40fff40 0x7f26a4102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.279+0000 7f26abb43700 1 --2- 172.21.15.31:0/1251279750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a4066260 0x7f26a40ffa00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.279+0000 7f26abb43700 1 -- 172.21.15.31:0/1251279750 >> 172.21.15.31:0/1251279750 conn(0x7f26a40f7430 msgr2=0x7f26a40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.279+0000 7f26abb43700 1 -- 172.21.15.31:0/1251279750 shutdown_connections 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.279+0000 7f26abb43700 1 -- 172.21.15.31:0/1251279750 wait complete. 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26abb43700 1 Processor -- start 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26abb43700 1 -- start start 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26abb43700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26a4066260 0x7f26a410da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26abb43700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a40fff40 0x7f26a410df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26abb43700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a410e540 con 0x7f26a40fff40 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26abb43700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a410ed40 con 0x7f26a4066260 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26a90de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a40fff40 0x7f26a410df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26a90de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a40fff40 0x7f26a410df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58920/0 (socket says 172.21.15.31:58920) 2024-04-24T23:14:14.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.280+0000 7f26a90de700 1 -- 172.21.15.31:0/2359692420 learned_addr learned my addr 172.21.15.31:0/2359692420 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:14.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f26a98df700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26a4066260 0x7f26a410da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f26a90de700 1 -- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26a4066260 msgr2=0x7f26a410da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f26a90de700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26a4066260 0x7f26a410da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f26a90de700 1 -- 172.21.15.31:0/2359692420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0005040 con 0x7f26a40fff40 2024-04-24T23:14:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f26a90de700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a40fff40 0x7f26a410df40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f269400eb40 tx=0x7f269400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f269a7fc700 1 -- 172.21.15.31:0/2359692420 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2694007cc0 con 0x7f26a40fff40 2024-04-24T23:14:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f269a7fc700 1 -- 172.21.15.31:0/2359692420 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f269400f040 con 0x7f26a40fff40 2024-04-24T23:14:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f269a7fc700 1 -- 172.21.15.31:0/2359692420 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f269400ca10 con 0x7f26a40fff40 2024-04-24T23:14:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a410efc0 con 0x7f26a40fff40 2024-04-24T23:14:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.281+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a410f500 con 0x7f26a40fff40 2024-04-24T23:14:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.283+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a405f5c0 con 0x7f26a40fff40 2024-04-24T23:14:14.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.283+0000 7f269a7fc700 1 -- 172.21.15.31:0/2359692420 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f269400cb70 con 0x7f26a40fff40 2024-04-24T23:14:14.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.283+0000 7f269a7fc700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f269006b860 0x7f269006dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.284+0000 7f269a7fc700 1 -- 172.21.15.31:0/2359692420 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f2694089fe0 con 0x7f26a40fff40 2024-04-24T23:14:14.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.284+0000 7f26a98df700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f269006b860 0x7f269006dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.284+0000 7f26a98df700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f269006b860 0x7f269006dd10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f26a0004890 tx=0x7f26a0009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.288+0000 7f269a7fc700 1 -- 172.21.15.31:0/2359692420 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f2694050100 con 0x7f26a40fff40 2024-04-24T23:14:14.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.671+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f26a41058a0 con 0x7f26a40fff40 2024-04-24T23:14:14.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.671+0000 7f269a7fc700 1 -- 172.21.15.31:0/2359692420 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f2694057680 con 0x7f26a40fff40 2024-04-24T23:14:14.673 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:14:14.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:14:14.674 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:14:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-24T23:14:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:14:14.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:14:14.675 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f269006b860 msgr2=0x7f269006dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f269006b860 0x7f269006dd10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f26a0004890 tx=0x7f26a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a40fff40 msgr2=0x7f26a410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a40fff40 0x7f26a410df40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f269400eb40 tx=0x7f269400b040 comp rx=0 tx=0).stop 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 shutdown_connections 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f269006b860 0x7f269006dd10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26a4066260 0x7f26a410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 --2- 172.21.15.31:0/2359692420 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26a40fff40 0x7f26a410df40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 >> 172.21.15.31:0/2359692420 conn(0x7f26a40f7430 msgr2=0x7f26a40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 shutdown_connections 2024-04-24T23:14:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.675+0000 7f26abb43700 1 -- 172.21.15.31:0/2359692420 wait complete. 2024-04-24T23:14:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.822+0000 7f49c014f700 1 -- 172.21.15.31:0/88082759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b8100280 msgr2=0x7f49b81006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.822+0000 7f49c014f700 1 --2- 172.21.15.31:0/88082759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b8100280 0x7f49b81006f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f49a8004940 tx=0x7f49a8009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.822+0000 7f49c014f700 1 -- 172.21.15.31:0/88082759 shutdown_connections 2024-04-24T23:14:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.822+0000 7f49c014f700 1 --2- 172.21.15.31:0/88082759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b8100280 0x7f49b81006f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.822+0000 7f49c014f700 1 --2- 172.21.15.31:0/88082759 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b80fefd0 0x7f49b80ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.822+0000 7f49c014f700 1 -- 172.21.15.31:0/88082759 >> 172.21.15.31:0/88082759 conn(0x7f49b80fa8f0 msgr2=0x7f49b80fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:14.825 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:14 smithi031 ceph-mon[26411]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:14.825 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:14 smithi031 ceph-mon[26411]: pgmap v40: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 75 MiB/s rd, 164 MiB/s wr, 389 op/s 2024-04-24T23:14:14.825 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:14 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.822+0000 7f49c014f700 1 -- 172.21.15.31:0/88082759 shutdown_connections 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.823+0000 7f49c014f700 1 -- 172.21.15.31:0/88082759 wait complete. 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.823+0000 7f49c014f700 1 Processor -- start 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.823+0000 7f49c014f700 1 -- start start 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.823+0000 7f49c014f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b80fefd0 0x7f49b810fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.823+0000 7f49c014f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b8100280 0x7f49b81100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.823+0000 7f49c014f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b81106d0 con 0x7f49b80fefd0 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.823+0000 7f49c014f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b8110f20 con 0x7f49b8100280 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bdeeb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b80fefd0 0x7f49b810fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bd6ea700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b8100280 0x7f49b81100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bdeeb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b80fefd0 0x7f49b810fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58940/0 (socket says 172.21.15.31:58940) 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bdeeb700 1 -- 172.21.15.31:0/3687939304 learned_addr learned my addr 172.21.15.31:0/3687939304 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bd6ea700 1 -- 172.21.15.31:0/3687939304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b80fefd0 msgr2=0x7f49b810fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bd6ea700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b80fefd0 0x7f49b810fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bd6ea700 1 -- 172.21.15.31:0/3687939304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a8005040 con 0x7f49b8100280 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bdeeb700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b80fefd0 0x7f49b810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49bd6ea700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b8100280 0x7f49b81100d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49a8000c00 tx=0x7f49a8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.824+0000 7f49ae7fc700 1 -- 172.21.15.31:0/3687939304 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49a800e070 con 0x7f49b8100280 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.825+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b81111a0 con 0x7f49b8100280 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.825+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b8111680 con 0x7f49b8100280 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.825+0000 7f49ae7fc700 1 -- 172.21.15.31:0/3687939304 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49a801c400 con 0x7f49b8100280 2024-04-24T23:14:14.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.825+0000 7f49ae7fc700 1 -- 172.21.15.31:0/3687939304 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49a800b670 con 0x7f49b8100280 2024-04-24T23:14:14.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.826+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b805f5c0 con 0x7f49b8100280 2024-04-24T23:14:14.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.826+0000 7f49ae7fc700 1 -- 172.21.15.31:0/3687939304 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 24) v1 ==== 89379+0+0 (secure 0 0 0) 0x7f49a800b7d0 con 0x7f49b8100280 2024-04-24T23:14:14.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.827+0000 7f49ae7fc700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f49a406b790 0x7f49a406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:14.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.827+0000 7f49ae7fc700 1 -- 172.21.15.31:0/3687939304 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f49a808b6b0 con 0x7f49b8100280 2024-04-24T23:14:14.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.827+0000 7f49bdeeb700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f49a406b790 0x7f49a406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:14.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.828+0000 7f49bdeeb700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f49a406b790 0x7f49a406dc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f49b40041d0 tx=0x7f49b400b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:14.833 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:14.832+0000 7f49ae7fc700 1 -- 172.21.15.31:0/3687939304 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f49a8058ef0 con 0x7f49b8100280 2024-04-24T23:14:14.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:14 smithi040 ceph-mon[33128]: from='client.24449 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:14.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:14 smithi040 ceph-mon[33128]: pgmap v40: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 75 MiB/s rd, 164 MiB/s wr, 389 op/s 2024-04-24T23:14:14.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:14 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:15.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.171+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49b8111a00 con 0x7f49a406b790 2024-04-24T23:14:15.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.173+0000 7f49ae7fc700 1 -- 172.21.15.31:0/3687939304 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f49b8111a00 con 0x7f49a406b790 2024-04-24T23:14:15.174 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:14:15.174 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:14:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:14:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:14:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-24T23:14:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T23:14:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-04-24T23:14:15.175 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:14:15.175 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:14:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f49a406b790 msgr2=0x7f49a406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f49a406b790 0x7f49a406dc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f49b40041d0 tx=0x7f49b400b040 comp rx=0 tx=0).stop 2024-04-24T23:14:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b8100280 msgr2=0x7f49b81100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b8100280 0x7f49b81100d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f49a8000c00 tx=0x7f49a8009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:15.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 shutdown_connections 2024-04-24T23:14:15.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f49a406b790 0x7f49a406dc40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49b80fefd0 0x7f49b810fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 --2- 172.21.15.31:0/3687939304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49b8100280 0x7f49b81100d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:15.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.175+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 >> 172.21.15.31:0/3687939304 conn(0x7f49b80fa8f0 msgr2=0x7f49b80fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:15.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.176+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 shutdown_connections 2024-04-24T23:14:15.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:15.176+0000 7f49c014f700 1 -- 172.21.15.31:0/3687939304 wait complete. 2024-04-24T23:14:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:15 smithi031 ceph-mon[26411]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:15 smithi031 ceph-mon[26411]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:15.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:15 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2359692420' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:14:15.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:15 smithi040 ceph-mon[33128]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:15.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:15 smithi040 ceph-mon[33128]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:15.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:15 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2359692420' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:14:16.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:16 smithi031 ceph-mon[26411]: from='client.24465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:16.698 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:16 smithi031 ceph-mon[26411]: pgmap v41: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 84 MiB/s rd, 180 MiB/s wr, 458 op/s 2024-04-24T23:14:16.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:16 smithi040 ceph-mon[33128]: from='client.24465 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:16.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:16 smithi040 ceph-mon[33128]: pgmap v41: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 84 MiB/s rd, 180 MiB/s wr, 458 op/s 2024-04-24T23:14:18.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:18 smithi031 ceph-mon[26411]: pgmap v42: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 81 MiB/s rd, 173 MiB/s wr, 436 op/s 2024-04-24T23:14:18.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:18 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:18.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:18 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:18.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:18 smithi040 ceph-mon[33128]: pgmap v42: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 81 MiB/s rd, 173 MiB/s wr, 436 op/s 2024-04-24T23:14:18.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:18 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:18.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:18 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:19.643 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:19 smithi031 ceph-mon[26411]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-24T23:14:19.643 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:19 smithi031 ceph-mon[26411]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-24T23:14:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:19 smithi040 ceph-mon[33128]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-24T23:14:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:19 smithi040 ceph-mon[33128]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-24T23:14:20.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:20 smithi040 ceph-mon[33128]: pgmap v43: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 54 MiB/s rd, 113 MiB/s wr, 321 op/s 2024-04-24T23:14:20.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:20 smithi031 ceph-mon[26411]: pgmap v43: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 54 MiB/s rd, 113 MiB/s wr, 321 op/s 2024-04-24T23:14:22.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:22 smithi040 ceph-mon[33128]: pgmap v44: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 54 MiB/s rd, 114 MiB/s wr, 399 op/s 2024-04-24T23:14:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:22 smithi031 ceph-mon[26411]: pgmap v44: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 54 MiB/s rd, 114 MiB/s wr, 399 op/s 2024-04-24T23:14:24.805 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:24 smithi031 ceph-mon[26411]: pgmap v45: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 14 MiB/s rd, 29 MiB/s wr, 191 op/s 2024-04-24T23:14:24.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:24 smithi040 ceph-mon[33128]: pgmap v45: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 14 MiB/s rd, 29 MiB/s wr, 191 op/s 2024-04-24T23:14:26.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:26 smithi040 ceph-mon[33128]: pgmap v46: 65 pgs: 65 active+clean; 2.3 GiB data, 8.9 GiB used, 528 GiB / 536 GiB avail; 14 MiB/s rd, 30 MiB/s wr, 235 op/s 2024-04-24T23:14:26.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:26 smithi031 ceph-mon[26411]: pgmap v46: 65 pgs: 65 active+clean; 2.3 GiB data, 8.9 GiB used, 528 GiB / 536 GiB avail; 14 MiB/s rd, 30 MiB/s wr, 235 op/s 2024-04-24T23:14:27.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:27 smithi031 ceph-mon[26411]: pgmap v47: 65 pgs: 65 active+clean; 2.0 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 4.3 MiB/s rd, 13 MiB/s wr, 191 op/s 2024-04-24T23:14:27.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:27 smithi040 ceph-mon[33128]: pgmap v47: 65 pgs: 65 active+clean; 2.0 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 4.3 MiB/s rd, 13 MiB/s wr, 191 op/s 2024-04-24T23:14:29.133 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:28 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:29.133 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:28 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:29.133 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:28 smithi040 ceph-mon[33128]: Reconfiguring node-exporter.smithi040 (dependencies changed)... 2024-04-24T23:14:29.133 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:28 smithi040 ceph-mon[33128]: Reconfiguring daemon node-exporter.smithi040 on smithi040 2024-04-24T23:14:29.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:28 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:29.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:28 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:29.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:28 smithi031 ceph-mon[26411]: Reconfiguring node-exporter.smithi040 (dependencies changed)... 2024-04-24T23:14:29.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:28 smithi031 ceph-mon[26411]: Reconfiguring daemon node-exporter.smithi040 on smithi040 2024-04-24T23:14:30.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:29 smithi031 ceph-mon[26411]: pgmap v48: 65 pgs: 65 active+clean; 2.0 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-24T23:14:30.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:29 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:30.266 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:29 smithi040 ceph-mon[33128]: pgmap v48: 65 pgs: 65 active+clean; 2.0 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 147 op/s 2024-04-24T23:14:30.266 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:29 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:30.441 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T23:14:30.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-24T23:14:31.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:31.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:31.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:31 smithi040 ceph-mon[33128]: mgrmap e25: smithi040.qtdwuq(active, since 91s), standbys: smithi031.rrnmwr 2024-04-24T23:14:31.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:31.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:31.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:14:31.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T23:14:31.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:31.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:14:31.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T23:14:32.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:32.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:14:32.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T23:14:32.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:32.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:32.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:31 smithi031 ceph-mon[26411]: mgrmap e25: smithi040.qtdwuq(active, since 91s), standbys: smithi031.rrnmwr 2024-04-24T23:14:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:14:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T23:14:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:14:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T23:14:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:14:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T23:14:32.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: Upgrade: Need to upgrade myself (mgr.smithi040.qtdwuq) 2024-04-24T23:14:32.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: Upgrade: Need to upgrade myself (mgr.smithi040.qtdwuq) 2024-04-24T23:14:32.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:32 smithi031 ceph-mon[26411]: pgmap v49: 65 pgs: 65 active+clean; 1006 MiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 32 KiB/s rd, 1.6 MiB/s wr, 210 op/s 2024-04-24T23:14:32.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:14:32.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031.front.sepia.ceph.com:9093"}]: dispatch 2024-04-24T23:14:32.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:14:32.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031.front.sepia.ceph.com:3000"}]: dispatch 2024-04-24T23:14:32.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:14:32.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-04-24T23:14:32.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: Upgrade: Need to upgrade myself (mgr.smithi040.qtdwuq) 2024-04-24T23:14:32.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: Upgrade: Need to upgrade myself (mgr.smithi040.qtdwuq) 2024-04-24T23:14:32.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:32 smithi040 ceph-mon[33128]: pgmap v49: 65 pgs: 65 active+clean; 1006 MiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 32 KiB/s rd, 1.6 MiB/s wr, 210 op/s 2024-04-24T23:14:33.063 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-24T23:14:33.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-24T23:14:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:34 smithi040 ceph-mon[33128]: pgmap v50: 65 pgs: 65 active+clean; 1006 MiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-24T23:14:34.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:34 smithi031 ceph-mon[26411]: pgmap v50: 65 pgs: 65 active+clean; 1006 MiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 23 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-24T23:14:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:36 smithi040 ceph-mon[33128]: pgmap v51: 65 pgs: 65 active+clean; 500 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 33 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-04-24T23:14:36.922 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:36 smithi031 ceph-mon[26411]: pgmap v51: 65 pgs: 65 active+clean; 500 MiB data, 3.6 GiB used, 533 GiB / 536 GiB avail; 33 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-04-24T23:14:38.910 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:38 smithi031 ceph-mon[26411]: pgmap v52: 65 pgs: 65 active+clean; 279 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 162 op/s 2024-04-24T23:14:38.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:38 smithi040 ceph-mon[33128]: pgmap v52: 65 pgs: 65 active+clean; 279 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 1.1 MiB/s wr, 162 op/s 2024-04-24T23:14:40.845 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:40 smithi031 ceph-mon[26411]: pgmap v53: 65 pgs: 65 active+clean; 279 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 952 KiB/s wr, 137 op/s 2024-04-24T23:14:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:40 smithi040 ceph-mon[33128]: pgmap v53: 65 pgs: 65 active+clean; 279 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 952 KiB/s wr, 137 op/s 2024-04-24T23:14:42.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:42 smithi040 ceph-mon[33128]: pgmap v54: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-04-24T23:14:42.944 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:42 smithi031 ceph-mon[26411]: pgmap v54: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-04-24T23:14:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:44 smithi040 ceph-mon[33128]: pgmap v55: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 949 KiB/s wr, 212 op/s 2024-04-24T23:14:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:44 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:44 smithi031 ceph-mon[26411]: pgmap v55: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 949 KiB/s wr, 212 op/s 2024-04-24T23:14:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:44 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.317+0000 7f5daed2c700 1 -- 172.21.15.31:0/2735630374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fbcb0 msgr2=0x7f5da80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.317+0000 7f5daed2c700 1 --2- 172.21.15.31:0/2735630374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fbcb0 0x7f5da80fc0c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c004940 tx=0x7f5d9c009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.318+0000 7f5daed2c700 1 -- 172.21.15.31:0/2735630374 shutdown_connections 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.318+0000 7f5daed2c700 1 --2- 172.21.15.31:0/2735630374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da80fcf60 0x7f5da80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.318+0000 7f5daed2c700 1 --2- 172.21.15.31:0/2735630374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fbcb0 0x7f5da80fc0c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.318+0000 7f5daed2c700 1 -- 172.21.15.31:0/2735630374 >> 172.21.15.31:0/2735630374 conn(0x7f5da80f7430 msgr2=0x7f5da80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.318+0000 7f5daed2c700 1 -- 172.21.15.31:0/2735630374 shutdown_connections 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.318+0000 7f5daed2c700 1 -- 172.21.15.31:0/2735630374 wait complete. 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.318+0000 7f5daed2c700 1 Processor -- start 2024-04-24T23:14:45.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.318+0000 7f5daed2c700 1 -- start start 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5daed2c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da80fbcb0 0x7f5da810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5daed2c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fcf60 0x7f5da810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5daed2c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da810e590 con 0x7f5da80fcf60 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5daed2c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da810ed90 con 0x7f5da80fbcb0 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5da7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fcf60 0x7f5da810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5dacac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da80fbcb0 0x7f5da810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5dacac8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da80fbcb0 0x7f5da810da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:48264/0 (socket says 172.21.15.31:48264) 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5da7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fcf60 0x7f5da810df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34990/0 (socket says 172.21.15.31:34990) 2024-04-24T23:14:45.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5dacac8700 1 -- 172.21.15.31:0/135515010 learned_addr learned my addr 172.21.15.31:0/135515010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5dacac8700 1 -- 172.21.15.31:0/135515010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fcf60 msgr2=0x7f5da810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5dacac8700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fcf60 0x7f5da810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5dacac8700 1 -- 172.21.15.31:0/135515010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c005040 con 0x7f5da80fbcb0 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5da7fff700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fcf60 0x7f5da810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5dacac8700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da80fbcb0 0x7f5da810da50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009150 tx=0x7f5d9c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5da57fa700 1 -- 172.21.15.31:0/135515010 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d9c00e040 con 0x7f5da80fbcb0 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da810f010 con 0x7f5da80fbcb0 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5da57fa700 1 -- 172.21.15.31:0/135515010 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d9c00bbf0 con 0x7f5da80fbcb0 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5da57fa700 1 -- 172.21.15.31:0/135515010 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d9c01c630 con 0x7f5da80fbcb0 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.319+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da810f550 con 0x7f5da80fbcb0 2024-04-24T23:14:45.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.320+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da805f5c0 con 0x7f5da80fbcb0 2024-04-24T23:14:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.321+0000 7f5da57fa700 1 -- 172.21.15.31:0/135515010 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f5d9c00bd60 con 0x7f5da80fbcb0 2024-04-24T23:14:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.321+0000 7f5da57fa700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5d9806b680 0x7f5d9806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.322+0000 7f5da57fa700 1 -- 172.21.15.31:0/135515010 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f5d9c08b4e0 con 0x7f5da80fbcb0 2024-04-24T23:14:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.322+0000 7f5da7fff700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5d9806b680 0x7f5d9806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.322+0000 7f5da7fff700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5d9806b680 0x7f5d9806db30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400f7c0 tx=0x7f5d94011040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:45.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.326+0000 7f5da57fa700 1 -- 172.21.15.31:0/135515010 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f5d9c058e20 con 0x7f5da80fbcb0 2024-04-24T23:14:45.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.607+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5da810f990 con 0x7f5d9806b680 2024-04-24T23:14:45.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.609+0000 7f5da57fa700 1 -- 172.21.15.31:0/135515010 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f5da810f990 con 0x7f5d9806b680 2024-04-24T23:14:45.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5d9806b680 msgr2=0x7f5d9806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5d9806b680 0x7f5d9806db30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400f7c0 tx=0x7f5d94011040 comp rx=0 tx=0).stop 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da80fbcb0 msgr2=0x7f5da810da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da80fbcb0 0x7f5da810da50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009150 tx=0x7f5d9c009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 shutdown_connections 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f5d9806b680 0x7f5d9806db30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5da80fbcb0 0x7f5da810da50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 --2- 172.21.15.31:0/135515010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da80fcf60 0x7f5da810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 >> 172.21.15.31:0/135515010 conn(0x7f5da80f7430 msgr2=0x7f5da8065b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 shutdown_connections 2024-04-24T23:14:45.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.611+0000 7f5daed2c700 1 -- 172.21.15.31:0/135515010 wait complete. 2024-04-24T23:14:45.623 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:14:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.729+0000 7fd7e7c29700 1 -- 172.21.15.31:0/141804167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e0066260 msgr2=0x7fd7e00ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.729+0000 7fd7e7c29700 1 --2- 172.21.15.31:0/141804167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e0066260 0x7fd7e00ffa00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc004940 tx=0x7fd7dc009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.729+0000 7fd7e7c29700 1 -- 172.21.15.31:0/141804167 shutdown_connections 2024-04-24T23:14:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.729+0000 7fd7e7c29700 1 --2- 172.21.15.31:0/141804167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e00fff40 0x7fd7e0102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.729+0000 7fd7e7c29700 1 --2- 172.21.15.31:0/141804167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e0066260 0x7fd7e00ffa00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.729+0000 7fd7e7c29700 1 -- 172.21.15.31:0/141804167 >> 172.21.15.31:0/141804167 conn(0x7fd7e00f7430 msgr2=0x7fd7e00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.729+0000 7fd7e7c29700 1 -- 172.21.15.31:0/141804167 shutdown_connections 2024-04-24T23:14:45.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.729+0000 7fd7e7c29700 1 -- 172.21.15.31:0/141804167 wait complete. 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e7c29700 1 Processor -- start 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e7c29700 1 -- start start 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e7c29700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e0066260 0x7fd7e010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e7c29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e00fff40 0x7fd7e010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e7c29700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e010e570 con 0x7fd7e00fff40 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e7c29700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e010edc0 con 0x7fd7e0066260 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e51c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e00fff40 0x7fd7e010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e51c4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e00fff40 0x7fd7e010df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35010/0 (socket says 172.21.15.31:35010) 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e51c4700 1 -- 172.21.15.31:0/1586088651 learned_addr learned my addr 172.21.15.31:0/1586088651 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e59c5700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e0066260 0x7fd7e010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e59c5700 1 -- 172.21.15.31:0/1586088651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e00fff40 msgr2=0x7fd7e010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:45.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e59c5700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e00fff40 0x7fd7e010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e59c5700 1 -- 172.21.15.31:0/1586088651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc005040 con 0x7fd7e0066260 2024-04-24T23:14:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.730+0000 7fd7e51c4700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e00fff40 0x7fd7e010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.731+0000 7fd7e59c5700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e0066260 0x7fd7e010da30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000c00 tx=0x7fd7dc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.731+0000 7fd7d67fc700 1 -- 172.21.15.31:0/1586088651 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7dc00e040 con 0x7fd7e0066260 2024-04-24T23:14:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.731+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e010f040 con 0x7fd7e0066260 2024-04-24T23:14:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.731+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e010f550 con 0x7fd7e0066260 2024-04-24T23:14:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.731+0000 7fd7d67fc700 1 -- 172.21.15.31:0/1586088651 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7dc00bbf0 con 0x7fd7e0066260 2024-04-24T23:14:45.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.731+0000 7fd7d67fc700 1 -- 172.21.15.31:0/1586088651 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7dc013660 con 0x7fd7e0066260 2024-04-24T23:14:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.732+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e005f5c0 con 0x7fd7e0066260 2024-04-24T23:14:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.733+0000 7fd7d67fc700 1 -- 172.21.15.31:0/1586088651 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 25) v1 ==== 89277+0+0 (secure 0 0 0) 0x7fd7dc013890 con 0x7fd7e0066260 2024-04-24T23:14:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.733+0000 7fd7d67fc700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fd7cc06b670 0x7fd7cc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.733+0000 7fd7d67fc700 1 -- 172.21.15.31:0/1586088651 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fd7dc08b410 con 0x7fd7e0066260 2024-04-24T23:14:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.733+0000 7fd7e51c4700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fd7cc06b670 0x7fd7cc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:45.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.734+0000 7fd7e51c4700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fd7cc06b670 0x7fd7cc06db20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00041d0 tx=0x7fd7d000b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:45.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:45.738+0000 7fd7d67fc700 1 -- 172.21.15.31:0/1586088651 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fd7dc058dc0 con 0x7fd7e0066260 2024-04-24T23:14:46.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.013+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7e010f8f0 con 0x7fd7cc06b670 2024-04-24T23:14:46.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.014+0000 7fd7d67fc700 1 -- 172.21.15.31:0/1586088651 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fd7e010f8f0 con 0x7fd7cc06b670 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fd7cc06b670 msgr2=0x7fd7cc06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fd7cc06b670 0x7fd7cc06db20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00041d0 tx=0x7fd7d000b040 comp rx=0 tx=0).stop 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e0066260 msgr2=0x7fd7e010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e0066260 0x7fd7e010da30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc000c00 tx=0x7fd7dc009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 shutdown_connections 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fd7cc06b670 0x7fd7cc06db20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7e0066260 0x7fd7e010da30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 --2- 172.21.15.31:0/1586088651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd7e00fff40 0x7fd7e010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 >> 172.21.15.31:0/1586088651 conn(0x7fd7e00f7430 msgr2=0x7fd7e00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 shutdown_connections 2024-04-24T23:14:46.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.017+0000 7fd7e7c29700 1 -- 172.21.15.31:0/1586088651 wait complete. 2024-04-24T23:14:46.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.152+0000 7f3fc20e2700 1 -- 172.21.15.31:0/4195352505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fee90 msgr2=0x7f3fbc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:46.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.152+0000 7f3fc20e2700 1 --2- 172.21.15.31:0/4195352505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fee90 0x7f3fbc101270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0004940 tx=0x7f3fb0009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:46.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.152+0000 7f3fc20e2700 1 -- 172.21.15.31:0/4195352505 shutdown_connections 2024-04-24T23:14:46.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.152+0000 7f3fc20e2700 1 --2- 172.21.15.31:0/4195352505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fee90 0x7f3fbc101270 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.152+0000 7f3fc20e2700 1 --2- 172.21.15.31:0/4195352505 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0fc570 0x7f3fbc0fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.152+0000 7f3fc20e2700 1 -- 172.21.15.31:0/4195352505 >> 172.21.15.31:0/4195352505 conn(0x7f3fbc0f6740 msgr2=0x7f3fbc0f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:46.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.152+0000 7f3fc20e2700 1 -- 172.21.15.31:0/4195352505 shutdown_connections 2024-04-24T23:14:46.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.152+0000 7f3fc20e2700 1 -- 172.21.15.31:0/4195352505 wait complete. 2024-04-24T23:14:46.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.153+0000 7f3fc20e2700 1 Processor -- start 2024-04-24T23:14:46.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.153+0000 7f3fc20e2700 1 -- start start 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.153+0000 7f3fc20e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fc570 0x7f3fbc103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.153+0000 7f3fc20e2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0fee90 0x7f3fbc101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.153+0000 7f3fc20e2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc103c30 con 0x7f3fbc0fc570 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.153+0000 7f3fc20e2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc102270 con 0x7f3fbc0fee90 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fc570 0x7f3fbc103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fc570 0x7f3fbc103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35032/0 (socket says 172.21.15.31:35032) 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbb7fe700 1 -- 172.21.15.31:0/3295953743 learned_addr learned my addr 172.21.15.31:0/3295953743 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbaffd700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0fee90 0x7f3fbc101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbb7fe700 1 -- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0fee90 msgr2=0x7f3fbc101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbb7fe700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0fee90 0x7f3fbc101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbb7fe700 1 -- 172.21.15.31:0/3295953743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0005040 con 0x7f3fbc0fc570 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbaffd700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0fee90 0x7f3fbc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:46.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fbb7fe700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fc570 0x7f3fbc103630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00eb40 tx=0x7f3fac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.154+0000 7f3fa3fff700 1 -- 172.21.15.31:0/3295953743 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fac007cc0 con 0x7f3fbc0fc570 2024-04-24T23:14:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.155+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1024f0 con 0x7f3fbc0fc570 2024-04-24T23:14:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.155+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc102a30 con 0x7f3fbc0fc570 2024-04-24T23:14:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.155+0000 7f3fa3fff700 1 -- 172.21.15.31:0/3295953743 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fac00f040 con 0x7f3fbc0fc570 2024-04-24T23:14:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.155+0000 7f3fa3fff700 1 -- 172.21.15.31:0/3295953743 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fac00cb60 con 0x7f3fbc0fc570 2024-04-24T23:14:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.156+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa80052e0 con 0x7f3fbc0fc570 2024-04-24T23:14:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.156+0000 7f3fa3fff700 1 -- 172.21.15.31:0/3295953743 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f3fac00ccc0 con 0x7f3fbc0fc570 2024-04-24T23:14:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.157+0000 7f3fa3fff700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fa406b670 0x7f3fa406db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.157+0000 7f3fa3fff700 1 -- 172.21.15.31:0/3295953743 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f3fac089ac0 con 0x7f3fbc0fc570 2024-04-24T23:14:46.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.157+0000 7f3fbaffd700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fa406b670 0x7f3fa406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:46.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.158+0000 7f3fbaffd700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fa406b670 0x7f3fa406db20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000b0d0 tx=0x7f3fb0009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:46.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.162+0000 7f3fa3fff700 1 -- 172.21.15.31:0/3295953743 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3fac050100 con 0x7f3fbc0fc570 2024-04-24T23:14:46.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.548+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fa8000bf0 con 0x7f3fa406b670 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.558+0000 7f3fa3fff700 1 -- 172.21.15.31:0/3295953743 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f3fa8000bf0 con 0x7f3fa406b670 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 starting - - - - 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (9m) 91s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (8m) 97s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 starting - - - - 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (3m) 91s ago 3m 135M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (4m) 91s ago 4m 11.6M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (4m) 97s ago 3m 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (3m) 97s ago 3m 13.2M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:14:46.560 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:9283 running (10m) 91s ago 10m 322M - 17.2.7-902-gc61472c8 0de407c1b3af 7b9a4ca2c955 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (2m) 97s ago 8m 491M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8b888d3a97fe 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (11m) 91s ago 11m 47.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (8m) 97s ago 8m 38.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 starting - - - - 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 starting - - - - 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 91s ago 7m 95.4M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (7m) 91s ago 7m 99.8M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (6m) 91s ago 6m 87.7M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (6m) 97s ago 6m 104M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (5m) 97s ago 5m 99.5M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (5m) 97s ago 5m 93.9M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:14:46.561 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 starting - - - - 2024-04-24T23:14:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fa406b670 msgr2=0x7f3fa406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fa406b670 0x7f3fa406db20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000b0d0 tx=0x7f3fb0009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fc570 msgr2=0x7f3fbc103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fc570 0x7f3fbc103630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00eb40 tx=0x7f3fac00b040 comp rx=0 tx=0).stop 2024-04-24T23:14:46.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 shutdown_connections 2024-04-24T23:14:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f3fa406b670 0x7f3fa406db20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3fbc0fc570 0x7f3fbc103630 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 --2- 172.21.15.31:0/3295953743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3fbc0fee90 0x7f3fbc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.560+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 >> 172.21.15.31:0/3295953743 conn(0x7f3fbc0f6740 msgr2=0x7f3fbc0f6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.561+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 shutdown_connections 2024-04-24T23:14:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.561+0000 7f3fc20e2700 1 -- 172.21.15.31:0/3295953743 wait complete. 2024-04-24T23:14:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 -- 172.21.15.31:0/532049791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 msgr2=0x7f6654101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 --2- 172.21.15.31:0/532049791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 0x7f6654101e00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6644004940 tx=0x7f6644009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 -- 172.21.15.31:0/532049791 shutdown_connections 2024-04-24T23:14:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 --2- 172.21.15.31:0/532049791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6654102340 0x7f6654104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 --2- 172.21.15.31:0/532049791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 0x7f6654101e00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 -- 172.21.15.31:0/532049791 >> 172.21.15.31:0/532049791 conn(0x7f66540f7430 msgr2=0x7f66540f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:46.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 -- 172.21.15.31:0/532049791 shutdown_connections 2024-04-24T23:14:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 -- 172.21.15.31:0/532049791 wait complete. 2024-04-24T23:14:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.694+0000 7f665a232700 1 Processor -- start 2024-04-24T23:14:46.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f665a232700 1 -- start start 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f665a232700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 0x7f665410fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f665a232700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6654102340 0x7f6654110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f665a232700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6654110780 con 0x7f6654066260 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f665a232700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6654110fd0 con 0x7f6654102340 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f66537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 0x7f665410fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f66537fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 0x7f665410fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35060/0 (socket says 172.21.15.31:35060) 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f6652ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6654102340 0x7f6654110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.695+0000 7f66537fe700 1 -- 172.21.15.31:0/4191688787 learned_addr learned my addr 172.21.15.31:0/4191688787 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f66537fe700 1 -- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6654102340 msgr2=0x7f6654110180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f66537fe700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6654102340 0x7f6654110180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f66537fe700 1 -- 172.21.15.31:0/4191688787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6644005040 con 0x7f6654066260 2024-04-24T23:14:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f6652ffd700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6654102340 0x7f6654110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:14:46.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f66537fe700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 0x7f665410fc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f664400b3b0 tx=0x7f6644009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:46.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f663bfff700 1 -- 172.21.15.31:0/4191688787 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f664400e040 con 0x7f6654066260 2024-04-24T23:14:46.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f663bfff700 1 -- 172.21.15.31:0/4191688787 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f664400bbf0 con 0x7f6654066260 2024-04-24T23:14:46.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6654111250 con 0x7f6654066260 2024-04-24T23:14:46.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f663bfff700 1 -- 172.21.15.31:0/4191688787 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f664401c630 con 0x7f6654066260 2024-04-24T23:14:46.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.696+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6654111790 con 0x7f6654066260 2024-04-24T23:14:46.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.697+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665405f5c0 con 0x7f6654066260 2024-04-24T23:14:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.698+0000 7f663bfff700 1 -- 172.21.15.31:0/4191688787 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f664400bd60 con 0x7f6654066260 2024-04-24T23:14:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.699+0000 7f663bfff700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f663c06b670 0x7f663c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.699+0000 7f663bfff700 1 -- 172.21.15.31:0/4191688787 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7f664408ac90 con 0x7f6654066260 2024-04-24T23:14:46.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.699+0000 7f6652ffd700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f663c06b670 0x7f663c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:46.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.699+0000 7f6652ffd700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f663c06b670 0x7f663c06db20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6648010520 tx=0x7f664800b040 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:46.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:46.703+0000 7f663bfff700 1 -- 172.21.15.31:0/4191688787 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6644058700 con 0x7f6654066260 2024-04-24T23:14:46.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:46 smithi031 ceph-mon[26411]: pgmap v56: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 298 op/s 2024-04-24T23:14:46.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:46 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:46.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:46 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.rrnmwr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:14:46.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:46 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.rrnmwr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:14:46.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:46 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:14:46.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:46 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:46 smithi040 ceph-mon[33128]: pgmap v56: 65 pgs: 65 active+clean; 283 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 298 op/s 2024-04-24T23:14:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:46 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:46 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.rrnmwr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:14:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:46 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.rrnmwr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:14:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:46 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:14:46.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:46 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.096+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6654107a90 con 0x7f6654066260 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.097+0000 7f663bfff700 1 -- 172.21.15.31:0/4191688787 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f6644058290 con 0x7f6654066260 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:14:47.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:14:47.100 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:14:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f663c06b670 msgr2=0x7f663c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f663c06b670 0x7f663c06db20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6648010520 tx=0x7f664800b040 comp rx=0 tx=0).stop 2024-04-24T23:14:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 msgr2=0x7f665410fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 0x7f665410fc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f664400b3b0 tx=0x7f6644009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:47.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 shutdown_connections 2024-04-24T23:14:47.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7f663c06b670 0x7f663c06db20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6654066260 0x7f665410fc40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 --2- 172.21.15.31:0/4191688787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6654102340 0x7f6654110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 >> 172.21.15.31:0/4191688787 conn(0x7f66540f7430 msgr2=0x7f66540f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:47.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 shutdown_connections 2024-04-24T23:14:47.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.099+0000 7f665a232700 1 -- 172.21.15.31:0/4191688787 wait complete. 2024-04-24T23:14:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.233+0000 7fe7bea2d700 1 -- 172.21.15.31:0/3305698319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fbce0 msgr2=0x7fe7b80fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.233+0000 7fe7bea2d700 1 --2- 172.21.15.31:0/3305698319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fbce0 0x7fe7b80fc0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8004940 tx=0x7fe7a8009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.234+0000 7fe7bea2d700 1 -- 172.21.15.31:0/3305698319 shutdown_connections 2024-04-24T23:14:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.234+0000 7fe7bea2d700 1 --2- 172.21.15.31:0/3305698319 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7b80fcf90 0x7fe7b80fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.234+0000 7fe7bea2d700 1 --2- 172.21.15.31:0/3305698319 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fbce0 0x7fe7b80fc0f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.234+0000 7fe7bea2d700 1 -- 172.21.15.31:0/3305698319 >> 172.21.15.31:0/3305698319 conn(0x7fe7b80f7440 msgr2=0x7fe7b80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.234+0000 7fe7bea2d700 1 -- 172.21.15.31:0/3305698319 shutdown_connections 2024-04-24T23:14:47.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.234+0000 7fe7bea2d700 1 -- 172.21.15.31:0/3305698319 wait complete. 2024-04-24T23:14:47.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.234+0000 7fe7bea2d700 1 Processor -- start 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.235+0000 7fe7bea2d700 1 -- start start 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.235+0000 7fe7bea2d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7b80fbce0 0x7fe7b810d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.235+0000 7fe7bea2d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fcf90 0x7fe7b810dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.235+0000 7fe7bea2d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b810e380 con 0x7fe7b80fcf90 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.235+0000 7fe7bea2d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b810eb80 con 0x7fe7b80fbce0 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.235+0000 7fe7b7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7b80fbce0 0x7fe7b810d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.235+0000 7fe7b7fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7b80fbce0 0x7fe7b810d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:48348/0 (socket says 172.21.15.31:48348) 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.235+0000 7fe7b7fff700 1 -- 172.21.15.31:0/2305312745 learned_addr learned my addr 172.21.15.31:0/2305312745 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.236+0000 7fe7b77fe700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fcf90 0x7fe7b810dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.236+0000 7fe7b77fe700 1 -- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7b80fbce0 msgr2=0x7fe7b810d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.236+0000 7fe7b77fe700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7b80fbce0 0x7fe7b810d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.236+0000 7fe7b77fe700 1 -- 172.21.15.31:0/2305312745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a8005040 con 0x7fe7b80fcf90 2024-04-24T23:14:47.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.236+0000 7fe7b77fe700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fcf90 0x7fe7b810dd80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009fd0 tx=0x7fe7ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:47.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.236+0000 7fe7b4ff9700 1 -- 172.21.15.31:0/2305312745 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7ac004e70 con 0x7fe7b80fcf90 2024-04-24T23:14:47.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.236+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b810ee00 con 0x7fe7b80fcf90 2024-04-24T23:14:47.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.236+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b810f340 con 0x7fe7b80fcf90 2024-04-24T23:14:47.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.237+0000 7fe7b4ff9700 1 -- 172.21.15.31:0/2305312745 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7ac00f040 con 0x7fe7b80fcf90 2024-04-24T23:14:47.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.237+0000 7fe7b4ff9700 1 -- 172.21.15.31:0/2305312745 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7ac00c980 con 0x7fe7b80fcf90 2024-04-24T23:14:47.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.238+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a40052e0 con 0x7fe7b80fcf90 2024-04-24T23:14:47.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.239+0000 7fe7b4ff9700 1 -- 172.21.15.31:0/2305312745 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 89277+0+0 (secure 0 0 0) 0x7fe7ac007cc0 con 0x7fe7b80fcf90 2024-04-24T23:14:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.239+0000 7fe7b4ff9700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fe7a006b680 0x7fe7a006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:14:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.240+0000 7fe7b4ff9700 1 -- 172.21.15.31:0/2305312745 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5808+0+0 (secure 0 0 0) 0x7fe7ac089c20 con 0x7fe7b80fcf90 2024-04-24T23:14:47.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.240+0000 7fe7b7fff700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fe7a006b680 0x7fe7a006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:14:47.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.240+0000 7fe7b7fff700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fe7a006b680 0x7fe7a006db30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8013870 tx=0x7fe7a8009e30 comp rx=0 tx=0).ready entity=mgr.24381 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:14:47.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.244+0000 7fe7b4ff9700 1 -- 172.21.15.31:0/2305312745 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fe7ac057430 con 0x7fe7b80fcf90 2024-04-24T23:14:47.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.584+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 --> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7a4000bf0 con 0x7fe7a006b680 2024-04-24T23:14:47.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.585+0000 7fe7b4ff9700 1 -- 172.21.15.31:0/2305312745 <== mgr.24381 v2:172.21.15.40:6828/2171597925 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe7a4000bf0 con 0x7fe7a006b680 2024-04-24T23:14:47.587 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:14:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:14:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:14:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:14:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-24T23:14:47.587 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "1/2 daemons upgraded", 2024-04-24T23:14:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-04-24T23:14:47.588 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:14:47.588 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:14:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fe7a006b680 msgr2=0x7fe7a006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fe7a006b680 0x7fe7a006db30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8013870 tx=0x7fe7a8009e30 comp rx=0 tx=0).stop 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fcf90 msgr2=0x7fe7b810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fcf90 0x7fe7b810dd80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009fd0 tx=0x7fe7ac00b040 comp rx=0 tx=0).stop 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 shutdown_connections 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:6828/2171597925,v1:172.21.15.40:6829/2171597925] conn(0x7fe7a006b680 0x7fe7a006db30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe7b80fbce0 0x7fe7b810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 --2- 172.21.15.31:0/2305312745 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b80fcf90 0x7fe7b810dd80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.587+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 >> 172.21.15.31:0/2305312745 conn(0x7fe7b80f7440 msgr2=0x7fe7b80f8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.588+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 shutdown_connections 2024-04-24T23:14:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:14:47.588+0000 7fe7bea2d700 1 -- 172.21.15.31:0/2305312745 wait complete. 2024-04-24T23:14:47.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:47 smithi040 ceph-mon[33128]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:47.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:47 smithi040 ceph-mon[33128]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:47.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:47 smithi040 ceph-mon[33128]: Upgrade: Updating mgr.smithi031.rrnmwr 2024-04-24T23:14:47.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:47 smithi040 ceph-mon[33128]: Deploying daemon mgr.smithi031.rrnmwr on smithi031 2024-04-24T23:14:47.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:47 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/4191688787' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:14:47.951 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:47 smithi031 ceph-mon[26411]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:47.951 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:47 smithi031 ceph-mon[26411]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:47.951 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:47 smithi031 ceph-mon[26411]: Upgrade: Updating mgr.smithi031.rrnmwr 2024-04-24T23:14:47.951 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:47 smithi031 ceph-mon[26411]: Deploying daemon mgr.smithi031.rrnmwr on smithi031 2024-04-24T23:14:47.951 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:47 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/4191688787' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:14:48.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:48 smithi031 ceph-mon[26411]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:48.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:48 smithi031 ceph-mon[26411]: pgmap v57: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 293 op/s 2024-04-24T23:14:48.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:48 smithi040 ceph-mon[33128]: from='client.14686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:48.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:48 smithi040 ceph-mon[33128]: pgmap v57: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 293 op/s 2024-04-24T23:14:49.849 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:49 smithi031 ceph-mon[26411]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:49.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:49 smithi040 ceph-mon[33128]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:14:50.692 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:50 smithi031 ceph-mon[26411]: pgmap v58: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 276 op/s 2024-04-24T23:14:50.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:50 smithi040 ceph-mon[33128]: pgmap v58: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 276 op/s 2024-04-24T23:14:52.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:52 smithi031 ceph-mon[26411]: pgmap v59: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.6 MiB/s wr, 419 op/s 2024-04-24T23:14:52.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:52 smithi040 ceph-mon[33128]: pgmap v59: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 1.6 MiB/s wr, 419 op/s 2024-04-24T23:14:54.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:54 smithi031 ceph-mon[26411]: pgmap v60: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 282 op/s 2024-04-24T23:14:54.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:54 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:54.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:54 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:54.642 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:54 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:54.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:54 smithi040 ceph-mon[33128]: pgmap v60: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 282 op/s 2024-04-24T23:14:54.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:54 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:54.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:54 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:54.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:54 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:14:56.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:56 smithi040 ceph-mon[33128]: pgmap v61: 65 pgs: 65 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 385 op/s 2024-04-24T23:14:56.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:56 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:56.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:56 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:56.908 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:56 smithi031 ceph-mon[26411]: pgmap v61: 65 pgs: 65 active+clean; 293 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 385 op/s 2024-04-24T23:14:56.909 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:56 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:56.909 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:56 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:58 smithi031 ceph-mon[26411]: pgmap v62: 65 pgs: 65 active+clean; 292 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-04-24T23:14:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:58 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:58.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:58 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:58.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:58 smithi040 ceph-mon[33128]: pgmap v62: 65 pgs: 65 active+clean; 292 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-04-24T23:14:58.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:58 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:58.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:58 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:14:59.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:59 smithi040 ceph-mon[33128]: pgmap v63: 65 pgs: 65 active+clean; 292 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 289 op/s 2024-04-24T23:14:59.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:14:59 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:14:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:59 smithi031 ceph-mon[26411]: pgmap v63: 65 pgs: 65 active+clean; 292 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.1 MiB/s wr, 289 op/s 2024-04-24T23:14:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:14:59 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:15:02.130 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:02 smithi031 ceph-mon[26411]: pgmap v64: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 412 op/s 2024-04-24T23:15:02.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:02 smithi040 ceph-mon[33128]: pgmap v64: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.6 MiB/s wr, 412 op/s 2024-04-24T23:15:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: pgmap v65: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.0 MiB/s wr, 269 op/s 2024-04-24T23:15:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: Standby manager daemon smithi031.rrnmwr restarted 2024-04-24T23:15:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: Standby manager daemon smithi031.rrnmwr started 2024-04-24T23:15:04.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.rrnmwr/crt"}]: dispatch 2024-04-24T23:15:04.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:15:04.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.rrnmwr/key"}]: dispatch 2024-04-24T23:15:04.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:15:04.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:04.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:04 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: pgmap v65: 65 pgs: 65 active+clean; 289 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.0 MiB/s wr, 269 op/s 2024-04-24T23:15:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: Standby manager daemon smithi031.rrnmwr restarted 2024-04-24T23:15:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: Standby manager daemon smithi031.rrnmwr started 2024-04-24T23:15:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.rrnmwr/crt"}]: dispatch 2024-04-24T23:15:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:15:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.rrnmwr/key"}]: dispatch 2024-04-24T23:15:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:15:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:04 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:05.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:05 smithi031 ceph-mon[26411]: mgrmap e26: smithi040.qtdwuq(active, since 2m), standbys: smithi031.rrnmwr 2024-04-24T23:15:05.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:05 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:05.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:05 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:05 smithi040 ceph-mon[33128]: mgrmap e26: smithi040.qtdwuq(active, since 2m), standbys: smithi031.rrnmwr 2024-04-24T23:15:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:05 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:05 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:06.583 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:06 smithi031 ceph-mon[26411]: pgmap v66: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.4 MiB/s wr, 382 op/s 2024-04-24T23:15:06.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:06 smithi040 ceph-mon[33128]: pgmap v66: 65 pgs: 65 active+clean; 280 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.4 MiB/s wr, 382 op/s 2024-04-24T23:15:08.796 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:08 smithi040 ceph-mon[33128]: pgmap v67: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 309 op/s 2024-04-24T23:15:08.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:08 smithi031 ceph-mon[26411]: pgmap v67: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 309 op/s 2024-04-24T23:15:10.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:10 smithi040 ceph-mon[33128]: pgmap v68: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 266 op/s 2024-04-24T23:15:10.970 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:10 smithi031 ceph-mon[26411]: pgmap v68: 65 pgs: 65 active+clean; 281 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.0 MiB/s wr, 266 op/s 2024-04-24T23:15:12.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:12 smithi031 ceph-mon[26411]: pgmap v69: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.6 MiB/s wr, 546 op/s 2024-04-24T23:15:12.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:12 smithi040 ceph-mon[33128]: pgmap v69: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.6 MiB/s wr, 546 op/s 2024-04-24T23:15:14.677 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: pgmap v70: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 424 op/s 2024-04-24T23:15:14.677 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:14.677 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:14.678 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:14.678 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:14.678 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:14.678 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:14.678 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr fail", "who": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:15:14.678 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr fail", "who": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:15:14.678 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:15:14.678 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd='[{"prefix": "mgr fail", "who": "smithi040.qtdwuq"}]': finished 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: mgrmap e27: smithi031.rrnmwr(active, starting, since 0.00643287s) 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:15:14.679 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:15:14.680 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:14.813 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: pgmap v70: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.2 MiB/s wr, 424 op/s 2024-04-24T23:15:14.813 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:14.813 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:14.813 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 172.21.15.40:0/2826765238' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr fail", "who": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "mgr fail", "who": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.24381 ' entity='mgr.smithi040.qtdwuq' cmd='[{"prefix": "mgr fail", "who": "smithi040.qtdwuq"}]': finished 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: mgrmap e27: smithi031.rrnmwr(active, starting, since 0.00643287s) 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:15:14.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:15:14.815 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:15.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:15 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:15:15.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:15 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:15:15.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:15 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:15:15.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:15 smithi040 ceph-mon[33128]: [24/Apr/2024:23:15:15] ENGINE Bus STARTING 2024-04-24T23:15:15.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:15 smithi040 ceph-mon[33128]: [24/Apr/2024:23:15:15] ENGINE Serving on http://172.21.15.31:8765 2024-04-24T23:15:15.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:15 smithi040 ceph-mon[33128]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T23:15:15.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:15 smithi040 ceph-mon[33128]: mgrmap e28: smithi031.rrnmwr(active, since 1.0092s) 2024-04-24T23:15:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:15 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:15:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:15 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:15:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:15 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:15:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:15 smithi031 ceph-mon[26411]: [24/Apr/2024:23:15:15] ENGINE Bus STARTING 2024-04-24T23:15:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:15 smithi031 ceph-mon[26411]: [24/Apr/2024:23:15:15] ENGINE Serving on http://172.21.15.31:8765 2024-04-24T23:15:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:15 smithi031 ceph-mon[26411]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T23:15:16.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:15 smithi031 ceph-mon[26411]: mgrmap e28: smithi031.rrnmwr(active, since 1.0092s) 2024-04-24T23:15:16.775 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:16 smithi031 ceph-mon[26411]: [24/Apr/2024:23:15:15] ENGINE Serving on https://172.21.15.31:7150 2024-04-24T23:15:16.775 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:16 smithi031 ceph-mon[26411]: [24/Apr/2024:23:15:15] ENGINE Bus STARTED 2024-04-24T23:15:16.775 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:16 smithi031 ceph-mon[26411]: pgmap v3: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:15:16.776 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:16 smithi031 ceph-mon[26411]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T23:15:16.776 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:16 smithi031 ceph-mon[26411]: Cluster is now healthy 2024-04-24T23:15:16.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:16 smithi040 ceph-mon[33128]: [24/Apr/2024:23:15:15] ENGINE Serving on https://172.21.15.31:7150 2024-04-24T23:15:16.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:16 smithi040 ceph-mon[33128]: [24/Apr/2024:23:15:15] ENGINE Bus STARTED 2024-04-24T23:15:16.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:16 smithi040 ceph-mon[33128]: pgmap v3: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:15:16.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:16 smithi040 ceph-mon[33128]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T23:15:16.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:16 smithi040 ceph-mon[33128]: Cluster is now healthy 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.749+0000 7f76e25e0700 1 -- 172.21.15.31:0/1579922113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc102340 msgr2=0x7f76dc104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.749+0000 7f76e25e0700 1 --2- 172.21.15.31:0/1579922113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc102340 0x7f76dc104720 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f76d0004940 tx=0x7f76d0009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.749+0000 7f76e25e0700 1 -- 172.21.15.31:0/1579922113 shutdown_connections 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.749+0000 7f76e25e0700 1 --2- 172.21.15.31:0/1579922113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc102340 0x7f76dc104720 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.749+0000 7f76e25e0700 1 --2- 172.21.15.31:0/1579922113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76dc066260 0x7f76dc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.749+0000 7f76e25e0700 1 -- 172.21.15.31:0/1579922113 >> 172.21.15.31:0/1579922113 conn(0x7f76dc0f7430 msgr2=0x7f76dc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.750+0000 7f76e25e0700 1 -- 172.21.15.31:0/1579922113 shutdown_connections 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.750+0000 7f76e25e0700 1 -- 172.21.15.31:0/1579922113 wait complete. 2024-04-24T23:15:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.750+0000 7f76e25e0700 1 Processor -- start 2024-04-24T23:15:17.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.750+0000 7f76e25e0700 1 -- start start 2024-04-24T23:15:17.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.750+0000 7f76e25e0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc066260 0x7f76dc10fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.750+0000 7f76e25e0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76dc102340 0x7f76dc110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.750+0000 7f76e25e0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc110700 con 0x7f76dc066260 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.750+0000 7f76e25e0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc110f00 con 0x7f76dc102340 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc066260 0x7f76dc10fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76db7fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76dc102340 0x7f76dc110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76dbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc066260 0x7f76dc10fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57034/0 (socket says 172.21.15.31:57034) 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76dbfff700 1 -- 172.21.15.31:0/3560416618 learned_addr learned my addr 172.21.15.31:0/3560416618 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76db7fe700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76dc102340 0x7f76dc110100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:42022/0 (socket says 172.21.15.31:42022) 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76db7fe700 1 -- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc066260 msgr2=0x7f76dc10fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:17.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76db7fe700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc066260 0x7f76dc10fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76db7fe700 1 -- 172.21.15.31:0/3560416618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0005040 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76db7fe700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76dc102340 0x7f76dc110100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76d0000c00 tx=0x7f76d0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76d8ff9700 1 -- 172.21.15.31:0/3560416618 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d000e030 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.751+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76dc111180 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.752+0000 7f76d8ff9700 1 -- 172.21.15.31:0/3560416618 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76d001c400 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.752+0000 7f76d8ff9700 1 -- 172.21.15.31:0/3560416618 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76d000b650 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.752+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76dc1116c0 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.752+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc05f5c0 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.753+0000 7f76d8ff9700 1 -- 172.21.15.31:0/3560416618 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 29) v1 ==== 46239+0+0 (secure 0 0 0) 0x7f76d000b840 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.753+0000 7f76d8ff9700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f76c4038e90 0x7f76c403b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.753+0000 7f76d8ff9700 1 -- 172.21.15.31:0/3560416618 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f76d004e490 con 0x7f76dc102340 2024-04-24T23:15:17.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.753+0000 7f76dbfff700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f76c4038e90 0x7f76c403b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.753+0000 7f76dbfff700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f76c4038e90 0x7f76c403b340 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f76cc004ba0 tx=0x7f76cc00b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:17.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:17.758+0000 7f76d8ff9700 1 -- 172.21.15.31:0/3560416618 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f76d0023350 con 0x7f76dc102340 2024-04-24T23:15:17.771 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:17 smithi040 ceph-mon[33128]: pgmap v4: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:15:17.771 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:17 smithi040 ceph-mon[33128]: mgrmap e29: smithi031.rrnmwr(active, since 2s) 2024-04-24T23:15:17.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:17 smithi031 ceph-mon[26411]: pgmap v4: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:15:17.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:17 smithi031 ceph-mon[26411]: mgrmap e29: smithi031.rrnmwr(active, since 2s) 2024-04-24T23:15:18.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.070+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76dc111a40 con 0x7f76c4038e90 2024-04-24T23:15:18.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.072+0000 7f76d8ff9700 1 -- 172.21.15.31:0/3560416618 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f76dc111a40 con 0x7f76c4038e90 2024-04-24T23:15:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f76c4038e90 msgr2=0x7f76c403b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f76c4038e90 0x7f76c403b340 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f76cc004ba0 tx=0x7f76cc00b040 comp rx=0 tx=0).stop 2024-04-24T23:15:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76dc102340 msgr2=0x7f76dc110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76dc102340 0x7f76dc110100 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76d0000c00 tx=0x7f76d0009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:18.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 shutdown_connections 2024-04-24T23:15:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f76c4038e90 0x7f76c403b340 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f76dc066260 0x7f76dc10fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 --2- 172.21.15.31:0/3560416618 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f76dc102340 0x7f76dc110100 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 >> 172.21.15.31:0/3560416618 conn(0x7f76dc0f7430 msgr2=0x7f76dc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 shutdown_connections 2024-04-24T23:15:18.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.074+0000 7f76e25e0700 1 -- 172.21.15.31:0/3560416618 wait complete. 2024-04-24T23:15:18.091 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:15:18.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.218+0000 7f374e118700 1 -- 172.21.15.31:0/1801966519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480fb420 msgr2=0x7f37480fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:18.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.218+0000 7f374e118700 1 --2- 172.21.15.31:0/1801966519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480fb420 0x7f37480fb830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3738009ce0 tx=0x7f3738004d10 comp rx=0 tx=0).stop 2024-04-24T23:15:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.218+0000 7f374e118700 1 -- 172.21.15.31:0/1801966519 shutdown_connections 2024-04-24T23:15:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.218+0000 7f374e118700 1 --2- 172.21.15.31:0/1801966519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37480fc6d0 0x7f37480fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.218+0000 7f374e118700 1 --2- 172.21.15.31:0/1801966519 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480fb420 0x7f37480fb830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.218+0000 7f374e118700 1 -- 172.21.15.31:0/1801966519 >> 172.21.15.31:0/1801966519 conn(0x7f37480f6b80 msgr2=0x7f37480f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.218+0000 7f374e118700 1 -- 172.21.15.31:0/1801966519 shutdown_connections 2024-04-24T23:15:18.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.218+0000 7f374e118700 1 -- 172.21.15.31:0/1801966519 wait complete. 2024-04-24T23:15:18.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f374e118700 1 Processor -- start 2024-04-24T23:15:18.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f374e118700 1 -- start start 2024-04-24T23:15:18.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f374e118700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37480fb420 0x7f374810b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:18.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f374e118700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480fc6d0 0x7f374810bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:18.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f374e118700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374810c330 con 0x7f37480fb420 2024-04-24T23:15:18.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f374e118700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374810c470 con 0x7f37480fc6d0 2024-04-24T23:15:18.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f37477fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37480fb420 0x7f374810b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:18.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f3746ffd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480fc6d0 0x7f374810bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:18.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f37477fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37480fb420 0x7f374810b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57048/0 (socket says 172.21.15.31:57048) 2024-04-24T23:15:18.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.219+0000 7f37477fe700 1 -- 172.21.15.31:0/1670982405 learned_addr learned my addr 172.21.15.31:0/1670982405 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:18.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.220+0000 7f37477fe700 1 -- 172.21.15.31:0/1670982405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480fc6d0 msgr2=0x7f374810bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.220+0000 7f37477fe700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480fc6d0 0x7f374810bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.220+0000 7f37477fe700 1 -- 172.21.15.31:0/1670982405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738009940 con 0x7f37480fb420 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.220+0000 7f37477fe700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37480fb420 0x7f374810b860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3738004540 tx=0x7f3738004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.220+0000 7f372ffff700 1 -- 172.21.15.31:0/1670982405 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f373800e040 con 0x7f37480fb420 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.221+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374810cd50 con 0x7f37480fb420 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.221+0000 7f372ffff700 1 -- 172.21.15.31:0/1670982405 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f373801cb20 con 0x7f37480fb420 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.221+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374810d290 con 0x7f37480fb420 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.221+0000 7f372ffff700 1 -- 172.21.15.31:0/1670982405 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3738012970 con 0x7f37480fb420 2024-04-24T23:15:18.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.222+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374805f5c0 con 0x7f37480fb420 2024-04-24T23:15:18.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.222+0000 7f372ffff700 1 -- 172.21.15.31:0/1670982405 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 29) v1 ==== 46239+0+0 (secure 0 0 0) 0x7f373801cc90 con 0x7f37480fb420 2024-04-24T23:15:18.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.222+0000 7f372ffff700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3730038e90 0x7f373003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:18.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.223+0000 7f372ffff700 1 -- 172.21.15.31:0/1670982405 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f373804ec20 con 0x7f37480fb420 2024-04-24T23:15:18.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.223+0000 7f3746ffd700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3730038e90 0x7f373003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:18.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.223+0000 7f3746ffd700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3730038e90 0x7f373003b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f374810c880 tx=0x7f373c00b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:18.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.230+0000 7f372ffff700 1 -- 172.21.15.31:0/1670982405 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f373801c650 con 0x7f37480fb420 2024-04-24T23:15:18.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.520+0000 7f372ffff700 1 -- 172.21.15.31:0/1670982405 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mgrmap(e 30) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f3738025400 con 0x7f37480fb420 2024-04-24T23:15:18.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.588+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f374810d630 con 0x7f3730038e90 2024-04-24T23:15:18.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.589+0000 7f372ffff700 1 -- 172.21.15.31:0/1670982405 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f374810d630 con 0x7f3730038e90 2024-04-24T23:15:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3730038e90 msgr2=0x7f373003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3730038e90 0x7f373003b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f374810c880 tx=0x7f373c00b040 comp rx=0 tx=0).stop 2024-04-24T23:15:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37480fb420 msgr2=0x7f374810b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37480fb420 0x7f374810b860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3738004540 tx=0x7f3738004d10 comp rx=0 tx=0).stop 2024-04-24T23:15:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 shutdown_connections 2024-04-24T23:15:18.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3730038e90 0x7f373003b340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37480fb420 0x7f374810b860 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 --2- 172.21.15.31:0/1670982405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f37480fc6d0 0x7f374810bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.592+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 >> 172.21.15.31:0/1670982405 conn(0x7f37480f6b80 msgr2=0x7f37480f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.593+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 shutdown_connections 2024-04-24T23:15:18.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.593+0000 7f374e118700 1 -- 172.21.15.31:0/1670982405 wait complete. 2024-04-24T23:15:18.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.748+0000 7f639de04700 1 -- 172.21.15.31:0/2844272432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fcf60 msgr2=0x7f63980fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:18.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.749+0000 7f639de04700 1 --2- 172.21.15.31:0/2844272432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fcf60 0x7f63980fd3d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f63880049c0 tx=0x7f6388009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:18.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.750+0000 7f639de04700 1 -- 172.21.15.31:0/2844272432 shutdown_connections 2024-04-24T23:15:18.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.750+0000 7f639de04700 1 --2- 172.21.15.31:0/2844272432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fcf60 0x7f63980fd3d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.750+0000 7f639de04700 1 --2- 172.21.15.31:0/2844272432 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63980fbcb0 0x7f63980fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.750+0000 7f639de04700 1 -- 172.21.15.31:0/2844272432 >> 172.21.15.31:0/2844272432 conn(0x7f63980f7430 msgr2=0x7f63980f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:18.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.751+0000 7f639de04700 1 -- 172.21.15.31:0/2844272432 shutdown_connections 2024-04-24T23:15:18.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.752+0000 7f639de04700 1 -- 172.21.15.31:0/2844272432 wait complete. 2024-04-24T23:15:18.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.752+0000 7f639de04700 1 Processor -- start 2024-04-24T23:15:18.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.753+0000 7f639de04700 1 -- start start 2024-04-24T23:15:18.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.753+0000 7f639de04700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fbcb0 0x7f6398113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:18.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.753+0000 7f639de04700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63980fcf60 0x7f6398114470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:18.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.753+0000 7f639de04700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398114a70 con 0x7f63980fbcb0 2024-04-24T23:15:18.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.753+0000 7f639de04700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398114be0 con 0x7f63980fcf60 2024-04-24T23:15:18.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.754+0000 7f63977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fbcb0 0x7f6398113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:18.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.754+0000 7f63977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fbcb0 0x7f6398113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49558/0 (socket says 172.21.15.31:49558) 2024-04-24T23:15:18.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.754+0000 7f63977fe700 1 -- 172.21.15.31:0/3105111513 learned_addr learned my addr 172.21.15.31:0/3105111513 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:18.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.754+0000 7f63977fe700 1 -- 172.21.15.31:0/3105111513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63980fcf60 msgr2=0x7f6398114470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-24T23:15:18.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.754+0000 7f63977fe700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63980fcf60 0x7f6398114470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:18.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.754+0000 7f63977fe700 1 -- 172.21.15.31:0/3105111513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388005040 con 0x7f63980fbcb0 2024-04-24T23:15:18.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.755+0000 7f63977fe700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fbcb0 0x7f6398113f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f639000b750 tx=0x7f63900079e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:18.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.756+0000 7f637ffff700 1 -- 172.21.15.31:0/3105111513 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6390010070 con 0x7f63980fbcb0 2024-04-24T23:15:18.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.756+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6398104c70 con 0x7f63980fbcb0 2024-04-24T23:15:18.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.757+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6398105230 con 0x7f63980fbcb0 2024-04-24T23:15:18.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.757+0000 7f637ffff700 1 -- 172.21.15.31:0/3105111513 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6390013370 con 0x7f63980fbcb0 2024-04-24T23:15:18.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.757+0000 7f637ffff700 1 -- 172.21.15.31:0/3105111513 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6390012550 con 0x7f63980fbcb0 2024-04-24T23:15:18.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.758+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63840052e0 con 0x7f63980fbcb0 2024-04-24T23:15:18.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.759+0000 7f637ffff700 1 -- 172.21.15.31:0/3105111513 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 30) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f63900139c0 con 0x7f63980fbcb0 2024-04-24T23:15:18.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.760+0000 7f637ffff700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6380038ff0 0x7f638003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:18.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.760+0000 7f637ffff700 1 -- 172.21.15.31:0/3105111513 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f639004cab0 con 0x7f63980fbcb0 2024-04-24T23:15:18.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.761+0000 7f6396ffd700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6380038ff0 0x7f638003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:18.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.763+0000 7f6396ffd700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6380038ff0 0x7f638003b4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6388005010 tx=0x7f638800b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:18.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:18.768+0000 7f637ffff700 1 -- 172.21.15.31:0/3105111513 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f639001e3b0 con 0x7f63980fbcb0 2024-04-24T23:15:18.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:18 smithi040 ceph-mon[33128]: from='client.24517 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:18.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:18.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:18 smithi031 ceph-mon[26411]: from='client.24517 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:19.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.084+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6384000bf0 con 0x7f6380038ff0 2024-04-24T23:15:19.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.102+0000 7f637ffff700 1 -- 172.21.15.31:0/3105111513 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6384000bf0 con 0x7f6380038ff0 2024-04-24T23:15:19.105 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:15:19.105 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (87s) 15s ago 10m 24.8M - 0.25.0 c8568f914cd2 3a7bccddf024 2024-04-24T23:15:19.105 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (10m) 15s ago 10m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:15:19.105 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (9m) 0s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:15:19.105 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 unknown 15s ago 9m - - 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (4m) 15s ago 4m 213M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (4m) 15s ago 4m 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (4m) 0s ago 4m 13.0M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (4m) 0s ago 4m 14.1M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (27s) 15s ago 11m 140M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (2m) 0s ago 9m 79.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8b888d3a97fe 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (11m) 15s ago 11m 50.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (9m) 0s ago 9m 42.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:15:19.106 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (118s) 15s ago 10m 15.9M - 1.5.0 0da6a335fe13 f6eda28e448e 2024-04-24T23:15:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (49s) 0s ago 9m 9605k - 1.5.0 0da6a335fe13 dfceecae86e1 2024-04-24T23:15:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (8m) 15s ago 8m 198M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:15:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (7m) 15s ago 7m 210M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:15:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (7m) 15s ago 7m 177M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:15:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (6m) 0s ago 6m 254M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:15:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (6m) 0s ago 6m 229M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:15:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (6m) 0s ago 6m 209M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:15:19.107 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (51s) 15s ago 9m 45.0M - 2.43.0 a07b618ecd1d 9466502b8e36 2024-04-24T23:15:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6380038ff0 msgr2=0x7f638003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6380038ff0 0x7f638003b4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6388005010 tx=0x7f638800b040 comp rx=0 tx=0).stop 2024-04-24T23:15:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fbcb0 msgr2=0x7f6398113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fbcb0 0x7f6398113f30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f639000b750 tx=0x7f63900079e0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 shutdown_connections 2024-04-24T23:15:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6380038ff0 0x7f638003b4a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f63980fbcb0 0x7f6398113f30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 --2- 172.21.15.31:0/3105111513 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63980fcf60 0x7f6398114470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 >> 172.21.15.31:0/3105111513 conn(0x7f63980f7430 msgr2=0x7f63980f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 shutdown_connections 2024-04-24T23:15:19.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.105+0000 7f639de04700 1 -- 172.21.15.31:0/3105111513 wait complete. 2024-04-24T23:15:19.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 -- 172.21.15.31:0/3675850051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0662c0 msgr2=0x7f3bbc0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 --2- 172.21.15.31:0/3675850051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0662c0 0x7f3bbc0ff9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3bac004940 tx=0x7f3bac009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 -- 172.21.15.31:0/3675850051 shutdown_connections 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 --2- 172.21.15.31:0/3675850051 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bbc0ffee0 0x7f3bbc102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 --2- 172.21.15.31:0/3675850051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0662c0 0x7f3bbc0ff9a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 -- 172.21.15.31:0/3675850051 >> 172.21.15.31:0/3675850051 conn(0x7f3bbc0f7430 msgr2=0x7f3bbc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 -- 172.21.15.31:0/3675850051 shutdown_connections 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 -- 172.21.15.31:0/3675850051 wait complete. 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.258+0000 7f3bc25e2700 1 Processor -- start 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bc25e2700 1 -- start start 2024-04-24T23:15:19.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bc25e2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bbc0662c0 0x7f3bbc0fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:19.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bc25e2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0ffee0 0x7f3bbc0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:19.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bc25e2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0fbe30 con 0x7f3bbc0ffee0 2024-04-24T23:15:19.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bc25e2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0fbf70 con 0x7f3bbc0662c0 2024-04-24T23:15:19.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bbb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0ffee0 0x7f3bbc0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:19.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bbc0662c0 0x7f3bbc0fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:19.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bbbfff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bbc0662c0 0x7f3bbc0fd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:35816/0 (socket says 172.21.15.31:35816) 2024-04-24T23:15:19.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bbb7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0ffee0 0x7f3bbc0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49570/0 (socket says 172.21.15.31:49570) 2024-04-24T23:15:19.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.259+0000 7f3bbbfff700 1 -- 172.21.15.31:0/4192342772 learned_addr learned my addr 172.21.15.31:0/4192342772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.260+0000 7f3bbbfff700 1 -- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0ffee0 msgr2=0x7f3bbc0fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.260+0000 7f3bbbfff700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0ffee0 0x7f3bbc0fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.260+0000 7f3bbbfff700 1 -- 172.21.15.31:0/4192342772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac005040 con 0x7f3bbc0662c0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.260+0000 7f3bbbfff700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bbc0662c0 0x7f3bbc0fd380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3bac000c00 tx=0x7f3bac009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.261+0000 7f3bb8ff9700 1 -- 172.21.15.31:0/4192342772 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bac00e030 con 0x7f3bbc0662c0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.261+0000 7f3bb8ff9700 1 -- 172.21.15.31:0/4192342772 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3bac013de0 con 0x7f3bbc0662c0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.261+0000 7f3bb8ff9700 1 -- 172.21.15.31:0/4192342772 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bac01ce50 con 0x7f3bbc0662c0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.261+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc0fc1f0 con 0x7f3bbc0662c0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.261+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc0fc620 con 0x7f3bbc0662c0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.262+0000 7f3bb8ff9700 1 -- 172.21.15.31:0/4192342772 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f3bac00b460 con 0x7f3bbc0662c0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.262+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba80052e0 con 0x7f3bbc0662c0 2024-04-24T23:15:19.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.262+0000 7f3bb8ff9700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3ba4038f10 0x7f3ba403b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:19.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.263+0000 7f3bb8ff9700 1 -- 172.21.15.31:0/4192342772 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f3bac04f690 con 0x7f3bbc0662c0 2024-04-24T23:15:19.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.263+0000 7f3bbb7fe700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3ba4038f10 0x7f3ba403b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:19.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.264+0000 7f3bbb7fe700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3ba4038f10 0x7f3ba403b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0fd960 tx=0x7f3bb000b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:19.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.268+0000 7f3bb8ff9700 1 -- 172.21.15.31:0/4192342772 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3bac023350 con 0x7f3bbc0662c0 2024-04-24T23:15:19.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.714+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3ba8005860 con 0x7f3bbc0662c0 2024-04-24T23:15:19.717 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:15:19.717 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:15:19.717 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:15:19.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:15:19.718 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:15:19.718 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:15:19.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:15:19.718 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:15:19.718 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:15:19.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:15:19.718 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:15:19.718 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:15:19.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:15:19.719 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:15:19.719 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T23:15:19.719 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:15:19.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:15:19.719 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:15:19.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.715+0000 7f3bb8ff9700 1 -- 172.21.15.31:0/4192342772 <== mon.1 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f3bac011c00 con 0x7f3bbc0662c0 2024-04-24T23:15:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.718+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3ba4038f10 msgr2=0x7f3ba403b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.718+0000 7f3bc25e2700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3ba4038f10 0x7f3ba403b3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc0fd960 tx=0x7f3bb000b040 comp rx=0 tx=0).stop 2024-04-24T23:15:19.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.719+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bbc0662c0 msgr2=0x7f3bbc0fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.719+0000 7f3bc25e2700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bbc0662c0 0x7f3bbc0fd380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3bac000c00 tx=0x7f3bac009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.719+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 shutdown_connections 2024-04-24T23:15:19.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.719+0000 7f3bc25e2700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f3ba4038f10 0x7f3ba403b3c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.719+0000 7f3bc25e2700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3bbc0662c0 0x7f3bbc0fd380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.719+0000 7f3bc25e2700 1 --2- 172.21.15.31:0/4192342772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bbc0ffee0 0x7f3bbc0fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.719+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 >> 172.21.15.31:0/4192342772 conn(0x7f3bbc0f7430 msgr2=0x7f3bbc0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.719+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 shutdown_connections 2024-04-24T23:15:19.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.720+0000 7f3bc25e2700 1 -- 172.21.15.31:0/4192342772 wait complete. 2024-04-24T23:15:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.865+0000 7f0b1361c700 1 -- 172.21.15.31:0/1238661853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b0c071de0 msgr2=0x7f0b0c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.865+0000 7f0b1361c700 1 --2- 172.21.15.31:0/1238661853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b0c071de0 0x7f0b0c0702f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0b08004940 tx=0x7f0b08009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.865+0000 7f0b1361c700 1 -- 172.21.15.31:0/1238661853 shutdown_connections 2024-04-24T23:15:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.865+0000 7f0b1361c700 1 --2- 172.21.15.31:0/1238661853 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b0c070830 0x7f0b0c070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.865+0000 7f0b1361c700 1 --2- 172.21.15.31:0/1238661853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b0c071de0 0x7f0b0c0702f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.865+0000 7f0b1361c700 1 -- 172.21.15.31:0/1238661853 >> 172.21.15.31:0/1238661853 conn(0x7f0b0c0f8590 msgr2=0x7f0b0c0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:19.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:19 smithi031 ceph-mon[26411]: pgmap v5: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:15:19.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:19 smithi031 ceph-mon[26411]: mgrmap e30: smithi031.rrnmwr(active, since 4s) 2024-04-24T23:15:19.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:19 smithi031 ceph-mon[26411]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:19.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:19 smithi031 ceph-mon[26411]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:19.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.865+0000 7f0b1361c700 1 -- 172.21.15.31:0/1238661853 shutdown_connections 2024-04-24T23:15:19.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.865+0000 7f0b1361c700 1 -- 172.21.15.31:0/1238661853 wait complete. 2024-04-24T23:15:19.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b1361c700 1 Processor -- start 2024-04-24T23:15:19.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b1361c700 1 -- start start 2024-04-24T23:15:19.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b1361c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b0c070830 0x7f0b0c10fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:19.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b1361c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b0c071de0 0x7f0b0c110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:19.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b1361c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c110700 con 0x7f0b0c071de0 2024-04-24T23:15:19.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b1361c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c110f50 con 0x7f0b0c070830 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b113b8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b0c070830 0x7f0b0c10fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b113b8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b0c070830 0x7f0b0c10fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.31:35832/0 (socket says 172.21.15.31:35832) 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b113b8700 1 -- 172.21.15.31:0/2540421797 learned_addr learned my addr 172.21.15.31:0/2540421797 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.866+0000 7f0b10bb7700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b0c071de0 0x7f0b0c110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b113b8700 1 -- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b0c071de0 msgr2=0x7f0b0c110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b113b8700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b0c071de0 0x7f0b0c110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b113b8700 1 -- 172.21.15.31:0/2540421797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08005040 con 0x7f0b0c070830 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b113b8700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b0c070830 0x7f0b0c10fbc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b08000c00 tx=0x7f0b08009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b01ffb700 1 -- 172.21.15.31:0/2540421797 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b0800e030 con 0x7f0b0c070830 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b01ffb700 1 -- 172.21.15.31:0/2540421797 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b08009430 con 0x7f0b0c070830 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1111d0 con 0x7f0b0c070830 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b01ffb700 1 -- 172.21.15.31:0/2540421797 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b0800b410 con 0x7f0b0c070830 2024-04-24T23:15:19.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.867+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c111710 con 0x7f0b0c070830 2024-04-24T23:15:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.868+0000 7f0b01ffb700 1 -- 172.21.15.31:0/2540421797 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 30) v1 ==== 46295+0+0 (secure 0 0 0) 0x7f0b0800b6d0 con 0x7f0b0c070830 2024-04-24T23:15:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.868+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c05f5c0 con 0x7f0b0c070830 2024-04-24T23:15:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.869+0000 7f0b01ffb700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0af8038f10 0x7f0af803b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:19.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.869+0000 7f0b01ffb700 1 -- 172.21.15.31:0/2540421797 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0b0804e240 con 0x7f0b0c070830 2024-04-24T23:15:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.869+0000 7f0b10bb7700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0af8038f10 0x7f0af803b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:19.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.870+0000 7f0b10bb7700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0af8038f10 0x7f0af803b3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009b80 tx=0x7f0afc00b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:19.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:19.874+0000 7f0b01ffb700 1 -- 172.21.15.31:0/2540421797 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0b08023360 con 0x7f0b0c070830 2024-04-24T23:15:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:19 smithi040 ceph-mon[33128]: pgmap v5: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:15:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:19 smithi040 ceph-mon[33128]: mgrmap e30: smithi031.rrnmwr(active, since 4s) 2024-04-24T23:15:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:19 smithi040 ceph-mon[33128]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:19 smithi040 ceph-mon[33128]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:20.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.211+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b0c1119d0 con 0x7f0af8038f10 2024-04-24T23:15:20.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.213+0000 7f0b01ffb700 1 -- 172.21.15.31:0/2540421797 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f0b0c1119d0 con 0x7f0af8038f10 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:15:20.215 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:15:20.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0af8038f10 msgr2=0x7f0af803b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:20.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0af8038f10 0x7f0af803b3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009b80 tx=0x7f0afc00b040 comp rx=0 tx=0).stop 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b0c070830 msgr2=0x7f0b0c10fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b0c070830 0x7f0b0c10fbc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0b08000c00 tx=0x7f0b08009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 shutdown_connections 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0af8038f10 0x7f0af803b3c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0b0c070830 0x7f0b0c10fbc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 --2- 172.21.15.31:0/2540421797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b0c071de0 0x7f0b0c110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 >> 172.21.15.31:0/2540421797 conn(0x7f0b0c0f8590 msgr2=0x7f0b0c0f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 shutdown_connections 2024-04-24T23:15:20.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:20.216+0000 7f0b1361c700 1 -- 172.21.15.31:0/2540421797 wait complete. 2024-04-24T23:15:20.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:20 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/4192342772' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:15:20.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:20 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:20.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:20 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:20.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:20 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:20.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:20 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:20.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:20 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/4192342772' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:15:20.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:20 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:20.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:20 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:20.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:20 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:20.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:20 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:22.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:21 smithi040 ceph-mon[33128]: from='client.24531 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:22.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:21 smithi040 ceph-mon[33128]: pgmap v6: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:15:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:21 smithi031 ceph-mon[26411]: from='client.24531 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:21 smithi031 ceph-mon[26411]: pgmap v6: 65 pgs: 65 active+clean; 272 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:15:23.209 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:23 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:23.209 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:23 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:23.209 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:23 smithi031 ceph-mon[26411]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:15:23.209 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:23 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/3589366715' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:15:23.209 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:23 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/3589366715' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:15:23.210 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:23 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/3589366715' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:15:23.210 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:23 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/3589366715' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:15:23.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:23 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:23.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:23 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:23.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:23 smithi040 ceph-mon[33128]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:15:23.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:23 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3589366715' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:15:23.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:23 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3589366715' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:15:23.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:23 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3589366715' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:15:23.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:23 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3589366715' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:15:24.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:24 smithi031 ceph-mon[26411]: pgmap v7: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 837 KiB/s wr, 452 op/s 2024-04-24T23:15:24.326 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:24 smithi031 ceph-mon[26411]: mgrmap e31: smithi031.rrnmwr(active, since 8s), standbys: smithi040.qtdwuq 2024-04-24T23:15:24.326 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:24 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi040.qtdwuq", "id": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:15:24.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:24 smithi040 ceph-mon[33128]: pgmap v7: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 52 KiB/s rd, 837 KiB/s wr, 452 op/s 2024-04-24T23:15:24.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:24 smithi040 ceph-mon[33128]: mgrmap e31: smithi031.rrnmwr(active, since 8s), standbys: smithi040.qtdwuq 2024-04-24T23:15:24.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:24 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi040.qtdwuq", "id": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:15:26.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:26 smithi031 ceph-mon[26411]: pgmap v8: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 651 KiB/s wr, 351 op/s 2024-04-24T23:15:26.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:26 smithi040 ceph-mon[33128]: pgmap v8: 65 pgs: 65 active+clean; 265 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 651 KiB/s wr, 351 op/s 2024-04-24T23:15:28.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:28 smithi040 ceph-mon[33128]: pgmap v9: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.1 MiB/s wr, 447 op/s 2024-04-24T23:15:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:28 smithi031 ceph-mon[26411]: pgmap v9: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.1 MiB/s wr, 447 op/s 2024-04-24T23:15:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: pgmap v10: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 998 KiB/s wr, 410 op/s 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:15:29.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:29 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: pgmap v10: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 998 KiB/s wr, 410 op/s 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:15:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:15:29.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:15:29.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:29 smithi031 ceph-mon[26411]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:15:30.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:15:30.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: pgmap v11: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.1 MiB/s wr, 445 op/s 2024-04-24T23:15:30.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:15:30.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-24T23:15:30.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:30 smithi040 ceph-mon[33128]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-24T23:15:30.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:15:30.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: pgmap v11: 65 pgs: 65 active+clean; 255 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 46 KiB/s rd, 1.1 MiB/s wr, 445 op/s 2024-04-24T23:15:30.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:30.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:15:30.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-04-24T23:15:30.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:30 smithi031 ceph-mon[26411]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-04-24T23:15:31.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:31 smithi040 ceph-mon[33128]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T23:15:31.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:31 smithi031 ceph-mon[26411]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T23:15:32.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:32 smithi040 ceph-mon[33128]: pgmap v12: 65 pgs: 65 active+clean; 247 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 61 KiB/s rd, 1.6 MiB/s wr, 606 op/s 2024-04-24T23:15:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:32 smithi031 ceph-mon[26411]: pgmap v12: 65 pgs: 65 active+clean; 247 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 61 KiB/s rd, 1.6 MiB/s wr, 606 op/s 2024-04-24T23:15:34.799 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:34 smithi031 ceph-mon[26411]: pgmap v13: 65 pgs: 65 active+clean; 247 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 320 op/s 2024-04-24T23:15:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:34 smithi040 ceph-mon[33128]: pgmap v13: 65 pgs: 65 active+clean; 247 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1.1 MiB/s wr, 320 op/s 2024-04-24T23:15:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:36 smithi040 ceph-mon[33128]: pgmap v14: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.6 MiB/s wr, 463 op/s 2024-04-24T23:15:36.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:36 smithi031 ceph-mon[26411]: pgmap v14: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.6 MiB/s wr, 463 op/s 2024-04-24T23:15:38.829 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:38 smithi031 ceph-mon[26411]: pgmap v15: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-04-24T23:15:38.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:38 smithi040 ceph-mon[33128]: pgmap v15: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-04-24T23:15:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:40 smithi040 ceph-mon[33128]: pgmap v16: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-04-24T23:15:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:40 smithi031 ceph-mon[26411]: pgmap v16: 65 pgs: 65 active+clean; 241 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.0 MiB/s wr, 304 op/s 2024-04-24T23:15:43.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:43 smithi040 ceph-mon[33128]: pgmap v17: 65 pgs: 65 active+clean; 236 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 402 op/s 2024-04-24T23:15:43.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:43 smithi031 ceph-mon[26411]: pgmap v17: 65 pgs: 65 active+clean; 236 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 402 op/s 2024-04-24T23:15:45.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:45 smithi040 ceph-mon[33128]: pgmap v18: 65 pgs: 65 active+clean; 236 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 882 KiB/s wr, 254 op/s 2024-04-24T23:15:45.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:45 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:45.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:45 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:15:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:45 smithi031 ceph-mon[26411]: pgmap v18: 65 pgs: 65 active+clean; 236 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 882 KiB/s wr, 254 op/s 2024-04-24T23:15:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:45 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:45 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:15:47.335 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:47 smithi031 ceph-mon[26411]: pgmap v19: 65 pgs: 65 active+clean; 225 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-04-24T23:15:47.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:47 smithi040 ceph-mon[33128]: pgmap v19: 65 pgs: 65 active+clean; 225 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-04-24T23:15:48.284 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-24T23:15:48.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-24T23:15:48.649 DEBUG:teuthology.parallel:result is None 2024-04-24T23:15:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:49 smithi031 ceph-mon[26411]: pgmap v20: 65 pgs: 65 active+clean; 225 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 899 KiB/s wr, 265 op/s 2024-04-24T23:15:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:49 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:49 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:49 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:15:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:49 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-24T23:15:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:49 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:49 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:49.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:49 smithi040 ceph-mon[33128]: pgmap v20: 65 pgs: 65 active+clean; 225 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 899 KiB/s wr, 265 op/s 2024-04-24T23:15:49.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:49 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:49.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:49 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:49.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:49 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:15:49.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:49 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-24T23:15:49.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:49 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:49.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:49 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:50.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.334+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/1712983453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fbcb0 msgr2=0x7fdfbc0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.334+0000 7fdfc0ecc700 1 --2- 172.21.15.31:0/1712983453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fbcb0 0x7fdfbc0fc0c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdfac004940 tx=0x7fdfac009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/1712983453 shutdown_connections 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 --2- 172.21.15.31:0/1712983453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc0fcf60 0x7fdfbc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 --2- 172.21.15.31:0/1712983453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fbcb0 0x7fdfbc0fc0c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/1712983453 >> 172.21.15.31:0/1712983453 conn(0x7fdfbc0f7430 msgr2=0x7fdfbc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/1712983453 shutdown_connections 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/1712983453 wait complete. 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 Processor -- start 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 -- start start 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc0fbcb0 0x7fdfbc110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:50.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fcf60 0x7fdfbc112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc066340 con 0x7fdfbc0fcf60 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfc0ecc700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc066480 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.335+0000 7fdfb9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fcf60 0x7fdfbc112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfba59c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc0fbcb0 0x7fdfbc110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfb9d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fcf60 0x7fdfbc112bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36420/0 (socket says 172.21.15.31:36420) 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfb9d9b700 1 -- 172.21.15.31:0/875346366 learned_addr learned my addr 172.21.15.31:0/875346366 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfba59c700 1 -- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fcf60 msgr2=0x7fdfbc112bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfba59c700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fcf60 0x7fdfbc112bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfba59c700 1 -- 172.21.15.31:0/875346366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac005040 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfba59c700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc0fbcb0 0x7fdfbc110680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdfac013870 tx=0x7fdfac009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfaaffd700 1 -- 172.21.15.31:0/875346366 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfac00e030 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfaaffd700 1 -- 172.21.15.31:0/875346366 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfac009430 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc113170 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfaaffd700 1 -- 172.21.15.31:0/875346366 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfac00b410 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.336+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc113610 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.337+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc05f5c0 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.338+0000 7fdfaaffd700 1 -- 172.21.15.31:0/875346366 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 31) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fdfac013af0 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.338+0000 7fdfaaffd700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdfa406db60 0x7fdfa4070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.339+0000 7fdfb9d9b700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdfa406db60 0x7fdfa4070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.339+0000 7fdfaaffd700 1 -- 172.21.15.31:0/875346366 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fdfac08eba0 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.339+0000 7fdfb9d9b700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdfa406db60 0x7fdfa4070010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0008fd0 tx=0x7fdfb000b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:50.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.343+0000 7fdfaaffd700 1 -- 172.21.15.31:0/875346366 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fdfac0584b0 con 0x7fdfbc0fbcb0 2024-04-24T23:15:50.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:50 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:15:50.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:50 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-24T23:15:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:50 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:15:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:50 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031:9095"}]: dispatch 2024-04-24T23:15:50.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.656+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfbc101250 con 0x7fdfa406db60 2024-04-24T23:15:50.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.657+0000 7fdfaaffd700 1 -- 172.21.15.31:0/875346366 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fdfbc101250 con 0x7fdfa406db60 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.659+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdfa406db60 msgr2=0x7fdfa4070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.659+0000 7fdfc0ecc700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdfa406db60 0x7fdfa4070010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0008fd0 tx=0x7fdfb000b040 comp rx=0 tx=0).stop 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.659+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc0fbcb0 msgr2=0x7fdfbc110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.659+0000 7fdfc0ecc700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc0fbcb0 0x7fdfbc110680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdfac013870 tx=0x7fdfac009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.659+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 shutdown_connections 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.660+0000 7fdfc0ecc700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdfa406db60 0x7fdfa4070010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.660+0000 7fdfc0ecc700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfbc0fbcb0 0x7fdfbc110680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.660+0000 7fdfc0ecc700 1 --2- 172.21.15.31:0/875346366 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfbc0fcf60 0x7fdfbc112bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.660+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 >> 172.21.15.31:0/875346366 conn(0x7fdfbc0f7430 msgr2=0x7fdfbc0f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:50.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.660+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 shutdown_connections 2024-04-24T23:15:50.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.660+0000 7fdfc0ecc700 1 -- 172.21.15.31:0/875346366 wait complete. 2024-04-24T23:15:50.675 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.776+0000 7f6584746700 1 -- 172.21.15.31:0/174659308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 msgr2=0x7f657c1046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.776+0000 7f6584746700 1 --2- 172.21.15.31:0/174659308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 0x7f657c1046d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f656c004940 tx=0x7f656c009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.776+0000 7f6584746700 1 -- 172.21.15.31:0/174659308 shutdown_connections 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.776+0000 7f6584746700 1 --2- 172.21.15.31:0/174659308 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 0x7f657c1046d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.776+0000 7f6584746700 1 --2- 172.21.15.31:0/174659308 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f657c066f70 0x7f657c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.776+0000 7f6584746700 1 -- 172.21.15.31:0/174659308 >> 172.21.15.31:0/174659308 conn(0x7f657c0f7420 msgr2=0x7f657c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.776+0000 7f6584746700 1 -- 172.21.15.31:0/174659308 shutdown_connections 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.776+0000 7f6584746700 1 -- 172.21.15.31:0/174659308 wait complete. 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6584746700 1 Processor -- start 2024-04-24T23:15:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6584746700 1 -- start start 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6584746700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f657c066f70 0x7f657c10fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6584746700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 0x7f657c1100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6584746700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c1106e0 con 0x7f657c1022f0 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6584746700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c110850 con 0x7f657c066f70 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6581ce1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 0x7f657c1100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6581ce1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 0x7f657c1100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36438/0 (socket says 172.21.15.31:36438) 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f6581ce1700 1 -- 172.21.15.31:0/1970691920 learned_addr learned my addr 172.21.15.31:0/1970691920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.777+0000 7f65824e2700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f657c066f70 0x7f657c10fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6581ce1700 1 -- 172.21.15.31:0/1970691920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f657c066f70 msgr2=0x7f657c10fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6581ce1700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f657c066f70 0x7f657c10fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6581ce1700 1 -- 172.21.15.31:0/1970691920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c005040 con 0x7f657c1022f0 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f65824e2700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f657c066f70 0x7f657c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:15:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6581ce1700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 0x7f657c1100e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f656c007f20 tx=0x7f656c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6572ffd700 1 -- 172.21.15.31:0/1970691920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f656c021070 con 0x7f657c1022f0 2024-04-24T23:15:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6572ffd700 1 -- 172.21.15.31:0/1970691920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f656c01c400 con 0x7f657c1022f0 2024-04-24T23:15:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c115200 con 0x7f657c1022f0 2024-04-24T23:15:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6572ffd700 1 -- 172.21.15.31:0/1970691920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f656c00e070 con 0x7f657c1022f0 2024-04-24T23:15:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.778+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c115760 con 0x7f657c1022f0 2024-04-24T23:15:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.779+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c04c710 con 0x7f657c1022f0 2024-04-24T23:15:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.780+0000 7f6572ffd700 1 -- 172.21.15.31:0/1970691920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f656c01ca70 con 0x7f657c1022f0 2024-04-24T23:15:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.780+0000 7f6572ffd700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f656806db70 0x7f6568070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.780+0000 7f6572ffd700 1 -- 172.21.15.31:0/1970691920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f656c08e570 con 0x7f657c1022f0 2024-04-24T23:15:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.780+0000 7f65824e2700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f656806db70 0x7f6568070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.781+0000 7f65824e2700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f656806db70 0x7f6568070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6578009b80 tx=0x7f657800b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:50.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:50.785+0000 7f6572ffd700 1 -- 172.21.15.31:0/1970691920 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f656c05bad0 con 0x7f657c1022f0 2024-04-24T23:15:50.914 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-24T23:15:50.915 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-24T23:15:51.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.099+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f657c0f8f60 con 0x7f656806db70 2024-04-24T23:15:51.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.101+0000 7f6572ffd700 1 -- 172.21.15.31:0/1970691920 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f657c0f8f60 con 0x7f656806db70 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f656806db70 msgr2=0x7f6568070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f656806db70 0x7f6568070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6578009b80 tx=0x7f657800b040 comp rx=0 tx=0).stop 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 msgr2=0x7f657c1100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 0x7f657c1100e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f656c007f20 tx=0x7f656c009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 shutdown_connections 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f656806db70 0x7f6568070020 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6578009b80 tx=0x7f657800b040 comp rx=0 tx=0).stop 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f657c066f70 0x7f657c10fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 --2- 172.21.15.31:0/1970691920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f657c1022f0 0x7f657c1100e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.105+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 >> 172.21.15.31:0/1970691920 conn(0x7f657c0f7420 msgr2=0x7f657c0f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.106+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 shutdown_connections 2024-04-24T23:15:51.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.106+0000 7f6584746700 1 -- 172.21.15.31:0/1970691920 wait complete. 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.211+0000 7f454030e700 1 -- 172.21.15.31:0/3719176197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 msgr2=0x7f45380fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.211+0000 7f454030e700 1 --2- 172.21.15.31:0/3719176197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 0x7f45380fd270 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4528009ce0 tx=0x7f4528004d10 comp rx=0 tx=0).stop 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.211+0000 7f454030e700 1 -- 172.21.15.31:0/3719176197 shutdown_connections 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.211+0000 7f454030e700 1 --2- 172.21.15.31:0/3719176197 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45380fdfd0 0x7f45380fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.211+0000 7f454030e700 1 --2- 172.21.15.31:0/3719176197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 0x7f45380fd270 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.211+0000 7f454030e700 1 -- 172.21.15.31:0/3719176197 >> 172.21.15.31:0/3719176197 conn(0x7f45380f8540 msgr2=0x7f45380fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.211+0000 7f454030e700 1 -- 172.21.15.31:0/3719176197 shutdown_connections 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.211+0000 7f454030e700 1 -- 172.21.15.31:0/3719176197 wait complete. 2024-04-24T23:15:51.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f454030e700 1 Processor -- start 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f454030e700 1 -- start start 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f454030e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 0x7f4538071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f454030e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45380fdfd0 0x7f45380702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f454030e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4538072200 con 0x7f45380fce60 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f454030e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45380707f0 con 0x7f45380fdfd0 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f453e0aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 0x7f4538071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f453e0aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 0x7f4538071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36458/0 (socket says 172.21.15.31:36458) 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f453e0aa700 1 -- 172.21.15.31:0/2792165845 learned_addr learned my addr 172.21.15.31:0/2792165845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f453d8a9700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45380fdfd0 0x7f45380702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f453e0aa700 1 -- 172.21.15.31:0/2792165845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45380fdfd0 msgr2=0x7f45380702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f453e0aa700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45380fdfd0 0x7f45380702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.212+0000 7f453e0aa700 1 -- 172.21.15.31:0/2792165845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528009940 con 0x7f45380fce60 2024-04-24T23:15:51.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.213+0000 7f453e0aa700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 0x7f4538071c00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4528009380 tx=0x7f4528004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:51.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.213+0000 7f4536ffd700 1 -- 172.21.15.31:0/2792165845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f452800e040 con 0x7f45380fce60 2024-04-24T23:15:51.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.213+0000 7f4536ffd700 1 -- 172.21.15.31:0/2792165845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45280128e0 con 0x7f45380fce60 2024-04-24T23:15:51.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.213+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4538070a40 con 0x7f45380fce60 2024-04-24T23:15:51.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.213+0000 7f4536ffd700 1 -- 172.21.15.31:0/2792165845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f452801c930 con 0x7f45380fce60 2024-04-24T23:15:51.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.213+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4538071000 con 0x7f45380fce60 2024-04-24T23:15:51.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.214+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453804c710 con 0x7f45380fce60 2024-04-24T23:15:51.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.214+0000 7f4536ffd700 1 -- 172.21.15.31:0/2792165845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f4528012a50 con 0x7f45380fce60 2024-04-24T23:15:51.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.215+0000 7f4536ffd700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f452006db70 0x7f4520070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.215+0000 7f4536ffd700 1 -- 172.21.15.31:0/2792165845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f452808ef80 con 0x7f45380fce60 2024-04-24T23:15:51.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.215+0000 7f453d8a9700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f452006db70 0x7f4520070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.215+0000 7f453d8a9700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f452006db70 0x7f4520070020 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f452c004f40 tx=0x7f452c00b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:51.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.220+0000 7f4536ffd700 1 -- 172.21.15.31:0/2792165845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f452805c440 con 0x7f45380fce60 2024-04-24T23:15:51.274 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:51 smithi040 ceph-mon[33128]: pgmap v21: 65 pgs: 65 active+clean; 225 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 899 KiB/s wr, 265 op/s 2024-04-24T23:15:51.274 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:51 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:15:51.274 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:51 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:15:51.274 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:51 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:51.275 DEBUG:teuthology.parallel:result is None 2024-04-24T23:15:51.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:15:51.308 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-24T23:15:51.309 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-24T23:15:51.345 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-24T23:15:51.345 DEBUG:teuthology.parallel:result is None 2024-04-24T23:15:51.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:51 smithi031 ceph-mon[26411]: pgmap v21: 65 pgs: 65 active+clean; 225 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 899 KiB/s wr, 265 op/s 2024-04-24T23:15:51.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:51 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:15:51.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:51 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:15:51.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:51 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:15:51.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.442+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4538102360 con 0x7f452006db70 2024-04-24T23:15:51.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.454+0000 7f4536ffd700 1 -- 172.21.15.31:0/2792165845 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4538102360 con 0x7f452006db70 2024-04-24T23:15:51.456 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:15:51.456 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (119s) 30s ago 10m 24.8M - 0.25.0 c8568f914cd2 3a7bccddf024 2024-04-24T23:15:51.456 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (10m) 30s ago 10m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:15:51.456 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (9m) 33s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 unknown 30s ago 10m - - 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (5m) 30s ago 5m 221M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (5m) 30s ago 5m 12.4M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (5m) 32s ago 5m 13.0M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (4m) 32s ago 4m 14.1M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (60s) 30s ago 12m 479M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283 running (3m) 33s ago 9m 79.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8b888d3a97fe 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (12m) 30s ago 12m 52.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (9m) 32s ago 9m 42.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (2m) 30s ago 10m 16.2M - 1.5.0 0da6a335fe13 f6eda28e448e 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (81s) 33s ago 9m 9605k - 1.5.0 0da6a335fe13 dfceecae86e1 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (8m) 30s ago 8m 208M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:15:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (8m) 30s ago 8m 223M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:15:51.458 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (7m) 30s ago 7m 188M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:15:51.458 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (7m) 32s ago 7m 254M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:15:51.458 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (7m) 32s ago 7m 229M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:15:51.458 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (6m) 32s ago 6m 209M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:15:51.458 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 starting - - - - 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f452006db70 msgr2=0x7f4520070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f452006db70 0x7f4520070020 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f452c004f40 tx=0x7f452c00b040 comp rx=0 tx=0).stop 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 msgr2=0x7f4538071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 0x7f4538071c00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4528009380 tx=0x7f4528004d10 comp rx=0 tx=0).stop 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 shutdown_connections 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f452006db70 0x7f4520070020 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45380fce60 0x7f4538071c00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 --2- 172.21.15.31:0/2792165845 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45380fdfd0 0x7f45380702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.456+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 >> 172.21.15.31:0/2792165845 conn(0x7f45380f8540 msgr2=0x7f45380f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.457+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 shutdown_connections 2024-04-24T23:15:51.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.457+0000 7f454030e700 1 -- 172.21.15.31:0/2792165845 wait complete. 2024-04-24T23:15:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.560+0000 7f60f8360700 1 -- 172.21.15.31:0/2859721713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 msgr2=0x7f60f00fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.560+0000 7f60f8360700 1 --2- 172.21.15.31:0/2859721713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 0x7f60f00fc0d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f60ec004940 tx=0x7f60ec009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.560+0000 7f60f8360700 1 -- 172.21.15.31:0/2859721713 shutdown_connections 2024-04-24T23:15:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.560+0000 7f60f8360700 1 --2- 172.21.15.31:0/2859721713 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00fcf70 0x7f60f00fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.560+0000 7f60f8360700 1 --2- 172.21.15.31:0/2859721713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 0x7f60f00fc0d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.560+0000 7f60f8360700 1 -- 172.21.15.31:0/2859721713 >> 172.21.15.31:0/2859721713 conn(0x7f60f00f7440 msgr2=0x7f60f00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.560+0000 7f60f8360700 1 -- 172.21.15.31:0/2859721713 shutdown_connections 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.560+0000 7f60f8360700 1 -- 172.21.15.31:0/2859721713 wait complete. 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f8360700 1 Processor -- start 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f8360700 1 -- start start 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f8360700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 0x7f60f00ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f8360700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00fcf70 0x7f60f00fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f8360700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f0100200 con 0x7f60f00fbcc0 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f8360700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00fe7f0 con 0x7f60f00fcf70 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f60fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 0x7f60f00ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f60fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 0x7f60f00ffc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36476/0 (socket says 172.21.15.31:36476) 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f60fc700 1 -- 172.21.15.31:0/584616943 learned_addr learned my addr 172.21.15.31:0/584616943 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f58fb700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00fcf70 0x7f60f00fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f60fc700 1 -- 172.21.15.31:0/584616943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00fcf70 msgr2=0x7f60f00fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f60fc700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00fcf70 0x7f60f00fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.561+0000 7f60f60fc700 1 -- 172.21.15.31:0/584616943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec005040 con 0x7f60f00fbcc0 2024-04-24T23:15:51.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.562+0000 7f60f60fc700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 0x7f60f00ffc00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009fd0 tx=0x7f60ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.562+0000 7f60e6ffd700 1 -- 172.21.15.31:0/584616943 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60ec00e040 con 0x7f60f00fbcc0 2024-04-24T23:15:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.562+0000 7f60e6ffd700 1 -- 172.21.15.31:0/584616943 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60ec00bcc0 con 0x7f60f00fbcc0 2024-04-24T23:15:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.562+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f00fea40 con 0x7f60f00fbcc0 2024-04-24T23:15:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.562+0000 7f60e6ffd700 1 -- 172.21.15.31:0/584616943 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60ec01c630 con 0x7f60f00fbcc0 2024-04-24T23:15:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.562+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f00ff000 con 0x7f60f00fbcc0 2024-04-24T23:15:51.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.563+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f0101b00 con 0x7f60f00fbcc0 2024-04-24T23:15:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.563+0000 7f60e6ffd700 1 -- 172.21.15.31:0/584616943 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f60ec013980 con 0x7f60f00fbcc0 2024-04-24T23:15:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.564+0000 7f60e6ffd700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f60dc06db70 0x7f60dc070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.564+0000 7f60e6ffd700 1 -- 172.21.15.31:0/584616943 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f60ec08e5a0 con 0x7f60f00fbcc0 2024-04-24T23:15:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.564+0000 7f60f58fb700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f60dc06db70 0x7f60dc070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.564+0000 7f60f58fb700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f60dc06db70 0x7f60dc070020 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f60e0004f40 tx=0x7f60e000b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:51.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.569+0000 7f60e6ffd700 1 -- 172.21.15.31:0/584616943 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f60ec05bba0 con 0x7f60f00fbcc0 2024-04-24T23:15:51.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.863+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f60f01047b0 con 0x7f60f00fbcc0 2024-04-24T23:15:51.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.863+0000 7f60e6ffd700 1 -- 172.21.15.31:0/584616943 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f60ec05b730 con 0x7f60f00fbcc0 2024-04-24T23:15:51.865 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:15:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:15:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:15:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:15:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:15:51.865 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:15:51.866 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:15:51.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f60dc06db70 msgr2=0x7f60dc070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f60dc06db70 0x7f60dc070020 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f60e0004f40 tx=0x7f60e000b040 comp rx=0 tx=0).stop 2024-04-24T23:15:51.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 msgr2=0x7f60f00ffc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 0x7f60f00ffc00 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009fd0 tx=0x7f60ec009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:51.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 shutdown_connections 2024-04-24T23:15:51.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f60dc06db70 0x7f60dc070020 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60f00fbcc0 0x7f60f00ffc00 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 --2- 172.21.15.31:0/584616943 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f60f00fcf70 0x7f60f00fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 >> 172.21.15.31:0/584616943 conn(0x7f60f00f7440 msgr2=0x7f60f00668c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:51.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 shutdown_connections 2024-04-24T23:15:51.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.865+0000 7f60f8360700 1 -- 172.21.15.31:0/584616943 wait complete. 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.972+0000 7f43c4594700 1 -- 172.21.15.31:0/2623730743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 msgr2=0x7f43bc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.972+0000 7f43c4594700 1 --2- 172.21.15.31:0/2623730743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 0x7f43bc0fd270 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f43b8004940 tx=0x7f43b8009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.972+0000 7f43c4594700 1 -- 172.21.15.31:0/2623730743 shutdown_connections 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.972+0000 7f43c4594700 1 --2- 172.21.15.31:0/2623730743 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc0fdfd0 0x7f43bc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.972+0000 7f43c4594700 1 --2- 172.21.15.31:0/2623730743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 0x7f43bc0fd270 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.972+0000 7f43c4594700 1 -- 172.21.15.31:0/2623730743 >> 172.21.15.31:0/2623730743 conn(0x7f43bc0f8520 msgr2=0x7f43bc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.972+0000 7f43c4594700 1 -- 172.21.15.31:0/2623730743 shutdown_connections 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c4594700 1 -- 172.21.15.31:0/2623730743 wait complete. 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c4594700 1 Processor -- start 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c4594700 1 -- start start 2024-04-24T23:15:51.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c4594700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 0x7f43bc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c4594700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc0fdfd0 0x7f43bc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c4594700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc10e4b0 con 0x7f43bc0fce60 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c4594700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc10e620 con 0x7f43bc0fdfd0 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c2330700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 0x7f43bc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c2330700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 0x7f43bc10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36492/0 (socket says 172.21.15.31:36492) 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c2330700 1 -- 172.21.15.31:0/323653612 learned_addr learned my addr 172.21.15.31:0/323653612 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.973+0000 7f43c1b2f700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc0fdfd0 0x7f43bc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43c2330700 1 -- 172.21.15.31:0/323653612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc0fdfd0 msgr2=0x7f43bc10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43c2330700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc0fdfd0 0x7f43bc10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43c2330700 1 -- 172.21.15.31:0/323653612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8005040 con 0x7f43bc0fce60 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43c2330700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 0x7f43bc10d970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f43b800b3b0 tx=0x7f43b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43b2ffd700 1 -- 172.21.15.31:0/323653612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b800e040 con 0x7f43bc0fce60 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43b2ffd700 1 -- 172.21.15.31:0/323653612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43b800bbf0 con 0x7f43bc0fce60 2024-04-24T23:15:51.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc113020 con 0x7f43bc0fce60 2024-04-24T23:15:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43b2ffd700 1 -- 172.21.15.31:0/323653612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b801c630 con 0x7f43bc0fce60 2024-04-24T23:15:51.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.974+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc1135e0 con 0x7f43bc0fce60 2024-04-24T23:15:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.975+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc04c710 con 0x7f43bc0fce60 2024-04-24T23:15:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.976+0000 7f43b2ffd700 1 -- 172.21.15.31:0/323653612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f43b800bd60 con 0x7f43bc0fce60 2024-04-24T23:15:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.976+0000 7f43b2ffd700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f43a806db70 0x7f43a8070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:15:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.976+0000 7f43b2ffd700 1 -- 172.21.15.31:0/323653612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f43b808df20 con 0x7f43bc0fce60 2024-04-24T23:15:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.976+0000 7f43c1b2f700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f43a806db70 0x7f43a8070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:15:51.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.976+0000 7f43c1b2f700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f43a806db70 0x7f43a8070020 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0041d0 tx=0x7f43ac0043c0 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:15:51.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:51.984+0000 7f43b2ffd700 1 -- 172.21.15.31:0/323653612 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f43b80c2100 con 0x7f43bc0fce60 2024-04-24T23:15:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:52 smithi031 ceph-mon[26411]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:52 smithi031 ceph-mon[26411]: Upgrade: Updating mgr.smithi040.qtdwuq 2024-04-24T23:15:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:52 smithi031 ceph-mon[26411]: Deploying daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:15:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:52 smithi031 ceph-mon[26411]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:52 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/584616943' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:15:52.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.285+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43bc1023b0 con 0x7f43a806db70 2024-04-24T23:15:52.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.286+0000 7f43b2ffd700 1 -- 172.21.15.31:0/323653612 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f43bc1023b0 con 0x7f43a806db70 2024-04-24T23:15:52.288 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:15:52.288 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:15:52.288 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:15:52.288 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:15:52.288 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:15:52.288 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-24T23:15:52.289 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:15:52.289 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T23:15:52.289 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-04-24T23:15:52.289 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:15:52.289 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:15:52.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f43a806db70 msgr2=0x7f43a8070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:52.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f43a806db70 0x7f43a8070020 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0041d0 tx=0x7f43ac0043c0 comp rx=0 tx=0).stop 2024-04-24T23:15:52.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 msgr2=0x7f43bc10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:15:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 0x7f43bc10d970 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f43b800b3b0 tx=0x7f43b8009e30 comp rx=0 tx=0).stop 2024-04-24T23:15:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 shutdown_connections 2024-04-24T23:15:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f43a806db70 0x7f43a8070020 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43bc0fce60 0x7f43bc10d970 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 --2- 172.21.15.31:0/323653612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f43bc0fdfd0 0x7f43bc10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:15:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.288+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 >> 172.21.15.31:0/323653612 conn(0x7f43bc0f8520 msgr2=0x7f43bc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:15:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.289+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 shutdown_connections 2024-04-24T23:15:52.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:15:52.289+0000 7f43c4594700 1 -- 172.21.15.31:0/323653612 wait complete. 2024-04-24T23:15:52.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:52 smithi040 ceph-mon[33128]: from='client.24535 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:52.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:52 smithi040 ceph-mon[33128]: Upgrade: Updating mgr.smithi040.qtdwuq 2024-04-24T23:15:52.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:52 smithi040 ceph-mon[33128]: Deploying daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:15:52.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:52 smithi040 ceph-mon[33128]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:52.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:52 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/584616943' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:15:53.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:53 smithi040 ceph-mon[33128]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:53.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:53 smithi040 ceph-mon[33128]: pgmap v22: 65 pgs: 65 active+clean; 214 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 396 op/s 2024-04-24T23:15:53.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:53 smithi031 ceph-mon[26411]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:53.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:53 smithi031 ceph-mon[26411]: pgmap v22: 65 pgs: 65 active+clean; 214 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.4 MiB/s wr, 396 op/s 2024-04-24T23:15:54.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:54 smithi040 ceph-mon[33128]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:54 smithi031 ceph-mon[26411]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:15:55.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:55 smithi040 ceph-mon[33128]: pgmap v23: 65 pgs: 65 active+clean; 214 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 953 KiB/s wr, 274 op/s 2024-04-24T23:15:55.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:55 smithi031 ceph-mon[26411]: pgmap v23: 65 pgs: 65 active+clean; 214 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 953 KiB/s wr, 274 op/s 2024-04-24T23:15:56.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:56 smithi031 ceph-mon[26411]: pgmap v24: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 32 KiB/s rd, 1.0 MiB/s wr, 327 op/s 2024-04-24T23:15:56.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:56.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:56.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:56.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:56.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:56.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:56 smithi040 ceph-mon[33128]: pgmap v24: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 32 KiB/s rd, 1.0 MiB/s wr, 327 op/s 2024-04-24T23:15:56.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:56.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:56.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:15:56.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:56.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:15:58.597 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:15:58 smithi040 ceph-mon[33128]: pgmap v25: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 572 KiB/s wr, 183 op/s 2024-04-24T23:15:58.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:15:58 smithi031 ceph-mon[26411]: pgmap v25: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 572 KiB/s wr, 183 op/s 2024-04-24T23:16:00.801 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:00 smithi031 ceph-mon[26411]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 572 KiB/s wr, 183 op/s 2024-04-24T23:16:00.801 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:00 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:16:00.801 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:00 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:00.801 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:00 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:00.831 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:00 smithi040 ceph-mon[33128]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 572 KiB/s wr, 183 op/s 2024-04-24T23:16:00.831 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:00 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:16:00.831 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:00 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:00.831 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:00 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:02.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:02.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:02.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:02.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:02.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:02.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:02.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:02.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:03.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:02 smithi040 ceph-mon[33128]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 573 KiB/s wr, 184 op/s 2024-04-24T23:16:03.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:02 smithi040 ceph-mon[33128]: Standby manager daemon smithi040.qtdwuq restarted 2024-04-24T23:16:03.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:02 smithi040 ceph-mon[33128]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:16:03.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:02 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3913432784' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:16:03.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:02 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3913432784' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:16:03.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:02 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3913432784' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:16:03.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:02 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3913432784' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:16:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:02 smithi031 ceph-mon[26411]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s rd, 573 KiB/s wr, 184 op/s 2024-04-24T23:16:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:02 smithi031 ceph-mon[26411]: Standby manager daemon smithi040.qtdwuq restarted 2024-04-24T23:16:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:02 smithi031 ceph-mon[26411]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:16:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:02 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/3913432784' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:16:03.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:02 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/3913432784' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:16:03.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:02 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/3913432784' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:16:03.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:02 smithi031 ceph-mon[26411]: from='mgr.? 172.21.15.40:0/3913432784' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:16:04.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:04 smithi040 ceph-mon[33128]: mgrmap e32: smithi031.rrnmwr(active, since 48s), standbys: smithi040.qtdwuq 2024-04-24T23:16:04.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:04 smithi040 ceph-mon[33128]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 5.9 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-04-24T23:16:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:04 smithi031 ceph-mon[26411]: mgrmap e32: smithi031.rrnmwr(active, since 48s), standbys: smithi040.qtdwuq 2024-04-24T23:16:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:04 smithi031 ceph-mon[26411]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 5.9 KiB/s rd, 77 KiB/s wr, 53 op/s 2024-04-24T23:16:05.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: pgmap v29: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 6.5 KiB/s rd, 84 KiB/s wr, 58 op/s 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: pgmap v30: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 114 B/s rd, 574 B/s wr, 0 op/s 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all mgr 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.rrnmwr"}]: dispatch 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.rrnmwr"}]': finished 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi040.qtdwuq"}]: dispatch 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi040.qtdwuq"}]': finished 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all rgw 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T23:16:05.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all iscsi 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all nfs 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all nvmeof 2024-04-24T23:16:05.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:05 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: pgmap v29: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 6.5 KiB/s rd, 84 KiB/s wr, 58 op/s 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: pgmap v30: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 114 B/s rd, 574 B/s wr, 0 op/s 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all mgr 2024-04-24T23:16:05.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.rrnmwr"}]: dispatch 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.rrnmwr"}]': finished 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi040.qtdwuq"}]: dispatch 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi040.qtdwuq"}]': finished 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all rgw 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all iscsi 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all nfs 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all nvmeof 2024-04-24T23:16:05.665 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:05 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:07.545 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:07 smithi031 ceph-mon[26411]: Upgrade: Updating node-exporter.smithi031 (1/2) 2024-04-24T23:16:07.545 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:07 smithi031 ceph-mon[26411]: Deploying daemon node-exporter.smithi031 on smithi031 2024-04-24T23:16:07.545 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:07 smithi031 ceph-mon[26411]: pgmap v31: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 114 B/s rd, 574 B/s wr, 0 op/s 2024-04-24T23:16:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:07 smithi040 ceph-mon[33128]: Upgrade: Updating node-exporter.smithi031 (1/2) 2024-04-24T23:16:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:07 smithi040 ceph-mon[33128]: Deploying daemon node-exporter.smithi031 on smithi031 2024-04-24T23:16:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:07 smithi040 ceph-mon[33128]: pgmap v31: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 114 B/s rd, 574 B/s wr, 0 op/s 2024-04-24T23:16:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:09 smithi031 ceph-mon[26411]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 114 B/s rd, 574 B/s wr, 0 op/s 2024-04-24T23:16:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:09 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:09 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:09.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:09 smithi040 ceph-mon[33128]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 114 B/s rd, 574 B/s wr, 0 op/s 2024-04-24T23:16:09.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:09 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:09.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:09 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:11.656 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:11 smithi031 ceph-mon[26411]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:11.656 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:11 smithi031 ceph-mon[26411]: Upgrade: Updating node-exporter.smithi040 (2/2) 2024-04-24T23:16:11.656 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:11 smithi031 ceph-mon[26411]: Deploying daemon node-exporter.smithi040 on smithi040 2024-04-24T23:16:11.781 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:11 smithi040 ceph-mon[33128]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:11.782 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:11 smithi040 ceph-mon[33128]: Upgrade: Updating node-exporter.smithi040 (2/2) 2024-04-24T23:16:11.782 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:11 smithi040 ceph-mon[33128]: Deploying daemon node-exporter.smithi040 on smithi040 2024-04-24T23:16:13.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:13 smithi031 ceph-mon[26411]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:13.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:13 smithi040 ceph-mon[33128]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:14.767 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:14.767 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:14.767 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:14.767 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:16:14.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:14.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:14.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:14.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:16:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:15 smithi031 ceph-mon[26411]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:16.030 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:15 smithi040 ceph-mon[33128]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:17 smithi040 ceph-mon[33128]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:17 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:17 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:17 smithi031 ceph-mon[26411]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:17 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:17 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:18.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:19.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:19.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:20.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:19 smithi040 ceph-mon[33128]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:19 smithi031 ceph-mon[26411]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:21.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:20 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:21.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:20 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:21.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:20 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:21.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:20 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.366 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:22 smithi031 ceph-mon[26411]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:22.367 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:22 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.367 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:22 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.367 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:22 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.367 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:22 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.403+0000 7fa2a62da700 1 -- 172.21.15.31:0/941745964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00ffee0 msgr2=0x7fa2a0102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.403+0000 7fa2a62da700 1 --2- 172.21.15.31:0/941745964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00ffee0 0x7fa2a0102300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa294004940 tx=0x7fa294009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.403+0000 7fa2a62da700 1 -- 172.21.15.31:0/941745964 shutdown_connections 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.403+0000 7fa2a62da700 1 --2- 172.21.15.31:0/941745964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00ffee0 0x7fa2a0102300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.403+0000 7fa2a62da700 1 --2- 172.21.15.31:0/941745964 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a00662c0 0x7fa2a00ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.403+0000 7fa2a62da700 1 -- 172.21.15.31:0/941745964 >> 172.21.15.31:0/941745964 conn(0x7fa2a00f7430 msgr2=0x7fa2a00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.403+0000 7fa2a62da700 1 -- 172.21.15.31:0/941745964 shutdown_connections 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.403+0000 7fa2a62da700 1 -- 172.21.15.31:0/941745964 wait complete. 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa2a62da700 1 Processor -- start 2024-04-24T23:16:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa2a62da700 1 -- start start 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa2a62da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00662c0 0x7fa2a010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa2a62da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a00ffee0 0x7fa2a010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa2a62da700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a010e4f0 con 0x7fa2a00662c0 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa2a62da700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a010ed40 con 0x7fa2a00ffee0 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00662c0 0x7fa2a010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa29ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00662c0 0x7fa2a010d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40538/0 (socket says 172.21.15.31:40538) 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.404+0000 7fa29ffff700 1 -- 172.21.15.31:0/43938010 learned_addr learned my addr 172.21.15.31:0/43938010 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa29f7fe700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a00ffee0 0x7fa2a010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa29ffff700 1 -- 172.21.15.31:0/43938010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a00ffee0 msgr2=0x7fa2a010def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa29ffff700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a00ffee0 0x7fa2a010def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa29ffff700 1 -- 172.21.15.31:0/43938010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294005040 con 0x7fa2a00662c0 2024-04-24T23:16:22.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa29ffff700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00662c0 0x7fa2a010d9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa29000eb40 tx=0x7fa29000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:22.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa29cff9700 1 -- 172.21.15.31:0/43938010 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa290007cc0 con 0x7fa2a00662c0 2024-04-24T23:16:22.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a010efc0 con 0x7fa2a00662c0 2024-04-24T23:16:22.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa29cff9700 1 -- 172.21.15.31:0/43938010 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa29000f040 con 0x7fa2a00662c0 2024-04-24T23:16:22.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa29cff9700 1 -- 172.21.15.31:0/43938010 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29000c9d0 con 0x7fa2a00662c0 2024-04-24T23:16:22.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.405+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a010f500 con 0x7fa2a00662c0 2024-04-24T23:16:22.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.406+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a005f5c0 con 0x7fa2a00662c0 2024-04-24T23:16:22.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.407+0000 7fa29cff9700 1 -- 172.21.15.31:0/43938010 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fa29000cc40 con 0x7fa2a00662c0 2024-04-24T23:16:22.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.407+0000 7fa29cff9700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa28806db60 0x7fa288070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:22.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.407+0000 7fa29cff9700 1 -- 172.21.15.31:0/43938010 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa29008ce90 con 0x7fa2a00662c0 2024-04-24T23:16:22.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.407+0000 7fa29f7fe700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa28806db60 0x7fa288070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:22.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.408+0000 7fa29f7fe700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa28806db60 0x7fa288070010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa294009b30 tx=0x7fa294009e30 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:22.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:22 smithi040 ceph-mon[33128]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:22.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:22 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:22 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:22 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:22 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:22.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.412+0000 7fa29cff9700 1 -- 172.21.15.31:0/43938010 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa29005a420 con 0x7fa2a00662c0 2024-04-24T23:16:22.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.641+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2a010f830 con 0x7fa28806db60 2024-04-24T23:16:22.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.643+0000 7fa29cff9700 1 -- 172.21.15.31:0/43938010 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fa2a010f830 con 0x7fa28806db60 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa28806db60 msgr2=0x7fa288070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa28806db60 0x7fa288070010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa294009b30 tx=0x7fa294009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00662c0 msgr2=0x7fa2a010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00662c0 0x7fa2a010d9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa29000eb40 tx=0x7fa29000b040 comp rx=0 tx=0).stop 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 shutdown_connections 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa28806db60 0x7fa288070010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2a00662c0 0x7fa2a010d9b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 --2- 172.21.15.31:0/43938010 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2a00ffee0 0x7fa2a010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 >> 172.21.15.31:0/43938010 conn(0x7fa2a00f7430 msgr2=0x7fa2a00f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 shutdown_connections 2024-04-24T23:16:22.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.645+0000 7fa2a62da700 1 -- 172.21.15.31:0/43938010 wait complete. 2024-04-24T23:16:22.657 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:16:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.750+0000 7f0ea034a700 1 -- 172.21.15.31:0/4275388376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 msgr2=0x7f0e980faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:22.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.750+0000 7f0ea034a700 1 --2- 172.21.15.31:0/4275388376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 0x7f0e980faea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c004940 tx=0x7f0e8c009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.750+0000 7f0ea034a700 1 -- 172.21.15.31:0/4275388376 shutdown_connections 2024-04-24T23:16:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.750+0000 7f0ea034a700 1 --2- 172.21.15.31:0/4275388376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 0x7f0e980faea0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.750+0000 7f0ea034a700 1 --2- 172.21.15.31:0/4275388376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980fe410 0x7f0e980fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.750+0000 7f0ea034a700 1 -- 172.21.15.31:0/4275388376 >> 172.21.15.31:0/4275388376 conn(0x7f0e980f6bd0 msgr2=0x7f0e980f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.750+0000 7f0ea034a700 1 -- 172.21.15.31:0/4275388376 shutdown_connections 2024-04-24T23:16:22.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0ea034a700 1 -- 172.21.15.31:0/4275388376 wait complete. 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0ea034a700 1 Processor -- start 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0ea034a700 1 -- start start 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0ea034a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980fe410 0x7f0e9810d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0ea034a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 0x7f0e9810de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0ea034a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9810e420 con 0x7f0e980fed60 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0ea034a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9810e590 con 0x7f0e980fe410 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0e9d8e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 0x7f0e9810de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0e9d8e5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 0x7f0e9810de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40564/0 (socket says 172.21.15.31:40564) 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.751+0000 7f0e9d8e5700 1 -- 172.21.15.31:0/47888963 learned_addr learned my addr 172.21.15.31:0/47888963 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0e9e0e6700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980fe410 0x7f0e9810d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:22.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0e9d8e5700 1 -- 172.21.15.31:0/47888963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980fe410 msgr2=0x7f0e9810d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0e9d8e5700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980fe410 0x7f0e9810d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0e9d8e5700 1 -- 172.21.15.31:0/47888963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c005040 con 0x7f0e980fed60 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0e9d8e5700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 0x7f0e9810de20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00b3b0 tx=0x7f0e8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0e96ffd700 1 -- 172.21.15.31:0/47888963 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e8c00e040 con 0x7f0e980fed60 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0e96ffd700 1 -- 172.21.15.31:0/47888963 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e8c00bbf0 con 0x7f0e980fed60 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e98112f90 con 0x7f0e980fed60 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0e96ffd700 1 -- 172.21.15.31:0/47888963 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e8c01c630 con 0x7f0e980fed60 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.752+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e98113520 con 0x7f0e980fed60 2024-04-24T23:16:22.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.753+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9804c710 con 0x7f0e980fed60 2024-04-24T23:16:22.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.754+0000 7f0e96ffd700 1 -- 172.21.15.31:0/47888963 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f0e8c00bd60 con 0x7f0e980fed60 2024-04-24T23:16:22.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.754+0000 7f0e96ffd700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0e8006db70 0x7f0e80070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:22.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.754+0000 7f0e96ffd700 1 -- 172.21.15.31:0/47888963 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0e8c08dfe0 con 0x7f0e980fed60 2024-04-24T23:16:22.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.754+0000 7f0e9e0e6700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0e8006db70 0x7f0e80070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:22.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.755+0000 7f0e9e0e6700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0e8006db70 0x7f0e80070020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0e88007a40 tx=0x7f0e88009da0 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:22.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.759+0000 7f0e96ffd700 1 -- 172.21.15.31:0/47888963 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0e8c05b540 con 0x7f0e980fed60 2024-04-24T23:16:22.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.993+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e981024a0 con 0x7f0e8006db70 2024-04-24T23:16:22.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.995+0000 7f0e96ffd700 1 -- 172.21.15.31:0/47888963 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f0e981024a0 con 0x7f0e8006db70 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0e8006db70 msgr2=0x7f0e80070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0e8006db70 0x7f0e80070020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0e88007a40 tx=0x7f0e88009da0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 msgr2=0x7f0e9810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 0x7f0e9810de20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00b3b0 tx=0x7f0e8c009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 shutdown_connections 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0e8006db70 0x7f0e80070020 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e980fe410 0x7f0e9810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 --2- 172.21.15.31:0/47888963 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e980fed60 0x7f0e9810de20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 >> 172.21.15.31:0/47888963 conn(0x7f0e980f6bd0 msgr2=0x7f0e980f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 shutdown_connections 2024-04-24T23:16:22.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:22.997+0000 7f0ea034a700 1 -- 172.21.15.31:0/47888963 wait complete. 2024-04-24T23:16:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.103+0000 7f5499896700 1 -- 172.21.15.31:0/1430469423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fbcc0 msgr2=0x7f54940fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.103+0000 7f5499896700 1 --2- 172.21.15.31:0/1430469423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fbcc0 0x7f54940fc0d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5484009ce0 tx=0x7f5484004d10 comp rx=0 tx=0).stop 2024-04-24T23:16:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.103+0000 7f5499896700 1 -- 172.21.15.31:0/1430469423 shutdown_connections 2024-04-24T23:16:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.103+0000 7f5499896700 1 --2- 172.21.15.31:0/1430469423 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54940fcf70 0x7f54940fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.103+0000 7f5499896700 1 --2- 172.21.15.31:0/1430469423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fbcc0 0x7f54940fc0d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.103+0000 7f5499896700 1 -- 172.21.15.31:0/1430469423 >> 172.21.15.31:0/1430469423 conn(0x7f54940f7440 msgr2=0x7f54940f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.103+0000 7f5499896700 1 -- 172.21.15.31:0/1430469423 shutdown_connections 2024-04-24T23:16:23.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.103+0000 7f5499896700 1 -- 172.21.15.31:0/1430469423 wait complete. 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f5499896700 1 Processor -- start 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f5499896700 1 -- start start 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f5499896700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54940fbcc0 0x7f549410db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f5499896700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fcf70 0x7f549410e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f5499896700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549410e6c0 con 0x7f54940fcf70 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f5499896700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549410e830 con 0x7f54940fbcc0 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f54927fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fcf70 0x7f549410e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f54927fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fcf70 0x7f549410e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40590/0 (socket says 172.21.15.31:40590) 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f54927fc700 1 -- 172.21.15.31:0/1074714536 learned_addr learned my addr 172.21.15.31:0/1074714536 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f5492ffd700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54940fbcc0 0x7f549410db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f54927fc700 1 -- 172.21.15.31:0/1074714536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54940fbcc0 msgr2=0x7f549410db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.104+0000 7f54927fc700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54940fbcc0 0x7f549410db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.105+0000 7f54927fc700 1 -- 172.21.15.31:0/1074714536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5484009940 con 0x7f54940fcf70 2024-04-24T23:16:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.105+0000 7f54927fc700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fcf70 0x7f549410e0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f548800c830 tx=0x7f5488004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.105+0000 7f547b7fe700 1 -- 172.21.15.31:0/1074714536 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54880106f0 con 0x7f54940fcf70 2024-04-24T23:16:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.105+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5494113240 con 0x7f54940fcf70 2024-04-24T23:16:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.105+0000 7f547b7fe700 1 -- 172.21.15.31:0/1074714536 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5488010d20 con 0x7f54940fcf70 2024-04-24T23:16:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.105+0000 7f547b7fe700 1 -- 172.21.15.31:0/1074714536 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54880113a0 con 0x7f54940fcf70 2024-04-24T23:16:23.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.105+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5494113800 con 0x7f54940fcf70 2024-04-24T23:16:23.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.106+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5494101b00 con 0x7f54940fcf70 2024-04-24T23:16:23.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.106+0000 7f547b7fe700 1 -- 172.21.15.31:0/1074714536 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f5488012400 con 0x7f54940fcf70 2024-04-24T23:16:23.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.107+0000 7f547b7fe700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f547c06db70 0x7f547c070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.107+0000 7f547b7fe700 1 -- 172.21.15.31:0/1074714536 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f548808d320 con 0x7f54940fcf70 2024-04-24T23:16:23.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.107+0000 7f5492ffd700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f547c06db70 0x7f547c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.107+0000 7f5492ffd700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f547c06db70 0x7f547c070020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f548400b740 tx=0x7f5484004d10 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:23.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.111+0000 7f547b7fe700 1 -- 172.21.15.31:0/1074714536 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f548805a8f0 con 0x7f54940fcf70 2024-04-24T23:16:23.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:23 smithi031 ceph-mon[26411]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:23.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:23 smithi031 ceph-mon[26411]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:23.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:23 smithi031 ceph-mon[26411]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:23.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.348+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f549402b340 con 0x7f547c06db70 2024-04-24T23:16:23.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.368+0000 7f547b7fe700 1 -- 172.21.15.31:0/1074714536 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f549402b340 con 0x7f547c06db70 2024-04-24T23:16:23.370 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:16:23.370 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (2m) 3s ago 11m 24.8M - 0.25.0 c8568f914cd2 3a7bccddf024 2024-04-24T23:16:23.370 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (11m) 3s ago 11m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:16:23.370 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (10m) 6s ago 10m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:16:23.370 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 unknown 3s ago 10m - - 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (5m) 3s ago 5m 139M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (5m) 3s ago 5m 12.7M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (5m) 6s ago 5m 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (5m) 6s ago 5m 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (92s) 3s ago 12m 501M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (28s) 6s ago 10m 448M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (12m) 3s ago 12m 56.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (10m) 6s ago 10m 46.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (14s) 3s ago 11m 3598k - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (10s) 6s ago 10m 3456k - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (9m) 3s ago 9m 222M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (8m) 3s ago 8m 238M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (8m) 3s ago 8m 197M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:16:23.371 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (7m) 6s ago 7m 276M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:16:23.372 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (7m) 6s ago 7m 245M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:16:23.372 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (7m) 6s ago 7m 223M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:16:23.372 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (35s) 3s ago 10m 46.8M - 2.43.0 a07b618ecd1d 93d7f3118e8c 2024-04-24T23:16:23.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f547c06db70 msgr2=0x7f547c070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f547c06db70 0x7f547c070020 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f548400b740 tx=0x7f5484004d10 comp rx=0 tx=0).stop 2024-04-24T23:16:23.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fcf70 msgr2=0x7f549410e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fcf70 0x7f549410e0c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f548800c830 tx=0x7f5488004470 comp rx=0 tx=0).stop 2024-04-24T23:16:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 shutdown_connections 2024-04-24T23:16:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f547c06db70 0x7f547c070020 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f54940fbcc0 0x7f549410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 --2- 172.21.15.31:0/1074714536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f54940fcf70 0x7f549410e0c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 >> 172.21.15.31:0/1074714536 conn(0x7f54940f7440 msgr2=0x7f54940668c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 shutdown_connections 2024-04-24T23:16:23.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.370+0000 7f5499896700 1 -- 172.21.15.31:0/1074714536 wait complete. 2024-04-24T23:16:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.480+0000 7fb9e985a700 1 -- 172.21.15.31:0/1752203296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fce60 msgr2=0x7fb9e40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.480+0000 7fb9e985a700 1 --2- 172.21.15.31:0/1752203296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fce60 0x7fb9e40fd270 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4009ce0 tx=0x7fb9d4004d10 comp rx=0 tx=0).stop 2024-04-24T23:16:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.480+0000 7fb9e985a700 1 -- 172.21.15.31:0/1752203296 shutdown_connections 2024-04-24T23:16:23.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.480+0000 7fb9e985a700 1 --2- 172.21.15.31:0/1752203296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40fdfd0 0x7fb9e40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.480+0000 7fb9e985a700 1 --2- 172.21.15.31:0/1752203296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fce60 0x7fb9e40fd270 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.480+0000 7fb9e985a700 1 -- 172.21.15.31:0/1752203296 >> 172.21.15.31:0/1752203296 conn(0x7fb9e40f8500 msgr2=0x7fb9e40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:23.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.480+0000 7fb9e985a700 1 -- 172.21.15.31:0/1752203296 shutdown_connections 2024-04-24T23:16:23.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.480+0000 7fb9e985a700 1 -- 172.21.15.31:0/1752203296 wait complete. 2024-04-24T23:16:23.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.481+0000 7fb9e985a700 1 Processor -- start 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.481+0000 7fb9e985a700 1 -- start start 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.481+0000 7fb9e985a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40fce60 0x7fb9e410db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.481+0000 7fb9e985a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fdfd0 0x7fb9e410e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.481+0000 7fb9e985a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e410e760 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.481+0000 7fb9e985a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e410ef50 con 0x7fb9e40fce60 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.481+0000 7fb9e27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fdfd0 0x7fb9e410e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fdfd0 0x7fb9e410e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40606/0 (socket says 172.21.15.31:40606) 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e27fc700 1 -- 172.21.15.31:0/1103953767 learned_addr learned my addr 172.21.15.31:0/1103953767 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e2ffd700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40fce60 0x7fb9e410db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e27fc700 1 -- 172.21.15.31:0/1103953767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40fce60 msgr2=0x7fb9e410db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e27fc700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40fce60 0x7fb9e410db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e27fc700 1 -- 172.21.15.31:0/1103953767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4009940 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e2ffd700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40fce60 0x7fb9e410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:23.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e27fc700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fdfd0 0x7fb9e410e0d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00b790 tx=0x7fb9cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:23.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9db7fe700 1 -- 172.21.15.31:0/1103953767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9cc00f770 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e410f230 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9db7fe700 1 -- 172.21.15.31:0/1103953767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9cc00fda0 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e410f7f0 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.482+0000 7fb9db7fe700 1 -- 172.21.15.31:0/1103953767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9cc00d310 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.483+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e405f5c0 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.484+0000 7fb9db7fe700 1 -- 172.21.15.31:0/1103953767 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fb9cc010420 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.485+0000 7fb9db7fe700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fb9d006db60 0x7fb9d0070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.485+0000 7fb9db7fe700 1 -- 172.21.15.31:0/1103953767 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fb9cc08d330 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.485+0000 7fb9e2ffd700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fb9d006db60 0x7fb9d0070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.485+0000 7fb9e2ffd700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fb9d006db60 0x7fb9d0070010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40095a0 tx=0x7fb9d4004d10 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:23.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.489+0000 7fb9db7fe700 1 -- 172.21.15.31:0/1103953767 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fb9cc05a9a0 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:23 smithi040 ceph-mon[33128]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:23 smithi040 ceph-mon[33128]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:23.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:23 smithi040 ceph-mon[33128]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:23.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.790+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb9e404bed0 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.791+0000 7fb9db7fe700 1 -- 172.21.15.31:0/1103953767 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fb9cc05a530 con 0x7fb9e40fdfd0 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:16:23.793 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:16:23.794 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:16:23.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fb9d006db60 msgr2=0x7fb9d0070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fb9d006db60 0x7fb9d0070010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40095a0 tx=0x7fb9d4004d10 comp rx=0 tx=0).stop 2024-04-24T23:16:23.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fdfd0 msgr2=0x7fb9e410e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fdfd0 0x7fb9e410e0d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00b790 tx=0x7fb9cc009250 comp rx=0 tx=0).stop 2024-04-24T23:16:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 shutdown_connections 2024-04-24T23:16:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fb9d006db60 0x7fb9d0070010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb9e40fce60 0x7fb9e410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 --2- 172.21.15.31:0/1103953767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9e40fdfd0 0x7fb9e410e0d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 >> 172.21.15.31:0/1103953767 conn(0x7fb9e40f8500 msgr2=0x7fb9e40f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 shutdown_connections 2024-04-24T23:16:23.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.793+0000 7fb9e985a700 1 -- 172.21.15.31:0/1103953767 wait complete. 2024-04-24T23:16:23.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.915+0000 7f31a35dd700 1 -- 172.21.15.31:0/2045118539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c066260 msgr2=0x7f319c0ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.915+0000 7f31a35dd700 1 --2- 172.21.15.31:0/2045118539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c066260 0x7f319c0ffa00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3198004940 tx=0x7f3198009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.915+0000 7f31a35dd700 1 -- 172.21.15.31:0/2045118539 shutdown_connections 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.915+0000 7f31a35dd700 1 --2- 172.21.15.31:0/2045118539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c0fff40 0x7f319c102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.915+0000 7f31a35dd700 1 --2- 172.21.15.31:0/2045118539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c066260 0x7f319c0ffa00 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3198004940 tx=0x7f3198009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.915+0000 7f31a35dd700 1 -- 172.21.15.31:0/2045118539 >> 172.21.15.31:0/2045118539 conn(0x7f319c0f7430 msgr2=0x7f319c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.916+0000 7f31a35dd700 1 -- 172.21.15.31:0/2045118539 shutdown_connections 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.916+0000 7f31a35dd700 1 -- 172.21.15.31:0/2045118539 wait complete. 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.917+0000 7f31a35dd700 1 Processor -- start 2024-04-24T23:16:23.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.917+0000 7f31a35dd700 1 -- start start 2024-04-24T23:16:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.917+0000 7f31a35dd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c0fff40 0x7f319c1141f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.917+0000 7f31a35dd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c114730 0x7f319c115580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a1379700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c0fff40 0x7f319c1141f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a0b78700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c114730 0x7f319c115580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a1379700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c0fff40 0x7f319c1141f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:40616/0 (socket says 172.21.15.31:40616) 2024-04-24T23:16:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a1379700 1 -- 172.21.15.31:0/2837501639 learned_addr learned my addr 172.21.15.31:0/2837501639 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c114bf0 con 0x7f319c0fff40 2024-04-24T23:16:23.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c114d30 con 0x7f319c114730 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a1379700 1 -- 172.21.15.31:0/2837501639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c114730 msgr2=0x7f319c115580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a1379700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c114730 0x7f319c115580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.918+0000 7f31a1379700 1 -- 172.21.15.31:0/2837501639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3198005040 con 0x7f319c0fff40 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.919+0000 7f31a1379700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c0fff40 0x7f319c1141f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f319800b3b0 tx=0x7f3198009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.919+0000 7f3191ffb700 1 -- 172.21.15.31:0/2837501639 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319800e040 con 0x7f319c0fff40 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.919+0000 7f3191ffb700 1 -- 172.21.15.31:0/2837501639 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3198007e90 con 0x7f319c0fff40 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.919+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c115b80 con 0x7f319c0fff40 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.919+0000 7f3191ffb700 1 -- 172.21.15.31:0/2837501639 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f319800e040 con 0x7f319c0fff40 2024-04-24T23:16:23.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.919+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c116110 con 0x7f319c0fff40 2024-04-24T23:16:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.920+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319c05f5c0 con 0x7f319c0fff40 2024-04-24T23:16:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.921+0000 7f3191ffb700 1 -- 172.21.15.31:0/2837501639 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f319800bbf0 con 0x7f319c0fff40 2024-04-24T23:16:23.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.922+0000 7f3191ffb700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f318806dbd0 0x7f3188070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.922+0000 7f3191ffb700 1 -- 172.21.15.31:0/2837501639 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f319808e260 con 0x7f319c0fff40 2024-04-24T23:16:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.922+0000 7f31a0b78700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f318806dbd0 0x7f3188070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:23.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.923+0000 7f31a0b78700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f318806dbd0 0x7f3188070080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f318c004f40 tx=0x7f318c00b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:23.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:23.927+0000 7f3191ffb700 1 -- 172.21.15.31:0/2837501639 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f319805b7c0 con 0x7f319c0fff40 2024-04-24T23:16:24.187 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.185+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f319c0f7d60 con 0x7f318806dbd0 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.187+0000 7f3191ffb700 1 -- 172.21.15.31:0/2837501639 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f319c0f7d60 con 0x7f318806dbd0 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-24T23:16:24.189 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:16:24.190 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:16:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f318806dbd0 msgr2=0x7f3188070080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f318806dbd0 0x7f3188070080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f318c004f40 tx=0x7f318c00b040 comp rx=0 tx=0).stop 2024-04-24T23:16:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c0fff40 msgr2=0x7f319c1141f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c0fff40 0x7f319c1141f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f319800b3b0 tx=0x7f3198009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 shutdown_connections 2024-04-24T23:16:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f318806dbd0 0x7f3188070080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f319c0fff40 0x7f319c1141f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 --2- 172.21.15.31:0/2837501639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f319c114730 0x7f319c115580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 >> 172.21.15.31:0/2837501639 conn(0x7f319c0f7430 msgr2=0x7f319c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 shutdown_connections 2024-04-24T23:16:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:24.189+0000 7f31a35dd700 1 -- 172.21.15.31:0/2837501639 wait complete. 2024-04-24T23:16:24.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:24 smithi031 ceph-mon[26411]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:24.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:24 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/1103953767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:24.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:24 smithi040 ceph-mon[33128]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:24.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:24 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/1103953767' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:25.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:25 smithi031 ceph-mon[26411]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:25.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:25 smithi031 ceph-mon[26411]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:25.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:25 smithi040 ceph-mon[33128]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:25.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:25 smithi040 ceph-mon[33128]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.444 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:26 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.856 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:26.856 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:26.856 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:26.856 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:26.856 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:26.857 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:26 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:27.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:27 smithi031 ceph-mon[26411]: Upgrade: Updating prometheus.smithi031 2024-04-24T23:16:27.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:27 smithi040 ceph-mon[33128]: Upgrade: Updating prometheus.smithi031 2024-04-24T23:16:28.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:28 smithi031 ceph-mon[26411]: Deploying daemon prometheus.smithi031 on smithi031 2024-04-24T23:16:28.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:28 smithi031 ceph-mon[26411]: pgmap v43: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:28.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:28 smithi040 ceph-mon[33128]: Deploying daemon prometheus.smithi031 on smithi031 2024-04-24T23:16:28.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:28 smithi040 ceph-mon[33128]: pgmap v43: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:30.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:30 smithi031 ceph-mon[26411]: pgmap v44: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:30.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:30 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:16:30.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:30 smithi040 ceph-mon[33128]: pgmap v44: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:30.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:30 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:16:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:32 smithi031 ceph-mon[26411]: pgmap v45: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:32.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:32 smithi040 ceph-mon[33128]: pgmap v45: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:33.735 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:33 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:33.736 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:33 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:33.736 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:33 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:33.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:33 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:33.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:33 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:33.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:33 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:34 smithi040 ceph-mon[33128]: pgmap v46: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:34.925 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:34 smithi031 ceph-mon[26411]: pgmap v46: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:36.808 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:36 smithi031 ceph-mon[26411]: pgmap v47: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:36 smithi040 ceph-mon[33128]: pgmap v47: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:38.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:38 smithi040 ceph-mon[33128]: pgmap v48: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:38.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:38 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:38.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:38 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:38.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:38 smithi031 ceph-mon[26411]: pgmap v48: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:38.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:38 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:38.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:38 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:40 smithi031 ceph-mon[26411]: pgmap v49: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:40 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:40 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:40 smithi040 ceph-mon[33128]: pgmap v49: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:40 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:40 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:42.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:42 smithi031 ceph-mon[26411]: pgmap v50: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:43.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:42 smithi040 ceph-mon[33128]: pgmap v50: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: pgmap v51: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: pgmap v52: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:44.203 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.204 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: pgmap v51: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: pgmap v52: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-24T23:16:44.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:44.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:45 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:16:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:45 smithi031 ceph-mon[26411]: Upgrade: Updating alertmanager.smithi031 2024-04-24T23:16:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:45 smithi031 ceph-mon[26411]: Deploying daemon alertmanager.smithi031 on smithi031 2024-04-24T23:16:45.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:45 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:16:45.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:45 smithi040 ceph-mon[33128]: Upgrade: Updating alertmanager.smithi031 2024-04-24T23:16:45.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:45 smithi040 ceph-mon[33128]: Deploying daemon alertmanager.smithi031 on smithi031 2024-04-24T23:16:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:46 smithi031 ceph-mon[26411]: pgmap v53: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:46.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:46 smithi040 ceph-mon[33128]: pgmap v53: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:16:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:47 smithi031 ceph-mon[26411]: mgrmap e33: smithi031.rrnmwr(active, since 91s), standbys: smithi040.qtdwuq 2024-04-24T23:16:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:47 smithi040 ceph-mon[33128]: mgrmap e33: smithi031.rrnmwr(active, since 91s), standbys: smithi040.qtdwuq 2024-04-24T23:16:48.477 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:48 smithi031 ceph-mon[26411]: pgmap v54: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 314 B/s wr, 0 op/s 2024-04-24T23:16:48.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:48 smithi040 ceph-mon[33128]: pgmap v54: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 314 B/s wr, 0 op/s 2024-04-24T23:16:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:50 smithi031 ceph-mon[26411]: pgmap v55: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 314 B/s wr, 0 op/s 2024-04-24T23:16:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:50 smithi040 ceph-mon[33128]: pgmap v55: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 314 B/s wr, 0 op/s 2024-04-24T23:16:52.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:52 smithi031 ceph-mon[26411]: pgmap v56: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 314 B/s wr, 0 op/s 2024-04-24T23:16:52.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:52 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:52.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:52 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:52.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:52 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:52.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:52 smithi040 ceph-mon[33128]: pgmap v56: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 314 B/s wr, 0 op/s 2024-04-24T23:16:52.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:52 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:52.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:52 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:52.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:52 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:16:54.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.301+0000 7f47da38c700 1 -- 172.21.15.31:0/353258310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 msgr2=0x7f47d40fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.301+0000 7f47da38c700 1 --2- 172.21.15.31:0/353258310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 0x7f47d40fd270 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f47c4004940 tx=0x7f47c4009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.301+0000 7f47da38c700 1 -- 172.21.15.31:0/353258310 shutdown_connections 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.301+0000 7f47da38c700 1 --2- 172.21.15.31:0/353258310 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47d40fdfd0 0x7f47d40fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.301+0000 7f47da38c700 1 --2- 172.21.15.31:0/353258310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 0x7f47d40fd270 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.301+0000 7f47da38c700 1 -- 172.21.15.31:0/353258310 >> 172.21.15.31:0/353258310 conn(0x7f47d40f8560 msgr2=0x7f47d40fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.301+0000 7f47da38c700 1 -- 172.21.15.31:0/353258310 shutdown_connections 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.301+0000 7f47da38c700 1 -- 172.21.15.31:0/353258310 wait complete. 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47da38c700 1 Processor -- start 2024-04-24T23:16:54.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47da38c700 1 -- start start 2024-04-24T23:16:54.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47da38c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 0x7f47d410d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:54.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47da38c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47d40fdfd0 0x7f47d410df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:54.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47da38c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d410e5b0 con 0x7f47d40fce60 2024-04-24T23:16:54.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47da38c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d410eda0 con 0x7f47d40fdfd0 2024-04-24T23:16:54.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47d3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 0x7f47d410d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:54.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47d3fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 0x7f47d410d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41150/0 (socket says 172.21.15.31:41150) 2024-04-24T23:16:54.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47d3fff700 1 -- 172.21.15.31:0/2936890741 learned_addr learned my addr 172.21.15.31:0/2936890741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:54.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47d37fe700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47d40fdfd0 0x7f47d410df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47d3fff700 1 -- 172.21.15.31:0/2936890741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47d40fdfd0 msgr2=0x7f47d410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47d3fff700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47d40fdfd0 0x7f47d410df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47d3fff700 1 -- 172.21.15.31:0/2936890741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c4005040 con 0x7f47d40fce60 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.302+0000 7f47d37fe700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47d40fdfd0 0x7f47d410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.303+0000 7f47d3fff700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 0x7f47d410d9e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b3b0 tx=0x7f47c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.303+0000 7f47d0ff9700 1 -- 172.21.15.31:0/2936890741 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c400e040 con 0x7f47d40fce60 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.303+0000 7f47d0ff9700 1 -- 172.21.15.31:0/2936890741 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47c400bcc0 con 0x7f47d40fce60 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.303+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d410f020 con 0x7f47d40fce60 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.303+0000 7f47d0ff9700 1 -- 172.21.15.31:0/2936890741 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c401c630 con 0x7f47d40fce60 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.303+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d410f5e0 con 0x7f47d40fce60 2024-04-24T23:16:54.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.304+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d405f5c0 con 0x7f47d40fce60 2024-04-24T23:16:54.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.305+0000 7f47d0ff9700 1 -- 172.21.15.31:0/2936890741 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f47c4013980 con 0x7f47d40fce60 2024-04-24T23:16:54.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.305+0000 7f47d0ff9700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f47bc06db10 0x7f47bc06ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:54.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.305+0000 7f47d0ff9700 1 -- 172.21.15.31:0/2936890741 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f47c408deb0 con 0x7f47d40fce60 2024-04-24T23:16:54.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.305+0000 7f47d37fe700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f47bc06db10 0x7f47bc06ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:54.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.306+0000 7f47d37fe700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f47bc06db10 0x7f47bc06ffc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f47c8006fd0 tx=0x7f47c8008040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:54.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.310+0000 7f47d0ff9700 1 -- 172.21.15.31:0/2936890741 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f47c405b590 con 0x7f47d40fce60 2024-04-24T23:16:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:54 smithi031 ceph-mon[26411]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2024-04-24T23:16:54.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.538+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47d410f960 con 0x7f47bc06db10 2024-04-24T23:16:54.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.539+0000 7f47d0ff9700 1 -- 172.21.15.31:0/2936890741 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f47d410f960 con 0x7f47bc06db10 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f47bc06db10 msgr2=0x7f47bc06ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f47bc06db10 0x7f47bc06ffc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f47c8006fd0 tx=0x7f47c8008040 comp rx=0 tx=0).stop 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 msgr2=0x7f47d410d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 0x7f47d410d9e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b3b0 tx=0x7f47c4009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 shutdown_connections 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f47bc06db10 0x7f47bc06ffc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47d40fce60 0x7f47d410d9e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 --2- 172.21.15.31:0/2936890741 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f47d40fdfd0 0x7f47d410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 >> 172.21.15.31:0/2936890741 conn(0x7f47d40f8560 msgr2=0x7f47d40f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 shutdown_connections 2024-04-24T23:16:54.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.541+0000 7f47da38c700 1 -- 172.21.15.31:0/2936890741 wait complete. 2024-04-24T23:16:54.553 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:16:54.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.646+0000 7fd891c16700 1 -- 172.21.15.31:0/1327320537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 msgr2=0x7fd88c0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.646+0000 7fd891c16700 1 --2- 172.21.15.31:0/1327320537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 0x7fd88c0fe440 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd880004940 tx=0x7fd880009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.646+0000 7fd891c16700 1 -- 172.21.15.31:0/1327320537 shutdown_connections 2024-04-24T23:16:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.646+0000 7fd891c16700 1 --2- 172.21.15.31:0/1327320537 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 0x7fd88c0fe440 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.646+0000 7fd891c16700 1 --2- 172.21.15.31:0/1327320537 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88c0fce60 0x7fd88c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.646+0000 7fd891c16700 1 -- 172.21.15.31:0/1327320537 >> 172.21.15.31:0/1327320537 conn(0x7fd88c0f8540 msgr2=0x7fd88c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.647+0000 7fd891c16700 1 -- 172.21.15.31:0/1327320537 shutdown_connections 2024-04-24T23:16:54.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.647+0000 7fd891c16700 1 -- 172.21.15.31:0/1327320537 wait complete. 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.647+0000 7fd891c16700 1 Processor -- start 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.647+0000 7fd891c16700 1 -- start start 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd891c16700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88c0fce60 0x7fd88c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd891c16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 0x7fd88c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd891c16700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c10e4d0 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd891c16700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c112e70 con 0x7fd88c0fce60 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd88affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 0x7fd88c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd88affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 0x7fd88c10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41180/0 (socket says 172.21.15.31:41180) 2024-04-24T23:16:54.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd88affd700 1 -- 172.21.15.31:0/3611830609 learned_addr learned my addr 172.21.15.31:0/3611830609 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd88b7fe700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88c0fce60 0x7fd88c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd88affd700 1 -- 172.21.15.31:0/3611830609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88c0fce60 msgr2=0x7fd88c10d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd88affd700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88c0fce60 0x7fd88c10d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd88affd700 1 -- 172.21.15.31:0/3611830609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd880005040 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.648+0000 7fd88affd700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 0x7fd88c10de40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd88000b3b0 tx=0x7fd880009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.649+0000 7fd873fff700 1 -- 172.21.15.31:0/3611830609 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd88000e040 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.649+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c113070 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.649+0000 7fd873fff700 1 -- 172.21.15.31:0/3611830609 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd88000bcc0 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.649+0000 7fd873fff700 1 -- 172.21.15.31:0/3611830609 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd88001c630 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.649+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c1135e0 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.650+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c04c710 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.651+0000 7fd873fff700 1 -- 172.21.15.31:0/3611830609 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fd880013980 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.651+0000 7fd873fff700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fd87406da40 0x7fd87406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.651+0000 7fd873fff700 1 -- 172.21.15.31:0/3611830609 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fd88008ded0 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.651+0000 7fd88b7fe700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fd87406da40 0x7fd87406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:54.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.652+0000 7fd88b7fe700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fd87406da40 0x7fd87406fef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd87c006fd0 tx=0x7fd87c008040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.656+0000 7fd873fff700 1 -- 172.21.15.31:0/3611830609 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fd88005b5d0 con 0x7fd88c0fdfd0 2024-04-24T23:16:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:54 smithi040 ceph-mon[33128]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2024-04-24T23:16:54.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.940+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd88c102270 con 0x7fd87406da40 2024-04-24T23:16:54.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.942+0000 7fd873fff700 1 -- 172.21.15.31:0/3611830609 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fd88c102270 con 0x7fd87406da40 2024-04-24T23:16:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fd87406da40 msgr2=0x7fd87406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fd87406da40 0x7fd87406fef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd87c006fd0 tx=0x7fd87c008040 comp rx=0 tx=0).stop 2024-04-24T23:16:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 msgr2=0x7fd88c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 0x7fd88c10de40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd88000b3b0 tx=0x7fd880009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:54.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 shutdown_connections 2024-04-24T23:16:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fd87406da40 0x7fd87406fef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd88c0fce60 0x7fd88c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 --2- 172.21.15.31:0/3611830609 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd88c0fdfd0 0x7fd88c10de40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 >> 172.21.15.31:0/3611830609 conn(0x7fd88c0f8540 msgr2=0x7fd88c0f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 shutdown_connections 2024-04-24T23:16:54.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:54.945+0000 7fd891c16700 1 -- 172.21.15.31:0/3611830609 wait complete. 2024-04-24T23:16:55.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 -- 172.21.15.31:0/3825434669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 msgr2=0x7f2290102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 --2- 172.21.15.31:0/3825434669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 0x7f2290102300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2280004940 tx=0x7f2280009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:55.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 -- 172.21.15.31:0/3825434669 shutdown_connections 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 --2- 172.21.15.31:0/3825434669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 0x7f2290102300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 --2- 172.21.15.31:0/3825434669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22900662c0 0x7f22900ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 -- 172.21.15.31:0/3825434669 >> 172.21.15.31:0/3825434669 conn(0x7f22900f7430 msgr2=0x7f22900f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 -- 172.21.15.31:0/3825434669 shutdown_connections 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 -- 172.21.15.31:0/3825434669 wait complete. 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 Processor -- start 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 -- start start 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22900662c0 0x7f229010dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 0x7f229010dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229010e680 con 0x7f22900ffee0 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.075+0000 7f2297699700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229010ee20 con 0x7f22900662c0 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2294c34700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 0x7f229010dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2294c34700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 0x7f229010dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41204/0 (socket says 172.21.15.31:41204) 2024-04-24T23:16:55.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2294c34700 1 -- 172.21.15.31:0/2875436205 learned_addr learned my addr 172.21.15.31:0/2875436205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2295435700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22900662c0 0x7f229010dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2294c34700 1 -- 172.21.15.31:0/2875436205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22900662c0 msgr2=0x7f229010dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2294c34700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22900662c0 0x7f229010dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2294c34700 1 -- 172.21.15.31:0/2875436205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2280005040 con 0x7f22900ffee0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2294c34700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 0x7f229010dff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f228000b3b0 tx=0x7f2280009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2285ffb700 1 -- 172.21.15.31:0/2875436205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f228000e040 con 0x7f22900ffee0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2285ffb700 1 -- 172.21.15.31:0/2875436205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f228000bcc0 con 0x7f22900ffee0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f229010f0a0 con 0x7f22900ffee0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.076+0000 7f2285ffb700 1 -- 172.21.15.31:0/2875436205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f228001c630 con 0x7f22900ffee0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.077+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229010f660 con 0x7f22900ffee0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.078+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229005f5c0 con 0x7f22900ffee0 2024-04-24T23:16:55.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.078+0000 7f2285ffb700 1 -- 172.21.15.31:0/2875436205 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f2280013980 con 0x7f22900ffee0 2024-04-24T23:16:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.079+0000 7f2285ffb700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f227c06da40 0x7f227c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.079+0000 7f2285ffb700 1 -- 172.21.15.31:0/2875436205 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f228008df30 con 0x7f22900ffee0 2024-04-24T23:16:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.079+0000 7f2295435700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f227c06da40 0x7f227c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.079+0000 7f2295435700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f227c06da40 0x7f227c06fef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f228c006fd0 tx=0x7f228c008040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:55.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.083+0000 7f2285ffb700 1 -- 172.21.15.31:0/2875436205 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f228005b5d0 con 0x7f22900ffee0 2024-04-24T23:16:55.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.304+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f229010f8f0 con 0x7f227c06da40 2024-04-24T23:16:55.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.314+0000 7f2285ffb700 1 -- 172.21.15.31:0/2875436205 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f229010f8f0 con 0x7f227c06da40 2024-04-24T23:16:55.316 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:16:55.316 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 starting - - - - 2024-04-24T23:16:55.316 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (11m) 16s ago 11m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:16:55.316 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (10m) 38s ago 10m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 unknown 16s ago 11m - - 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (6m) 16s ago 6m 139M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (6m) 16s ago 6m 12.8M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (6m) 38s ago 6m 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (6m) 38s ago 6m 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (2m) 16s ago 13m 504M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (60s) 38s ago 10m 448M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (13m) 16s ago 13m 54.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (10m) 38s ago 10m 46.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (46s) 16s ago 11m 7340k - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (42s) 38s ago 10m 3456k - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (9m) 16s ago 9m 222M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (9m) 16s ago 9m 238M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (8m) 16s ago 8m 198M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:16:55.317 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (8m) 38s ago 8m 276M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:16:55.318 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (8m) 38s ago 8m 245M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:16:55.318 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (7m) 38s ago 7m 223M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:16:55.318 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (23s) 16s ago 11m 38.0M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f227c06da40 msgr2=0x7f227c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f227c06da40 0x7f227c06fef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f228c006fd0 tx=0x7f228c008040 comp rx=0 tx=0).stop 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 msgr2=0x7f229010dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 0x7f229010dff0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f228000b3b0 tx=0x7f2280009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 shutdown_connections 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f227c06da40 0x7f227c06fef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22900662c0 0x7f229010dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 --2- 172.21.15.31:0/2875436205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22900ffee0 0x7f229010dff0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 >> 172.21.15.31:0/2875436205 conn(0x7f22900f7430 msgr2=0x7f22900f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 shutdown_connections 2024-04-24T23:16:55.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.317+0000 7f2297699700 1 -- 172.21.15.31:0/2875436205 wait complete. 2024-04-24T23:16:55.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.421+0000 7faa65e6e700 1 -- 172.21.15.31:0/3713511559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa600feff0 msgr2=0x7faa600ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.421+0000 7faa65e6e700 1 --2- 172.21.15.31:0/3713511559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa600feff0 0x7faa600ff400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7faa50004940 tx=0x7faa50009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:55.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.421+0000 7faa65e6e700 1 -- 172.21.15.31:0/3713511559 shutdown_connections 2024-04-24T23:16:55.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.421+0000 7faa65e6e700 1 --2- 172.21.15.31:0/3713511559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa601002a0 0x7faa60100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.421+0000 7faa65e6e700 1 --2- 172.21.15.31:0/3713511559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa600feff0 0x7faa600ff400 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.421+0000 7faa65e6e700 1 -- 172.21.15.31:0/3713511559 >> 172.21.15.31:0/3713511559 conn(0x7faa600fa8f0 msgr2=0x7faa600fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:55.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.421+0000 7faa65e6e700 1 -- 172.21.15.31:0/3713511559 shutdown_connections 2024-04-24T23:16:55.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.421+0000 7faa65e6e700 1 -- 172.21.15.31:0/3713511559 wait complete. 2024-04-24T23:16:55.424 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa65e6e700 1 Processor -- start 2024-04-24T23:16:55.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa65e6e700 1 -- start start 2024-04-24T23:16:55.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa65e6e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600feff0 0x7faa6010fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa65e6e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa601002a0 0x7faa60110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa65e6e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa60110650 con 0x7faa601002a0 2024-04-24T23:16:55.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa65e6e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa601107c0 con 0x7faa600feff0 2024-04-24T23:16:55.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa5effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa601002a0 0x7faa60110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa5effd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa601002a0 0x7faa60110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41212/0 (socket says 172.21.15.31:41212) 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa5effd700 1 -- 172.21.15.31:0/3653979071 learned_addr learned my addr 172.21.15.31:0/3653979071 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.422+0000 7faa5f7fe700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600feff0 0x7faa6010fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa5effd700 1 -- 172.21.15.31:0/3653979071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600feff0 msgr2=0x7faa6010fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa5effd700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600feff0 0x7faa6010fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa5effd700 1 -- 172.21.15.31:0/3653979071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa50005040 con 0x7faa601002a0 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa5effd700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa601002a0 0x7faa60110050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa54002a50 tx=0x7faa540046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa47fff700 1 -- 172.21.15.31:0/3653979071 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa54007cc0 con 0x7faa601002a0 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa60115220 con 0x7faa601002a0 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa47fff700 1 -- 172.21.15.31:0/3653979071 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa5400f040 con 0x7faa601002a0 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa47fff700 1 -- 172.21.15.31:0/3653979071 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5400da30 con 0x7faa601002a0 2024-04-24T23:16:55.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.423+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa601157e0 con 0x7faa601002a0 2024-04-24T23:16:55.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.424+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6004c710 con 0x7faa601002a0 2024-04-24T23:16:55.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.425+0000 7faa47fff700 1 -- 172.21.15.31:0/3653979071 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7faa5400db90 con 0x7faa601002a0 2024-04-24T23:16:55.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.425+0000 7faa47fff700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7faa4806da40 0x7faa4806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.425+0000 7faa47fff700 1 -- 172.21.15.31:0/3653979071 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7faa5408cf60 con 0x7faa601002a0 2024-04-24T23:16:55.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.425+0000 7faa5f7fe700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7faa4806da40 0x7faa4806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.426+0000 7faa5f7fe700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7faa4806da40 0x7faa4806fef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b970 tx=0x7faa50009e30 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:55.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.430+0000 7faa47fff700 1 -- 172.21.15.31:0/3653979071 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7faa540c1100 con 0x7faa601002a0 2024-04-24T23:16:55.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:55 smithi031 ceph-mon[26411]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:55.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:55 smithi031 ceph-mon[26411]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:55.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:55 smithi040 ceph-mon[33128]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:55.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:55 smithi040 ceph-mon[33128]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:55.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.746+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faa60115c10 con 0x7faa601002a0 2024-04-24T23:16:55.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.746+0000 7faa47fff700 1 -- 172.21.15.31:0/3653979071 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7faa5405a3d0 con 0x7faa601002a0 2024-04-24T23:16:55.748 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:16:55.748 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T23:16:55.749 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:16:55.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:16:55.750 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:16:55.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.748+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7faa4806da40 msgr2=0x7faa4806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.748+0000 7faa65e6e700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7faa4806da40 0x7faa4806fef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b970 tx=0x7faa50009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.748+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa601002a0 msgr2=0x7faa60110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.748+0000 7faa65e6e700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa601002a0 0x7faa60110050 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa54002a50 tx=0x7faa540046b0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.749+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 shutdown_connections 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.749+0000 7faa65e6e700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7faa4806da40 0x7faa4806fef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.749+0000 7faa65e6e700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faa600feff0 0x7faa6010fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.749+0000 7faa65e6e700 1 --2- 172.21.15.31:0/3653979071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa601002a0 0x7faa60110050 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.749+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 >> 172.21.15.31:0/3653979071 conn(0x7faa600fa8f0 msgr2=0x7faa600fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.749+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 shutdown_connections 2024-04-24T23:16:55.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.749+0000 7faa65e6e700 1 -- 172.21.15.31:0/3653979071 wait complete. 2024-04-24T23:16:55.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.864+0000 7f8f59f11700 1 -- 172.21.15.31:0/1824928337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 msgr2=0x7f8f540fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.864+0000 7f8f59f11700 1 --2- 172.21.15.31:0/1824928337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 0x7f8f540fd460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c004940 tx=0x7f8f3c009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.864+0000 7f8f59f11700 1 -- 172.21.15.31:0/1824928337 shutdown_connections 2024-04-24T23:16:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.864+0000 7f8f59f11700 1 --2- 172.21.15.31:0/1824928337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 0x7f8f540fd460 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.864+0000 7f8f59f11700 1 --2- 172.21.15.31:0/1824928337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f540fbd40 0x7f8f540fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.864+0000 7f8f59f11700 1 -- 172.21.15.31:0/1824928337 >> 172.21.15.31:0/1824928337 conn(0x7f8f540f74c0 msgr2=0x7f8f540f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.865+0000 7f8f59f11700 1 -- 172.21.15.31:0/1824928337 shutdown_connections 2024-04-24T23:16:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.865+0000 7f8f59f11700 1 -- 172.21.15.31:0/1824928337 wait complete. 2024-04-24T23:16:55.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.865+0000 7f8f59f11700 1 Processor -- start 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.865+0000 7f8f59f11700 1 -- start start 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.865+0000 7f8f59f11700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f540fbd40 0x7f8f5410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.865+0000 7f8f59f11700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 0x7f8f5410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.865+0000 7f8f59f11700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5410e580 con 0x7f8f540fcff0 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.865+0000 7f8f59f11700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5410ed80 con 0x7f8f540fbd40 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f4affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 0x7f8f5410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f4affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 0x7f8f5410df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41220/0 (socket says 172.21.15.31:41220) 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f4affd700 1 -- 172.21.15.31:0/3854551069 learned_addr learned my addr 172.21.15.31:0/3854551069 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f537fe700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f540fbd40 0x7f8f5410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f4affd700 1 -- 172.21.15.31:0/3854551069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f540fbd40 msgr2=0x7f8f5410da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f4affd700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f540fbd40 0x7f8f5410da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f4affd700 1 -- 172.21.15.31:0/3854551069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c005040 con 0x7f8f540fcff0 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f537fe700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f540fbd40 0x7f8f5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:16:55.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f4affd700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 0x7f8f5410df80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00b3b0 tx=0x7f8f3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f50ff9700 1 -- 172.21.15.31:0/3854551069 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f3c00e040 con 0x7f8f540fcff0 2024-04-24T23:16:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5410f000 con 0x7f8f540fcff0 2024-04-24T23:16:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f50ff9700 1 -- 172.21.15.31:0/3854551069 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f3c00bcc0 con 0x7f8f540fcff0 2024-04-24T23:16:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f50ff9700 1 -- 172.21.15.31:0/3854551069 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f3c01c630 con 0x7f8f540fcff0 2024-04-24T23:16:55.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.866+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5410f540 con 0x7f8f540fcff0 2024-04-24T23:16:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.868+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f54101b80 con 0x7f8f540fcff0 2024-04-24T23:16:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.868+0000 7f8f50ff9700 1 -- 172.21.15.31:0/3854551069 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f8f3c013980 con 0x7f8f540fcff0 2024-04-24T23:16:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.868+0000 7f8f50ff9700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8f4006da40 0x7f8f4006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:16:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.869+0000 7f8f50ff9700 1 -- 172.21.15.31:0/3854551069 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8f3c08ded0 con 0x7f8f540fcff0 2024-04-24T23:16:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.869+0000 7f8f537fe700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8f4006da40 0x7f8f4006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:16:55.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.869+0000 7f8f537fe700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8f4006da40 0x7f8f4006fef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009fd0 tx=0x7f8f44009ee0 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:16:55.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:55.874+0000 7f8f50ff9700 1 -- 172.21.15.31:0/3854551069 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8f3c05b5d0 con 0x7f8f540fcff0 2024-04-24T23:16:56.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.108+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f5410f760 con 0x7f8f4006da40 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.109+0000 7f8f50ff9700 1 -- 172.21.15.31:0/3854551069 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f8f5410f760 con 0x7f8f4006da40 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/2 daemons upgraded", 2024-04-24T23:16:56.111 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-24T23:16:56.112 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:16:56.112 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:16:56.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.111+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8f4006da40 msgr2=0x7f8f4006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.111+0000 7f8f59f11700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8f4006da40 0x7f8f4006fef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009fd0 tx=0x7f8f44009ee0 comp rx=0 tx=0).stop 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.111+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 msgr2=0x7f8f5410df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.111+0000 7f8f59f11700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 0x7f8f5410df80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00b3b0 tx=0x7f8f3c009e30 comp rx=0 tx=0).stop 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.111+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 shutdown_connections 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.111+0000 7f8f59f11700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8f4006da40 0x7f8f4006fef0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.111+0000 7f8f59f11700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8f540fbd40 0x7f8f5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.111+0000 7f8f59f11700 1 --2- 172.21.15.31:0/3854551069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f540fcff0 0x7f8f5410df80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.112+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 >> 172.21.15.31:0/3854551069 conn(0x7f8f540f74c0 msgr2=0x7f8f54066430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.112+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 shutdown_connections 2024-04-24T23:16:56.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:16:56.112+0000 7f8f59f11700 1 -- 172.21.15.31:0/3854551069 wait complete. 2024-04-24T23:16:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:56 smithi031 ceph-mon[26411]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-24T23:16:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:56 smithi031 ceph-mon[26411]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:56 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/3653979071' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:56 smithi031 ceph-mon[26411]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:56.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:56 smithi040 ceph-mon[33128]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-24T23:16:56.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:56 smithi040 ceph-mon[33128]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:56.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:56 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/3653979071' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:16:56.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:56.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:56.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:56 smithi040 ceph-mon[33128]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:16:58.661 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:58 smithi031 ceph-mon[26411]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-24T23:16:58.661 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:58 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:58.661 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:16:58 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:58.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:58 smithi040 ceph-mon[33128]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-24T23:16:58.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:58 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:16:58.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:16:58 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:00 smithi031 ceph-mon[26411]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:00.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:00 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:17:00.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:00 smithi040 ceph-mon[33128]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:00.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:00 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:17:02.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:02.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:02.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:02.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:02.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:02.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:02.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:02.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:02.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:02.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031:9093"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:02.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:04.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:04 smithi031 ceph-mon[26411]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:04.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:04 smithi040 ceph-mon[33128]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:05 smithi031 ceph-mon[26411]: Upgrade: Updating grafana.smithi031 2024-04-24T23:17:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:05 smithi031 ceph-mon[26411]: Deploying daemon grafana.smithi031 on smithi031 2024-04-24T23:17:05.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:05 smithi040 ceph-mon[33128]: Upgrade: Updating grafana.smithi031 2024-04-24T23:17:05.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:05 smithi040 ceph-mon[33128]: Deploying daemon grafana.smithi031 on smithi031 2024-04-24T23:17:06.357 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:06 smithi031 ceph-mon[26411]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:06.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:06 smithi040 ceph-mon[33128]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:08 smithi031 ceph-mon[26411]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:08 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:08 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:08.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:08 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:08.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:08 smithi040 ceph-mon[33128]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:08.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:08 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:08.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:08 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:08.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:08 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:10.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:10 smithi031 ceph-mon[26411]: pgmap v66: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:10.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:10 smithi040 ceph-mon[33128]: pgmap v66: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:12 smithi031 ceph-mon[26411]: pgmap v67: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:12.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:12 smithi040 ceph-mon[33128]: pgmap v67: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:13.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:13 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:13.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:13 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:13.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:13 smithi040 ceph-mon[33128]: pgmap v68: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:13.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:13 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:13.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:13 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:13.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:13 smithi031 ceph-mon[26411]: pgmap v68: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:15.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:15.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:15.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:14 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:17:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:14 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:17:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:15 smithi031 ceph-mon[26411]: pgmap v69: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:15 smithi040 ceph-mon[33128]: pgmap v69: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: pgmap v70: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T23:17:18.416 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T23:17:18.417 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:18 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: pgmap v70: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:18.501 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T23:17:18.502 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T23:17:18.503 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:18 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:19.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: pgmap v71: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: Upgrade: Finalizing container_image settings 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: Upgrade: Complete! 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: Cluster is now healthy 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:19.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:19.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:19 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: pgmap v71: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-24T23:17:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031:3000"}]: dispatch 2024-04-24T23:17:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: Upgrade: Finalizing container_image settings 2024-04-24T23:17:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: Upgrade: Complete! 2024-04-24T23:17:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-24T23:17:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: Cluster is now healthy 2024-04-24T23:17:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:19.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:19 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:20.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:20 smithi040 ceph-mon[33128]: pgmap v72: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:20.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:20 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:20.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:20 smithi031 ceph-mon[26411]: pgmap v72: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:20.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:20 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:22.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:22 smithi040 ceph-mon[33128]: pgmap v73: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:22 smithi031 ceph-mon[26411]: pgmap v73: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:24.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:24 smithi040 ceph-mon[33128]: pgmap v74: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:25.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:24 smithi031 ceph-mon[26411]: pgmap v74: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.238+0000 7f507230c700 1 -- 172.21.15.31:0/2833412962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 msgr2=0x7f506c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.238+0000 7f507230c700 1 --2- 172.21.15.31:0/2833412962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 0x7f506c102390 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5054004940 tx=0x7f5054009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.238+0000 7f507230c700 1 -- 172.21.15.31:0/2833412962 shutdown_connections 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.238+0000 7f507230c700 1 --2- 172.21.15.31:0/2833412962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 0x7f506c102390 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.238+0000 7f507230c700 1 --2- 172.21.15.31:0/2833412962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f506c066eb0 0x7f506c0ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.238+0000 7f507230c700 1 -- 172.21.15.31:0/2833412962 >> 172.21.15.31:0/2833412962 conn(0x7f506c0f7480 msgr2=0x7f506c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.238+0000 7f507230c700 1 -- 172.21.15.31:0/2833412962 shutdown_connections 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.238+0000 7f507230c700 1 -- 172.21.15.31:0/2833412962 wait complete. 2024-04-24T23:17:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f507230c700 1 Processor -- start 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f507230c700 1 -- start start 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f507230c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f506c066eb0 0x7f506c10d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f507230c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 0x7f506c10de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f507230c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c10e490 con 0x7f506c0fffb0 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f507230c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c10e600 con 0x7f506c066eb0 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f506b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 0x7f506c10de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f506b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 0x7f506c10de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48580/0 (socket says 172.21.15.31:48580) 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f506b7fe700 1 -- 172.21.15.31:0/3333716669 learned_addr learned my addr 172.21.15.31:0/3333716669 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.239+0000 7f506bfff700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f506c066eb0 0x7f506c10d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f506b7fe700 1 -- 172.21.15.31:0/3333716669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f506c066eb0 msgr2=0x7f506c10d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f506b7fe700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f506c066eb0 0x7f506c10d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f506b7fe700 1 -- 172.21.15.31:0/3333716669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5054005040 con 0x7f506c0fffb0 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f506b7fe700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 0x7f506c10de90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f505400b3b0 tx=0x7f5054009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f5068ff9700 1 -- 172.21.15.31:0/3333716669 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f505400e040 con 0x7f506c0fffb0 2024-04-24T23:17:26.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f5068ff9700 1 -- 172.21.15.31:0/3333716669 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f505400bbf0 con 0x7f506c0fffb0 2024-04-24T23:17:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c113000 con 0x7f506c0fffb0 2024-04-24T23:17:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f5068ff9700 1 -- 172.21.15.31:0/3333716669 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f505401c630 con 0x7f506c0fffb0 2024-04-24T23:17:26.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.240+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c0fba90 con 0x7f506c0fffb0 2024-04-24T23:17:26.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.241+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c04c710 con 0x7f506c0fffb0 2024-04-24T23:17:26.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.242+0000 7f5068ff9700 1 -- 172.21.15.31:0/3333716669 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f505400bd60 con 0x7f506c0fffb0 2024-04-24T23:17:26.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.242+0000 7f5068ff9700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f5058071e50 0x7f5058074300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:26.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.242+0000 7f5068ff9700 1 -- 172.21.15.31:0/3333716669 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f505408dd80 con 0x7f506c0fffb0 2024-04-24T23:17:26.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.242+0000 7f506bfff700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f5058071e50 0x7f5058074300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:26.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.243+0000 7f506bfff700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f5058071e50 0x7f5058074300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f506c066bf0 tx=0x7f505c00b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:26.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.247+0000 7f5068ff9700 1 -- 172.21.15.31:0/3333716669 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f505405b500 con 0x7f506c0fffb0 2024-04-24T23:17:26.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.475+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f506c0f8f70 con 0x7f5058071e50 2024-04-24T23:17:26.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.476+0000 7f5068ff9700 1 -- 172.21.15.31:0/3333716669 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f506c0f8f70 con 0x7f5058071e50 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f5058071e50 msgr2=0x7f5058074300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f5058071e50 0x7f5058074300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f506c066bf0 tx=0x7f505c00b040 comp rx=0 tx=0).stop 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 msgr2=0x7f506c10de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 0x7f506c10de90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f505400b3b0 tx=0x7f5054009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 shutdown_connections 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f5058071e50 0x7f5058074300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f506c066eb0 0x7f506c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 --2- 172.21.15.31:0/3333716669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f506c0fffb0 0x7f506c10de90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 >> 172.21.15.31:0/3333716669 conn(0x7f506c0f7480 msgr2=0x7f506c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 shutdown_connections 2024-04-24T23:17:26.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:26.478+0000 7f507230c700 1 -- 172.21.15.31:0/3333716669 wait complete. 2024-04-24T23:17:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:26 smithi031 ceph-mon[26411]: pgmap v75: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:26.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:26 smithi040 ceph-mon[33128]: pgmap v75: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:27.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-24T23:17:27.389 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:27.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:27 smithi031 ceph-mon[26411]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:27.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:27 smithi040 ceph-mon[33128]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.808+0000 7f15f1e6b700 1 -- 172.21.15.31:0/1962820455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec0fbd40 msgr2=0x7f15ec0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.808+0000 7f15f1e6b700 1 --2- 172.21.15.31:0/1962820455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec0fbd40 0x7f15ec0fc1b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f15e0004940 tx=0x7f15e0009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.808+0000 7f15f1e6b700 1 -- 172.21.15.31:0/1962820455 shutdown_connections 2024-04-24T23:17:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.808+0000 7f15f1e6b700 1 --2- 172.21.15.31:0/1962820455 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec0fbd40 0x7f15ec0fc1b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.808+0000 7f15f1e6b700 1 --2- 172.21.15.31:0/1962820455 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec101d90 0x7f15ec102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.808+0000 7f15f1e6b700 1 -- 172.21.15.31:0/1962820455 >> 172.21.15.31:0/1962820455 conn(0x7f15ec071d00 msgr2=0x7f15ec072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.809+0000 7f15f1e6b700 1 -- 172.21.15.31:0/1962820455 shutdown_connections 2024-04-24T23:17:28.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.809+0000 7f15f1e6b700 1 -- 172.21.15.31:0/1962820455 wait complete. 2024-04-24T23:17:28.812 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:28 smithi031 ceph-mon[26411]: pgmap v76: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.809+0000 7f15f1e6b700 1 Processor -- start 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.809+0000 7f15f1e6b700 1 -- start start 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15f1e6b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec0fbd40 0x7f15ec113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15f1e6b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec101d90 0x7f15ec114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15f1e6b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec114680 con 0x7f15ec101d90 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15f1e6b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec10da70 con 0x7f15ec0fbd40 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15eaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec101d90 0x7f15ec114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15eaffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec101d90 0x7f15ec114030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43718/0 (socket says 172.21.15.31:43718) 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15eaffd700 1 -- 172.21.15.31:0/2242962390 learned_addr learned my addr 172.21.15.31:0/2242962390 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:28.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15eb7fe700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec0fbd40 0x7f15ec113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15eaffd700 1 -- 172.21.15.31:0/2242962390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec0fbd40 msgr2=0x7f15ec113af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15eaffd700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec0fbd40 0x7f15ec113af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15eaffd700 1 -- 172.21.15.31:0/2242962390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e0005040 con 0x7f15ec101d90 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.810+0000 7f15eaffd700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec101d90 0x7f15ec114030 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009b30 tx=0x7f15e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.811+0000 7f15d3fff700 1 -- 172.21.15.31:0/2242962390 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e0021070 con 0x7f15ec101d90 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.811+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec10dcf0 con 0x7f15ec101d90 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.811+0000 7f15d3fff700 1 -- 172.21.15.31:0/2242962390 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15e0013c00 con 0x7f15ec101d90 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.811+0000 7f15d3fff700 1 -- 172.21.15.31:0/2242962390 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e000e070 con 0x7f15ec101d90 2024-04-24T23:17:28.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.811+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec10e1d0 con 0x7f15ec101d90 2024-04-24T23:17:28.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.812+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec0fd430 con 0x7f15ec101d90 2024-04-24T23:17:28.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.812+0000 7f15d3fff700 1 -- 172.21.15.31:0/2242962390 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f15e001cb60 con 0x7f15ec101d90 2024-04-24T23:17:28.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.813+0000 7f15d3fff700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f15d406da40 0x7f15d406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:28.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.813+0000 7f15d3fff700 1 -- 172.21.15.31:0/2242962390 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f15e008e680 con 0x7f15ec101d90 2024-04-24T23:17:28.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.813+0000 7f15eb7fe700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f15d406da40 0x7f15d406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:28.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.813+0000 7f15eb7fe700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f15d406da40 0x7f15d406fef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009b80 tx=0x7f15dc00b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:28.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:28.818+0000 7f15d3fff700 1 -- 172.21.15.31:0/2242962390 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f15e005be00 con 0x7f15ec101d90 2024-04-24T23:17:28.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:28 smithi040 ceph-mon[33128]: pgmap v76: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:29.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.112+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f15ec108df0 con 0x7f15ec101d90 2024-04-24T23:17:29.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.112+0000 7f15d3fff700 1 -- 172.21.15.31:0/2242962390 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f15e0004490 con 0x7f15ec101d90 2024-04-24T23:17:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f15d406da40 msgr2=0x7f15d406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f15d406da40 0x7f15d406fef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009b80 tx=0x7f15dc00b040 comp rx=0 tx=0).stop 2024-04-24T23:17:29.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec101d90 msgr2=0x7f15ec114030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec101d90 0x7f15ec114030 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009b30 tx=0x7f15e0009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 shutdown_connections 2024-04-24T23:17:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f15d406da40 0x7f15d406fef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15ec0fbd40 0x7f15ec113af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 --2- 172.21.15.31:0/2242962390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15ec101d90 0x7f15ec114030 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.114+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 >> 172.21.15.31:0/2242962390 conn(0x7f15ec071d00 msgr2=0x7f15ec0fa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.115+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 shutdown_connections 2024-04-24T23:17:29.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:29.115+0000 7f15f1e6b700 1 -- 172.21.15.31:0/2242962390 wait complete. 2024-04-24T23:17:29.127 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:17:29.746 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:29 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2242962390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:29.746 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:29 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:29.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:29 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:17:29.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-24T23:17:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:29 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2242962390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:29 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:29.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:29 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:17:30.044 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:30.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:30 smithi031 ceph-mon[26411]: pgmap v77: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:30.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:30 smithi040 ceph-mon[33128]: pgmap v77: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 -- 172.21.15.31:0/1357481824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70181026a0 msgr2=0x7f70180fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 --2- 172.21.15.31:0/1357481824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70181026a0 0x7f70180fbd10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7000004940 tx=0x7f7000009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 -- 172.21.15.31:0/1357481824 shutdown_connections 2024-04-24T23:17:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 --2- 172.21.15.31:0/1357481824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70181026a0 0x7f70180fbd10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 --2- 172.21.15.31:0/1357481824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7018101d00 0x7f70181020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:31.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 -- 172.21.15.31:0/1357481824 >> 172.21.15.31:0/1357481824 conn(0x7f70180f7870 msgr2=0x7f70180f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 -- 172.21.15.31:0/1357481824 shutdown_connections 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 -- 172.21.15.31:0/1357481824 wait complete. 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.593+0000 7f701e09b700 1 Processor -- start 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f701e09b700 1 -- start start 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f701e09b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018101d00 0x7f70181128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f701e09b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70181026a0 0x7f7018112e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f701e09b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70181134f0 con 0x7f7018101d00 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f701e09b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701810ca60 con 0x7f70181026a0 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f70177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018101d00 0x7f70181128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f70177fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018101d00 0x7f70181128d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43734/0 (socket says 172.21.15.31:43734) 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f70177fe700 1 -- 172.21.15.31:0/2616073080 learned_addr learned my addr 172.21.15.31:0/2616073080 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:31.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f7016ffd700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70181026a0 0x7f7018112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f70177fe700 1 -- 172.21.15.31:0/2616073080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70181026a0 msgr2=0x7f7018112e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f70177fe700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70181026a0 0x7f7018112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f70177fe700 1 -- 172.21.15.31:0/2616073080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000005040 con 0x7f7018101d00 2024-04-24T23:17:31.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.594+0000 7f70177fe700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018101d00 0x7f70181128d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f700800bbb0 tx=0x7f7008009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.595+0000 7f700ffff700 1 -- 172.21.15.31:0/2616073080 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70080092e0 con 0x7f7018101d00 2024-04-24T23:17:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.595+0000 7f700ffff700 1 -- 172.21.15.31:0/2616073080 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f700800cbd0 con 0x7f7018101d00 2024-04-24T23:17:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.595+0000 7f700ffff700 1 -- 172.21.15.31:0/2616073080 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7008007400 con 0x7f7018101d00 2024-04-24T23:17:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.595+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701810cd40 con 0x7f7018101d00 2024-04-24T23:17:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.595+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701810d300 con 0x7f7018101d00 2024-04-24T23:17:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.596+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70180fd450 con 0x7f7018101d00 2024-04-24T23:17:31.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.596+0000 7f700ffff700 1 -- 172.21.15.31:0/2616073080 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f7008012070 con 0x7f7018101d00 2024-04-24T23:17:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.597+0000 7f700ffff700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f700406da40 0x7f700406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.597+0000 7f700ffff700 1 -- 172.21.15.31:0/2616073080 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f700808ce90 con 0x7f7018101d00 2024-04-24T23:17:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.597+0000 7f7016ffd700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f700406da40 0x7f700406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.597+0000 7f7016ffd700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f700406da40 0x7f700406fef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f700000b3b0 tx=0x7f7000009e30 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:31.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.601+0000 7f700ffff700 1 -- 172.21.15.31:0/2616073080 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f700805a590 con 0x7f7018101d00 2024-04-24T23:17:31.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.896+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7018108e50 con 0x7f7018101d00 2024-04-24T23:17:31.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.897+0000 7f700ffff700 1 -- 172.21.15.31:0/2616073080 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f700805a120 con 0x7f7018101d00 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.898+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f700406da40 msgr2=0x7f700406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.898+0000 7f701e09b700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f700406da40 0x7f700406fef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f700000b3b0 tx=0x7f7000009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.898+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018101d00 msgr2=0x7f70181128d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.898+0000 7f701e09b700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018101d00 0x7f70181128d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f700800bbb0 tx=0x7f7008009840 comp rx=0 tx=0).stop 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.899+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 shutdown_connections 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.899+0000 7f701e09b700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f700406da40 0x7f700406fef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.899+0000 7f701e09b700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7018101d00 0x7f70181128d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.899+0000 7f701e09b700 1 --2- 172.21.15.31:0/2616073080 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f70181026a0 0x7f7018112e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.899+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 >> 172.21.15.31:0/2616073080 conn(0x7f70180f7870 msgr2=0x7f70180f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:31.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.899+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 shutdown_connections 2024-04-24T23:17:31.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:31.899+0000 7f701e09b700 1 -- 172.21.15.31:0/2616073080 wait complete. 2024-04-24T23:17:31.911 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)" 2024-04-24T23:17:32.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-24T23:17:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:32 smithi031 ceph-mon[26411]: pgmap v78: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:32.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:32 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2616073080' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:32.790 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:32.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:32 smithi040 ceph-mon[33128]: pgmap v78: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:32.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:32 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2616073080' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:34.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 -- 172.21.15.31:0/2493255023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc102de0 msgr2=0x7fdebc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 --2- 172.21.15.31:0/2493255023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc102de0 0x7fdebc1031b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004940 tx=0x7fdeb8009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:34.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 -- 172.21.15.31:0/2493255023 shutdown_connections 2024-04-24T23:17:34.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 --2- 172.21.15.31:0/2493255023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdebc103780 0x7fdebc0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 --2- 172.21.15.31:0/2493255023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc102de0 0x7fdebc1031b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 -- 172.21.15.31:0/2493255023 >> 172.21.15.31:0/2493255023 conn(0x7fdebc0f8990 msgr2=0x7fdebc0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 -- 172.21.15.31:0/2493255023 shutdown_connections 2024-04-24T23:17:34.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 -- 172.21.15.31:0/2493255023 wait complete. 2024-04-24T23:17:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 Processor -- start 2024-04-24T23:17:34.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.461+0000 7fdec3cfd700 1 -- start start 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec3cfd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdebc102de0 0x7fdebc113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec3cfd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc103780 0x7fdebc113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec3cfd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc114550 con 0x7fdebc103780 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec3cfd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc10dac0 con 0x7fdebc102de0 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec1298700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc103780 0x7fdebc113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec1298700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc103780 0x7fdebc113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43758/0 (socket says 172.21.15.31:43758) 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec1298700 1 -- 172.21.15.31:0/885293172 learned_addr learned my addr 172.21.15.31:0/885293172 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec1a99700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdebc102de0 0x7fdebc113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.466 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec1298700 1 -- 172.21.15.31:0/885293172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdebc102de0 msgr2=0x7fdebc113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec1298700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdebc102de0 0x7fdebc113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec1298700 1 -- 172.21.15.31:0/885293172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8005040 con 0x7fdebc103780 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.462+0000 7fdec1298700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc103780 0x7fdebc113e70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b790 tx=0x7fdeac009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.463+0000 7fdeb27fc700 1 -- 172.21.15.31:0/885293172 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeac00f6f0 con 0x7fdebc103780 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.463+0000 7fdeb27fc700 1 -- 172.21.15.31:0/885293172 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdeac00fd20 con 0x7fdebc103780 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.463+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc10dda0 con 0x7fdebc103780 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.463+0000 7fdeb27fc700 1 -- 172.21.15.31:0/885293172 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdeac0103a0 con 0x7fdebc103780 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.463+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc10e360 con 0x7fdebc103780 2024-04-24T23:17:34.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.464+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc1000f0 con 0x7fdebc103780 2024-04-24T23:17:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.465+0000 7fdeb27fc700 1 -- 172.21.15.31:0/885293172 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fdeac011400 con 0x7fdebc103780 2024-04-24T23:17:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.465+0000 7fdeb27fc700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdea806da40 0x7fdea806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.465+0000 7fdeb27fc700 1 -- 172.21.15.31:0/885293172 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fdeac08d1c0 con 0x7fdebc103780 2024-04-24T23:17:34.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.465+0000 7fdec1a99700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdea806da40 0x7fdea806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:34.469 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.466+0000 7fdec1a99700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdea806da40 0x7fdea806fef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004f90 tx=0x7fdeb8009e30 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:34.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.469+0000 7fdeb27fc700 1 -- 172.21.15.31:0/885293172 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fdeac05a920 con 0x7fdebc103780 2024-04-24T23:17:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.762+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdebc0fe530 con 0x7fdebc103780 2024-04-24T23:17:34.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.763+0000 7fdeb27fc700 1 -- 172.21.15.31:0/885293172 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fdeac05a4b0 con 0x7fdebc103780 2024-04-24T23:17:34.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdea806da40 msgr2=0x7fdea806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdea806da40 0x7fdea806fef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8004f90 tx=0x7fdeb8009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc103780 msgr2=0x7fdebc113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc103780 0x7fdebc113e70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00b790 tx=0x7fdeac009220 comp rx=0 tx=0).stop 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 shutdown_connections 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fdea806da40 0x7fdea806fef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdebc102de0 0x7fdebc113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 --2- 172.21.15.31:0/885293172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdebc103780 0x7fdebc113e70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 >> 172.21.15.31:0/885293172 conn(0x7fdebc0f8990 msgr2=0x7fdebc0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 shutdown_connections 2024-04-24T23:17:34.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:34.765+0000 7fdec3cfd700 1 -- 172.21.15.31:0/885293172 wait complete. 2024-04-24T23:17:34.777 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:17:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:34 smithi040 ceph-mon[33128]: pgmap v79: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:34.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:34 smithi031 ceph-mon[26411]: pgmap v79: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:35.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-24T23:17:35.658 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:35.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:35 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/885293172' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:35 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/885293172' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:36.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:36 smithi031 ceph-mon[26411]: pgmap v80: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:36 smithi040 ceph-mon[33128]: pgmap v80: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 -- 172.21.15.31:0/3550516549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c102de0 msgr2=0x7f278c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 --2- 172.21.15.31:0/3550516549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c102de0 0x7f278c1031b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2784009ce0 tx=0x7f2784004d10 comp rx=0 tx=0).stop 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 -- 172.21.15.31:0/3550516549 shutdown_connections 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 --2- 172.21.15.31:0/3550516549 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f278c103780 0x7f278c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 --2- 172.21.15.31:0/3550516549 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c102de0 0x7f278c1031b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 -- 172.21.15.31:0/3550516549 >> 172.21.15.31:0/3550516549 conn(0x7f278c0f8990 msgr2=0x7f278c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 -- 172.21.15.31:0/3550516549 shutdown_connections 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 -- 172.21.15.31:0/3550516549 wait complete. 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.064+0000 7f2791df7700 1 Processor -- start 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f2791df7700 1 -- start start 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f2791df7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f278c102de0 0x7f278c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f2791df7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c103780 0x7f278c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f2791df7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0721b0 con 0x7f278c103780 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f2791df7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c070ce0 con 0x7f278c102de0 2024-04-24T23:17:37.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f278affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c103780 0x7f278c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f278affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c103780 0x7f278c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43782/0 (socket says 172.21.15.31:43782) 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f278affd700 1 -- 172.21.15.31:0/4173679988 learned_addr learned my addr 172.21.15.31:0/4173679988 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f278b7fe700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f278c102de0 0x7f278c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f278affd700 1 -- 172.21.15.31:0/4173679988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f278c102de0 msgr2=0x7f278c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f278affd700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f278c102de0 0x7f278c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f278affd700 1 -- 172.21.15.31:0/4173679988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784009940 con 0x7f278c103780 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f278affd700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c103780 0x7f278c0707a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2774002a50 tx=0x7f2774004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.065+0000 7f2783fff700 1 -- 172.21.15.31:0/4173679988 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2774007cc0 con 0x7f278c103780 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.066+0000 7f2783fff700 1 -- 172.21.15.31:0/4173679988 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f277400f040 con 0x7f278c103780 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.066+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c070fc0 con 0x7f278c103780 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.066+0000 7f2783fff700 1 -- 172.21.15.31:0/4173679988 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f277400da40 con 0x7f278c103780 2024-04-24T23:17:37.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.066+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c11a2d0 con 0x7f278c103780 2024-04-24T23:17:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.067+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c1000f0 con 0x7f278c103780 2024-04-24T23:17:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.067+0000 7f2783fff700 1 -- 172.21.15.31:0/4173679988 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f27740097c0 con 0x7f278c103780 2024-04-24T23:17:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.068+0000 7f2783fff700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f277806da40 0x7f277806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.068+0000 7f2783fff700 1 -- 172.21.15.31:0/4173679988 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f277408d090 con 0x7f278c103780 2024-04-24T23:17:37.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.068+0000 7f278b7fe700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f277806da40 0x7f277806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:37.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.068+0000 7f278b7fe700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f277806da40 0x7f277806fef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2784005010 tx=0x7f2784004d10 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:37.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.072+0000 7f2783fff700 1 -- 172.21.15.31:0/4173679988 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f277405a6f0 con 0x7f278c103780 2024-04-24T23:17:37.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:37.296+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}) v1 -- 0x7f278c109ee0 con 0x7f277806da40 2024-04-24T23:17:38.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:38 smithi040 ceph-mon[33128]: pgmap v81: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:38.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:38 smithi040 ceph-mon[33128]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:38.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:38 smithi031 ceph-mon[26411]: pgmap v81: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:38.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:38 smithi031 ceph-mon[26411]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:39.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.567+0000 7f2783fff700 1 -- 172.21.15.31:0/4173679988 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f278c109ee0 con 0x7f277806da40 2024-04-24T23:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f277806da40 msgr2=0x7f277806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f277806da40 0x7f277806fef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2784005010 tx=0x7f2784004d10 comp rx=0 tx=0).stop 2024-04-24T23:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c103780 msgr2=0x7f278c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c103780 0x7f278c0707a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2774002a50 tx=0x7f2774004600 comp rx=0 tx=0).stop 2024-04-24T23:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 shutdown_connections 2024-04-24T23:17:39.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f277806da40 0x7f277806fef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f278c102de0 0x7f278c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 --2- 172.21.15.31:0/4173679988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f278c103780 0x7f278c0707a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 >> 172.21.15.31:0/4173679988 conn(0x7f278c0f8990 msgr2=0x7f278c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 shutdown_connections 2024-04-24T23:17:39.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:39.569+0000 7f2791df7700 1 -- 172.21.15.31:0/4173679988 wait complete. 2024-04-24T23:17:39.583 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:17:40.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-24T23:17:40.479 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:40 smithi040 ceph-mon[33128]: pgmap v82: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:40.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:40 smithi031 ceph-mon[26411]: pgmap v82: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.084+0000 7f5977dc4700 1 -- 172.21.15.31:0/3272550637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 msgr2=0x7f59701020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.084+0000 7f5977dc4700 1 --2- 172.21.15.31:0/3272550637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 0x7f59701020d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5960004940 tx=0x7f5960009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.084+0000 7f5977dc4700 1 -- 172.21.15.31:0/3272550637 shutdown_connections 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.084+0000 7f5977dc4700 1 --2- 172.21.15.31:0/3272550637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59701026a0 0x7f59700fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.084+0000 7f5977dc4700 1 --2- 172.21.15.31:0/3272550637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 0x7f59701020d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.084+0000 7f5977dc4700 1 -- 172.21.15.31:0/3272550637 >> 172.21.15.31:0/3272550637 conn(0x7f59700f7870 msgr2=0x7f59700f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.084+0000 7f5977dc4700 1 -- 172.21.15.31:0/3272550637 shutdown_connections 2024-04-24T23:17:42.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.084+0000 7f5977dc4700 1 -- 172.21.15.31:0/3272550637 wait complete. 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5977dc4700 1 Processor -- start 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5977dc4700 1 -- start start 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5977dc4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 0x7f5970102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5977dc4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59701026a0 0x7f59701030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5977dc4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5970104b00 con 0x7f5970101d00 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5977dc4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59701035e0 con 0x7f59701026a0 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5975b60700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 0x7f5970102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5975b60700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 0x7f5970102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58538/0 (socket says 172.21.15.31:58538) 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.085+0000 7f5975b60700 1 -- 172.21.15.31:0/4110053936 learned_addr learned my addr 172.21.15.31:0/4110053936 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f597535f700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59701026a0 0x7f59701030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f5975b60700 1 -- 172.21.15.31:0/4110053936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59701026a0 msgr2=0x7f59701030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f5975b60700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59701026a0 0x7f59701030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f5975b60700 1 -- 172.21.15.31:0/4110053936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5960005040 con 0x7f5970101d00 2024-04-24T23:17:42.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f5975b60700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 0x7f5970102b60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f596000b3b0 tx=0x7f5960009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f59667fc700 1 -- 172.21.15.31:0/4110053936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f596000e040 con 0x7f5970101d00 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f59667fc700 1 -- 172.21.15.31:0/4110053936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5960013420 con 0x7f5970101d00 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f59667fc700 1 -- 172.21.15.31:0/4110053936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f596001c630 con 0x7f5970101d00 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5970103830 con 0x7f5970101d00 2024-04-24T23:17:42.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.086+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597011a090 con 0x7f5970101d00 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.087+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59700fd450 con 0x7f5970101d00 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.088+0000 7f59667fc700 1 -- 172.21.15.31:0/4110053936 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f5960013590 con 0x7f5970101d00 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.088+0000 7f59667fc700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f595c06da40 0x7f595c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.088+0000 7f59667fc700 1 -- 172.21.15.31:0/4110053936 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f596008e390 con 0x7f5970101d00 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.088+0000 7f597535f700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f595c06da40 0x7f595c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:42.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.089+0000 7f597535f700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f595c06da40 0x7f595c06fef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5970104290 tx=0x7f596c0043c0 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:42.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.093+0000 7f59667fc700 1 -- 172.21.15.31:0/4110053936 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f5960057fe0 con 0x7f5970101d00 2024-04-24T23:17:42.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.310+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f59700f8ef0 con 0x7f595c06da40 2024-04-24T23:17:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.321+0000 7f59667fc700 1 -- 172.21.15.31:0/4110053936 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f59700f8ef0 con 0x7f595c06da40 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (51s) 29s ago 12m 16.4M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (12m) 29s ago 12m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (11m) 85s ago 11m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (35s) 29s ago 12m 52.7M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (6m) 29s ago 6m 139M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (6m) 29s ago 6m 13.0M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (6m) 85s ago 6m 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (6m) 85s ago 6m 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (2m) 29s ago 13m 511M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (107s) 85s ago 11m 448M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (13m) 29s ago 14m 55.8M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (11m) 85s ago 11m 46.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:17:42.323 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (93s) 29s ago 12m 16.8M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (89s) 85s ago 11m 3456k - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (10m) 29s ago 10m 221M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (10m) 29s ago 10m 238M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (9m) 29s ago 9m 198M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (9m) 85s ago 9m 276M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (8m) 85s ago 8m 245M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (8m) 85s ago 8m 223M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:17:42.324 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (70s) 29s ago 12m 48.6M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f595c06da40 msgr2=0x7f595c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f595c06da40 0x7f595c06fef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5970104290 tx=0x7f596c0043c0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 msgr2=0x7f5970102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 0x7f5970102b60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f596000b3b0 tx=0x7f5960009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 shutdown_connections 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f595c06da40 0x7f595c06fef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5970101d00 0x7f5970102b60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 --2- 172.21.15.31:0/4110053936 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f59701026a0 0x7f59701030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 >> 172.21.15.31:0/4110053936 conn(0x7f59700f7870 msgr2=0x7f59700f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 shutdown_connections 2024-04-24T23:17:42.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:42.323+0000 7f5977dc4700 1 -- 172.21.15.31:0/4110053936 wait complete. 2024-04-24T23:17:42.902 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:42 smithi031 ceph-mon[26411]: pgmap v83: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:42.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:42 smithi040 ceph-mon[33128]: pgmap v83: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:42.929 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T23:17:42.930 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:17:42.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-24T23:17:43.199 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:43.843 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:43 smithi031 ceph-mon[26411]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:43.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:43 smithi040 ceph-mon[33128]: from='client.14830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:44.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:44 smithi031 ceph-mon[26411]: pgmap v84: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:44.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:44 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.852+0000 7f26e4b8d700 1 -- 172.21.15.31:0/191053641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e00fbd40 msgr2=0x7f26e00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.852+0000 7f26e4b8d700 1 --2- 172.21.15.31:0/191053641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e00fbd40 0x7f26e00fc1b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f26c8004940 tx=0x7f26c8009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.852+0000 7f26e4b8d700 1 -- 172.21.15.31:0/191053641 shutdown_connections 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.852+0000 7f26e4b8d700 1 --2- 172.21.15.31:0/191053641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e00fbd40 0x7f26e00fc1b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.852+0000 7f26e4b8d700 1 --2- 172.21.15.31:0/191053641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e0101d90 0x7f26e0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.852+0000 7f26e4b8d700 1 -- 172.21.15.31:0/191053641 >> 172.21.15.31:0/191053641 conn(0x7f26e0071d00 msgr2=0x7f26e0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.852+0000 7f26e4b8d700 1 -- 172.21.15.31:0/191053641 shutdown_connections 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.852+0000 7f26e4b8d700 1 -- 172.21.15.31:0/191053641 wait complete. 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26e4b8d700 1 Processor -- start 2024-04-24T23:17:44.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26e4b8d700 1 -- start start 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26e4b8d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e00fbd40 0x7f26e010ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26e4b8d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0101d90 0x7f26e010f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26e4b8d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e010f750 con 0x7f26e0101d90 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26e4b8d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e010f890 con 0x7f26e00fbd40 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26ddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0101d90 0x7f26e010f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26ddd9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0101d90 0x7f26e010f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58552/0 (socket says 172.21.15.31:58552) 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26ddd9b700 1 -- 172.21.15.31:0/2976121793 learned_addr learned my addr 172.21.15.31:0/2976121793 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.853+0000 7f26de59c700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e00fbd40 0x7f26e010ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26ddd9b700 1 -- 172.21.15.31:0/2976121793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e00fbd40 msgr2=0x7f26e010ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26ddd9b700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e00fbd40 0x7f26e010ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26ddd9b700 1 -- 172.21.15.31:0/2976121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8005040 con 0x7f26e0101d90 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26ddd9b700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0101d90 0x7f26e010f210 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b3b0 tx=0x7f26c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26d6ffd700 1 -- 172.21.15.31:0/2976121793 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c800e040 con 0x7f26e0101d90 2024-04-24T23:17:44.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26d6ffd700 1 -- 172.21.15.31:0/2976121793 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26c8013420 con 0x7f26e0101d90 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e0117b80 con 0x7f26e0101d90 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26d6ffd700 1 -- 172.21.15.31:0/2976121793 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c801c630 con 0x7f26e0101d90 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.854+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e0118110 con 0x7f26e0101d90 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.855+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e00fd430 con 0x7f26e0101d90 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.856+0000 7f26d6ffd700 1 -- 172.21.15.31:0/2976121793 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f26c8013590 con 0x7f26e0101d90 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.857+0000 7f26d6ffd700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f26cc06da40 0x7f26cc06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.857+0000 7f26de59c700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f26cc06da40 0x7f26cc06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.857+0000 7f26d6ffd700 1 -- 172.21.15.31:0/2976121793 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f26c808e310 con 0x7f26e0101d90 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.857+0000 7f26de59c700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f26cc06da40 0x7f26cc06fef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26d8007a40 tx=0x7f26d8009da0 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:44.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:44.861+0000 7f26d6ffd700 1 -- 172.21.15.31:0/2976121793 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f26c8058000 con 0x7f26e0101d90 2024-04-24T23:17:44.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:44 smithi040 ceph-mon[33128]: pgmap v84: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:44.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:44 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:17:45.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.068+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f26e004bed0 con 0x7f26e0101d90 2024-04-24T23:17:45.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.072+0000 7f26d6ffd700 1 -- 172.21.15.31:0/2976121793 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v52)=0 v52) v1 ==== 125+0+0 (secure 0 0 0) 0x7f26c805b6c0 con 0x7f26e0101d90 2024-04-24T23:17:45.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.074+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f26cc06da40 msgr2=0x7f26cc06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.074+0000 7f26e4b8d700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f26cc06da40 0x7f26cc06fef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26d8007a40 tx=0x7f26d8009da0 comp rx=0 tx=0).stop 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.074+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0101d90 msgr2=0x7f26e010f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.074+0000 7f26e4b8d700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0101d90 0x7f26e010f210 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b3b0 tx=0x7f26c8009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.075+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 shutdown_connections 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.075+0000 7f26e4b8d700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f26cc06da40 0x7f26cc06fef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.075+0000 7f26e4b8d700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f26e00fbd40 0x7f26e010ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.075+0000 7f26e4b8d700 1 --2- 172.21.15.31:0/2976121793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26e0101d90 0x7f26e010f210 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.075+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 >> 172.21.15.31:0/2976121793 conn(0x7f26e0071d00 msgr2=0x7f26e0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.075+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 shutdown_connections 2024-04-24T23:17:45.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:45.075+0000 7f26e4b8d700 1 -- 172.21.15.31:0/2976121793 wait complete. 2024-04-24T23:17:45.714 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T23:17:45.715 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:17:45.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-24T23:17:45.984 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:46.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:46 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2976121793' entity='client.admin' 2024-04-24T23:17:46.406 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:46 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:46.407 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:46 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:46.407 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:46 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:46.407 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:46 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:46.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:46 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2976121793' entity='client.admin' 2024-04-24T23:17:46.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:46 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:46.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:46 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:46.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:46 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:46.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:46 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:47.360 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:47 smithi031 ceph-mon[26411]: pgmap v85: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:47.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:47 smithi040 ceph-mon[33128]: pgmap v85: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.571+0000 7f8c52992700 1 -- 172.21.15.31:0/2389503470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c101d90 msgr2=0x7f8c4c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.571+0000 7f8c52992700 1 --2- 172.21.15.31:0/2389503470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c101d90 0x7f8c4c102160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c004940 tx=0x7f8c3c009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.571+0000 7f8c52992700 1 -- 172.21.15.31:0/2389503470 shutdown_connections 2024-04-24T23:17:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.571+0000 7f8c52992700 1 --2- 172.21.15.31:0/2389503470 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4c0fbd40 0x7f8c4c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.571+0000 7f8c52992700 1 --2- 172.21.15.31:0/2389503470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c101d90 0x7f8c4c102160 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:47.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.571+0000 7f8c52992700 1 -- 172.21.15.31:0/2389503470 >> 172.21.15.31:0/2389503470 conn(0x7f8c4c071d00 msgr2=0x7f8c4c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.571+0000 7f8c52992700 1 -- 172.21.15.31:0/2389503470 shutdown_connections 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.571+0000 7f8c52992700 1 -- 172.21.15.31:0/2389503470 wait complete. 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c52992700 1 Processor -- start 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c52992700 1 -- start start 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c52992700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c0fbd40 0x7f8c4c113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c52992700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4c101d90 0x7f8c4c114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c52992700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c114770 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c52992700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c10dad0 con 0x7f8c4c101d90 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c4bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c0fbd40 0x7f8c4c113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c4bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c0fbd40 0x7f8c4c113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58578/0 (socket says 172.21.15.31:58578) 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c4bfff700 1 -- 172.21.15.31:0/3821332685 learned_addr learned my addr 172.21.15.31:0/3821332685 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:47.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c4b7fe700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4c101d90 0x7f8c4c114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c4bfff700 1 -- 172.21.15.31:0/3821332685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4c101d90 msgr2=0x7f8c4c114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c4bfff700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4c101d90 0x7f8c4c114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c4bfff700 1 -- 172.21.15.31:0/3821332685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c005040 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.572+0000 7f8c4b7fe700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4c101d90 0x7f8c4c114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.573+0000 7f8c4bfff700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c0fbd40 0x7f8c4c113b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b3b0 tx=0x7f8c3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.573+0000 7f8c48ff9700 1 -- 172.21.15.31:0/3821332685 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3c00e040 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.573+0000 7f8c48ff9700 1 -- 172.21.15.31:0/3821332685 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c3c00bbc0 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.573+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c10dd50 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.573+0000 7f8c48ff9700 1 -- 172.21.15.31:0/3821332685 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c3c01c3f0 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.573+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c10e310 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.574+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c1063c0 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.575+0000 7f8c48ff9700 1 -- 172.21.15.31:0/3821332685 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f8c3c00bd30 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.575+0000 7f8c48ff9700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8c3406da40 0x7f8c3406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.575+0000 7f8c48ff9700 1 -- 172.21.15.31:0/3821332685 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f8c3c08e4f0 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.575+0000 7f8c4b7fe700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8c3406da40 0x7f8c3406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:47.580 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.575+0000 7f8c4b7fe700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8c3406da40 0x7f8c3406fef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c10ec90 tx=0x7f8c40008040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:47.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.580+0000 7f8c48ff9700 1 -- 172.21.15.31:0/3821332685 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8c3c05bbf0 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.788+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f8c4c10f1a0 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.788+0000 7f8c48ff9700 1 -- 172.21.15.31:0/3821332685 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v52)=0 v52) v1 ==== 155+0+0 (secure 0 0 0) 0x7f8c3c05b780 con 0x7f8c4c0fbd40 2024-04-24T23:17:47.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.790+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8c3406da40 msgr2=0x7f8c3406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.790+0000 7f8c52992700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8c3406da40 0x7f8c3406fef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c10ec90 tx=0x7f8c40008040 comp rx=0 tx=0).stop 2024-04-24T23:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.790+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c0fbd40 msgr2=0x7f8c4c113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.790+0000 7f8c52992700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c0fbd40 0x7f8c4c113b50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b3b0 tx=0x7f8c3c009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.790+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 shutdown_connections 2024-04-24T23:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.791+0000 7f8c52992700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f8c3406da40 0x7f8c3406fef0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.791+0000 7f8c52992700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8c4c0fbd40 0x7f8c4c113b50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.791+0000 7f8c52992700 1 --2- 172.21.15.31:0/3821332685 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8c4c101d90 0x7f8c4c114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:47.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.791+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 >> 172.21.15.31:0/3821332685 conn(0x7f8c4c071d00 msgr2=0x7f8c4c0fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.791+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 shutdown_connections 2024-04-24T23:17:47.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:47.791+0000 7f8c52992700 1 -- 172.21.15.31:0/3821332685 wait complete. 2024-04-24T23:17:48.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-24T23:17:48.663 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:49.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:49 smithi040 ceph-mon[33128]: pgmap v86: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:49.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:49 smithi031 ceph-mon[26411]: pgmap v86: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:50.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 -- 172.21.15.31:0/4054926216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 msgr2=0x7ff0601031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:50.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 --2- 172.21.15.31:0/4054926216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 0x7ff0601031b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff054004940 tx=0x7ff054009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:50.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 -- 172.21.15.31:0/4054926216 shutdown_connections 2024-04-24T23:17:50.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 --2- 172.21.15.31:0/4054926216 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff060103780 0x7ff0600fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:50.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 --2- 172.21.15.31:0/4054926216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 0x7ff0601031b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:50.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 -- 172.21.15.31:0/4054926216 >> 172.21.15.31:0/4054926216 conn(0x7ff0600f89b0 msgr2=0x7ff0600fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:50.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 -- 172.21.15.31:0/4054926216 shutdown_connections 2024-04-24T23:17:50.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 -- 172.21.15.31:0/4054926216 wait complete. 2024-04-24T23:17:50.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 Processor -- start 2024-04-24T23:17:50.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.258+0000 7ff06701d700 1 -- start start 2024-04-24T23:17:50.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff06701d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 0x7ff060070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:50.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff06701d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff060103780 0x7ff0600707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff06701d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff060070d70 con 0x7ff060102de0 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff06701d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff060070eb0 con 0x7ff060103780 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff064db9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 0x7ff060070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff064db9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 0x7ff060070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49478/0 (socket says 172.21.15.31:49478) 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff064db9700 1 -- 172.21.15.31:0/3545390139 learned_addr learned my addr 172.21.15.31:0/3545390139 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff05ffff700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff060103780 0x7ff0600707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff064db9700 1 -- 172.21.15.31:0/3545390139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff060103780 msgr2=0x7ff0600707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff064db9700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff060103780 0x7ff0600707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff064db9700 1 -- 172.21.15.31:0/3545390139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff054005040 con 0x7ff060102de0 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.259+0000 7ff064db9700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 0x7ff060070260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff05400b3b0 tx=0x7ff054009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.260+0000 7ff05d7fa700 1 -- 172.21.15.31:0/3545390139 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05400e040 con 0x7ff060102de0 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.260+0000 7ff05d7fa700 1 -- 172.21.15.31:0/3545390139 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff054013420 con 0x7ff060102de0 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.260+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff060119ea0 con 0x7ff060102de0 2024-04-24T23:17:50.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.260+0000 7ff05d7fa700 1 -- 172.21.15.31:0/3545390139 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff05401c630 con 0x7ff060102de0 2024-04-24T23:17:50.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.260+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06011a320 con 0x7ff060102de0 2024-04-24T23:17:50.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.261+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0601000f0 con 0x7ff060102de0 2024-04-24T23:17:50.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.261+0000 7ff05d7fa700 1 -- 172.21.15.31:0/3545390139 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7ff054013590 con 0x7ff060102de0 2024-04-24T23:17:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.262+0000 7ff05d7fa700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff05006da40 0x7ff05006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.262+0000 7ff05d7fa700 1 -- 172.21.15.31:0/3545390139 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff05408e3a0 con 0x7ff060102de0 2024-04-24T23:17:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.262+0000 7ff05ffff700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff05006da40 0x7ff05006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:50.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.262+0000 7ff05ffff700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff05006da40 0x7ff05006fef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff060071b50 tx=0x7ff04c0091a0 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:50.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.266+0000 7ff05d7fa700 1 -- 172.21.15.31:0/3545390139 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff054057ff0 con 0x7ff060102de0 2024-04-24T23:17:50.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.472+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7ff0600fe530 con 0x7ff060102de0 2024-04-24T23:17:50.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.472+0000 7ff05d7fa700 1 -- 172.21.15.31:0/3545390139 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v52)=0 v52) v1 ==== 163+0+0 (secure 0 0 0) 0x7ff05405b6b0 con 0x7ff060102de0 2024-04-24T23:17:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff05006da40 msgr2=0x7ff05006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff05006da40 0x7ff05006fef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff060071b50 tx=0x7ff04c0091a0 comp rx=0 tx=0).stop 2024-04-24T23:17:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 msgr2=0x7ff060070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 0x7ff060070260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff05400b3b0 tx=0x7ff054009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 shutdown_connections 2024-04-24T23:17:50.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff05006da40 0x7ff05006fef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff060102de0 0x7ff060070260 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 --2- 172.21.15.31:0/3545390139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff060103780 0x7ff0600707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 >> 172.21.15.31:0/3545390139 conn(0x7ff0600f89b0 msgr2=0x7ff060107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 shutdown_connections 2024-04-24T23:17:50.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:50.474+0000 7ff06701d700 1 -- 172.21.15.31:0/3545390139 wait complete. 2024-04-24T23:17:51.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-24T23:17:51.336 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:51.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:51 smithi040 ceph-mon[33128]: pgmap v87: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:51.450 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:51 smithi031 ceph-mon[26411]: pgmap v87: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:52.453 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:52 smithi031 ceph-mon[26411]: pgmap v88: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:52.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:52 smithi040 ceph-mon[33128]: pgmap v88: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:52.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.880+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2199427144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 msgr2=0x7f0ae40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:52.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.880+0000 7f0aeb4a2700 1 --2- 172.21.15.31:0/2199427144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 0x7f0ae40fcdf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4004940 tx=0x7f0ad4009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:52.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.880+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2199427144 shutdown_connections 2024-04-24T23:17:52.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.880+0000 7f0aeb4a2700 1 --2- 172.21.15.31:0/2199427144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 0x7f0ae40fcdf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.880+0000 7f0aeb4a2700 1 --2- 172.21.15.31:0/2199427144 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ae4102de0 0x7f0ae41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.880+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2199427144 >> 172.21.15.31:0/2199427144 conn(0x7f0ae40f8990 msgr2=0x7f0ae40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.880+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2199427144 shutdown_connections 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.880+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2199427144 wait complete. 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0aeb4a2700 1 Processor -- start 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0aeb4a2700 1 -- start start 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0aeb4a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ae4102de0 0x7f0ae4113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0aeb4a2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 0x7f0ae4113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0aeb4a2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae4114510 con 0x7f0ae4103780 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0aeb4a2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae410db10 con 0x7f0ae4102de0 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0ae8a3d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 0x7f0ae4113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0ae8a3d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 0x7f0ae4113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49502/0 (socket says 172.21.15.31:49502) 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0ae8a3d700 1 -- 172.21.15.31:0/2690775131 learned_addr learned my addr 172.21.15.31:0/2690775131 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:52.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0ae923e700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ae4102de0 0x7f0ae4113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0ae8a3d700 1 -- 172.21.15.31:0/2690775131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ae4102de0 msgr2=0x7f0ae4113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0ae8a3d700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ae4102de0 0x7f0ae4113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0ae8a3d700 1 -- 172.21.15.31:0/2690775131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad4005040 con 0x7f0ae4103780 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.881+0000 7f0ae8a3d700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 0x7f0ae4113ec0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009100 tx=0x7f0ad4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.882+0000 7f0ad9ffb700 1 -- 172.21.15.31:0/2690775131 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad400e040 con 0x7f0ae4103780 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.882+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae410dd90 con 0x7f0ae4103780 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.882+0000 7f0ad9ffb700 1 -- 172.21.15.31:0/2690775131 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ad4009af0 con 0x7f0ae4103780 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.882+0000 7f0ad9ffb700 1 -- 172.21.15.31:0/2690775131 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ad401c3f0 con 0x7f0ae4103780 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.882+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae410e350 con 0x7f0ae4103780 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.883+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae41052b0 con 0x7f0ae4103780 2024-04-24T23:17:52.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.884+0000 7f0ad9ffb700 1 -- 172.21.15.31:0/2690775131 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f0ad400baf0 con 0x7f0ae4103780 2024-04-24T23:17:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.884+0000 7f0ad9ffb700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0ad006da40 0x7f0ad006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.884+0000 7f0ad9ffb700 1 -- 172.21.15.31:0/2690775131 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f0ad408e400 con 0x7f0ae4103780 2024-04-24T23:17:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.884+0000 7f0ae923e700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0ad006da40 0x7f0ad006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:52.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.885+0000 7f0ae923e700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0ad006da40 0x7f0ad006fef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0004ed0 tx=0x7f0ae000b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:52.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:52.889+0000 7f0ad9ffb700 1 -- 172.21.15.31:0/2690775131 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0ad405bb80 con 0x7f0ae4103780 2024-04-24T23:17:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.098+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f0ae410efd0 con 0x7f0ae4103780 2024-04-24T23:17:53.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.098+0000 7f0ad9ffb700 1 -- 172.21.15.31:0/2690775131 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v52)=0 v52) v1 ==== 135+0+0 (secure 0 0 0) 0x7f0ad405b710 con 0x7f0ae4103780 2024-04-24T23:17:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0ad006da40 msgr2=0x7f0ad006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0ad006da40 0x7f0ad006fef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0004ed0 tx=0x7f0ae000b040 comp rx=0 tx=0).stop 2024-04-24T23:17:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 msgr2=0x7f0ae4113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 0x7f0ae4113ec0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009100 tx=0x7f0ad4009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:53.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 shutdown_connections 2024-04-24T23:17:53.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f0ad006da40 0x7f0ad006fef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ae4102de0 0x7f0ae4113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 --2- 172.21.15.31:0/2690775131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ae4103780 0x7f0ae4113ec0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:53.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 >> 172.21.15.31:0/2690775131 conn(0x7f0ae40f8990 msgr2=0x7f0ae4102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:53.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.101+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 shutdown_connections 2024-04-24T23:17:53.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:53.102+0000 7f0aeb4a2700 1 -- 172.21.15.31:0/2690775131 wait complete. 2024-04-24T23:17:53.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-24T23:17:53.924 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:54.342 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:54 smithi031 ceph-mon[26411]: pgmap v89: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:54 smithi040 ceph-mon[33128]: pgmap v89: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.431+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/3326461391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28102de0 msgr2=0x7f6c281031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.431+0000 7f6c2d1cf700 1 --2- 172.21.15.31:0/3326461391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28102de0 0x7f6c281031b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6c18004940 tx=0x7f6c18009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.431+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/3326461391 shutdown_connections 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.431+0000 7f6c2d1cf700 1 --2- 172.21.15.31:0/3326461391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c28103780 0x7f6c280fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.431+0000 7f6c2d1cf700 1 --2- 172.21.15.31:0/3326461391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28102de0 0x7f6c281031b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.431+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/3326461391 >> 172.21.15.31:0/3326461391 conn(0x7f6c280f8990 msgr2=0x7f6c280fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.431+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/3326461391 shutdown_connections 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.431+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/3326461391 wait complete. 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.432+0000 7f6c2d1cf700 1 Processor -- start 2024-04-24T23:17:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.432+0000 7f6c2d1cf700 1 -- start start 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.432+0000 7f6c2d1cf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c28102de0 0x7f6c28113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.432+0000 7f6c2d1cf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28103780 0x7f6c28113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.432+0000 7f6c2d1cf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c28114550 con 0x7f6c28103780 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.432+0000 7f6c2d1cf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2810dac0 con 0x7f6c28102de0 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.432+0000 7f6c2659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28103780 0x7f6c28113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c2659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28103780 0x7f6c28113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49516/0 (socket says 172.21.15.31:49516) 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c2659c700 1 -- 172.21.15.31:0/2203765440 learned_addr learned my addr 172.21.15.31:0/2203765440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c26d9d700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c28102de0 0x7f6c28113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c2659c700 1 -- 172.21.15.31:0/2203765440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c28102de0 msgr2=0x7f6c28113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c2659c700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c28102de0 0x7f6c28113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c2659c700 1 -- 172.21.15.31:0/2203765440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c18005040 con 0x7f6c28103780 2024-04-24T23:17:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c2659c700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28103780 0x7f6c28113e70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b790 tx=0x7f6c1c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c0f7fe700 1 -- 172.21.15.31:0/2203765440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1c00f6f0 con 0x7f6c28103780 2024-04-24T23:17:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2810dda0 con 0x7f6c28103780 2024-04-24T23:17:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2810e360 con 0x7f6c28103780 2024-04-24T23:17:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c0f7fe700 1 -- 172.21.15.31:0/2203765440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c1c00fd20 con 0x7f6c28103780 2024-04-24T23:17:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.433+0000 7f6c0f7fe700 1 -- 172.21.15.31:0/2203765440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1c0103a0 con 0x7f6c28103780 2024-04-24T23:17:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.434+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c140052e0 con 0x7f6c28103780 2024-04-24T23:17:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.435+0000 7f6c0f7fe700 1 -- 172.21.15.31:0/2203765440 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f6c1c00f850 con 0x7f6c28103780 2024-04-24T23:17:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.435+0000 7f6c0f7fe700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6c1006da40 0x7f6c1006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.435+0000 7f6c0f7fe700 1 -- 172.21.15.31:0/2203765440 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f6c1c08ce10 con 0x7f6c28103780 2024-04-24T23:17:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.435+0000 7f6c26d9d700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6c1006da40 0x7f6c1006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.436+0000 7f6c26d9d700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6c1006da40 0x7f6c1006fef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c18004f90 tx=0x7f6c18009e30 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:55.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.440+0000 7f6c0f7fe700 1 -- 172.21.15.31:0/2203765440 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6c1c05a630 con 0x7f6c28103780 2024-04-24T23:17:55.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.670+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c14000c90 con 0x7f6c1006da40 2024-04-24T23:17:55.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.677+0000 7f6c0f7fe700 1 -- 172.21.15.31:0/2203765440 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f6c14000c90 con 0x7f6c1006da40 2024-04-24T23:17:55.680 INFO:teuthology.orchestra.run.smithi031.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:17:55.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6c1006da40 msgr2=0x7f6c1006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:55.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6c1006da40 0x7f6c1006fef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c18004f90 tx=0x7f6c18009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:55.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28103780 msgr2=0x7f6c28113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:55.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28103780 0x7f6c28113e70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c00b790 tx=0x7f6c1c009220 comp rx=0 tx=0).stop 2024-04-24T23:17:55.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 shutdown_connections 2024-04-24T23:17:55.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f6c1006da40 0x7f6c1006fef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c28102de0 0x7f6c28113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 --2- 172.21.15.31:0/2203765440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c28103780 0x7f6c28113e70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 >> 172.21.15.31:0/2203765440 conn(0x7f6c280f8990 msgr2=0x7f6c280fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 shutdown_connections 2024-04-24T23:17:55.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:55.680+0000 7f6c2d1cf700 1 -- 172.21.15.31:0/2203765440 wait complete. 2024-04-24T23:17:56.309 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-24T23:17:56.309 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:17:56.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-24T23:17:56.587 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: pgmap v90: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:56.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:56 smithi031 ceph-mon[26411]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:17:57.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: pgmap v90: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:17:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:17:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:17:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:57.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:56 smithi040 ceph-mon[33128]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:17:58.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.365+0000 7f7b7075d700 1 -- 172.21.15.31:0/2322602967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 msgr2=0x7f7b681031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:58.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.365+0000 7f7b7075d700 1 --2- 172.21.15.31:0/2322602967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 0x7f7b681031b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7b64004940 tx=0x7f7b64009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:58.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.365+0000 7f7b7075d700 1 -- 172.21.15.31:0/2322602967 shutdown_connections 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.365+0000 7f7b7075d700 1 --2- 172.21.15.31:0/2322602967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b68103780 0x7f7b680fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.365+0000 7f7b7075d700 1 --2- 172.21.15.31:0/2322602967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 0x7f7b681031b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.365+0000 7f7b7075d700 1 -- 172.21.15.31:0/2322602967 >> 172.21.15.31:0/2322602967 conn(0x7f7b680f8990 msgr2=0x7f7b680fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.365+0000 7f7b7075d700 1 -- 172.21.15.31:0/2322602967 shutdown_connections 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.365+0000 7f7b7075d700 1 -- 172.21.15.31:0/2322602967 wait complete. 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b7075d700 1 Processor -- start 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b7075d700 1 -- start start 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b7075d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 0x7f7b68113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b7075d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b68103780 0x7f7b68113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b7075d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b68114500 con 0x7f7b68102de0 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b7075d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6810db50 con 0x7f7b68103780 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b6e4f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 0x7f7b68113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b6e4f9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 0x7f7b68113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49524/0 (socket says 172.21.15.31:49524) 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.366+0000 7f7b6e4f9700 1 -- 172.21.15.31:0/3554347230 learned_addr learned my addr 172.21.15.31:0/3554347230 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:58.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b6dcf8700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b68103780 0x7f7b68113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:58.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b6e4f9700 1 -- 172.21.15.31:0/3554347230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b68103780 msgr2=0x7f7b68113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:58.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b6e4f9700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b68103780 0x7f7b68113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.370 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b6e4f9700 1 -- 172.21.15.31:0/3554347230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64005040 con 0x7f7b68102de0 2024-04-24T23:17:58.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b6e4f9700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 0x7f7b68113970 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b8b0 tx=0x7f7b64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:58.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b5affd700 1 -- 172.21.15.31:0/3554347230 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6400e040 con 0x7f7b68102de0 2024-04-24T23:17:58.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6810ddd0 con 0x7f7b68102de0 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6810e390 con 0x7f7b68102de0 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.367+0000 7f7b5affd700 1 -- 172.21.15.31:0/3554347230 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b64004290 con 0x7f7b68102de0 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.368+0000 7f7b5affd700 1 -- 172.21.15.31:0/3554347230 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b64025400 con 0x7f7b68102de0 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.368+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c0052b0 con 0x7f7b68102de0 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.369+0000 7f7b5affd700 1 -- 172.21.15.31:0/3554347230 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f7b64013420 con 0x7f7b68102de0 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.369+0000 7f7b5affd700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f7b5406da40 0x7f7b5406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.369+0000 7f7b5affd700 1 -- 172.21.15.31:0/3554347230 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f7b6408e470 con 0x7f7b68102de0 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.369+0000 7f7b6dcf8700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f7b5406da40 0x7f7b5406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:58.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.370+0000 7f7b6dcf8700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f7b5406da40 0x7f7b5406fef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c007fd0 tx=0x7f7b5c009040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:58.375 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.374+0000 7f7b5affd700 1 -- 172.21.15.31:0/3554347230 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f7b6405bbf0 con 0x7f7b68102de0 2024-04-24T23:17:58.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.603+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b4c000bc0 con 0x7f7b5406da40 2024-04-24T23:17:58.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.604+0000 7f7b5affd700 1 -- 172.21.15.31:0/3554347230 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f7b4c000bc0 con 0x7f7b5406da40 2024-04-24T23:17:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f7b5406da40 msgr2=0x7f7b5406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f7b5406da40 0x7f7b5406fef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c007fd0 tx=0x7f7b5c009040 comp rx=0 tx=0).stop 2024-04-24T23:17:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 msgr2=0x7f7b68113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 0x7f7b68113970 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400b8b0 tx=0x7f7b64009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 shutdown_connections 2024-04-24T23:17:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f7b5406da40 0x7f7b5406fef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b68102de0 0x7f7b68113970 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 --2- 172.21.15.31:0/3554347230 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7b68103780 0x7f7b68113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 >> 172.21.15.31:0/3554347230 conn(0x7f7b680f8990 msgr2=0x7f7b68107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.606+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 shutdown_connections 2024-04-24T23:17:58.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.607+0000 7f7b7075d700 1 -- 172.21.15.31:0/3554347230 wait complete. 2024-04-24T23:17:58.619 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:17:58.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.717+0000 7fa8d54d9700 1 -- 172.21.15.31:0/1539521481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 msgr2=0x7fa8d01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.717+0000 7fa8d54d9700 1 --2- 172.21.15.31:0/1539521481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 0x7fa8d01020d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0004940 tx=0x7fa8c0009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.717+0000 7fa8d54d9700 1 -- 172.21.15.31:0/1539521481 shutdown_connections 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.717+0000 7fa8d54d9700 1 --2- 172.21.15.31:0/1539521481 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8d01026a0 0x7fa8d00fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.717+0000 7fa8d54d9700 1 --2- 172.21.15.31:0/1539521481 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 0x7fa8d01020d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.717+0000 7fa8d54d9700 1 -- 172.21.15.31:0/1539521481 >> 172.21.15.31:0/1539521481 conn(0x7fa8d00f7870 msgr2=0x7fa8d00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.717+0000 7fa8d54d9700 1 -- 172.21.15.31:0/1539521481 shutdown_connections 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.717+0000 7fa8d54d9700 1 -- 172.21.15.31:0/1539521481 wait complete. 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8d54d9700 1 Processor -- start 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8d54d9700 1 -- start start 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8d54d9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 0x7fa8d01139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8d54d9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8d01026a0 0x7fa8d0113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8d54d9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0114530 con 0x7fa8d0101d00 2024-04-24T23:17:58.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8d54d9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d010db30 con 0x7fa8d01026a0 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8ceffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 0x7fa8d01139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8ceffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 0x7fa8d01139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42836/0 (socket says 172.21.15.31:42836) 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8ceffd700 1 -- 172.21.15.31:0/2884285065 learned_addr learned my addr 172.21.15.31:0/2884285065 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8ce7fc700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8d01026a0 0x7fa8d0113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8ceffd700 1 -- 172.21.15.31:0/2884285065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8d01026a0 msgr2=0x7fa8d0113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8ceffd700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8d01026a0 0x7fa8d0113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8ceffd700 1 -- 172.21.15.31:0/2884285065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0005040 con 0x7fa8d0101d00 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.718+0000 7fa8ceffd700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 0x7fa8d01139a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000b3b0 tx=0x7fa8c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.719+0000 7fa8b77fe700 1 -- 172.21.15.31:0/2884285065 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8c000e040 con 0x7fa8d0101d00 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.719+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d010ddb0 con 0x7fa8d0101d00 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.719+0000 7fa8b77fe700 1 -- 172.21.15.31:0/2884285065 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8c0013420 con 0x7fa8d0101d00 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.719+0000 7fa8b77fe700 1 -- 172.21.15.31:0/2884285065 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8c001c630 con 0x7fa8d0101d00 2024-04-24T23:17:58.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.719+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d010e370 con 0x7fa8d0101d00 2024-04-24T23:17:58.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.720+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d00fd450 con 0x7fa8d0101d00 2024-04-24T23:17:58.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.720+0000 7fa8b77fe700 1 -- 172.21.15.31:0/2884285065 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fa8c0013590 con 0x7fa8d0101d00 2024-04-24T23:17:58.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.721+0000 7fa8b77fe700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa8b806da40 0x7fa8b806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:58.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.721+0000 7fa8b77fe700 1 -- 172.21.15.31:0/2884285065 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7fa8c008e350 con 0x7fa8d0101d00 2024-04-24T23:17:58.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.721+0000 7fa8ce7fc700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa8b806da40 0x7fa8b806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:58.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.721+0000 7fa8ce7fc700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa8b806da40 0x7fa8b806fef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa8d010ecf0 tx=0x7fa8c400b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:58.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.725+0000 7fa8b77fe700 1 -- 172.21.15.31:0/2884285065 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa8c005bab0 con 0x7fa8d0101d00 2024-04-24T23:17:58.966 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:58 smithi031 ceph-mon[26411]: pgmap v91: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:58.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.970+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8d0108e70 con 0x7fa8b806da40 2024-04-24T23:17:58.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.971+0000 7fa8b77fe700 1 -- 172.21.15.31:0/2884285065 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fa8d0108e70 con 0x7fa8b806da40 2024-04-24T23:17:58.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa8b806da40 msgr2=0x7fa8b806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:58.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa8b806da40 0x7fa8b806fef0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa8d010ecf0 tx=0x7fa8c400b040 comp rx=0 tx=0).stop 2024-04-24T23:17:58.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 msgr2=0x7fa8d01139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 0x7fa8d01139a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000b3b0 tx=0x7fa8c0009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 shutdown_connections 2024-04-24T23:17:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7fa8b806da40 0x7fa8b806fef0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8d0101d00 0x7fa8d01139a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 --2- 172.21.15.31:0/2884285065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8d01026a0 0x7fa8d0113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.973+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 >> 172.21.15.31:0/2884285065 conn(0x7fa8d00f7870 msgr2=0x7fa8d0066280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.974+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 shutdown_connections 2024-04-24T23:17:58.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:58.974+0000 7fa8d54d9700 1 -- 172.21.15.31:0/2884285065 wait complete. 2024-04-24T23:17:59.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 -- 172.21.15.31:0/2261636052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 msgr2=0x7f07d00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 --2- 172.21.15.31:0/2261636052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 0x7f07d00fcdf0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f07c0004940 tx=0x7f07c0009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:59.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 -- 172.21.15.31:0/2261636052 shutdown_connections 2024-04-24T23:17:59.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 --2- 172.21.15.31:0/2261636052 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 0x7f07d00fcdf0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 --2- 172.21.15.31:0/2261636052 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07d0102de0 0x7f07d01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 -- 172.21.15.31:0/2261636052 >> 172.21.15.31:0/2261636052 conn(0x7f07d00f8990 msgr2=0x7f07d00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:59.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 -- 172.21.15.31:0/2261636052 shutdown_connections 2024-04-24T23:17:59.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 -- 172.21.15.31:0/2261636052 wait complete. 2024-04-24T23:17:59.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.078+0000 7f07d69c0700 1 Processor -- start 2024-04-24T23:17:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07d69c0700 1 -- start start 2024-04-24T23:17:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07d69c0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07d0102de0 0x7f07d0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07d69c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 0x7f07d00707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07d69c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d00721f0 con 0x7f07d0103780 2024-04-24T23:17:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07d69c0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0070ce0 con 0x7f07d0102de0 2024-04-24T23:17:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 0x7f07d00707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07cf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 0x7f07d00707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42868/0 (socket says 172.21.15.31:42868) 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07cf7fe700 1 -- 172.21.15.31:0/1057736534 learned_addr learned my addr 172.21.15.31:0/1057736534 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07cffff700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07d0102de0 0x7f07d0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07cf7fe700 1 -- 172.21.15.31:0/1057736534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07d0102de0 msgr2=0x7f07d0070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07cf7fe700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07d0102de0 0x7f07d0070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.079+0000 7f07cf7fe700 1 -- 172.21.15.31:0/1057736534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c0005040 con 0x7f07d0103780 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.080+0000 7f07cf7fe700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 0x7f07d00707a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009180 tx=0x7f07c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.080+0000 7f07ccff9700 1 -- 172.21.15.31:0/1057736534 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07c000e040 con 0x7f07d0103780 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.080+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d0070f60 con 0x7f07d0103780 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.080+0000 7f07ccff9700 1 -- 172.21.15.31:0/1057736534 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07c0013420 con 0x7f07d0103780 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.080+0000 7f07ccff9700 1 -- 172.21.15.31:0/1057736534 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07c001c610 con 0x7f07d0103780 2024-04-24T23:17:59.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.080+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d011a170 con 0x7f07d0103780 2024-04-24T23:17:59.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.081+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d01000f0 con 0x7f07d0103780 2024-04-24T23:17:59.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.082+0000 7f07ccff9700 1 -- 172.21.15.31:0/1057736534 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f07c0013a00 con 0x7f07d0103780 2024-04-24T23:17:59.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.082+0000 7f07ccff9700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f07bc06da40 0x7f07bc06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.082+0000 7f07ccff9700 1 -- 172.21.15.31:0/1057736534 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f07c008e2f0 con 0x7f07d0103780 2024-04-24T23:17:59.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.082+0000 7f07cffff700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f07bc06da40 0x7f07bc06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.083+0000 7f07cffff700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f07bc06da40 0x7f07bc06fef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009bf0 tx=0x7f07b80091a0 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:59.088 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.087+0000 7f07ccff9700 1 -- 172.21.15.31:0/1057736534 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f07c0057f90 con 0x7f07d0103780 2024-04-24T23:17:59.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:58 smithi040 ceph-mon[33128]: pgmap v91: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:17:59.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.306+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f07d0109f30 con 0x7f07bc06da40 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.317+0000 7f07ccff9700 1 -- 172.21.15.31:0/1057736534 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f07d0109f30 con 0x7f07bc06da40 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (68s) 46s ago 12m 16.4M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (12m) 46s ago 12m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (11m) 102s ago 11m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (52s) 46s ago 12m 52.7M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (7m) 46s ago 7m 139M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (7m) 46s ago 7m 13.0M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:17:59.319 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (7m) 102s ago 7m 13.4M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (7m) 102s ago 7m 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (3m) 46s ago 14m 511M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (2m) 102s ago 11m 448M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (14m) 46s ago 14m 55.8M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 8572f02a7ab3 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (11m) 102s ago 11m 46.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af bf68fa6cba29 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (110s) 46s ago 12m 16.8M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (106s) 102s ago 11m 3456k - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (10m) 46s ago 10m 221M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (10m) 46s ago 10m 238M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (10m) 46s ago 10m 198M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (9m) 102s ago 9m 276M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (9m) 102s ago 9m 245M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (8m) 102s ago 8m 223M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:17:59.320 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (87s) 46s ago 12m 48.6M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:17:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f07bc06da40 msgr2=0x7f07bc06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f07bc06da40 0x7f07bc06fef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009bf0 tx=0x7f07b80091a0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 msgr2=0x7f07d00707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 0x7f07d00707a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009180 tx=0x7f07c0009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 shutdown_connections 2024-04-24T23:17:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f07bc06da40 0x7f07bc06fef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07d0102de0 0x7f07d0070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 --2- 172.21.15.31:0/1057736534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07d0103780 0x7f07d00707a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.320+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 >> 172.21.15.31:0/1057736534 conn(0x7f07d00f8990 msgr2=0x7f07d0107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.321+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 shutdown_connections 2024-04-24T23:17:59.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.321+0000 7f07d69c0700 1 -- 172.21.15.31:0/1057736534 wait complete. 2024-04-24T23:17:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.427+0000 7ff89d9d1700 1 -- 172.21.15.31:0/3908368698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898103780 msgr2=0x7ff8980fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.427+0000 7ff89d9d1700 1 --2- 172.21.15.31:0/3908368698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898103780 0x7ff8980fcdf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff88c004940 tx=0x7ff88c009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.427+0000 7ff89d9d1700 1 -- 172.21.15.31:0/3908368698 shutdown_connections 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.427+0000 7ff89d9d1700 1 --2- 172.21.15.31:0/3908368698 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898103780 0x7ff8980fcdf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.427+0000 7ff89d9d1700 1 --2- 172.21.15.31:0/3908368698 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff898102de0 0x7ff8981031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.427+0000 7ff89d9d1700 1 -- 172.21.15.31:0/3908368698 >> 172.21.15.31:0/3908368698 conn(0x7ff8980f8990 msgr2=0x7ff8980fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.427+0000 7ff89d9d1700 1 -- 172.21.15.31:0/3908368698 shutdown_connections 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.427+0000 7ff89d9d1700 1 -- 172.21.15.31:0/3908368698 wait complete. 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff89d9d1700 1 Processor -- start 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff89d9d1700 1 -- start start 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff89d9d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898102de0 0x7ff898115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff89d9d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff898103780 0x7ff8981160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff89d9d1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff898116730 con 0x7ff898102de0 2024-04-24T23:17:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff89d9d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89810fd10 con 0x7ff898103780 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff896ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898102de0 0x7ff898115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff896ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898102de0 0x7ff898115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42886/0 (socket says 172.21.15.31:42886) 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff896ffd700 1 -- 172.21.15.31:0/2667193280 learned_addr learned my addr 172.21.15.31:0/2667193280 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff8967fc700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff898103780 0x7ff8981160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff896ffd700 1 -- 172.21.15.31:0/2667193280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff898103780 msgr2=0x7ff8981160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff896ffd700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff898103780 0x7ff8981160e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.428+0000 7ff896ffd700 1 -- 172.21.15.31:0/2667193280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c005040 con 0x7ff898102de0 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.429+0000 7ff896ffd700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898102de0 0x7ff898115ba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff888002a50 tx=0x7ff8880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.429+0000 7ff87f7fe700 1 -- 172.21.15.31:0/2667193280 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff888007cc0 con 0x7ff898102de0 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.429+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89810fff0 con 0x7ff898102de0 2024-04-24T23:17:59.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.429+0000 7ff87f7fe700 1 -- 172.21.15.31:0/2667193280 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff88800f040 con 0x7ff898102de0 2024-04-24T23:17:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.429+0000 7ff87f7fe700 1 -- 172.21.15.31:0/2667193280 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff88800da30 con 0x7ff898102de0 2024-04-24T23:17:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.429+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff898110580 con 0x7ff898102de0 2024-04-24T23:17:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.430+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8980fe530 con 0x7ff898102de0 2024-04-24T23:17:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.430+0000 7ff87f7fe700 1 -- 172.21.15.31:0/2667193280 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7ff88800db90 con 0x7ff898102de0 2024-04-24T23:17:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.431+0000 7ff87f7fe700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff88006da40 0x7ff88006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.431+0000 7ff87f7fe700 1 -- 172.21.15.31:0/2667193280 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff88808cf60 con 0x7ff898102de0 2024-04-24T23:17:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.431+0000 7ff8967fc700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff88006da40 0x7ff88006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.431+0000 7ff8967fc700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff88006da40 0x7ff88006fef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff88c004890 tx=0x7ff88c009e30 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:59.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.436+0000 7ff87f7fe700 1 -- 172.21.15.31:0/2667193280 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff8880c1100 con 0x7ff898102de0 2024-04-24T23:17:59.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.732+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff8981111f0 con 0x7ff898102de0 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.733+0000 7ff87f7fe700 1 -- 172.21.15.31:0/2667193280 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff88805a3d0 con 0x7ff898102de0 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:17:59.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:17:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:17:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:17:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:17:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:17:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-24T23:17:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:17:59.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:17:59.736 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:17:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff88006da40 msgr2=0x7ff88006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff88006da40 0x7ff88006fef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff88c004890 tx=0x7ff88c009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898102de0 msgr2=0x7ff898115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898102de0 0x7ff898115ba0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff888002a50 tx=0x7ff8880046b0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 shutdown_connections 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff88006da40 0x7ff88006fef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff898102de0 0x7ff898115ba0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 --2- 172.21.15.31:0/2667193280 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff898103780 0x7ff8981160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 >> 172.21.15.31:0/2667193280 conn(0x7ff8980f8990 msgr2=0x7ff898070490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 shutdown_connections 2024-04-24T23:17:59.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.735+0000 7ff89d9d1700 1 -- 172.21.15.31:0/2667193280 wait complete. 2024-04-24T23:17:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.842+0000 7f36001d0700 1 -- 172.21.15.31:0/1453531434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 msgr2=0x7f35f81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.842+0000 7f36001d0700 1 --2- 172.21.15.31:0/1453531434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 0x7f35f81031c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f35f4004940 tx=0x7f35f4009e30 comp rx=0 tx=0).stop 2024-04-24T23:17:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.843+0000 7f36001d0700 1 -- 172.21.15.31:0/1453531434 shutdown_connections 2024-04-24T23:17:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.843+0000 7f36001d0700 1 --2- 172.21.15.31:0/1453531434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f8103790 0x7f35f80fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.843+0000 7f36001d0700 1 --2- 172.21.15.31:0/1453531434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 0x7f35f81031c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.843+0000 7f36001d0700 1 -- 172.21.15.31:0/1453531434 >> 172.21.15.31:0/1453531434 conn(0x7f35f80f8980 msgr2=0x7f35f80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:17:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.843+0000 7f36001d0700 1 -- 172.21.15.31:0/1453531434 shutdown_connections 2024-04-24T23:17:59.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.843+0000 7f36001d0700 1 -- 172.21.15.31:0/1453531434 wait complete. 2024-04-24T23:17:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f36001d0700 1 Processor -- start 2024-04-24T23:17:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f36001d0700 1 -- start start 2024-04-24T23:17:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f36001d0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 0x7f35f8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f36001d0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f8103790 0x7f35f80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f36001d0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8070d30 con 0x7f35f8102df0 2024-04-24T23:17:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f36001d0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8070e70 con 0x7f35f8103790 2024-04-24T23:17:59.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f35fdf6c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 0x7f35f8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f35fdf6c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 0x7f35f8070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42904/0 (socket says 172.21.15.31:42904) 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f35fdf6c700 1 -- 172.21.15.31:0/702415762 learned_addr learned my addr 172.21.15.31:0/702415762 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.844+0000 7f35fd76b700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f8103790 0x7f35f80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f35fdf6c700 1 -- 172.21.15.31:0/702415762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f8103790 msgr2=0x7f35f80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f35fdf6c700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f8103790 0x7f35f80707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f35fdf6c700 1 -- 172.21.15.31:0/702415762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4005040 con 0x7f35f8102df0 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f35fdf6c700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 0x7f35f8070260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f35f400b8b0 tx=0x7f35f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f35ee7fc700 1 -- 172.21.15.31:0/702415762 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f400e040 con 0x7f35f8102df0 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f35ee7fc700 1 -- 172.21.15.31:0/702415762 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35f4004290 con 0x7f35f8102df0 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f80710c0 con 0x7f35f8102df0 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f35ee7fc700 1 -- 172.21.15.31:0/702415762 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f401c3f0 con 0x7f35f8102df0 2024-04-24T23:17:59.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.845+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f811a310 con 0x7f35f8102df0 2024-04-24T23:17:59.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.846+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f8100100 con 0x7f35f8102df0 2024-04-24T23:17:59.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.847+0000 7f35ee7fc700 1 -- 172.21.15.31:0/702415762 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f35f401c660 con 0x7f35f8102df0 2024-04-24T23:17:59.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.847+0000 7f35ee7fc700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f35e406da40 0x7f35e406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:17:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.847+0000 7f35ee7fc700 1 -- 172.21.15.31:0/702415762 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f35f408e340 con 0x7f35f8102df0 2024-04-24T23:17:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.847+0000 7f35fd76b700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f35e406da40 0x7f35e406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:17:59.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.848+0000 7f35fd76b700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f35e406da40 0x7f35e406fef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35f8071ba0 tx=0x7f35e800b040 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:17:59.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:17:59.853+0000 7f35ee7fc700 1 -- 172.21.15.31:0/702415762 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f35f4057f90 con 0x7f35f8102df0 2024-04-24T23:17:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:59 smithi031 ceph-mon[26411]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:59 smithi031 ceph-mon[26411]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:17:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:59 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:17:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:59 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:17:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:59 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:17:59.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:59 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:00.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:59 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T23:18:00.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:17:59 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.114+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f35f8109f50 con 0x7f35f8102df0 2024-04-24T23:18:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.114+0000 7f35ee7fc700 1 -- 172.21.15.31:0/702415762 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f35f405b650 con 0x7f35f8102df0 2024-04-24T23:18:00.116 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:18:00.116 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:18:00.116 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:18:00.116 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:18:00.116 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:18:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:00.118 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:18:00.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:18:00.119 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:18:00.119 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:18:00.119 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:00.119 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:00.119 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f35e406da40 msgr2=0x7f35e406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f35e406da40 0x7f35e406fef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35f8071ba0 tx=0x7f35e800b040 comp rx=0 tx=0).stop 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 msgr2=0x7f35f8070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 0x7f35f8070260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f35f400b8b0 tx=0x7f35f4009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 shutdown_connections 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f35e406da40 0x7f35e406fef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f35f8102df0 0x7f35f8070260 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 --2- 172.21.15.31:0/702415762 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f35f8103790 0x7f35f80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 >> 172.21.15.31:0/702415762 conn(0x7f35f80f8980 msgr2=0x7f35f81073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 shutdown_connections 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.116+0000 7f36001d0700 1 -- 172.21.15.31:0/702415762 wait complete. 2024-04-24T23:18:00.120 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:18:00.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:59 smithi040 ceph-mon[33128]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:00.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:59 smithi040 ceph-mon[33128]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:00.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:59 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:00.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:59 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:00.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:59 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:18:00.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:59 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:00.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:59 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T23:18:00.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:17:59 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.227+0000 7f16333bf700 1 -- 172.21.15.31:0/3138097053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1026b0 msgr2=0x7f162c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.227+0000 7f16333bf700 1 --2- 172.21.15.31:0/3138097053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1026b0 0x7f162c0fbd20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f161c004940 tx=0x7f161c009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 -- 172.21.15.31:0/3138097053 shutdown_connections 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 --2- 172.21.15.31:0/3138097053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c1026b0 0x7f162c0fbd20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 --2- 172.21.15.31:0/3138097053 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f162c101d10 0x7f162c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 -- 172.21.15.31:0/3138097053 >> 172.21.15.31:0/3138097053 conn(0x7f162c0f7860 msgr2=0x7f162c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 -- 172.21.15.31:0/3138097053 shutdown_connections 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 -- 172.21.15.31:0/3138097053 wait complete. 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 Processor -- start 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 -- start start 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c101d10 0x7f162c10eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:00.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f162c1026b0 0x7f162c10f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c10f5b0 con 0x7f162c101d10 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.228+0000 7f16333bf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c10f6f0 con 0x7f162c1026b0 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f163115b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c101d10 0x7f162c10eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f163115b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c101d10 0x7f162c10eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42912/0 (socket says 172.21.15.31:42912) 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f163115b700 1 -- 172.21.15.31:0/1121226997 learned_addr learned my addr 172.21.15.31:0/1121226997 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f163115b700 1 -- 172.21.15.31:0/1121226997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f162c1026b0 msgr2=0x7f162c10f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f163115b700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f162c1026b0 0x7f162c10f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f163115b700 1 -- 172.21.15.31:0/1121226997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c005040 con 0x7f162c101d10 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f163115b700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c101d10 0x7f162c10eb30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f162800b790 tx=0x7f1628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f1621ffb700 1 -- 172.21.15.31:0/1121226997 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162800f770 con 0x7f162c101d10 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f1621ffb700 1 -- 172.21.15.31:0/1121226997 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f162800fda0 con 0x7f162c101d10 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c10f980 con 0x7f162c101d10 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.229+0000 7f1621ffb700 1 -- 172.21.15.31:0/1121226997 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162800d310 con 0x7f162c101d10 2024-04-24T23:18:00.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.230+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c117fa0 con 0x7f162c101d10 2024-04-24T23:18:00.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.231+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c0fd460 con 0x7f162c101d10 2024-04-24T23:18:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.231+0000 7f1621ffb700 1 -- 172.21.15.31:0/1121226997 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f1628010420 con 0x7f162c101d10 2024-04-24T23:18:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.231+0000 7f1621ffb700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f161806da40 0x7f161806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.231+0000 7f1621ffb700 1 -- 172.21.15.31:0/1121226997 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7f162808d0d0 con 0x7f162c101d10 2024-04-24T23:18:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.231+0000 7f163095a700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f161806da40 0x7f161806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:00.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.232+0000 7f163095a700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f161806da40 0x7f161806fef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f161c004f90 tx=0x7f161c009e30 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:00.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.236+0000 7f1621ffb700 1 -- 172.21.15.31:0/1121226997 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f1628056dc0 con 0x7f162c101d10 2024-04-24T23:18:00.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.469+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 --> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f162c0f8e80 con 0x7f161806da40 2024-04-24T23:18:00.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.470+0000 7f1621ffb700 1 -- 172.21.15.31:0/1121226997 <== mgr.14698 v2:172.21.15.31:6800/1306034850 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f162c0f8e80 con 0x7f161806da40 2024-04-24T23:18:00.472 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:18:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:18:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:18:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:18:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:18:00.472 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-24T23:18:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:18:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/21 daemons upgraded", 2024-04-24T23:18:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T23:18:00.473 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:18:00.473 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:18:00.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f161806da40 msgr2=0x7f161806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f161806da40 0x7f161806fef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f161c004f90 tx=0x7f161c009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:00.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c101d10 msgr2=0x7f162c10eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c101d10 0x7f162c10eb30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f162800b790 tx=0x7f1628009250 comp rx=0 tx=0).stop 2024-04-24T23:18:00.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 shutdown_connections 2024-04-24T23:18:00.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7f161806da40 0x7f161806fef0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f162c101d10 0x7f162c10eb30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 --2- 172.21.15.31:0/1121226997 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f162c1026b0 0x7f162c10f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 >> 172.21.15.31:0/1121226997 conn(0x7f162c0f7860 msgr2=0x7f162c0f8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:00.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 shutdown_connections 2024-04-24T23:18:00.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.473+0000 7f16333bf700 1 -- 172.21.15.31:0/1121226997 wait complete. 2024-04-24T23:18:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.580+0000 7ff7f05d3700 1 -- 172.21.15.31:0/2232442603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 msgr2=0x7ff7e80fba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.580+0000 7ff7f05d3700 1 --2- 172.21.15.31:0/2232442603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 0x7ff7e80fba10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8004a50 tx=0x7ff7d800af40 comp rx=0 tx=0).stop 2024-04-24T23:18:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.580+0000 7ff7f05d3700 1 -- 172.21.15.31:0/2232442603 shutdown_connections 2024-04-24T23:18:00.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.580+0000 7ff7f05d3700 1 --2- 172.21.15.31:0/2232442603 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 0x7ff7e80fba10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.580+0000 7ff7f05d3700 1 --2- 172.21.15.31:0/2232442603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e81004d0 0x7ff7e81008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.580+0000 7ff7f05d3700 1 -- 172.21.15.31:0/2232442603 >> 172.21.15.31:0/2232442603 conn(0x7ff7e80714d0 msgr2=0x7ff7e80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:00.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.580+0000 7ff7f05d3700 1 -- 172.21.15.31:0/2232442603 shutdown_connections 2024-04-24T23:18:00.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.580+0000 7ff7f05d3700 1 -- 172.21.15.31:0/2232442603 wait complete. 2024-04-24T23:18:00.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7f05d3700 1 Processor -- start 2024-04-24T23:18:00.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7f05d3700 1 -- start start 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7f05d3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 0x7ff7e8113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7f05d3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e81004d0 0x7ff7e8113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7f05d3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e81144c0 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7f05d3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e810db10 con 0x7ff7e81004d0 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7ee36f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 0x7ff7e8113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7ee36f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 0x7ff7e8113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42924/0 (socket says 172.21.15.31:42924) 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.581+0000 7ff7ee36f700 1 -- 172.21.15.31:0/69785480 learned_addr learned my addr 172.21.15.31:0/69785480 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7edb6e700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e81004d0 0x7ff7e8113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7ee36f700 1 -- 172.21.15.31:0/69785480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e81004d0 msgr2=0x7ff7e8113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7ee36f700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e81004d0 0x7ff7e8113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7ee36f700 1 -- 172.21.15.31:0/69785480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d8005040 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7ee36f700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 0x7ff7e8113930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c830 tx=0x7ff7e4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7deffd700 1 -- 172.21.15.31:0/69785480 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e40106f0 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7deffd700 1 -- 172.21.15.31:0/69785480 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7e4010d20 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7deffd700 1 -- 172.21.15.31:0/69785480 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e40113a0 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e810ddf0 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.582+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e810e3b0 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.583+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e8103a60 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.584+0000 7ff7deffd700 1 -- 172.21.15.31:0/69785480 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 33) v1 ==== 91925+0+0 (secure 0 0 0) 0x7ff7e4012400 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.584+0000 7ff7deffd700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff7d406da40 0x7ff7d406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.584+0000 7ff7deffd700 1 -- 172.21.15.31:0/69785480 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6066+0+0 (secure 0 0 0) 0x7ff7e408d050 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.584+0000 7ff7edb6e700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff7d406da40 0x7ff7d406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:00.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.585+0000 7ff7edb6e700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff7d406da40 0x7ff7d406fef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff7e810ec80 tx=0x7ff7d800af40 comp rx=0 tx=0).ready entity=mgr.14698 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:00.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.589+0000 7ff7deffd700 1 -- 172.21.15.31:0/69785480 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff7e405a870 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.887+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff7e810ef80 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.888+0000 7ff7deffd700 1 -- 172.21.15.31:0/69785480 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff7e405a400 con 0x7ff7e80fb5a0 2024-04-24T23:18:00.891 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff7d406da40 msgr2=0x7ff7d406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff7d406da40 0x7ff7d406fef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff7e810ec80 tx=0x7ff7d800af40 comp rx=0 tx=0).stop 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 msgr2=0x7ff7e8113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 0x7ff7e8113930 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c830 tx=0x7ff7e4004470 comp rx=0 tx=0).stop 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 shutdown_connections 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:6800/1306034850,v1:172.21.15.31:6801/1306034850] conn(0x7ff7d406da40 0x7ff7d406fef0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7e80fb5a0 0x7ff7e8113930 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 --2- 172.21.15.31:0/69785480 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e81004d0 0x7ff7e8113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 >> 172.21.15.31:0/69785480 conn(0x7ff7e80714d0 msgr2=0x7ff7e80f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 shutdown_connections 2024-04-24T23:18:00.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:00.890+0000 7ff7f05d3700 1 -- 172.21.15.31:0/69785480 wait complete. 2024-04-24T23:18:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:00 smithi031 ceph-mon[26411]: pgmap v92: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:00 smithi031 ceph-mon[26411]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:00 smithi031 ceph-mon[26411]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (squid) 2024-04-24T23:18:01.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:00 smithi031 ceph-mon[26411]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-24T23:18:01.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:00 smithi031 ceph-mon[26411]: Upgrade: Setting container_image for all mgr 2024-04-24T23:18:01.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:00 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/2667193280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:18:01.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:00 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/702415762' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:18:01.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:00 smithi040 ceph-mon[33128]: pgmap v92: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:01.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:00 smithi040 ceph-mon[33128]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:01.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:00 smithi040 ceph-mon[33128]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (squid) 2024-04-24T23:18:01.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:00 smithi040 ceph-mon[33128]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-24T23:18:01.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:00 smithi040 ceph-mon[33128]: Upgrade: Setting container_image for all mgr 2024-04-24T23:18:01.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:00 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/2667193280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:18:01.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:00 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/702415762' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:18:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:01 smithi031 ceph-mon[26411]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:01 smithi031 ceph-mon[26411]: from='client.? 172.21.15.31:0/69785480' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:18:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:18:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:18:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:01 smithi031 ceph-mon[26411]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:02.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:01 smithi040 ceph-mon[33128]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:02.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:01 smithi040 ceph-mon[33128]: from='client.? 172.21.15.31:0/69785480' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:18:02.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:02.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:18:02.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:18:02.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:01 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:02.910 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:02 smithi031 ceph-mon[26411]: pgmap v93: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:02.911 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:02 smithi031 ceph-mon[26411]: Upgrade: Updating mon.smithi031 2024-04-24T23:18:02.911 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:02 smithi031 ceph-mon[26411]: Deploying daemon mon.smithi031 on smithi031 2024-04-24T23:18:03.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:02 smithi040 ceph-mon[33128]: pgmap v93: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:03.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:02 smithi040 ceph-mon[33128]: Upgrade: Updating mon.smithi031 2024-04-24T23:18:03.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:02 smithi040 ceph-mon[33128]: Deploying daemon mon.smithi031 on smithi031 2024-04-24T23:18:03.469 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:03 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:03.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:03 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[26407]: 2024-04-24T23:18:03.454+0000 7f0d83a22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:03.470 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:03 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[26407]: 2024-04-24T23:18:03.454+0000 7f0d83a22700 -1 mon.smithi031@0(leader) e2 *** Got Signal Terminated *** 2024-04-24T23:18:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:03 smithi031 podman[111217]: 2024-04-24 23:18:03.629822466 +0000 UTC m=+0.268348458 container died 8572f02a7ab3ac176a810a34fd394c693afdf17d552f3a87f49b7d919f5e1957 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=quincy-c61472c) 2024-04-24T23:18:04.233 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 podman[111217]: 2024-04-24 23:18:04.030632924 +0000 UTC m=+0.669158905 container cleanup 8572f02a7ab3ac176a810a34fd394c693afdf17d552f3a87f49b7d919f5e1957 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:18:04.233 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 bash[111217]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031 2024-04-24T23:18:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 podman[111243]: 2024-04-24 23:18:04.231967344 +0000 UTC m=+0.596650494 container remove 8572f02a7ab3ac176a810a34fd394c693afdf17d552f3a87f49b7d919f5e1957 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=quincy-c61472c, ceph=True, org.label-schema.build-date=20240424) 2024-04-24T23:18:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031.service: Deactivated successfully. 2024-04-24T23:18:04.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 systemd[1]: Stopped Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:18:04.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031.service: Consumed 14.355s CPU time. 2024-04-24T23:18:04.911 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 systemd[1]: Starting Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:04.911 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 podman[111375]: 2024-04-24 23:18:04.783317088 +0000 UTC m=+0.108439927 container create 78d60c8cc2f3290b86d7685b62db4330ab347b0772a816dbd29d8a60c6167018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-24T23:18:04.911 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:04 smithi031 podman[111375]: 2024-04-24 23:18:04.698526767 +0000 UTC m=+0.023649627 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 podman[111375]: 2024-04-24 23:18:05.053553455 +0000 UTC m=+0.378676305 container init 78d60c8cc2f3290b86d7685b62db4330ab347b0772a816dbd29d8a60c6167018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 podman[111375]: 2024-04-24 23:18:05.058164275 +0000 UTC m=+0.383287123 container start 78d60c8cc2f3290b86d7685b62db4330ab347b0772a816dbd29d8a60c6167018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: pidfile_write: ignore empty --pid-file 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: load: jerasure load: lrc 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Git sha 0 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: DB SUMMARY 2024-04-24T23:18:05.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: DB Session ID: JAMUCSFG8WAJVRIZAJH4 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1159 Bytes 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi031/store.db dir, Total Num: 1, files: 000033.sst 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi031/store.db: 000031.log size: 2121678 ; 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.env: 0x55cc999b5c60 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.info_log: 0x55cc9bcb9960 2024-04-24T23:18:05.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:18:05.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.statistics: (nil) 2024-04-24T23:18:05.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.use_fsync: 0 2024-04-24T23:18:05.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:18:05.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:18:05.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:18:05.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:18:05.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.db_log_dir: 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.wal_dir: 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:18:05.254 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.write_buffer_manager: 0x55cc9bc8c1e0 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.unordered_write: 0 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:18:05.255 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.row_cache: None 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.wal_filter: None 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.wal_compression: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:18:05.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:18:05.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_open_files: -1 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Compression algorithms supported: 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: kZSTD supported: 0 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:18:05.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000015 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.merge_operator: 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_filter: None 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:18:05.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cc9bcb9a60) 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_type: 0 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_index_type: 0 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_shortening: 1 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: checksum: 4 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: no_block_cache: 0 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache: 0x55cc9bcbd090 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_options: 2024-04-24T23:18:05.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: capacity : 536870912 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_shard_bits : 4 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: strict_capacity_limit : 0 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_compressed: (nil) 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: persistent_cache: (nil) 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size: 4096 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size_deviation: 10 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_restart_interval: 16 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_block_restart_interval: 1 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_block_size: 4096 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: partition_filters: 0 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: use_delta_encoding: 1 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: filter_policy: bloomfilter 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: whole_key_filtering: 1 2024-04-24T23:18:05.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: verify_compression: 0 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: format_version: 5 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_index_compression: 1 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_align: 0 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_auto_readahead_size: 262144 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: prepopulate_block_cache: 0 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression: NoCompression 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:18:05.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.num_levels: 7 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:18:05.263 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:18:05.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:18:05.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:18:05.266 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.ttl: 2592000 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:18:05.267 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 12019, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-04-24T23:18:05.268 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b0de33d7-6780-4b15-8538-06e0d36070d3 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000685099003, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000685106147, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1862145, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12020, "largest_seqno": 13662, "table_properties": {"data_size": 1854629, "index_size": 4644, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1861, "raw_key_size": 17807, "raw_average_key_size": 23, "raw_value_size": 1838839, "raw_average_value_size": 2478, "num_data_blocks": 212, "num_entries": 742, "num_filter_entries": 742, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714000685, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b0de33d7-6780-4b15-8538-06e0d36070d3", "db_session_id": "JAMUCSFG8WAJVRIZAJH4", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000685106262, "job": 1, "event": "recovery_finished"} 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi031/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cc9bdae000 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: DB pointer 0x55cc9bd98000 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:18:05.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** DB Stats ** 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:18:05.270 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L0 1/0 1.78 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 509.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L6 1/0 6.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Sum 2/0 8.60 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 509.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 509.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 509.1 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-24T23:18:05.271 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative compaction: 0.00 GB write, 88.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval compaction: 0.00 GB write, 88.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache BinnedLRUCache@0x55cc9bcbd090#2 capacity: 512.00 MB usage: 7.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,5.08 KB,0.000968575%) Misc(1,0.00 KB,0%) 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: starting mon.smithi031 rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi031 fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: mon.smithi031@-1(???) e2 preinit fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: mon.smithi031@-1(???).mds e15 new map 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: mon.smithi031@-1(???).mds e15 print_map 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: e15 2024-04-24T23:18:05.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: legacy client fscid: 1 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Filesystem 'cephfs' (1) 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: fs_name cephfs 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: epoch 15 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: tableserver 0 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: root 0 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: session_timeout 60 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: session_autoclose 300 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_file_size 1099511627776 2024-04-24T23:18:05.273 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_xattr_size 65536 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: required_client_features {} 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: last_failure 0 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: last_failure_osd_epoch 46 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_mds 1 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: in 0 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: up {0=14464} 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: failed 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: damaged 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: stopped 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_pools [3] 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_pool 2 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: inline_data disabled 2024-04-24T23:18:05.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: balancer 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: bal_rank_mask -1 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: standby_count_wanted 1 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: qdb_cluster leader: 0 members: 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Standby daemons: 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:05.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:05.276 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: mon.smithi031@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-24T23:18:05.276 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: mon.smithi031@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T23:18:05.276 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: mon.smithi031@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T23:18:05.276 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: mon.smithi031@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T23:18:05.276 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 ceph-mon[111404]: mon.smithi031@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-04-24T23:18:05.276 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 bash[111375]: 78d60c8cc2f3290b86d7685b62db4330ab347b0772a816dbd29d8a60c6167018 2024-04-24T23:18:05.276 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:05 smithi031 systemd[1]: Started Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:18:06.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: pgmap v95: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: mon.smithi031 calling monitor election 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: mon.smithi031 is new leader, mons smithi031,smithi040 in quorum (ranks 0,1) 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: monmap epoch 2 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: last_changed 2024-04-24T23:06:15.229793+0000 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: created 2024-04-24T23:03:38.159717+0000 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: min_mon_release 17 (quincy) 2024-04-24T23:18:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: election_strategy: 1 2024-04-24T23:18:06.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-04-24T23:18:06.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: 1: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.smithi040 2024-04-24T23:18:06.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:18:06.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T23:18:06.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: mgrmap e33: smithi031.rrnmwr(active, since 2m), standbys: smithi040.qtdwuq 2024-04-24T23:18:06.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: overall HEALTH_OK 2024-04-24T23:18:06.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: from='mgr.14698 ' entity='' 2024-04-24T23:18:06.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:06 smithi040 ceph-mon[33128]: mgrmap e34: smithi031.rrnmwr(active, since 2m), standbys: smithi040.qtdwuq 2024-04-24T23:18:06.916 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:18:06.916 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: from='mgr.14698 172.21.15.31:0/203285039' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:18:06.916 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: pgmap v95: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:06.916 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: mon.smithi031 calling monitor election 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: mon.smithi031 is new leader, mons smithi031,smithi040 in quorum (ranks 0,1) 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: monmap epoch 2 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: last_changed 2024-04-24T23:06:15.229793+0000 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: created 2024-04-24T23:03:38.159717+0000 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: min_mon_release 17 (quincy) 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: election_strategy: 1 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: 1: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.smithi040 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: osdmap e48: 6 total, 6 up, 6 in 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: mgrmap e33: smithi031.rrnmwr(active, since 2m), standbys: smithi040.qtdwuq 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: overall HEALTH_OK 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: from='mgr.14698 ' entity='' 2024-04-24T23:18:06.917 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:06 smithi031 ceph-mon[111404]: mgrmap e34: smithi031.rrnmwr(active, since 2m), standbys: smithi040.qtdwuq 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: Active manager daemon smithi031.rrnmwr restarted 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: mgrmap e35: smithi031.rrnmwr(active, starting, since 0.0058399s), standbys: smithi040.qtdwuq 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:18:14.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: Standby manager daemon smithi040.qtdwuq restarted 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi040.qtdwuq", "id": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.? 172.21.15.40:0/3199415726' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.? 172.21.15.40:0/3199415726' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:18:14.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:18:14.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:18:14.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.? 172.21.15.40:0/3199415726' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:18:14.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.? 172.21.15.40:0/3199415726' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:18:14.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:18:14.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: Active manager daemon smithi031.rrnmwr restarted 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: Activating manager daemon smithi031.rrnmwr 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: mgrmap e35: smithi031.rrnmwr(active, starting, since 0.0058399s), standbys: smithi040.qtdwuq 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:18:14.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi031.rrnmwr", "id": "smithi031.rrnmwr"}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: Standby manager daemon smithi040.qtdwuq restarted 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: Standby manager daemon smithi040.qtdwuq started 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr metadata", "who": "smithi040.qtdwuq", "id": "smithi040.qtdwuq"}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3199415726' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/crt"}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3199415726' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-24T23:18:14.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3199415726' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi040.qtdwuq/key"}]: dispatch 2024-04-24T23:18:14.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.? 172.21.15.40:0/3199415726' entity='mgr.smithi040.qtdwuq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-24T23:18:14.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: Manager daemon smithi031.rrnmwr is now available 2024-04-24T23:18:14.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:13 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:14.992 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:14 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:14.992 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:14 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:18:14.992 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:14 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:18:14.992 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:14 smithi040 ceph-mon[33128]: [24/Apr/2024:23:18:14] ENGINE Bus STARTING 2024-04-24T23:18:14.992 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:14 smithi040 ceph-mon[33128]: mgrmap e36: smithi031.rrnmwr(active, since 1.01078s), standbys: smithi040.qtdwuq 2024-04-24T23:18:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/mirror_snapshot_schedule"}]: dispatch 2024-04-24T23:18:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.rrnmwr/trash_purge_schedule"}]: dispatch 2024-04-24T23:18:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:14 smithi031 ceph-mon[111404]: [24/Apr/2024:23:18:14] ENGINE Bus STARTING 2024-04-24T23:18:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:14 smithi031 ceph-mon[111404]: mgrmap e36: smithi031.rrnmwr(active, since 1.01078s), standbys: smithi040.qtdwuq 2024-04-24T23:18:15.849 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:15 smithi040 ceph-mon[33128]: pgmap v3: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:15.849 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:15 smithi040 ceph-mon[33128]: [24/Apr/2024:23:18:14] ENGINE Serving on http://172.21.15.31:8765 2024-04-24T23:18:15.850 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:15 smithi040 ceph-mon[33128]: [24/Apr/2024:23:18:14] ENGINE Serving on https://172.21.15.31:7150 2024-04-24T23:18:15.850 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:15 smithi040 ceph-mon[33128]: [24/Apr/2024:23:18:14] ENGINE Bus STARTED 2024-04-24T23:18:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:15 smithi031 ceph-mon[111404]: pgmap v3: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:15 smithi031 ceph-mon[111404]: [24/Apr/2024:23:18:14] ENGINE Serving on http://172.21.15.31:8765 2024-04-24T23:18:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:15 smithi031 ceph-mon[111404]: [24/Apr/2024:23:18:14] ENGINE Serving on https://172.21.15.31:7150 2024-04-24T23:18:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:15 smithi031 ceph-mon[111404]: [24/Apr/2024:23:18:14] ENGINE Bus STARTED 2024-04-24T23:18:16.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:16 smithi031 ceph-mon[111404]: pgmap v4: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:16.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:16 smithi031 ceph-mon[111404]: mgrmap e37: smithi031.rrnmwr(active, since 2s), standbys: smithi040.qtdwuq 2024-04-24T23:18:17.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:16 smithi040 ceph-mon[33128]: pgmap v4: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:17.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:16 smithi040 ceph-mon[33128]: mgrmap e37: smithi031.rrnmwr(active, since 2s), standbys: smithi040.qtdwuq 2024-04-24T23:18:18.215 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:17 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:18.215 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:17 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:18.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:17 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:18.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:17 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:19.016 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:18 smithi040 ceph-mon[33128]: pgmap v5: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:19.016 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:18 smithi040 ceph-mon[33128]: mgrmap e38: smithi031.rrnmwr(active, since 4s), standbys: smithi040.qtdwuq 2024-04-24T23:18:19.017 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:18 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:19.017 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:18 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:18 smithi031 ceph-mon[111404]: pgmap v5: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:18 smithi031 ceph-mon[111404]: mgrmap e38: smithi031.rrnmwr(active, since 4s), standbys: smithi040.qtdwuq 2024-04-24T23:18:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:18 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:18 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:20.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:20 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:20.524 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:20 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:20.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:20 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:20.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:20 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:21 smithi031 ceph-mon[111404]: pgmap v6: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:21 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:21.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:21 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:21 smithi040 ceph-mon[33128]: pgmap v6: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:21 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:21 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:22.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:22 smithi031 ceph-mon[111404]: pgmap v7: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-24T23:18:22.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:22 smithi031 ceph-mon[111404]: Detected new or changed devices on smithi040 2024-04-24T23:18:22.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:22 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:22.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:22 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:23.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:22 smithi040 ceph-mon[33128]: pgmap v7: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-24T23:18:23.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:22 smithi040 ceph-mon[33128]: Detected new or changed devices on smithi040 2024-04-24T23:18:23.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:22 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:23.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:22 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:25.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: pgmap v8: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: Detected new or changed devices on smithi031 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:18:25.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:25 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:18:25.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: pgmap v8: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-24T23:18:25.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: Detected new or changed devices on smithi031 2024-04-24T23:18:25.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:25.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:25.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:25.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:25.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: Updating smithi031:/etc/ceph/ceph.conf 2024-04-24T23:18:25.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: Updating smithi040:/etc/ceph/ceph.conf 2024-04-24T23:18:25.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:18:25.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:25 smithi031 ceph-mon[111404]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-24T23:18:26.256 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:18:26.257 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:26 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T23:18:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: Updating smithi040:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:18:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-04-24T23:18:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: Updating smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:18:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: Updating smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-24T23:18:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:26.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:26.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:18:26.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-24T23:18:27.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:27 smithi040 ceph-mon[33128]: pgmap v9: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-24T23:18:27.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:27 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:27.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:27 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:18:27.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:27 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:18:27.495 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:27 smithi040 ceph-mon[33128]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:27 smithi031 ceph-mon[111404]: pgmap v9: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-24T23:18:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:27 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:27 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:18:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:27 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:18:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:27 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:28 smithi031 ceph-mon[111404]: Upgrade: Updating mon.smithi040 2024-04-24T23:18:28.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:28 smithi031 ceph-mon[111404]: Deploying daemon mon.smithi040 on smithi040 2024-04-24T23:18:28.511 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:28 smithi040 ceph-mon[33128]: Upgrade: Updating mon.smithi040 2024-04-24T23:18:28.511 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:28 smithi040 ceph-mon[33128]: Deploying daemon mon.smithi040 on smithi040 2024-04-24T23:18:29.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:28 smithi040 systemd[1]: Stopping Ceph mon.smithi040 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:29.516 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:29 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040[33124]: 2024-04-24T23:18:29.188+0000 7f95cc6e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi040 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:18:29.516 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:29 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040[33124]: 2024-04-24T23:18:29.188+0000 7f95cc6e8700 -1 mon.smithi040@1(peon) e2 *** Got Signal Terminated *** 2024-04-24T23:18:29.516 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:29 smithi040 podman[82877]: 2024-04-24 23:18:29.281435269 +0000 UTC m=+0.217821448 container died bf68fa6cba2920982fabee711fc9966b26475039df38b819188d015d0210bf48 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c) 2024-04-24T23:18:29.833 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:29 smithi040 podman[82877]: 2024-04-24 23:18:29.614296845 +0000 UTC m=+0.550683066 container cleanup bf68fa6cba2920982fabee711fc9966b26475039df38b819188d015d0210bf48 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-24T23:18:29.833 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:29 smithi040 bash[82877]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040 2024-04-24T23:18:30.091 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:29 smithi040 podman[82892]: 2024-04-24 23:18:29.832273631 +0000 UTC m=+0.550079161 container remove bf68fa6cba2920982fabee711fc9966b26475039df38b819188d015d0210bf48 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=quincy-c61472c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-24T23:18:30.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi040.service: Deactivated successfully. 2024-04-24T23:18:30.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 systemd[1]: Stopped Ceph mon.smithi040 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:18:30.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi040.service: Consumed 7.558s CPU time. 2024-04-24T23:18:30.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 systemd[1]: Starting Ceph mon.smithi040 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:18:30.772 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 podman[83028]: 2024-04-24 23:18:30.414144427 +0000 UTC m=+0.018532844 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:18:30.772 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 podman[83028]: 2024-04-24 23:18:30.541712695 +0000 UTC m=+0.146101127 container create 62515ef0bf5349fd1f895bb8ef3e442b05242ccc9057114432e7d03df3e66080 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T23:18:30.772 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 podman[83028]: 2024-04-24 23:18:30.770228481 +0000 UTC m=+0.374616876 container init 62515ef0bf5349fd1f895bb8ef3e442b05242ccc9057114432e7d03df3e66080 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 podman[83028]: 2024-04-24 23:18:30.774045803 +0000 UTC m=+0.378434193 container start 62515ef0bf5349fd1f895bb8ef3e442b05242ccc9057114432e7d03df3e66080 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: set uid:gid to 167:167 (ceph:ceph) 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: pidfile_write: ignore empty --pid-file 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: load: jerasure load: lrc 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: RocksDB version: 7.9.2 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Git sha 0 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: DB SUMMARY 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: DB Session ID: BTYTMSKCEE28YX2CTPKJ 2024-04-24T23:18:30.809 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: CURRENT file: CURRENT 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: IDENTITY file: IDENTITY 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi040/store.db dir, Total Num: 1, files: 000027.sst 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi040/store.db: 000025.log size: 5833488 ; 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.error_if_exists: 0 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.create_if_missing: 0 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.paranoid_checks: 1 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.env: 0x556faed0cc60 2024-04-24T23:18:30.810 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.fs: PosixFileSystem 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.info_log: 0x556fb02a5960 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_file_opening_threads: 16 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.statistics: (nil) 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.use_fsync: 0 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_log_file_size: 0 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.keep_log_file_num: 1000 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.recycle_log_file_num: 0 2024-04-24T23:18:30.811 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.allow_fallocate: 1 2024-04-24T23:18:30.812 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.allow_mmap_reads: 0 2024-04-24T23:18:30.812 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.allow_mmap_writes: 0 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.use_direct_reads: 0 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.create_missing_column_families: 0 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.db_log_dir: 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.wal_dir: 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.advise_random_on_open: 1 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.db_write_buffer_size: 0 2024-04-24T23:18:30.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.write_buffer_manager: 0x556fb02781e0 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.rate_limiter: (nil) 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.wal_recovery_mode: 2 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.enable_thread_tracking: 0 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.enable_pipelined_write: 0 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.unordered_write: 0 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-24T23:18:30.814 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.row_cache: None 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.wal_filter: None 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.allow_ingest_behind: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.two_write_queues: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.manual_wal_flush: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.wal_compression: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.atomic_flush: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.log_readahead_size: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.best_efforts_recovery: 0 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-24T23:18:30.815 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.allow_data_in_errors: 0 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.db_host_id: __hostname__ 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_background_jobs: 2 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_background_compactions: -1 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_subcompactions: 1 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_total_wal_size: 0 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-24T23:18:30.816 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_open_files: -1 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bytes_per_sync: 0 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_readahead_size: 0 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_background_flushes: -1 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Compression algorithms supported: 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: kZSTD supported: 0 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: kXpressCompression supported: 0 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: kLZ4HCCompression supported: 1 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: kZlibCompression supported: 1 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: kSnappyCompression supported: 1 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: kLZ4Compression supported: 1 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: kBZip2Compression supported: 0 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-24T23:18:30.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi040/store.db/MANIFEST-000009 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.merge_operator: 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_filter: None 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_filter_factory: None 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.sst_partitioner_factory: None 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556fb02a5a60) 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: cache_index_and_filter_blocks: 1 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: pin_top_level_index_and_filter: 1 2024-04-24T23:18:30.818 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_type: 0 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_block_index_type: 0 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_shortening: 1 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: checksum: 4 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: no_block_cache: 0 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache: 0x556fb02a9090 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_name: BinnedLRUCache 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_options: 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: capacity : 536870912 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: num_shard_bits : 4 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: strict_capacity_limit : 0 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: high_pri_pool_ratio: 0.000 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_cache_compressed: (nil) 2024-04-24T23:18:30.819 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: persistent_cache: (nil) 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_size: 4096 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_size_deviation: 10 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_restart_interval: 16 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: index_block_restart_interval: 1 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: metadata_block_size: 4096 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: partition_filters: 0 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: use_delta_encoding: 1 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: filter_policy: bloomfilter 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: whole_key_filtering: 1 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: verify_compression: 0 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: read_amp_bytes_per_bit: 0 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: format_version: 5 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: enable_index_compression: 1 2024-04-24T23:18:30.820 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: block_align: 0 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: max_auto_readahead_size: 262144 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: prepopulate_block_cache: 0 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: initial_auto_readahead_size: 8192 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: num_file_reads_for_auto_readahead: 2 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.write_buffer_size: 33554432 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_write_buffer_number: 2 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression: NoCompression 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression: Disabled 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.prefix_extractor: nullptr 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.num_levels: 7 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-24T23:18:30.821 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.level: 32767 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.strategy: 0 2024-04-24T23:18:30.822 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.enabled: false 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.target_file_size_base: 67108864 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-24T23:18:30.823 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.arena_block_size: 1048576 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.disable_auto_compactions: 0 2024-04-24T23:18:30.824 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.table_properties_collectors: 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.inplace_update_support: 0 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-24T23:18:30.825 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.bloom_locality: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.max_successive_merges: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.paranoid_file_checks: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.force_consistency_checks: 1 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.report_bg_io_stats: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.ttl: 2592000 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.enable_blob_files: false 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.min_blob_size: 0 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.blob_file_size: 268435456 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-24T23:18:30.826 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.blob_file_starting_level: 0 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi040/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12289, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d42dca37-49fb-4208-82fb-014dcee596d3 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000710804160, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-24T23:18:30.827 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-24T23:18:31.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.014+0000 7fc3f01b2700 1 -- 172.21.15.31:0/2435465821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8103780 msgr2=0x7fc3e80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.014+0000 7fc3f01b2700 1 --2- 172.21.15.31:0/2435465821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8103780 0x7fc3e80fcdf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8004940 tx=0x7fc3d8009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:31.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.014+0000 7fc3f01b2700 1 -- 172.21.15.31:0/2435465821 shutdown_connections 2024-04-24T23:18:31.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.014+0000 7fc3f01b2700 1 --2- 172.21.15.31:0/2435465821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8103780 0x7fc3e80fcdf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.014+0000 7fc3f01b2700 1 --2- 172.21.15.31:0/2435465821 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8102de0 0x7fc3e81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.014+0000 7fc3f01b2700 1 -- 172.21.15.31:0/2435465821 >> 172.21.15.31:0/2435465821 conn(0x7fc3e80f8990 msgr2=0x7fc3e80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.014+0000 7fc3f01b2700 1 -- 172.21.15.31:0/2435465821 shutdown_connections 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.014+0000 7fc3f01b2700 1 -- 172.21.15.31:0/2435465821 wait complete. 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3f01b2700 1 Processor -- start 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3f01b2700 1 -- start start 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3f01b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8102de0 0x7fc3e8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3f01b2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 0x7fc3e80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3f01b2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e80721b0 con 0x7fc3e8102de0 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3f01b2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8070ce0 con 0x7fc3e8103780 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3edf4e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8102de0 0x7fc3e8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3edf4e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8102de0 0x7fc3e8070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38296/0 (socket says 172.21.15.31:38296) 2024-04-24T23:18:31.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3edf4e700 1 -- 172.21.15.31:0/214167483 learned_addr learned my addr 172.21.15.31:0/214167483 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3ed74d700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 0x7fc3e80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3ed74d700 1 -- 172.21.15.31:0/214167483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 msgr2=0x7fc3e80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3ed74d700 1 -- 172.21.15.31:0/214167483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 msgr2=0x7fc3e80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.015+0000 7fc3ed74d700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 0x7fc3e80707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3ed74d700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 0x7fc3e80707a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3edf4e700 1 -- 172.21.15.31:0/214167483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 msgr2=0x7fc3e80707a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3edf4e700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 0x7fc3e80707a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3edf4e700 1 -- 172.21.15.31:0/214167483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8005040 con 0x7fc3e8102de0 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3edf4e700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8102de0 0x7fc3e8070260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40041d0 tx=0x7fc3e40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3de7fc700 1 -- 172.21.15.31:0/214167483 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3e4007cc0 con 0x7fc3e8102de0 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e8070fc0 con 0x7fc3e8102de0 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3de7fc700 1 -- 172.21.15.31:0/214167483 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3e400f040 con 0x7fc3e8102de0 2024-04-24T23:18:31.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3de7fc700 1 -- 172.21.15.31:0/214167483 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3e400dbe0 con 0x7fc3e8102de0 2024-04-24T23:18:31.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.016+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e811a170 con 0x7fc3e8102de0 2024-04-24T23:18:31.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.017+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e81052b0 con 0x7fc3e8102de0 2024-04-24T23:18:31.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.018+0000 7fc3de7fc700 1 -- 172.21.15.31:0/214167483 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fc3e400dd40 con 0x7fc3e8102de0 2024-04-24T23:18:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.018+0000 7fc3de7fc700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc3d406e270 0x7fc3d4070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.018+0000 7fc3de7fc700 1 -- 172.21.15.31:0/214167483 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc3e408d4d0 con 0x7fc3e8102de0 2024-04-24T23:18:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.018+0000 7fc3ed74d700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc3d406e270 0x7fc3d4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.019+0000 7fc3ed74d700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc3d406e270 0x7fc3d4070720 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8071990 tx=0x7fc3d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:31.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.023+0000 7fc3de7fc700 1 -- 172.21.15.31:0/214167483 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc3e4056680 con 0x7fc3e8102de0 2024-04-24T23:18:31.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000710819923, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3518955, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12294, "largest_seqno": 14624, "table_properties": {"data_size": 3509303, "index_size": 6200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2437, "raw_key_size": 22903, "raw_average_key_size": 23, "raw_value_size": 3488787, "raw_average_value_size": 3634, "num_data_blocks": 281, "num_entries": 960, "num_filter_entries": 960, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714000710, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d42dca37-49fb-4208-82fb-014dcee596d3", "db_session_id": "BTYTMSKCEE28YX2CTPKJ", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-24T23:18:31.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714000710820003, "job": 1, "event": "recovery_finished"} 2024-04-24T23:18:31.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-24T23:18:31.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-24T23:18:31.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi040/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-24T23:18:31.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556fb039a000 2024-04-24T23:18:31.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: DB pointer 0x556fb0384000 2024-04-24T23:18:31.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** DB Stats ** 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-24T23:18:31.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: L0 1/0 3.36 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 643.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: L6 1/0 6.83 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Sum 2/0 10.18 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 643.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 643.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** Compaction Stats [default] ** 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 643.9 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-24T23:18:31.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-24T23:18:31.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-24T23:18:31.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-24T23:18:31.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-24T23:18:31.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-24T23:18:31.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Cumulative compaction: 0.00 GB write, 169.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:18:31.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Interval compaction: 0.00 GB write, 169.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-24T23:18:31.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-24T23:18:31.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Block cache BinnedLRUCache@0x556fb02a9090#2 capacity: 512.00 MB usage: 9.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-24T23:18:31.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.53 KB,0.000482798%) IndexBlock(1,6.58 KB,0.00125468%) Misc(1,0.00 KB,0%) 2024-04-24T23:18:31.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-24T23:18:31.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: starting mon.smithi040 rank 1 at public addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] at bind addrs [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi040 fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:18:31.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: mon.smithi040@-1(???) e2 preinit fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:18:31.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: mon.smithi040@-1(???).mds e15 new map 2024-04-24T23:18:31.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: mon.smithi040@-1(???).mds e15 print_map 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: e15 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: legacy client fscid: 1 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Filesystem 'cephfs' (1) 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: fs_name cephfs 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: epoch 15 2024-04-24T23:18:31.168 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:18:31.169 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:18:31.169 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:18:31.169 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: tableserver 0 2024-04-24T23:18:31.169 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: root 0 2024-04-24T23:18:31.169 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: session_timeout 60 2024-04-24T23:18:31.169 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: session_autoclose 300 2024-04-24T23:18:31.170 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: max_file_size 1099511627776 2024-04-24T23:18:31.170 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: max_xattr_size 65536 2024-04-24T23:18:31.170 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: required_client_features {} 2024-04-24T23:18:31.170 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: last_failure 0 2024-04-24T23:18:31.170 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: last_failure_osd_epoch 46 2024-04-24T23:18:31.170 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:18:31.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: max_mds 1 2024-04-24T23:18:31.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: in 0 2024-04-24T23:18:31.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: up {0=14464} 2024-04-24T23:18:31.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: failed 2024-04-24T23:18:31.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: damaged 2024-04-24T23:18:31.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: stopped 2024-04-24T23:18:31.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: data_pools [3] 2024-04-24T23:18:31.171 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: metadata_pool 2 2024-04-24T23:18:31.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: inline_data disabled 2024-04-24T23:18:31.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: balancer 2024-04-24T23:18:31.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: bal_rank_mask -1 2024-04-24T23:18:31.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: standby_count_wanted 1 2024-04-24T23:18:31.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: qdb_cluster leader: 0 members: 2024-04-24T23:18:31.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: [mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:31.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.172 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: Standby daemons: 2024-04-24T23:18:31.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: 2024-04-24T23:18:31.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: [mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:31.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: [mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:31.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout: [mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:31.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: mon.smithi040@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-24T23:18:31.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: mon.smithi040@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T23:18:31.173 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: mon.smithi040@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T23:18:31.174 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: mon.smithi040@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-24T23:18:31.174 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: mon.smithi040@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-24T23:18:31.174 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: pgmap v10: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T23:18:31.174 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:31.174 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 bash[83028]: 62515ef0bf5349fd1f895bb8ef3e442b05242ccc9057114432e7d03df3e66080 2024-04-24T23:18:31.174 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:30 smithi040 systemd[1]: Started Ceph mon.smithi040 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:18:31.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:30 smithi031 ceph-mon[111404]: pgmap v10: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T23:18:31.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:30 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:31.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.252+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3e8109ee0 con 0x7fc3d406e270 2024-04-24T23:18:31.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.253+0000 7fc3de7fc700 1 -- 172.21.15.31:0/214167483 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc3e8109ee0 con 0x7fc3d406e270 2024-04-24T23:18:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc3d406e270 msgr2=0x7fc3d4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc3d406e270 0x7fc3d4070720 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8071990 tx=0x7fc3d8009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8102de0 msgr2=0x7fc3e8070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8102de0 0x7fc3e8070260 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40041d0 tx=0x7fc3e40046b0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 shutdown_connections 2024-04-24T23:18:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3e8102de0 0x7fc3e8070260 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc3d406e270 0x7fc3d4070720 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 --2- 172.21.15.31:0/214167483 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc3e8103780 0x7fc3e80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 >> 172.21.15.31:0/214167483 conn(0x7fc3e80f8990 msgr2=0x7fc3e81021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 shutdown_connections 2024-04-24T23:18:31.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.256+0000 7fc3f01b2700 1 -- 172.21.15.31:0/214167483 wait complete. 2024-04-24T23:18:31.268 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:18:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 -- 172.21.15.31:0/943795790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fbf40 msgr2=0x7f7164101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 --2- 172.21.15.31:0/943795790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fbf40 0x7f7164101a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7154004940 tx=0x7f7154009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 -- 172.21.15.31:0/943795790 shutdown_connections 2024-04-24T23:18:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 --2- 172.21.15.31:0/943795790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fbf40 0x7f7164101a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 --2- 172.21.15.31:0/943795790 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fb5a0 0x7f71640fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 -- 172.21.15.31:0/943795790 >> 172.21.15.31:0/943795790 conn(0x7f7164073350 msgr2=0x7f7164073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 -- 172.21.15.31:0/943795790 shutdown_connections 2024-04-24T23:18:31.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 -- 172.21.15.31:0/943795790 wait complete. 2024-04-24T23:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 Processor -- start 2024-04-24T23:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.362+0000 7f716b3eb700 1 -- start start 2024-04-24T23:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f716b3eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fb5a0 0x7f7164113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f716b3eb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 0x7f7164113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f716b3eb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71641145d0 con 0x7f71640fb5a0 2024-04-24T23:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f716b3eb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716410dbb0 con 0x7f71640fbf40 2024-04-24T23:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7169187700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fb5a0 0x7f7164113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7169187700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fb5a0 0x7f7164113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38310/0 (socket says 172.21.15.31:38310) 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7169187700 1 -- 172.21.15.31:0/3851644477 learned_addr learned my addr 172.21.15.31:0/3851644477 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7168986700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 0x7f7164113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7168986700 1 -- 172.21.15.31:0/3851644477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 msgr2=0x7f7164113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7168986700 1 -- 172.21.15.31:0/3851644477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 msgr2=0x7f7164113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7168986700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 0x7f7164113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7169187700 1 -- 172.21.15.31:0/3851644477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 msgr2=0x7f7164113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7168986700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 0x7f7164113f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7169187700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 0x7f7164113f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7169187700 1 -- 172.21.15.31:0/3851644477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154005040 con 0x7f71640fb5a0 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7169187700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fb5a0 0x7f7164113a40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7160002a50 tx=0x7f71600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7159ffb700 1 -- 172.21.15.31:0/3851644477 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7160007cc0 con 0x7f71640fb5a0 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716410de90 con 0x7f71640fb5a0 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7159ffb700 1 -- 172.21.15.31:0/3851644477 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f716000f040 con 0x7f71640fb5a0 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.363+0000 7f7159ffb700 1 -- 172.21.15.31:0/3851644477 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7160009640 con 0x7f71640fb5a0 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.364+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716410e3d0 con 0x7f71640fb5a0 2024-04-24T23:18:31.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.364+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71640fe670 con 0x7f71640fb5a0 2024-04-24T23:18:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.365+0000 7f7159ffb700 1 -- 172.21.15.31:0/3851644477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f71600097a0 con 0x7f71640fb5a0 2024-04-24T23:18:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.365+0000 7f7159ffb700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f715006e270 0x7f7150070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.365+0000 7f7159ffb700 1 -- 172.21.15.31:0/3851644477 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f716008db80 con 0x7f71640fb5a0 2024-04-24T23:18:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.365+0000 7f7168986700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f715006e270 0x7f7150070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.366+0000 7f7168986700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f715006e270 0x7f7150070720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f716410f2e0 tx=0x7f7154009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:31.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.370+0000 7f7159ffb700 1 -- 172.21.15.31:0/3851644477 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f716005a860 con 0x7f71640fb5a0 2024-04-24T23:18:31.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.598+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7164109f70 con 0x7f715006e270 2024-04-24T23:18:31.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.599+0000 7f7159ffb700 1 -- 172.21.15.31:0/3851644477 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f7164109f70 con 0x7f715006e270 2024-04-24T23:18:31.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f715006e270 msgr2=0x7f7150070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f715006e270 0x7f7150070720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f716410f2e0 tx=0x7f7154009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:31.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fb5a0 msgr2=0x7f7164113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fb5a0 0x7f7164113a40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7160002a50 tx=0x7f71600046b0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 shutdown_connections 2024-04-24T23:18:31.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71640fb5a0 0x7f7164113a40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f715006e270 0x7f7150070720 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 --2- 172.21.15.31:0/3851644477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71640fbf40 0x7f7164113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 >> 172.21.15.31:0/3851644477 conn(0x7f7164073350 msgr2=0x7f7164100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:31.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 shutdown_connections 2024-04-24T23:18:31.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.602+0000 7f716b3eb700 1 -- 172.21.15.31:0/3851644477 wait complete. 2024-04-24T23:18:31.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.706+0000 7f29d2c0e700 1 -- 172.21.15.31:0/480553933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc0fbd40 msgr2=0x7f29cc0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.706+0000 7f29d2c0e700 1 --2- 172.21.15.31:0/480553933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc0fbd40 0x7f29cc0fc1b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f29b8004940 tx=0x7f29b8009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:31.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.706+0000 7f29d2c0e700 1 -- 172.21.15.31:0/480553933 shutdown_connections 2024-04-24T23:18:31.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.706+0000 7f29d2c0e700 1 --2- 172.21.15.31:0/480553933 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc0fbd40 0x7f29cc0fc1b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.706+0000 7f29d2c0e700 1 --2- 172.21.15.31:0/480553933 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc101d90 0x7f29cc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.706+0000 7f29d2c0e700 1 -- 172.21.15.31:0/480553933 >> 172.21.15.31:0/480553933 conn(0x7f29cc071d00 msgr2=0x7f29cc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.706+0000 7f29d2c0e700 1 -- 172.21.15.31:0/480553933 shutdown_connections 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.706+0000 7f29d2c0e700 1 -- 172.21.15.31:0/480553933 wait complete. 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.707+0000 7f29d2c0e700 1 Processor -- start 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.707+0000 7f29d2c0e700 1 -- start start 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.707+0000 7f29d2c0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 0x7f29cc113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.707+0000 7f29d2c0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc101d90 0x7f29cc1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.707+0000 7f29d2c0e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc114710 con 0x7f29cc101d90 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.707+0000 7f29d2c0e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc10db00 con 0x7f29cc0fbd40 2024-04-24T23:18:31.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc101d90 0x7f29cc1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc101d90 0x7f29cc1140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38326/0 (socket says 172.21.15.31:38326) 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29cbfff700 1 -- 172.21.15.31:0/3957648042 learned_addr learned my addr 172.21.15.31:0/3957648042 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29d09aa700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 0x7f29cc113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29d09aa700 1 -- 172.21.15.31:0/3957648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 msgr2=0x7f29cc113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29d09aa700 1 -- 172.21.15.31:0/3957648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 msgr2=0x7f29cc113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29d09aa700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 0x7f29cc113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29d09aa700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 0x7f29cc113b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29cbfff700 1 -- 172.21.15.31:0/3957648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 msgr2=0x7f29cc113b80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:18:31.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29cbfff700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 0x7f29cc113b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29cbfff700 1 -- 172.21.15.31:0/3957648042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8005040 con 0x7f29cc101d90 2024-04-24T23:18:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29cbfff700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc101d90 0x7f29cc1140c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009150 tx=0x7f29b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29c97fa700 1 -- 172.21.15.31:0/3957648042 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29b800e040 con 0x7f29cc101d90 2024-04-24T23:18:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29c97fa700 1 -- 172.21.15.31:0/3957648042 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29b800bbc0 con 0x7f29cc101d90 2024-04-24T23:18:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc10dd80 con 0x7f29cc101d90 2024-04-24T23:18:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29c97fa700 1 -- 172.21.15.31:0/3957648042 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29b801c630 con 0x7f29cc101d90 2024-04-24T23:18:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.708+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc10e340 con 0x7f29cc101d90 2024-04-24T23:18:31.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.709+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc0fd430 con 0x7f29cc101d90 2024-04-24T23:18:31.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.710+0000 7f29c97fa700 1 -- 172.21.15.31:0/3957648042 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f29b800bd30 con 0x7f29cc101d90 2024-04-24T23:18:31.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.710+0000 7f29c97fa700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f29bc06e260 0x7f29bc070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:31.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.710+0000 7f29d09aa700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f29bc06e260 0x7f29bc070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:31.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.710+0000 7f29c97fa700 1 -- 172.21.15.31:0/3957648042 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f29b808ef20 con 0x7f29cc101d90 2024-04-24T23:18:31.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.711+0000 7f29d09aa700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f29bc06e260 0x7f29bc070710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009e50 tx=0x7f29c0009f10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:31.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.715+0000 7f29c97fa700 1 -- 172.21.15.31:0/3957648042 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f29b80c3740 con 0x7f29cc101d90 2024-04-24T23:18:31.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.935+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f29cc108df0 con 0x7f29bc06e260 2024-04-24T23:18:31.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.946+0000 7f29c97fa700 1 -- 172.21.15.31:0/3957648042 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f29cc108df0 con 0x7f29bc06e260 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (101s) 12s ago 13m 24.1M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (13m) 12s ago 13m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (12m) 15s ago 12m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (85s) 12s ago 13m 73.7M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (7m) 12s ago 7m 140M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (7m) 12s ago 7m 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (7m) 15s ago 7m 14.0M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (7m) 15s ago 7m 15.1M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (3m) 12s ago 14m 471M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:18:31.950 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (2m) 15s ago 12m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (27s) 12s ago 14m 27.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 starting - - - 2048M 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (2m) 12s ago 13m 16.7M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (2m) 15s ago 12m 14.9M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (11m) 12s ago 11m 222M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (11m) 12s ago 11m 239M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (10m) 12s ago 10m 198M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (10m) 15s ago 10m 278M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (9m) 15s ago 9m 245M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (9m) 15s ago 9m 225M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:18:31.951 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (119s) 12s ago 12m 56.3M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:18:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f29bc06e260 msgr2=0x7f29bc070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f29bc06e260 0x7f29bc070710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009e50 tx=0x7f29c0009f10 comp rx=0 tx=0).stop 2024-04-24T23:18:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc101d90 msgr2=0x7f29cc1140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc101d90 0x7f29cc1140c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009150 tx=0x7f29b8009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 shutdown_connections 2024-04-24T23:18:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f29bc06e260 0x7f29bc070710 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f29cc0fbd40 0x7f29cc113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 --2- 172.21.15.31:0/3957648042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29cc101d90 0x7f29cc1140c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:31.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.948+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 >> 172.21.15.31:0/3957648042 conn(0x7f29cc071d00 msgr2=0x7f29cc0fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:31.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.949+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 shutdown_connections 2024-04-24T23:18:31.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:31.949+0000 7f29d2c0e700 1 -- 172.21.15.31:0/3957648042 wait complete. 2024-04-24T23:18:32.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.052+0000 7f8b09ded700 1 -- 172.21.15.31:0/2353044146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04102e50 msgr2=0x7f8b04103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.052+0000 7f8b09ded700 1 --2- 172.21.15.31:0/2353044146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04102e50 0x7f8b04103220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8af4004940 tx=0x7f8af4009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.052+0000 7f8b09ded700 1 -- 172.21.15.31:0/2353044146 shutdown_connections 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.052+0000 7f8b09ded700 1 --2- 172.21.15.31:0/2353044146 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04065640 0x7f8b04065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.052+0000 7f8b09ded700 1 --2- 172.21.15.31:0/2353044146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04102e50 0x7f8b04103220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.052+0000 7f8b09ded700 1 -- 172.21.15.31:0/2353044146 >> 172.21.15.31:0/2353044146 conn(0x7f8b04071d00 msgr2=0x7f8b04072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.052+0000 7f8b09ded700 1 -- 172.21.15.31:0/2353044146 shutdown_connections 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.052+0000 7f8b09ded700 1 -- 172.21.15.31:0/2353044146 wait complete. 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b09ded700 1 Processor -- start 2024-04-24T23:18:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b09ded700 1 -- start start 2024-04-24T23:18:32.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b09ded700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04065640 0x7f8b04113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b09ded700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 0x7f8b04114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b09ded700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b041146e0 con 0x7f8b04065640 2024-04-24T23:18:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b09ded700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0410dad0 con 0x7f8b04102e50 2024-04-24T23:18:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04065640 0x7f8b04113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b037fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04065640 0x7f8b04113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38342/0 (socket says 172.21.15.31:38342) 2024-04-24T23:18:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b037fe700 1 -- 172.21.15.31:0/1740859427 learned_addr learned my addr 172.21.15.31:0/1740859427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.053+0000 7f8b02ffd700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 0x7f8b04114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.054+0000 7f8b02ffd700 1 -- 172.21.15.31:0/1740859427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 msgr2=0x7f8b04114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:18:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.054+0000 7f8b02ffd700 1 -- 172.21.15.31:0/1740859427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 msgr2=0x7f8b04114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:18:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.054+0000 7f8b02ffd700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 0x7f8b04114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T23:18:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.054+0000 7f8b02ffd700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 0x7f8b04114090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:18:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.054+0000 7f8b037fe700 1 -- 172.21.15.31:0/1740859427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 msgr2=0x7f8b04114090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:18:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.054+0000 7f8b037fe700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 0x7f8b04114090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.054+0000 7f8b037fe700 1 -- 172.21.15.31:0/1740859427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4005040 con 0x7f8b04065640 2024-04-24T23:18:32.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.054+0000 7f8b037fe700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04065640 0x7f8b04113b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009150 tx=0x7f8af4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.056+0000 7f8aebfff700 1 -- 172.21.15.31:0/1740859427 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8af400e040 con 0x7f8b04065640 2024-04-24T23:18:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.056+0000 7f8aebfff700 1 -- 172.21.15.31:0/1740859427 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8af4013420 con 0x7f8b04065640 2024-04-24T23:18:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.056+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0410dd50 con 0x7f8b04065640 2024-04-24T23:18:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.056+0000 7f8aebfff700 1 -- 172.21.15.31:0/1740859427 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8af401c630 con 0x7f8b04065640 2024-04-24T23:18:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.056+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0410e310 con 0x7f8b04065640 2024-04-24T23:18:32.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.057+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b04100100 con 0x7f8b04065640 2024-04-24T23:18:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.058+0000 7f8aebfff700 1 -- 172.21.15.31:0/1740859427 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f8af400baf0 con 0x7f8b04065640 2024-04-24T23:18:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.058+0000 7f8aebfff700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8aec06e260 0x7f8aec070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.058+0000 7f8aebfff700 1 -- 172.21.15.31:0/1740859427 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8af408ef00 con 0x7f8b04065640 2024-04-24T23:18:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.058+0000 7f8b02ffd700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8aec06e260 0x7f8aec070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.059+0000 7f8b02ffd700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8aec06e260 0x7f8aec070710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b0410ec90 tx=0x7f8af8009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:32.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.063+0000 7f8aebfff700 1 -- 172.21.15.31:0/1740859427 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8af405bd00 con 0x7f8b04065640 2024-04-24T23:18:32.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: 1: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.smithi040 2024-04-24T23:18:32.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:18:32.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T23:18:32.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: mgrmap e38: smithi031.rrnmwr(active, since 17s), standbys: smithi040.qtdwuq 2024-04-24T23:18:32.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: overall HEALTH_OK 2024-04-24T23:18:32.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: mon.smithi040 calling monitor election 2024-04-24T23:18:32.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: mon.smithi031 calling monitor election 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: mon.smithi031 is new leader, mons smithi031,smithi040 in quorum (ranks 0,1) 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: monmap epoch 3 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: last_changed 2024-04-24T23:18:30.935663+0000 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: created 2024-04-24T23:03:38.159717+0000 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: min_mon_release 19 (squid) 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: election_strategy: 1 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: 1: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.smithi040 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T23:18:32.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: mgrmap e38: smithi031.rrnmwr(active, since 17s), standbys: smithi040.qtdwuq 2024-04-24T23:18:32.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: overall HEALTH_OK 2024-04-24T23:18:32.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:32.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:32.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:32.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:32.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:31 smithi040 ceph-mon[83047]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:32.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: 1: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.smithi040 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: mgrmap e38: smithi031.rrnmwr(active, since 17s), standbys: smithi040.qtdwuq 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: overall HEALTH_OK 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: mon.smithi040 calling monitor election 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: mon.smithi031 calling monitor election 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: mon.smithi031 is new leader, mons smithi031,smithi040 in quorum (ranks 0,1) 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mon metadata", "id": "smithi040"}]: dispatch 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: monmap epoch 3 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: fsid ca7b26e4-028e-11ef-bc93-c7b262605968 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: last_changed 2024-04-24T23:18:30.935663+0000 2024-04-24T23:18:32.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: created 2024-04-24T23:03:38.159717+0000 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: min_mon_release 19 (squid) 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: election_strategy: 1 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: 1: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.smithi040 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: osdmap e49: 6 total, 6 up, 6 in 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: mgrmap e38: smithi031.rrnmwr(active, since 17s), standbys: smithi040.qtdwuq 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: overall HEALTH_OK 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:32.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:31 smithi031 ceph-mon[111404]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:32.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.361+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8b0410f1a0 con 0x7f8b04065640 2024-04-24T23:18:32.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.361+0000 7f8aebfff700 1 -- 172.21.15.31:0/1740859427 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f8af405b890 con 0x7f8b04065640 2024-04-24T23:18:32.364 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:18:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:18:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:18:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:18:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:18:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:18:32.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:18:32.365 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:18:32.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8aec06e260 msgr2=0x7f8aec070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:32.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8aec06e260 0x7f8aec070710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b0410ec90 tx=0x7f8af8009f70 comp rx=0 tx=0).stop 2024-04-24T23:18:32.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04065640 msgr2=0x7f8b04113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04065640 0x7f8b04113b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8af4009150 tx=0x7f8af4009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 shutdown_connections 2024-04-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b04065640 0x7f8b04113b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8aec06e260 0x7f8aec070710 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 --2- 172.21.15.31:0/1740859427 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8b04102e50 0x7f8b04114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 >> 172.21.15.31:0/1740859427 conn(0x7f8b04071d00 msgr2=0x7f8b040fafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 shutdown_connections 2024-04-24T23:18:32.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.364+0000 7f8b09ded700 1 -- 172.21.15.31:0/1740859427 wait complete. 2024-04-24T23:18:32.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.471+0000 7faf9ce84700 1 -- 172.21.15.31:0/4113818296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98103780 msgr2=0x7faf980fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:32.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.471+0000 7faf9ce84700 1 --2- 172.21.15.31:0/4113818296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98103780 0x7faf980fcdf0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf80004940 tx=0x7faf80009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:32.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.471+0000 7faf9ce84700 1 -- 172.21.15.31:0/4113818296 shutdown_connections 2024-04-24T23:18:32.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.471+0000 7faf9ce84700 1 --2- 172.21.15.31:0/4113818296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98103780 0x7faf980fcdf0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.471+0000 7faf9ce84700 1 --2- 172.21.15.31:0/4113818296 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98102de0 0x7faf981031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.471+0000 7faf9ce84700 1 -- 172.21.15.31:0/4113818296 >> 172.21.15.31:0/4113818296 conn(0x7faf980f8950 msgr2=0x7faf980fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:32.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.471+0000 7faf9ce84700 1 -- 172.21.15.31:0/4113818296 shutdown_connections 2024-04-24T23:18:32.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.471+0000 7faf9ce84700 1 -- 172.21.15.31:0/4113818296 wait complete. 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9ce84700 1 Processor -- start 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9ce84700 1 -- start start 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9ce84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98102de0 0x7faf98070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9ce84700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 0x7faf980707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9ce84700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98070d30 con 0x7faf98102de0 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9ce84700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98070e70 con 0x7faf98103780 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98102de0 0x7faf98070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98102de0 0x7faf98070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38358/0 (socket says 172.21.15.31:38358) 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9659c700 1 -- 172.21.15.31:0/821091641 learned_addr learned my addr 172.21.15.31:0/821091641 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf95d9b700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 0x7faf980707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf95d9b700 1 -- 172.21.15.31:0/821091641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 msgr2=0x7faf980707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf95d9b700 1 -- 172.21.15.31:0/821091641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 msgr2=0x7faf980707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf95d9b700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 0x7faf980707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf95d9b700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 0x7faf980707a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9659c700 1 -- 172.21.15.31:0/821091641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 msgr2=0x7faf980707a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9659c700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 0x7faf980707a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.472+0000 7faf9659c700 1 -- 172.21.15.31:0/821091641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80005040 con 0x7faf98102de0 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.473+0000 7faf9659c700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98102de0 0x7faf98070260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c8c0 tx=0x7faf880044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.473+0000 7faf8effd700 1 -- 172.21.15.31:0/821091641 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf8800a980 con 0x7faf98102de0 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.473+0000 7faf8effd700 1 -- 172.21.15.31:0/821091641 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf8800ce60 con 0x7faf98102de0 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.473+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf98119eb0 con 0x7faf98102de0 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.473+0000 7faf8effd700 1 -- 172.21.15.31:0/821091641 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf880115e0 con 0x7faf98102de0 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.473+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9811a330 con 0x7faf98102de0 2024-04-24T23:18:32.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.474+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf980fe530 con 0x7faf98102de0 2024-04-24T23:18:32.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.474+0000 7faf8effd700 1 -- 172.21.15.31:0/821091641 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7faf88012400 con 0x7faf98102de0 2024-04-24T23:18:32.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.475+0000 7faf8effd700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faf8406e260 0x7faf84070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.475+0000 7faf8effd700 1 -- 172.21.15.31:0/821091641 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7faf8808dc60 con 0x7faf98102de0 2024-04-24T23:18:32.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.475+0000 7faf95d9b700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faf8406e260 0x7faf84070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.476+0000 7faf95d9b700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faf8406e260 0x7faf84070710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf8000b3b0 tx=0x7faf80009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:32.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.480+0000 7faf8effd700 1 -- 172.21.15.31:0/821091641 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faf88057040 con 0x7faf98102de0 2024-04-24T23:18:32.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.736+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf9804bed0 con 0x7faf98102de0 2024-04-24T23:18:32.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.736+0000 7faf8effd700 1 -- 172.21.15.31:0/821091641 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1878 (secure 0 0 0) 0x7faf880126b0 con 0x7faf98102de0 2024-04-24T23:18:32.738 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:18:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:18:32.740 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:18:32.741 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:18:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:18:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:18:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:18:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:18:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:18:32.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faf8406e260 msgr2=0x7faf84070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:32.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faf8406e260 0x7faf84070710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf8000b3b0 tx=0x7faf80009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98102de0 msgr2=0x7faf98070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98102de0 0x7faf98070260 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c8c0 tx=0x7faf880044c0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 shutdown_connections 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf98102de0 0x7faf98070260 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faf8406e260 0x7faf84070710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 --2- 172.21.15.31:0/821091641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf98103780 0x7faf980707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 >> 172.21.15.31:0/821091641 conn(0x7faf980f8950 msgr2=0x7faf980f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 shutdown_connections 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.738+0000 7faf9ce84700 1 -- 172.21.15.31:0/821091641 wait complete. 2024-04-24T23:18:32.744 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.846+0000 7fb129685700 1 -- 172.21.15.31:0/3569344646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 msgr2=0x7fb1240fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.846+0000 7fb129685700 1 --2- 172.21.15.31:0/3569344646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 0x7fb1240fcdf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb114004940 tx=0x7fb114009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.846+0000 7fb129685700 1 -- 172.21.15.31:0/3569344646 shutdown_connections 2024-04-24T23:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.846+0000 7fb129685700 1 --2- 172.21.15.31:0/3569344646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 0x7fb1240fcdf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.846+0000 7fb129685700 1 --2- 172.21.15.31:0/3569344646 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 0x7fb1241031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.846+0000 7fb129685700 1 -- 172.21.15.31:0/3569344646 >> 172.21.15.31:0/3569344646 conn(0x7fb1240f8990 msgr2=0x7fb1240fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.846+0000 7fb129685700 1 -- 172.21.15.31:0/3569344646 shutdown_connections 2024-04-24T23:18:32.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.846+0000 7fb129685700 1 -- 172.21.15.31:0/3569344646 wait complete. 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb129685700 1 Processor -- start 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb129685700 1 -- start start 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb129685700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 0x7fb124070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb129685700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 0x7fb1240707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb129685700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240721b0 con 0x7fb124103780 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb129685700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb124070ce0 con 0x7fb124102de0 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb1227fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 0x7fb1240707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb1227fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 0x7fb1240707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38374/0 (socket says 172.21.15.31:38374) 2024-04-24T23:18:32.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb1227fc700 1 -- 172.21.15.31:0/2001169124 learned_addr learned my addr 172.21.15.31:0/2001169124 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb122ffd700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 0x7fb124070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb122ffd700 1 -- 172.21.15.31:0/2001169124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 msgr2=0x7fb124070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb122ffd700 1 -- 172.21.15.31:0/2001169124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 msgr2=0x7fb124070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb122ffd700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 0x7fb124070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb122ffd700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 0x7fb124070260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb1227fc700 1 -- 172.21.15.31:0/2001169124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 msgr2=0x7fb124070260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb1227fc700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 0x7fb124070260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.847+0000 7fb1227fc700 1 -- 172.21.15.31:0/2001169124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb114005040 con 0x7fb124103780 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.848+0000 7fb1227fc700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 0x7fb1240707a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb114009100 tx=0x7fb114009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.848+0000 7fb10b7fe700 1 -- 172.21.15.31:0/2001169124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb11400e040 con 0x7fb124103780 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.848+0000 7fb10b7fe700 1 -- 172.21.15.31:0/2001169124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb114004290 con 0x7fb124103780 2024-04-24T23:18:32.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.848+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb124070f60 con 0x7fb124103780 2024-04-24T23:18:32.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.848+0000 7fb10b7fe700 1 -- 172.21.15.31:0/2001169124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb11401c3f0 con 0x7fb124103780 2024-04-24T23:18:32.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.848+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12411a110 con 0x7fb124103780 2024-04-24T23:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.849+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1241052b0 con 0x7fb124103780 2024-04-24T23:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.850+0000 7fb10b7fe700 1 -- 172.21.15.31:0/2001169124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fb114013790 con 0x7fb124103780 2024-04-24T23:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.850+0000 7fb10b7fe700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb10c06e270 0x7fb10c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.851+0000 7fb10b7fe700 1 -- 172.21.15.31:0/2001169124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb11408ef10 con 0x7fb124103780 2024-04-24T23:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.851+0000 7fb122ffd700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb10c06e270 0x7fb10c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:32.853 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.851+0000 7fb122ffd700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb10c06e270 0x7fb10c070720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb118009ce0 tx=0x7fb118009f70 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:32.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:32.855+0000 7fb10b7fe700 1 -- 172.21.15.31:0/2001169124 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb114058290 con 0x7fb124103780 2024-04-24T23:18:33.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.084+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb124109ee0 con 0x7fb10c06e270 2024-04-24T23:18:33.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.085+0000 7fb10b7fe700 1 -- 172.21.15.31:0/2001169124 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fb124109ee0 con 0x7fb10c06e270 2024-04-24T23:18:33.087 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:18:33.087 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "3/21 daemons upgraded", 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:18:33.088 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb10c06e270 msgr2=0x7fb10c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb10c06e270 0x7fb10c070720 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb118009ce0 tx=0x7fb118009f70 comp rx=0 tx=0).stop 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 msgr2=0x7fb1240707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 0x7fb1240707a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb114009100 tx=0x7fb114009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 shutdown_connections 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb10c06e270 0x7fb10c070720 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb124102de0 0x7fb124070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 --2- 172.21.15.31:0/2001169124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb124103780 0x7fb1240707a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 >> 172.21.15.31:0/2001169124 conn(0x7fb1240f8990 msgr2=0x7fb1241021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 shutdown_connections 2024-04-24T23:18:33.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.088+0000 7fb129685700 1 -- 172.21.15.31:0/2001169124 wait complete. 2024-04-24T23:18:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.192+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/97171355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 msgr2=0x7f6c880fe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.193+0000 7f6c8e7e8700 1 --2- 172.21.15.31:0/97171355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 0x7f6c880fe1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c78004940 tx=0x7f6c78009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.193+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/97171355 shutdown_connections 2024-04-24T23:18:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.193+0000 7f6c8e7e8700 1 --2- 172.21.15.31:0/97171355 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 0x7f6c88106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.193+0000 7f6c8e7e8700 1 --2- 172.21.15.31:0/97171355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 0x7f6c880fe1a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.193+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/97171355 >> 172.21.15.31:0/97171355 conn(0x7f6c880f7850 msgr2=0x7f6c880f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:33 smithi031 ceph-mon[111404]: pgmap v12: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T23:18:33.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:33 smithi031 ceph-mon[111404]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:33.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:33 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1740859427' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:18:33.196 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:33 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/821091641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:18:33.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.193+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/97171355 shutdown_connections 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.193+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/97171355 wait complete. 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c8e7e8700 1 Processor -- start 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c8e7e8700 1 -- start start 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c8e7e8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 0x7f6c88113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c8e7e8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 0x7f6c881140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c8e7e8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88114700 con 0x7f6c880fddd0 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c8e7e8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8810daf0 con 0x7f6c880fe770 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 0x7f6c88113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 0x7f6c881140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c87fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 0x7f6c88113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38400/0 (socket says 172.21.15.31:38400) 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c87fff700 1 -- 172.21.15.31:0/1055901201 learned_addr learned my addr 172.21.15.31:0/1055901201 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c877fe700 1 -- 172.21.15.31:0/1055901201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 msgr2=0x7f6c881140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-24T23:18:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c877fe700 1 -- 172.21.15.31:0/1055901201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 msgr2=0x7f6c881140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.194+0000 7f6c877fe700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 0x7f6c881140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c877fe700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 0x7f6c881140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c87fff700 1 -- 172.21.15.31:0/1055901201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 msgr2=0x7f6c881140b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c87fff700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 0x7f6c881140b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c87fff700 1 -- 172.21.15.31:0/1055901201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78005040 con 0x7f6c880fddd0 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c87fff700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 0x7f6c88113b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009380 tx=0x7f6c78009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c84ff9700 1 -- 172.21.15.31:0/1055901201 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7800e040 con 0x7f6c880fddd0 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8810dd70 con 0x7f6c880fddd0 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c84ff9700 1 -- 172.21.15.31:0/1055901201 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c78009af0 con 0x7f6c880fddd0 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8810e2b0 con 0x7f6c880fddd0 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.195+0000 7f6c84ff9700 1 -- 172.21.15.31:0/1055901201 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7801c630 con 0x7f6c880fddd0 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.196+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c880666d0 con 0x7f6c880fddd0 2024-04-24T23:18:33.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.197+0000 7f6c84ff9700 1 -- 172.21.15.31:0/1055901201 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f6c7800bbc0 con 0x7f6c880fddd0 2024-04-24T23:18:33.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.197+0000 7f6c84ff9700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f6c7006e260 0x7f6c70070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:18:33.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.197+0000 7f6c84ff9700 1 -- 172.21.15.31:0/1055901201 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6c7808f0a0 con 0x7f6c880fddd0 2024-04-24T23:18:33.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.197+0000 7f6c877fe700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f6c7006e260 0x7f6c70070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:18:33.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.198+0000 7f6c877fe700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f6c7006e260 0x7f6c70070710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c8810ecb0 tx=0x7f6c7c009f10 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:18:33.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.202+0000 7f6c84ff9700 1 -- 172.21.15.31:0/1055901201 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6c7805be00 con 0x7f6c880fddd0 2024-04-24T23:18:33.372 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:33 smithi040 ceph-mon[83047]: pgmap v12: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-24T23:18:33.372 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:33 smithi040 ceph-mon[83047]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:33.372 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:33 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1740859427' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:18:33.372 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:33 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/821091641' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:18:33.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.494+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6c88109e60 con 0x7f6c880fddd0 2024-04-24T23:18:33.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.495+0000 7f6c84ff9700 1 -- 172.21.15.31:0/1055901201 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6c7805b990 con 0x7f6c880fddd0 2024-04-24T23:18:33.497 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:18:33.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f6c7006e260 msgr2=0x7f6c70070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f6c7006e260 0x7f6c70070710 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6c8810ecb0 tx=0x7f6c7c009f10 comp rx=0 tx=0).stop 2024-04-24T23:18:33.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 msgr2=0x7f6c88113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:18:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 0x7f6c88113b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009380 tx=0x7f6c78009e30 comp rx=0 tx=0).stop 2024-04-24T23:18:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 shutdown_connections 2024-04-24T23:18:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c880fddd0 0x7f6c88113b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f6c7006e260 0x7f6c70070710 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 --2- 172.21.15.31:0/1055901201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6c880fe770 0x7f6c881140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:18:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.497+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 >> 172.21.15.31:0/1055901201 conn(0x7f6c880f7850 msgr2=0x7f6c880f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:18:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.498+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 shutdown_connections 2024-04-24T23:18:33.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:18:33.498+0000 7f6c8e7e8700 1 -- 172.21.15.31:0/1055901201 wait complete. 2024-04-24T23:18:34.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:34 smithi040 ceph-mon[83047]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:34.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:34 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1055901201' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:18:34.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:34 smithi031 ceph-mon[111404]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:18:34.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:34 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1055901201' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:18:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:35 smithi040 ceph-mon[83047]: pgmap v13: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:35.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:35 smithi031 ceph-mon[111404]: pgmap v13: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:35.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:36 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:36.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:36 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:36.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:36 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:36.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:36 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:37.575 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:37 smithi040 ceph-mon[83047]: pgmap v14: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:37.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:37 smithi031 ceph-mon[111404]: pgmap v14: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:39.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: pgmap v15: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:39.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:39.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:39.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:39.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:39.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:39.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:18:39.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:18:39.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:39.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: pgmap v15: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:39.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:39.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:39.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:39.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:18:39.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:39.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:18:39.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:18:39.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:40.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:40 smithi031 ceph-mon[111404]: Reconfiguring mon.smithi031 (monmap changed)... 2024-04-24T23:18:40.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:40 smithi031 ceph-mon[111404]: Reconfiguring daemon mon.smithi031 on smithi031 2024-04-24T23:18:40.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:40 smithi040 ceph-mon[83047]: Reconfiguring mon.smithi031 (monmap changed)... 2024-04-24T23:18:40.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:40 smithi040 ceph-mon[83047]: Reconfiguring daemon mon.smithi031 on smithi031 2024-04-24T23:18:41.547 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:41 smithi031 ceph-mon[111404]: pgmap v16: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:41.547 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:41 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:41.548 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:41 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:41.548 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:41 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.rrnmwr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:18:41.548 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:41 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:18:41.548 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:41 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:41.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:41 smithi040 ceph-mon[83047]: pgmap v16: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:41.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:41 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:41.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:41 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:41.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:41 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.rrnmwr", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:18:41.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:41 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:18:41.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:41 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:42.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:42 smithi031 ceph-mon[111404]: Reconfiguring mgr.smithi031.rrnmwr (monmap changed)... 2024-04-24T23:18:42.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:42 smithi031 ceph-mon[111404]: Reconfiguring daemon mgr.smithi031.rrnmwr on smithi031 2024-04-24T23:18:42.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:42 smithi040 ceph-mon[83047]: Reconfiguring mgr.smithi031.rrnmwr (monmap changed)... 2024-04-24T23:18:42.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:42 smithi040 ceph-mon[83047]: Reconfiguring daemon mgr.smithi031.rrnmwr on smithi031 2024-04-24T23:18:43.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:43 smithi040 ceph-mon[83047]: pgmap v17: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:43.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:43.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:43.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:18:43.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:43.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:43 smithi031 ceph-mon[111404]: pgmap v17: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:43.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:43.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:44.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:18:44.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:44.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:44 smithi040 ceph-mon[83047]: Reconfiguring crash.smithi031 (monmap changed)... 2024-04-24T23:18:44.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:44 smithi040 ceph-mon[83047]: Reconfiguring daemon crash.smithi031 on smithi031 2024-04-24T23:18:44.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:44.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:44.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:44.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:18:44.915 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:44 smithi031 ceph-mon[111404]: Reconfiguring crash.smithi031 (monmap changed)... 2024-04-24T23:18:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:44 smithi031 ceph-mon[111404]: Reconfiguring daemon crash.smithi031 on smithi031 2024-04-24T23:18:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:45.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:45.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:18:45.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:45.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:45 smithi040 ceph-mon[83047]: pgmap v18: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:45.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:45 smithi040 ceph-mon[83047]: Reconfiguring osd.0 (monmap changed)... 2024-04-24T23:18:45.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:45 smithi040 ceph-mon[83047]: Reconfiguring daemon osd.0 on smithi031 2024-04-24T23:18:45.961 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:45 smithi031 ceph-mon[111404]: pgmap v18: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:45.961 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:45 smithi031 ceph-mon[111404]: Reconfiguring osd.0 (monmap changed)... 2024-04-24T23:18:45.961 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:45 smithi031 ceph-mon[111404]: Reconfiguring daemon osd.0 on smithi031 2024-04-24T23:18:47.377 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:47 smithi031 ceph-mon[111404]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:47.377 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:47 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:47.377 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:47 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:47.377 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:47 smithi031 ceph-mon[111404]: Reconfiguring osd.1 (monmap changed)... 2024-04-24T23:18:47.377 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:47 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:18:47.377 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:47 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:47.378 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:47 smithi031 ceph-mon[111404]: Reconfiguring daemon osd.1 on smithi031 2024-04-24T23:18:47.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:47 smithi040 ceph-mon[83047]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:47.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:47 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:47.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:47 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:47.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:47 smithi040 ceph-mon[83047]: Reconfiguring osd.1 (monmap changed)... 2024-04-24T23:18:47.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:47 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:18:47.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:47 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:47.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:47 smithi040 ceph-mon[83047]: Reconfiguring daemon osd.1 on smithi031 2024-04-24T23:18:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:48 smithi040 ceph-mon[83047]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:48 smithi040 ceph-mon[83047]: Reconfiguring osd.2 (monmap changed)... 2024-04-24T23:18:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:18:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:48 smithi040 ceph-mon[83047]: Reconfiguring daemon osd.2 on smithi031 2024-04-24T23:18:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:48 smithi031 ceph-mon[111404]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:48 smithi031 ceph-mon[111404]: Reconfiguring osd.2 (monmap changed)... 2024-04-24T23:18:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:18:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:49.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:48 smithi031 ceph-mon[111404]: Reconfiguring daemon osd.2 on smithi031 2024-04-24T23:18:50.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:50 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:50.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:50 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:50.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:50 smithi031 ceph-mon[111404]: Reconfiguring mds.cephfs.smithi031.xxejxr (monmap changed)... 2024-04-24T23:18:50.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:50 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.xxejxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:18:50.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:50 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:50.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:50 smithi031 ceph-mon[111404]: Reconfiguring daemon mds.cephfs.smithi031.xxejxr on smithi031 2024-04-24T23:18:50.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:50 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:50.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:50 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:50.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:50 smithi040 ceph-mon[83047]: Reconfiguring mds.cephfs.smithi031.xxejxr (monmap changed)... 2024-04-24T23:18:50.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:50 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.xxejxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:18:50.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:50 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:50.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:50 smithi040 ceph-mon[83047]: Reconfiguring daemon mds.cephfs.smithi031.xxejxr on smithi031 2024-04-24T23:18:51.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:51 smithi031 ceph-mon[111404]: pgmap v21: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:51.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:51 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:51.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:51 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:51.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:51 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.acqiox", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:18:51.814 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:51 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:51 smithi040 ceph-mon[83047]: pgmap v21: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:51.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:51 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:51.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:51 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:51.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:51 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.acqiox", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:18:51.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:51 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:52.801 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:52 smithi031 ceph-mon[111404]: Reconfiguring mds.cephfs.smithi031.acqiox (monmap changed)... 2024-04-24T23:18:52.801 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:52 smithi031 ceph-mon[111404]: Reconfiguring daemon mds.cephfs.smithi031.acqiox on smithi031 2024-04-24T23:18:52.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:52 smithi040 ceph-mon[83047]: Reconfiguring mds.cephfs.smithi031.acqiox (monmap changed)... 2024-04-24T23:18:52.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:52 smithi040 ceph-mon[83047]: Reconfiguring daemon mds.cephfs.smithi031.acqiox on smithi031 2024-04-24T23:18:53.744 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:53 smithi040 ceph-mon[83047]: pgmap v22: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:53.744 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:53.744 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:53.744 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:18:53.744 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:53.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:53 smithi031 ceph-mon[111404]: pgmap v22: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:53.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:53.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:54.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:18:54.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:54.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:54 smithi040 ceph-mon[83047]: Reconfiguring crash.smithi040 (monmap changed)... 2024-04-24T23:18:54.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:54 smithi040 ceph-mon[83047]: Reconfiguring daemon crash.smithi040 on smithi040 2024-04-24T23:18:54.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:54 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:54.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:54 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:54.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:54 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:18:54.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:54 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:18:54.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:54 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:54.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:54 smithi031 ceph-mon[111404]: Reconfiguring crash.smithi040 (monmap changed)... 2024-04-24T23:18:54.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:54 smithi031 ceph-mon[111404]: Reconfiguring daemon crash.smithi040 on smithi040 2024-04-24T23:18:54.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:54 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:55.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:54 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:55.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:54 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi040.qtdwuq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-24T23:18:55.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:54 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-24T23:18:55.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:54 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:55.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:55 smithi040 ceph-mon[83047]: pgmap v23: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:55.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:55 smithi040 ceph-mon[83047]: Reconfiguring mgr.smithi040.qtdwuq (monmap changed)... 2024-04-24T23:18:55.817 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:55 smithi040 ceph-mon[83047]: Reconfiguring daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:18:55.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:55 smithi031 ceph-mon[111404]: pgmap v23: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:56.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:55 smithi031 ceph-mon[111404]: Reconfiguring mgr.smithi040.qtdwuq (monmap changed)... 2024-04-24T23:18:56.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:55 smithi031 ceph-mon[111404]: Reconfiguring daemon mgr.smithi040.qtdwuq on smithi040 2024-04-24T23:18:57.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:57 smithi040 ceph-mon[83047]: pgmap v24: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:57.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:57 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:57.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:57 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:57.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:57 smithi040 ceph-mon[83047]: Reconfiguring mon.smithi040 (monmap changed)... 2024-04-24T23:18:57.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:57 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:18:57.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:57 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:18:57.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:57 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:57.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:57 smithi040 ceph-mon[83047]: Reconfiguring daemon mon.smithi040 on smithi040 2024-04-24T23:18:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:57 smithi031 ceph-mon[111404]: pgmap v24: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:57 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:57.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:57 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:57.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:57 smithi031 ceph-mon[111404]: Reconfiguring mon.smithi040 (monmap changed)... 2024-04-24T23:18:57.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:57 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-24T23:18:57.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:57 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-24T23:18:57.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:57 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:57.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:57 smithi031 ceph-mon[111404]: Reconfiguring daemon mon.smithi040 on smithi040 2024-04-24T23:18:58.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:58 smithi031 ceph-mon[111404]: pgmap v25: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:58.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:58.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:58.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:58 smithi031 ceph-mon[111404]: Reconfiguring osd.3 (monmap changed)... 2024-04-24T23:18:58.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:18:59.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:59.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:58 smithi031 ceph-mon[111404]: Reconfiguring daemon osd.3 on smithi040 2024-04-24T23:18:59.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:18:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:18:59.149 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:58 smithi040 ceph-mon[83047]: pgmap v25: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:18:59.149 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:59.150 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:18:59.150 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:58 smithi040 ceph-mon[83047]: Reconfiguring osd.3 (monmap changed)... 2024-04-24T23:18:59.150 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:18:59.150 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:18:59.150 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:58 smithi040 ceph-mon[83047]: Reconfiguring daemon osd.3 on smithi040 2024-04-24T23:18:59.150 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:18:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:00.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:00 smithi040 ceph-mon[83047]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:00.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:00 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:00.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:00 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:00.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:00 smithi040 ceph-mon[83047]: Reconfiguring osd.4 (monmap changed)... 2024-04-24T23:19:00.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:00 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:19:00.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:00 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:00.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:00 smithi040 ceph-mon[83047]: Reconfiguring daemon osd.4 on smithi040 2024-04-24T23:19:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:00 smithi031 ceph-mon[111404]: pgmap v26: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:00 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:00 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:00 smithi031 ceph-mon[111404]: Reconfiguring osd.4 (monmap changed)... 2024-04-24T23:19:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:00 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:19:01.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:00 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:01.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:00 smithi031 ceph-mon[111404]: Reconfiguring daemon osd.4 on smithi040 2024-04-24T23:19:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:02 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:02.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:02 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:02.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:02 smithi040 ceph-mon[83047]: Reconfiguring osd.5 (monmap changed)... 2024-04-24T23:19:02.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:02 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:19:02.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:02 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:02.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:02 smithi040 ceph-mon[83047]: Reconfiguring daemon osd.5 on smithi040 2024-04-24T23:19:02.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:02 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:02.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:02 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:02.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:02 smithi031 ceph-mon[111404]: Reconfiguring osd.5 (monmap changed)... 2024-04-24T23:19:02.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:02 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:19:02.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:02 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:02.670 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:02 smithi031 ceph-mon[111404]: Reconfiguring daemon osd.5 on smithi040 2024-04-24T23:19:03.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 -- 172.21.15.31:0/1865417463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fd0a0 msgr2=0x7f5038109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:03.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 --2- 172.21.15.31:0/1865417463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fd0a0 0x7f5038109350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f502c004940 tx=0x7f502c009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:03.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 -- 172.21.15.31:0/1865417463 shutdown_connections 2024-04-24T23:19:03.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 --2- 172.21.15.31:0/1865417463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fd0a0 0x7f5038109350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 --2- 172.21.15.31:0/1865417463 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50380fc790 0x7f50380fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 -- 172.21.15.31:0/1865417463 >> 172.21.15.31:0/1865417463 conn(0x7f5038071d00 msgr2=0x7f5038072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:03.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 -- 172.21.15.31:0/1865417463 shutdown_connections 2024-04-24T23:19:03.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 -- 172.21.15.31:0/1865417463 wait complete. 2024-04-24T23:19:03.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 Processor -- start 2024-04-24T23:19:03.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.607+0000 7f503caaa700 1 -- start start 2024-04-24T23:19:03.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503caaa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fc790 0x7f5038115dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:03.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503caaa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50380fd0a0 0x7f5038116300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:03.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503caaa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5038116950 con 0x7f50380fc790 2024-04-24T23:19:03.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503caaa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503810fd20 con 0x7f50380fd0a0 2024-04-24T23:19:03.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fc790 0x7f5038115dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:03.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fc790 0x7f5038115dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38634/0 (socket says 172.21.15.31:38634) 2024-04-24T23:19:03.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503659c700 1 -- 172.21.15.31:0/1341585749 learned_addr learned my addr 172.21.15.31:0/1341585749 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:03.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f5035d9b700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50380fd0a0 0x7f5038116300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:03.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503659c700 1 -- 172.21.15.31:0/1341585749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50380fd0a0 msgr2=0x7f5038116300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:03.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503659c700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50380fd0a0 0x7f5038116300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503659c700 1 -- 172.21.15.31:0/1341585749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c005040 con 0x7f50380fc790 2024-04-24T23:19:03.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503659c700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fc790 0x7f5038115dc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f502800eb40 tx=0x7f502800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:03.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f5026ffd700 1 -- 172.21.15.31:0/1341585749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5028007cc0 con 0x7f50380fc790 2024-04-24T23:19:03.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f5026ffd700 1 -- 172.21.15.31:0/1341585749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f502800f040 con 0x7f50380fc790 2024-04-24T23:19:03.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503810ffa0 con 0x7f50380fc790 2024-04-24T23:19:03.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.608+0000 7f5026ffd700 1 -- 172.21.15.31:0/1341585749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f502800ca10 con 0x7f50380fc790 2024-04-24T23:19:03.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.609+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50381104b0 con 0x7f50380fc790 2024-04-24T23:19:03.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.610+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50380fc330 con 0x7f50380fc790 2024-04-24T23:19:03.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.610+0000 7f5026ffd700 1 -- 172.21.15.31:0/1341585749 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f502800cb70 con 0x7f50380fc790 2024-04-24T23:19:03.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.610+0000 7f5026ffd700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f502006e260 0x7f5020070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:03.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.611+0000 7f5026ffd700 1 -- 172.21.15.31:0/1341585749 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f502808d860 con 0x7f50380fc790 2024-04-24T23:19:03.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.611+0000 7f5035d9b700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f502006e260 0x7f5020070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:03.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.611+0000 7f5035d9b700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f502006e260 0x7f5020070710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f502c009b30 tx=0x7f502c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:03.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.616+0000 7f5026ffd700 1 -- 172.21.15.31:0/1341585749 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f502805a670 con 0x7f50380fc790 2024-04-24T23:19:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:03 smithi031 ceph-mon[111404]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:03 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:03 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:03 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.dicevu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:19:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:03 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:03.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:03 smithi040 ceph-mon[83047]: pgmap v27: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:03.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:03 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:03.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:03 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:03.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:03 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.dicevu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:19:03.794 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:03 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:03.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.845+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f503804bed0 con 0x7f502006e260 2024-04-24T23:19:03.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.847+0000 7f5026ffd700 1 -- 172.21.15.31:0/1341585749 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f503804bed0 con 0x7f502006e260 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f502006e260 msgr2=0x7f5020070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f502006e260 0x7f5020070710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f502c009b30 tx=0x7f502c009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fc790 msgr2=0x7f5038115dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fc790 0x7f5038115dc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f502800eb40 tx=0x7f502800b040 comp rx=0 tx=0).stop 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 shutdown_connections 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50380fc790 0x7f5038115dc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f502006e260 0x7f5020070710 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 --2- 172.21.15.31:0/1341585749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f50380fd0a0 0x7f5038116300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 >> 172.21.15.31:0/1341585749 conn(0x7f5038071d00 msgr2=0x7f50380f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:03.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 shutdown_connections 2024-04-24T23:19:03.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.849+0000 7f503caaa700 1 -- 172.21.15.31:0/1341585749 wait complete. 2024-04-24T23:19:03.861 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:19:03.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.958+0000 7f9ac31e7700 1 -- 172.21.15.31:0/2507494428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc103780 msgr2=0x7f9abc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.958+0000 7f9ac31e7700 1 --2- 172.21.15.31:0/2507494428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc103780 0x7f9abc0fcdf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9aac004940 tx=0x7f9aac009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.958+0000 7f9ac31e7700 1 -- 172.21.15.31:0/2507494428 shutdown_connections 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.958+0000 7f9ac31e7700 1 --2- 172.21.15.31:0/2507494428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc103780 0x7f9abc0fcdf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.958+0000 7f9ac31e7700 1 --2- 172.21.15.31:0/2507494428 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9abc102de0 0x7f9abc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.958+0000 7f9ac31e7700 1 -- 172.21.15.31:0/2507494428 >> 172.21.15.31:0/2507494428 conn(0x7f9abc0f89b0 msgr2=0x7f9abc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.958+0000 7f9ac31e7700 1 -- 172.21.15.31:0/2507494428 shutdown_connections 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.958+0000 7f9ac31e7700 1 -- 172.21.15.31:0/2507494428 wait complete. 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac31e7700 1 Processor -- start 2024-04-24T23:19:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac31e7700 1 -- start start 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac31e7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc102de0 0x7f9abc113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac31e7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9abc103780 0x7f9abc114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac31e7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc114690 con 0x7f9abc102de0 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac31e7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc10da80 con 0x7f9abc103780 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac0f83700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc102de0 0x7f9abc113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac0f83700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc102de0 0x7f9abc113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38660/0 (socket says 172.21.15.31:38660) 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9ac0f83700 1 -- 172.21.15.31:0/3564779708 learned_addr learned my addr 172.21.15.31:0/3564779708 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.959+0000 7f9abbfff700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9abc103780 0x7f9abc114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9abbfff700 1 -- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc102de0 msgr2=0x7f9abc113b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9abbfff700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc102de0 0x7f9abc113b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9abbfff700 1 -- 172.21.15.31:0/3564779708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac005040 con 0x7f9abc103780 2024-04-24T23:19:03.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9ac0f83700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc102de0 0x7f9abc113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9abbfff700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9abc103780 0x7f9abc114040 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9aac000c00 tx=0x7f9aac009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9ab97fa700 1 -- 172.21.15.31:0/3564779708 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aac00e070 con 0x7f9abc103780 2024-04-24T23:19:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9abc10dd00 con 0x7f9abc103780 2024-04-24T23:19:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9ab97fa700 1 -- 172.21.15.31:0/3564779708 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9aac013c00 con 0x7f9abc103780 2024-04-24T23:19:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9ab97fa700 1 -- 172.21.15.31:0/3564779708 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aac00b650 con 0x7f9abc103780 2024-04-24T23:19:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.960+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9abc10e240 con 0x7f9abc103780 2024-04-24T23:19:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.961+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9abc0fe530 con 0x7f9abc103780 2024-04-24T23:19:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.961+0000 7f9ab97fa700 1 -- 172.21.15.31:0/3564779708 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9aac01d9c0 con 0x7f9abc103780 2024-04-24T23:19:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.962+0000 7f9ab97fa700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9aa806e260 0x7f9aa8070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.962+0000 7f9ab97fa700 1 -- 172.21.15.31:0/3564779708 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9aac08eef0 con 0x7f9abc103780 2024-04-24T23:19:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.962+0000 7f9ac0f83700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9aa806e260 0x7f9aa8070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.962+0000 7f9ac0f83700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9aa806e260 0x7f9aa8070710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40041d0 tx=0x7f9aa400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:03.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:03.967+0000 7f9ab97fa700 1 -- 172.21.15.31:0/3564779708 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9aac05bd00 con 0x7f9abc103780 2024-04-24T23:19:04.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.195+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9abc1137d0 con 0x7f9aa806e260 2024-04-24T23:19:04.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.196+0000 7f9ab97fa700 1 -- 172.21.15.31:0/3564779708 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f9abc1137d0 con 0x7f9aa806e260 2024-04-24T23:19:04.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9aa806e260 msgr2=0x7f9aa8070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9aa806e260 0x7f9aa8070710 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9aa40041d0 tx=0x7f9aa400b040 comp rx=0 tx=0).stop 2024-04-24T23:19:04.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9abc103780 msgr2=0x7f9abc114040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9abc103780 0x7f9abc114040 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9aac000c00 tx=0x7f9aac009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:04.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 shutdown_connections 2024-04-24T23:19:04.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9abc102de0 0x7f9abc113b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9aa806e260 0x7f9aa8070710 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 --2- 172.21.15.31:0/3564779708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9abc103780 0x7f9abc114040 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 >> 172.21.15.31:0/3564779708 conn(0x7f9abc0f89b0 msgr2=0x7f9abc0f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:04.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 shutdown_connections 2024-04-24T23:19:04.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.198+0000 7f9ac31e7700 1 -- 172.21.15.31:0/3564779708 wait complete. 2024-04-24T23:19:04.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.301+0000 7fd97310f700 1 -- 172.21.15.31:0/3020483961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 msgr2=0x7fd96c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.301+0000 7fd97310f700 1 --2- 172.21.15.31:0/3020483961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 0x7fd96c1031b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd960004940 tx=0x7fd960009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:04.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.301+0000 7fd97310f700 1 -- 172.21.15.31:0/3020483961 shutdown_connections 2024-04-24T23:19:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.301+0000 7fd97310f700 1 --2- 172.21.15.31:0/3020483961 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd96c103780 0x7fd96c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.301+0000 7fd97310f700 1 --2- 172.21.15.31:0/3020483961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 0x7fd96c1031b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.301+0000 7fd97310f700 1 -- 172.21.15.31:0/3020483961 >> 172.21.15.31:0/3020483961 conn(0x7fd96c0f8990 msgr2=0x7fd96c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.301+0000 7fd97310f700 1 -- 172.21.15.31:0/3020483961 shutdown_connections 2024-04-24T23:19:04.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.302+0000 7fd97310f700 1 -- 172.21.15.31:0/3020483961 wait complete. 2024-04-24T23:19:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.302+0000 7fd97310f700 1 Processor -- start 2024-04-24T23:19:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.302+0000 7fd97310f700 1 -- start start 2024-04-24T23:19:04.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.302+0000 7fd97310f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 0x7fd96c1138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.302+0000 7fd97310f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd96c103780 0x7fd96c113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.302+0000 7fd97310f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c114470 con 0x7fd96c102de0 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.302+0000 7fd97310f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c10da70 con 0x7fd96c103780 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd970eab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 0x7fd96c1138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd970eab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 0x7fd96c1138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38672/0 (socket says 172.21.15.31:38672) 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd970eab700 1 -- 172.21.15.31:0/1103621683 learned_addr learned my addr 172.21.15.31:0/1103621683 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd96bfff700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd96c103780 0x7fd96c113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd970eab700 1 -- 172.21.15.31:0/1103621683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd96c103780 msgr2=0x7fd96c113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd970eab700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd96c103780 0x7fd96c113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd970eab700 1 -- 172.21.15.31:0/1103621683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd960005040 con 0x7fd96c102de0 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd970eab700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 0x7fd96c1138e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd96000b8b0 tx=0x7fd960009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd9697fa700 1 -- 172.21.15.31:0/1103621683 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96000e040 con 0x7fd96c102de0 2024-04-24T23:19:04.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c10dcf0 con 0x7fd96c102de0 2024-04-24T23:19:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd9697fa700 1 -- 172.21.15.31:0/1103621683 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd960004290 con 0x7fd96c102de0 2024-04-24T23:19:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd9697fa700 1 -- 172.21.15.31:0/1103621683 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd96001c3f0 con 0x7fd96c102de0 2024-04-24T23:19:04.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.303+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c10e2b0 con 0x7fd96c102de0 2024-04-24T23:19:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.304+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96c1000f0 con 0x7fd96c102de0 2024-04-24T23:19:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.305+0000 7fd9697fa700 1 -- 172.21.15.31:0/1103621683 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fd960013420 con 0x7fd96c102de0 2024-04-24T23:19:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.305+0000 7fd9697fa700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd95c06e2c0 0x7fd95c070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.305+0000 7fd9697fa700 1 -- 172.21.15.31:0/1103621683 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd96008f000 con 0x7fd96c102de0 2024-04-24T23:19:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.305+0000 7fd96bfff700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd95c06e2c0 0x7fd95c070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:04.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.306+0000 7fd96bfff700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd95c06e2c0 0x7fd95c070770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd96c10ec30 tx=0x7fd95800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:04.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.310+0000 7fd9697fa700 1 -- 172.21.15.31:0/1103621683 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd96005bcc0 con 0x7fd96c102de0 2024-04-24T23:19:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:04 smithi031 ceph-mon[111404]: Reconfiguring mds.cephfs.smithi040.dicevu (monmap changed)... 2024-04-24T23:19:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:04 smithi031 ceph-mon[111404]: Reconfiguring daemon mds.cephfs.smithi040.dicevu on smithi040 2024-04-24T23:19:04.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.526+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd96c10ef30 con 0x7fd95c06e2c0 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.536+0000 7fd9697fa700 1 -- 172.21.15.31:0/1103621683 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fd96c10ef30 con 0x7fd95c06e2c0 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (2m) 45s ago 13m 24.1M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (14m) 45s ago 14m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af a88492ec9217 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (13m) 30s ago 13m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 9c37e7b995f6 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (117s) 45s ago 13m 73.7M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (8m) 45s ago 8m 140M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (8m) 45s ago 8m 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:19:04.538 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (8m) 30s ago 8m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (8m) 30s ago 8m 15.5M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (4m) 45s ago 15m 471M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (3m) 30s ago 12m 450M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (59s) 45s ago 15m 27.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (34s) 30s ago 12m 22.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (2m) 45s ago 13m 16.7M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (2m) 30s ago 12m 15.0M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (12m) 45s ago 12m 222M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (11m) 45s ago 11m 239M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (11m) 45s ago 11m 198M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (10m) 30s ago 10m 278M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (10m) 30s ago 10m 245M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:19:04.539 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (9m) 30s ago 9m 225M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:19:04.540 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (2m) 45s ago 13m 56.3M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd95c06e2c0 msgr2=0x7fd95c070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd95c06e2c0 0x7fd95c070770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd96c10ec30 tx=0x7fd95800b040 comp rx=0 tx=0).stop 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 msgr2=0x7fd96c1138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 0x7fd96c1138e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd96000b8b0 tx=0x7fd960009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 shutdown_connections 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd96c102de0 0x7fd96c1138e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd95c06e2c0 0x7fd95c070770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 --2- 172.21.15.31:0/1103621683 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd96c103780 0x7fd96c113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 >> 172.21.15.31:0/1103621683 conn(0x7fd96c0f8990 msgr2=0x7fd96c0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 shutdown_connections 2024-04-24T23:19:04.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.538+0000 7fd97310f700 1 -- 172.21.15.31:0/1103621683 wait complete. 2024-04-24T23:19:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 -- 172.21.15.31:0/2188032434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 msgr2=0x7f19d81020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 --2- 172.21.15.31:0/2188032434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 0x7f19d81020d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f19d4004940 tx=0x7f19d4009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 -- 172.21.15.31:0/2188032434 shutdown_connections 2024-04-24T23:19:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 --2- 172.21.15.31:0/2188032434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d81026a0 0x7f19d80fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 --2- 172.21.15.31:0/2188032434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 0x7f19d81020d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 -- 172.21.15.31:0/2188032434 >> 172.21.15.31:0/2188032434 conn(0x7f19d80f7870 msgr2=0x7f19d80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 -- 172.21.15.31:0/2188032434 shutdown_connections 2024-04-24T23:19:04.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 -- 172.21.15.31:0/2188032434 wait complete. 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.642+0000 7f19e01b3700 1 Processor -- start 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19e01b3700 1 -- start start 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19e01b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 0x7f19d8112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19e01b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d81026a0 0x7f19d8112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19e01b3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d8113410 con 0x7f19d8101d00 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19e01b3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d810ca10 con 0x7f19d81026a0 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19ddf4f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 0x7f19d8112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19ddf4f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 0x7f19d8112880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38704/0 (socket says 172.21.15.31:38704) 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19ddf4f700 1 -- 172.21.15.31:0/3671551847 learned_addr learned my addr 172.21.15.31:0/3671551847 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19dd74e700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d81026a0 0x7f19d8112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19ddf4f700 1 -- 172.21.15.31:0/3671551847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d81026a0 msgr2=0x7f19d8112dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19ddf4f700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d81026a0 0x7f19d8112dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19ddf4f700 1 -- 172.21.15.31:0/3671551847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4005040 con 0x7f19d8101d00 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.643+0000 7f19ddf4f700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 0x7f19d8112880 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009150 tx=0x7f19d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.644+0000 7f19ce7fc700 1 -- 172.21.15.31:0/3671551847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d400e040 con 0x7f19d8101d00 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.644+0000 7f19ce7fc700 1 -- 172.21.15.31:0/3671551847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19d400bd00 con 0x7f19d8101d00 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.644+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d810cc90 con 0x7f19d8101d00 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.644+0000 7f19ce7fc700 1 -- 172.21.15.31:0/3671551847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19d4013660 con 0x7f19d8101d00 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.644+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d810d1f0 con 0x7f19d8101d00 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.644+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d80fd450 con 0x7f19d8101d00 2024-04-24T23:19:04.648 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.645+0000 7f19ce7fc700 1 -- 172.21.15.31:0/3671551847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f19d401e420 con 0x7f19d8101d00 2024-04-24T23:19:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.645+0000 7f19ce7fc700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f19c406e270 0x7f19c4070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.646+0000 7f19ce7fc700 1 -- 172.21.15.31:0/3671551847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f19d4091bb0 con 0x7f19d8101d00 2024-04-24T23:19:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.646+0000 7f19dd74e700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f19c406e270 0x7f19c4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:04.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.646+0000 7f19dd74e700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f19c406e270 0x7f19c4070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f19c8007fd0 tx=0x7f19c8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:04.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.650+0000 7f19ce7fc700 1 -- 172.21.15.31:0/3671551847 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f19d4098080 con 0x7f19d8101d00 2024-04-24T23:19:04.729 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:04 smithi040 ceph-mon[83047]: Reconfiguring mds.cephfs.smithi040.dicevu (monmap changed)... 2024-04-24T23:19:04.730 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:04 smithi040 ceph-mon[83047]: Reconfiguring daemon mds.cephfs.smithi040.dicevu on smithi040 2024-04-24T23:19:04.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.945+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f19d8108e00 con 0x7f19d8101d00 2024-04-24T23:19:04.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.945+0000 7f19ce7fc700 1 -- 172.21.15.31:0/3671551847 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f19d405e910 con 0x7f19d8101d00 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:19:04.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:19:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:19:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-24T23:19:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T23:19:04.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:19:04.950 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f19c406e270 msgr2=0x7f19c4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f19c406e270 0x7f19c4070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f19c8007fd0 tx=0x7f19c8009040 comp rx=0 tx=0).stop 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 msgr2=0x7f19d8112880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 0x7f19d8112880 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f19d4009150 tx=0x7f19d4009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 shutdown_connections 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d8101d00 0x7f19d8112880 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f19c406e270 0x7f19c4070720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 --2- 172.21.15.31:0/3671551847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19d81026a0 0x7f19d8112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 >> 172.21.15.31:0/3671551847 conn(0x7f19d80f7870 msgr2=0x7f19d80f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:04.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 shutdown_connections 2024-04-24T23:19:04.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:04.948+0000 7f19e01b3700 1 -- 172.21.15.31:0/3671551847 wait complete. 2024-04-24T23:19:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.052+0000 7fd005e0e700 1 -- 172.21.15.31:0/3867224118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 msgr2=0x7fd0001031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.052+0000 7fd005e0e700 1 --2- 172.21.15.31:0/3867224118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 0x7fd0001031b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcff0004940 tx=0x7fcff0009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.052+0000 7fd005e0e700 1 -- 172.21.15.31:0/3867224118 shutdown_connections 2024-04-24T23:19:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.052+0000 7fd005e0e700 1 --2- 172.21.15.31:0/3867224118 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd000103780 0x7fd0000fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.052+0000 7fd005e0e700 1 --2- 172.21.15.31:0/3867224118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 0x7fd0001031b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.053+0000 7fd005e0e700 1 -- 172.21.15.31:0/3867224118 >> 172.21.15.31:0/3867224118 conn(0x7fd0000f8990 msgr2=0x7fd0000fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:05.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.053+0000 7fd005e0e700 1 -- 172.21.15.31:0/3867224118 shutdown_connections 2024-04-24T23:19:05.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.053+0000 7fd005e0e700 1 -- 172.21.15.31:0/3867224118 wait complete. 2024-04-24T23:19:05.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.053+0000 7fd005e0e700 1 Processor -- start 2024-04-24T23:19:05.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.053+0000 7fd005e0e700 1 -- start start 2024-04-24T23:19:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fd005e0e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 0x7fd000070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fd005e0e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd000103780 0x7fd0000707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fd005e0e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0000721b0 con 0x7fd000102de0 2024-04-24T23:19:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fd005e0e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd000070ce0 con 0x7fd000103780 2024-04-24T23:19:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 0x7fd000070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfff7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 0x7fd000070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38720/0 (socket says 172.21.15.31:38720) 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfff7fe700 1 -- 172.21.15.31:0/746708767 learned_addr learned my addr 172.21.15.31:0/746708767 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcffeffd700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd000103780 0x7fd0000707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfff7fe700 1 -- 172.21.15.31:0/746708767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd000103780 msgr2=0x7fd0000707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfff7fe700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd000103780 0x7fd0000707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfff7fe700 1 -- 172.21.15.31:0/746708767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff0005040 con 0x7fd000102de0 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfff7fe700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 0x7fd000070260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcff000b8b0 tx=0x7fcff0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfe7fff700 1 -- 172.21.15.31:0/746708767 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff000e040 con 0x7fd000102de0 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfe7fff700 1 -- 172.21.15.31:0/746708767 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcff0004290 con 0x7fd000102de0 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd000070f60 con 0x7fd000102de0 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fcfe7fff700 1 -- 172.21.15.31:0/746708767 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcff001c3f0 con 0x7fd000102de0 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.054+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00011a030 con 0x7fd000102de0 2024-04-24T23:19:05.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.055+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0001000f0 con 0x7fd000102de0 2024-04-24T23:19:05.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.056+0000 7fcfe7fff700 1 -- 172.21.15.31:0/746708767 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fcff0013420 con 0x7fd000102de0 2024-04-24T23:19:05.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.056+0000 7fcfe7fff700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcfe806e270 0x7fcfe8070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.057+0000 7fcffeffd700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcfe806e270 0x7fcfe8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.057+0000 7fcfe7fff700 1 -- 172.21.15.31:0/746708767 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcff008ef60 con 0x7fd000102de0 2024-04-24T23:19:05.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.057+0000 7fcffeffd700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcfe806e270 0x7fcfe8070720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd000071990 tx=0x7fcff400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:05.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.061+0000 7fcfe7fff700 1 -- 172.21.15.31:0/746708767 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcff00c3740 con 0x7fd000102de0 2024-04-24T23:19:05.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.320+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd000071800 con 0x7fd000102de0 2024-04-24T23:19:05.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.320+0000 7fcfe7fff700 1 -- 172.21.15.31:0/746708767 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1878 (secure 0 0 0) 0x7fcff0058190 con 0x7fd000102de0 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:19:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:19:05.325 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:19:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:19:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:19:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.322+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcfe806e270 msgr2=0x7fcfe8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.322+0000 7fd005e0e700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcfe806e270 0x7fcfe8070720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd000071990 tx=0x7fcff400b040 comp rx=0 tx=0).stop 2024-04-24T23:19:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 msgr2=0x7fd000070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 0x7fd000070260 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcff000b8b0 tx=0x7fcff0009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:05.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 shutdown_connections 2024-04-24T23:19:05.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd000102de0 0x7fd000070260 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcfe806e270 0x7fcfe8070720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 --2- 172.21.15.31:0/746708767 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd000103780 0x7fd0000707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 >> 172.21.15.31:0/746708767 conn(0x7fd0000f8990 msgr2=0x7fd0000fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:05.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 shutdown_connections 2024-04-24T23:19:05.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.323+0000 7fd005e0e700 1 -- 172.21.15.31:0/746708767 wait complete. 2024-04-24T23:19:05.328 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:19:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.427+0000 7f1322ab4700 1 -- 172.21.15.31:0/406992182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c102f60 msgr2=0x7f131c103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.427+0000 7f1322ab4700 1 --2- 172.21.15.31:0/406992182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c102f60 0x7f131c103330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1310004940 tx=0x7f1310009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:05.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.427+0000 7f1322ab4700 1 -- 172.21.15.31:0/406992182 shutdown_connections 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.427+0000 7f1322ab4700 1 --2- 172.21.15.31:0/406992182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c0fb520 0x7f131c0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 --2- 172.21.15.31:0/406992182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c102f60 0x7f131c103330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 -- 172.21.15.31:0/406992182 >> 172.21.15.31:0/406992182 conn(0x7f131c0714d0 msgr2=0x7f131c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 -- 172.21.15.31:0/406992182 shutdown_connections 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 -- 172.21.15.31:0/406992182 wait complete. 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 Processor -- start 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 -- start start 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c0fb520 0x7f131c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c102f60 0x7f131c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c114510 con 0x7f131c0fb520 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.428+0000 7f1322ab4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c10db30 con 0x7f131c102f60 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1320850700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c0fb520 0x7f131c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1320850700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c0fb520 0x7f131c113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38734/0 (socket says 172.21.15.31:38734) 2024-04-24T23:19:05.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1320850700 1 -- 172.21.15.31:0/1120411694 learned_addr learned my addr 172.21.15.31:0/1120411694 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:05.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f131bfff700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c102f60 0x7f131c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1320850700 1 -- 172.21.15.31:0/1120411694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c102f60 msgr2=0x7f131c113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1320850700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c102f60 0x7f131c113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1320850700 1 -- 172.21.15.31:0/1120411694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310005040 con 0x7f131c0fb520 2024-04-24T23:19:05.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f131bfff700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c102f60 0x7f131c113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:19:05.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1320850700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c0fb520 0x7f131c113980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1310009b30 tx=0x7f1310009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:05.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f13197fa700 1 -- 172.21.15.31:0/1120411694 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1310021070 con 0x7f131c0fb520 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f13197fa700 1 -- 172.21.15.31:0/1120411694 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1310013c00 con 0x7f131c0fb520 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c10ddb0 con 0x7f131c0fb520 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f13197fa700 1 -- 172.21.15.31:0/1120411694 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f131000e070 con 0x7f131c0fb520 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.429+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c10e290 con 0x7f131c0fb520 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.430+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c100250 con 0x7f131c0fb520 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.431+0000 7f13197fa700 1 -- 172.21.15.31:0/1120411694 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f131001cad0 con 0x7f131c0fb520 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.431+0000 7f13197fa700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f130c06e270 0x7f130c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.431+0000 7f13197fa700 1 -- 172.21.15.31:0/1120411694 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f131008f100 con 0x7f131c0fb520 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.432+0000 7f131bfff700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f130c06e270 0x7f130c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.432+0000 7f131bfff700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f130c06e270 0x7f130c070720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f131c10f220 tx=0x7f130800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:05.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.436+0000 7f13197fa700 1 -- 172.21.15.31:0/1120411694 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1310058330 con 0x7f131c0fb520 2024-04-24T23:19:05.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.665+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f131c109fe0 con 0x7f130c06e270 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.osunbp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3671551847' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:05.669 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:05 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/746708767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:19:05.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.667+0000 7f13197fa700 1 -- 172.21.15.31:0/1120411694 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f131c109fe0 con 0x7f130c06e270 2024-04-24T23:19:05.670 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:19:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:19:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:19:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:19:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:19:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:19:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:19:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:19:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "4/21 daemons upgraded", 2024-04-24T23:19:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mon daemons", 2024-04-24T23:19:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:19:05.671 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f130c06e270 msgr2=0x7f130c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f130c06e270 0x7f130c070720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f131c10f220 tx=0x7f130800b040 comp rx=0 tx=0).stop 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c0fb520 msgr2=0x7f131c113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c0fb520 0x7f131c113980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1310009b30 tx=0x7f1310009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 shutdown_connections 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f131c0fb520 0x7f131c113980 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f130c06e270 0x7f130c070720 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 --2- 172.21.15.31:0/1120411694 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f131c102f60 0x7f131c113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 >> 172.21.15.31:0/1120411694 conn(0x7f131c0714d0 msgr2=0x7f131c0fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 shutdown_connections 2024-04-24T23:19:05.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.670+0000 7f1322ab4700 1 -- 172.21.15.31:0/1120411694 wait complete. 2024-04-24T23:19:05.748 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: pgmap v28: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='client.44105 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.osunbp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3671551847' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:05.749 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:05 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/746708767' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:19:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 -- 172.21.15.31:0/979091002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 msgr2=0x7f3510070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 --2- 172.21.15.31:0/979091002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 0x7f3510070cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3500004970 tx=0x7f3500009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 -- 172.21.15.31:0/979091002 shutdown_connections 2024-04-24T23:19:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 --2- 172.21.15.31:0/979091002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 0x7f3510070cb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 --2- 172.21.15.31:0/979091002 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3510071f30 0x7f3510070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 -- 172.21.15.31:0/979091002 >> 172.21.15.31:0/979091002 conn(0x7f35100f8990 msgr2=0x7f35100fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 -- 172.21.15.31:0/979091002 shutdown_connections 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 -- 172.21.15.31:0/979091002 wait complete. 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 Processor -- start 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.776+0000 7f351694a700 1 -- start start 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f351694a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 0x7f3510119f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f351694a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3510071f30 0x7f351011a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f351694a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351011ab00 con 0x7f3510070840 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f351694a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35101140e0 con 0x7f3510071f30 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 0x7f3510119f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 0x7f3510119f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38750/0 (socket says 172.21.15.31:38750) 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350ffff700 1 -- 172.21.15.31:0/3774528847 learned_addr learned my addr 172.21.15.31:0/3774528847 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350f7fe700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3510071f30 0x7f351011a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350ffff700 1 -- 172.21.15.31:0/3774528847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3510071f30 msgr2=0x7f351011a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:05.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350ffff700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3510071f30 0x7f351011a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:05.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350ffff700 1 -- 172.21.15.31:0/3774528847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500005040 con 0x7f3510070840 2024-04-24T23:19:05.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350ffff700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 0x7f3510119f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34f800eb40 tx=0x7f34f800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:05.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350cff9700 1 -- 172.21.15.31:0/3774528847 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34f8007cc0 con 0x7f3510070840 2024-04-24T23:19:05.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350cff9700 1 -- 172.21.15.31:0/3774528847 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34f800f040 con 0x7f3510070840 2024-04-24T23:19:05.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3510114360 con 0x7f3510070840 2024-04-24T23:19:05.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.777+0000 7f350cff9700 1 -- 172.21.15.31:0/3774528847 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34f800ca10 con 0x7f3510070840 2024-04-24T23:19:05.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.778+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35101148a0 con 0x7f3510070840 2024-04-24T23:19:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.778+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3510071a40 con 0x7f3510070840 2024-04-24T23:19:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.779+0000 7f350cff9700 1 -- 172.21.15.31:0/3774528847 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f34f800cc80 con 0x7f3510070840 2024-04-24T23:19:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.779+0000 7f350cff9700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f34fc06e270 0x7f34fc070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.779+0000 7f350cff9700 1 -- 172.21.15.31:0/3774528847 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f34f808d920 con 0x7f3510070840 2024-04-24T23:19:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.779+0000 7f350f7fe700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f34fc06e270 0x7f34fc070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:05.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.780+0000 7f350f7fe700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f34fc06e270 0x7f34fc070720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3510115810 tx=0x7f3500009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:05.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:05.784+0000 7f350cff9700 1 -- 172.21.15.31:0/3774528847 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f34f805a6b0 con 0x7f3510070840 2024-04-24T23:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.075+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f351004c710 con 0x7f3510070840 2024-04-24T23:19:06.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.075+0000 7f350cff9700 1 -- 172.21.15.31:0/3774528847 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f34f805a240 con 0x7f3510070840 2024-04-24T23:19:06.078 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:19:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f34fc06e270 msgr2=0x7f34fc070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f34fc06e270 0x7f34fc070720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3510115810 tx=0x7f3500009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 msgr2=0x7f3510119f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 0x7f3510119f70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f34f800eb40 tx=0x7f34f800b040 comp rx=0 tx=0).stop 2024-04-24T23:19:06.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 shutdown_connections 2024-04-24T23:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3510070840 0x7f3510119f70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f34fc06e270 0x7f34fc070720 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 --2- 172.21.15.31:0/3774528847 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3510071f30 0x7f351011a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 >> 172.21.15.31:0/3774528847 conn(0x7f35100f8990 msgr2=0x7f35100f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 shutdown_connections 2024-04-24T23:19:06.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:06.077+0000 7f351694a700 1 -- 172.21.15.31:0/3774528847 wait complete. 2024-04-24T23:19:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:06 smithi031 ceph-mon[111404]: Reconfiguring mds.cephfs.smithi040.osunbp (monmap changed)... 2024-04-24T23:19:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:06 smithi031 ceph-mon[111404]: Reconfiguring daemon mds.cephfs.smithi040.osunbp on smithi040 2024-04-24T23:19:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:06 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3774528847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:19:06.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:06 smithi040 ceph-mon[83047]: Reconfiguring mds.cephfs.smithi040.osunbp (monmap changed)... 2024-04-24T23:19:06.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:06 smithi040 ceph-mon[83047]: Reconfiguring daemon mds.cephfs.smithi040.osunbp on smithi040 2024-04-24T23:19:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:06 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3774528847' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:19:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: pgmap v29: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all mon 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]: dispatch 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]': finished 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi040"}]: dispatch 2024-04-24T23:19:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi040"}]': finished 2024-04-24T23:19:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: pgmap v29: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all mon 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]: dispatch 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]': finished 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi040"}]: dispatch 2024-04-24T23:19:07.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi040"}]': finished 2024-04-24T23:19:09.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:09 smithi031 ceph-mon[111404]: pgmap v30: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:09.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:09 smithi031 ceph-mon[111404]: Upgrade: Updating crash.smithi031 (1/2) 2024-04-24T23:19:09.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:09.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:19:09.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:09.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:09 smithi031 ceph-mon[111404]: Deploying daemon crash.smithi031 on smithi031 2024-04-24T23:19:09.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:09 smithi040 ceph-mon[83047]: pgmap v30: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:09.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:09 smithi040 ceph-mon[83047]: Upgrade: Updating crash.smithi031 (1/2) 2024-04-24T23:19:09.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:09.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:19:09.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:09.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:09 smithi040 ceph-mon[83047]: Deploying daemon crash.smithi031 on smithi031 2024-04-24T23:19:11.590 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:11 smithi031 ceph-mon[111404]: pgmap v31: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:11 smithi040 ceph-mon[83047]: pgmap v31: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:13.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:12 smithi031 ceph-mon[111404]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:13.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:12 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:13.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:12 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:13.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:12 smithi040 ceph-mon[83047]: pgmap v32: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:13.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:12 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:13.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:12 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:14.885 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:14 smithi040 ceph-mon[83047]: Upgrade: Updating crash.smithi040 (2/2) 2024-04-24T23:19:14.885 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:14.886 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:19:14.886 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:14.886 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:14 smithi040 ceph-mon[83047]: Deploying daemon crash.smithi040 on smithi040 2024-04-24T23:19:14.886 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:14 smithi031 ceph-mon[111404]: Upgrade: Updating crash.smithi040 (2/2) 2024-04-24T23:19:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi040", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-24T23:19:15.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:15.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:14 smithi031 ceph-mon[111404]: Deploying daemon crash.smithi040 on smithi040 2024-04-24T23:19:15.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:15.704 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:15 smithi040 ceph-mon[83047]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:15 smithi031 ceph-mon[111404]: pgmap v33: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:17 smithi040 ceph-mon[83047]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:17 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:17 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:17.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:17 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:17 smithi031 ceph-mon[111404]: pgmap v34: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:17 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:17 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:17.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:17 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:19.905 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:19 smithi031 ceph-mon[111404]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:19.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:19 smithi040 ceph-mon[83047]: pgmap v35: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:21 smithi031 ceph-mon[111404]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:21 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:21 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:21 smithi040 ceph-mon[83047]: pgmap v36: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:21 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:21 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:22.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:22 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:22.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:22 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:22.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:22 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:22.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:22 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:22 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:22 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:22 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:22.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:22 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:23.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:23 smithi040 ceph-mon[83047]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:23 smithi031 ceph-mon[111404]: pgmap v37: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:24.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:24 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:24.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:24 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:24.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:24 smithi031 ceph-mon[111404]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:25.076 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:24 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:25.076 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:24 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:25.076 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:24 smithi040 ceph-mon[83047]: pgmap v38: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:26.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:26.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:27 smithi031 ceph-mon[111404]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:27.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:27 smithi040 ceph-mon[83047]: pgmap v39: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:28.392 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all crash 2024-04-24T23:19:28.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:28.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]: dispatch 2024-04-24T23:19:28.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]': finished 2024-04-24T23:19:28.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi040"}]: dispatch 2024-04-24T23:19:28.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi040"}]': finished 2024-04-24T23:19:28.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T23:19:28.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T23:19:28.393 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:28 smithi031 ceph-mon[111404]: Upgrade: osd.0 is safe to restart 2024-04-24T23:19:28.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all crash 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]: dispatch 2024-04-24T23:19:28.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]': finished 2024-04-24T23:19:28.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi040"}]: dispatch 2024-04-24T23:19:28.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi040"}]': finished 2024-04-24T23:19:28.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T23:19:28.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-24T23:19:28.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:28 smithi040 ceph-mon[83047]: Upgrade: osd.0 is safe to restart 2024-04-24T23:19:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:29 smithi031 ceph-mon[111404]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:19:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:29.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:29 smithi040 ceph-mon[83047]: pgmap v40: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:29.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:29.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:29.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-24T23:19:29.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:30.665 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:30 smithi031 ceph-mon[111404]: Upgrade: Updating osd.0 2024-04-24T23:19:30.665 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:30 smithi031 ceph-mon[111404]: Deploying daemon osd.0 on smithi031 2024-04-24T23:19:30.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:30 smithi040 ceph-mon[83047]: Upgrade: Updating osd.0 2024-04-24T23:19:30.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:30 smithi040 ceph-mon[83047]: Deploying daemon osd.0 on smithi031 2024-04-24T23:19:31.673 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:31 smithi031 ceph-mon[111404]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:31.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:31 smithi040 ceph-mon[83047]: pgmap v41: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:32.603 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:32 smithi031 systemd[1]: Stopping Ceph osd.0 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:32.999 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:32 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[46025]: 2024-04-24T23:19:32.601+0000 7f46127c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:19:32.999 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:32 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[46025]: 2024-04-24T23:19:32.601+0000 7f46127c2700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-24T23:19:32.999 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:32 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[46025]: 2024-04-24T23:19:32.601+0000 7f46127c2700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:19:33.590 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:33 smithi031 ceph-mon[111404]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:33.590 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:33 smithi031 ceph-mon[111404]: osd.0 marked itself down and dead 2024-04-24T23:19:33.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:33 smithi040 ceph-mon[83047]: pgmap v42: 65 pgs: 65 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:33.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:33 smithi040 ceph-mon[83047]: osd.0 marked itself down and dead 2024-04-24T23:19:34.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:33 smithi031 podman[122600]: 2024-04-24 23:19:33.988050745 +0000 UTC m=+1.512976750 container died 2463ed4dbf7f0d38a1882097538d239c04ee486a98ff2001fc3a9fd738ecfc08 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:34.583 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:34 smithi031 ceph-mon[111404]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:19:34.583 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:34 smithi031 ceph-mon[111404]: osdmap e50: 6 total, 5 up, 6 in 2024-04-24T23:19:34.583 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:34 smithi031 podman[122600]: 2024-04-24 23:19:34.351084774 +0000 UTC m=+1.876010780 container cleanup 2463ed4dbf7f0d38a1882097538d239c04ee486a98ff2001fc3a9fd738ecfc08 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, ceph=True, GIT_BRANCH=HEAD, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-24T23:19:34.583 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:34 smithi031 bash[122600]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0 2024-04-24T23:19:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:34 smithi040 ceph-mon[83047]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:19:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:34 smithi040 ceph-mon[83047]: osdmap e50: 6 total, 5 up, 6 in 2024-04-24T23:19:34.971 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:34 smithi031 podman[122627]: 2024-04-24 23:19:34.644788183 +0000 UTC m=+0.655083758 container remove 2463ed4dbf7f0d38a1882097538d239c04ee486a98ff2001fc3a9fd738ecfc08 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-24T23:19:35.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:34 smithi031 podman[122721]: 2024-04-24 23:19:34.873874405 +0000 UTC m=+0.018766534 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:35.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:34 smithi031 podman[122721]: 2024-04-24 23:19:34.970774486 +0000 UTC m=+0.115666592 container create c8746da9e7af8f5853d1b805a859e6a5adf24f1016b52a21570023efe3c465cc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:35.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:35 smithi031 podman[122721]: 2024-04-24 23:19:35.173643561 +0000 UTC m=+0.318535668 container init c8746da9e7af8f5853d1b805a859e6a5adf24f1016b52a21570023efe3c465cc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-24T23:19:35.250 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:35 smithi031 podman[122721]: 2024-04-24 23:19:35.17734565 +0000 UTC m=+0.322237755 container start c8746da9e7af8f5853d1b805a859e6a5adf24f1016b52a21570023efe3c465cc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:19:35.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:35 smithi031 ceph-mon[111404]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:35.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:35 smithi031 ceph-mon[111404]: osdmap e51: 6 total, 5 up, 6 in 2024-04-24T23:19:35.750 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:35 smithi031 podman[122721]: 2024-04-24 23:19:35.281631104 +0000 UTC m=+0.426523221 container attach c8746da9e7af8f5853d1b805a859e6a5adf24f1016b52a21570023efe3c465cc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:19:35.750 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:35 smithi031 podman[122721]: 2024-04-24 23:19:35.475817472 +0000 UTC m=+0.620709580 container died c8746da9e7af8f5853d1b805a859e6a5adf24f1016b52a21570023efe3c465cc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T23:19:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:35 smithi040 ceph-mon[83047]: pgmap v44: 65 pgs: 9 stale+active+clean, 56 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:19:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:35 smithi040 ceph-mon[83047]: osdmap e51: 6 total, 5 up, 6 in 2024-04-24T23:19:36.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.198+0000 7f765b221700 1 -- 172.21.15.31:0/213868381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 msgr2=0x7f7654103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.198+0000 7f765b221700 1 --2- 172.21.15.31:0/213868381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 0x7f7654103220 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7648004940 tx=0x7f7648009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.198+0000 7f765b221700 1 -- 172.21.15.31:0/213868381 shutdown_connections 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.198+0000 7f765b221700 1 --2- 172.21.15.31:0/213868381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654065640 0x7f7654065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.198+0000 7f765b221700 1 --2- 172.21.15.31:0/213868381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 0x7f7654103220 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.198+0000 7f765b221700 1 -- 172.21.15.31:0/213868381 >> 172.21.15.31:0/213868381 conn(0x7f7654071d00 msgr2=0x7f7654072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.198+0000 7f765b221700 1 -- 172.21.15.31:0/213868381 shutdown_connections 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.198+0000 7f765b221700 1 -- 172.21.15.31:0/213868381 wait complete. 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f765b221700 1 Processor -- start 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f765b221700 1 -- start start 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f765b221700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654065640 0x7f7654115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f765b221700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 0x7f76541162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f765b221700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654116920 con 0x7f7654102e50 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f765b221700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765410fcf0 con 0x7f7654065640 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f7653fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 0x7f76541162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f7653fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 0x7f76541162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39832/0 (socket says 172.21.15.31:39832) 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f7653fff700 1 -- 172.21.15.31:0/2431783057 learned_addr learned my addr 172.21.15.31:0/2431783057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.199+0000 7f7658fbd700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654065640 0x7f7654115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f7653fff700 1 -- 172.21.15.31:0/2431783057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654065640 msgr2=0x7f7654115d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f7653fff700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654065640 0x7f7654115d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f7653fff700 1 -- 172.21.15.31:0/2431783057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7648005040 con 0x7f7654102e50 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f7653fff700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 0x7f76541162d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7640002a50 tx=0x7f76400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f76517fa700 1 -- 172.21.15.31:0/2431783057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7640007cc0 con 0x7f7654102e50 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f76517fa700 1 -- 172.21.15.31:0/2431783057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f764000f040 con 0x7f7654102e50 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f76517fa700 1 -- 172.21.15.31:0/2431783057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764000da30 con 0x7f7654102e50 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765410ffd0 con 0x7f7654102e50 2024-04-24T23:19:36.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.200+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7654110590 con 0x7f7654102e50 2024-04-24T23:19:36.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.202+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7654107260 con 0x7f7654102e50 2024-04-24T23:19:36.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.202+0000 7f76517fa700 1 -- 172.21.15.31:0/2431783057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f764000db90 con 0x7f7654102e50 2024-04-24T23:19:36.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.202+0000 7f76517fa700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f764406e260 0x7f7644070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.202+0000 7f76517fa700 1 -- 172.21.15.31:0/2431783057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f764008da30 con 0x7f7654102e50 2024-04-24T23:19:36.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.202+0000 7f7658fbd700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f764406e260 0x7f7644070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.203+0000 7f7658fbd700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f764406e260 0x7f7644070710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7648004f90 tx=0x7f7648009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:36.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.207+0000 7f76517fa700 1 -- 172.21.15.31:0/2431783057 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f764005a790 con 0x7f7654102e50 2024-04-24T23:19:36.328 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 podman[122747]: 2024-04-24 23:19:36.056061233 +0000 UTC m=+0.574759675 container remove c8746da9e7af8f5853d1b805a859e6a5adf24f1016b52a21570023efe3c465cc (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-24T23:19:36.328 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-24T23:19:36.328 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 systemd[1]: Stopped Ceph osd.0 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:19:36.328 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.0.service: Consumed 1min 4.076s CPU time. 2024-04-24T23:19:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.470+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7654109ef0 con 0x7f764406e260 2024-04-24T23:19:36.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.472+0000 7f76517fa700 1 -- 172.21.15.31:0/2431783057 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7654109ef0 con 0x7f764406e260 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f764406e260 msgr2=0x7f7644070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f764406e260 0x7f7644070710 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7648004f90 tx=0x7f7648009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 msgr2=0x7f76541162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 0x7f76541162d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7640002a50 tx=0x7f76400046b0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 shutdown_connections 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f764406e260 0x7f7644070710 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7654065640 0x7f7654115d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 --2- 172.21.15.31:0/2431783057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7654102e50 0x7f76541162d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 >> 172.21.15.31:0/2431783057 conn(0x7f7654071d00 msgr2=0x7f76540fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 shutdown_connections 2024-04-24T23:19:36.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.474+0000 7f765b221700 1 -- 172.21.15.31:0/2431783057 wait complete. 2024-04-24T23:19:36.486 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:19:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.582+0000 7f0f36380700 1 -- 172.21.15.31:0/2181424627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 msgr2=0x7f0f300fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.582+0000 7f0f36380700 1 --2- 172.21.15.31:0/2181424627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 0x7f0f300fb3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0f24004940 tx=0x7f0f24009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:36.584 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:36 smithi031 ceph-mon[111404]: Health check failed: Degraded data redundancy: 40/225 objects degraded (17.778%), 18 pgs degraded (PG_DEGRADED) 2024-04-24T23:19:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.582+0000 7f0f36380700 1 -- 172.21.15.31:0/2181424627 shutdown_connections 2024-04-24T23:19:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.582+0000 7f0f36380700 1 --2- 172.21.15.31:0/2181424627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 0x7f0f300fb3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.582+0000 7f0f36380700 1 --2- 172.21.15.31:0/2181424627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f30102e20 0x7f0f301031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.582+0000 7f0f36380700 1 -- 172.21.15.31:0/2181424627 >> 172.21.15.31:0/2181424627 conn(0x7f0f30072380 msgr2=0x7f0f30072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:36.585 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 systemd[1]: Starting Ceph osd.0 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:19:36.585 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 podman[122874]: 2024-04-24 23:19:36.455410381 +0000 UTC m=+0.018743163 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:36.585 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 podman[122874]: 2024-04-24 23:19:36.553666483 +0000 UTC m=+0.116999274 container create d23570262bdf206feba52269ac075b7d0d1ffb31e5711392e64b758f70e78b33 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-24T23:19:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.582+0000 7f0f36380700 1 -- 172.21.15.31:0/2181424627 shutdown_connections 2024-04-24T23:19:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.582+0000 7f0f36380700 1 -- 172.21.15.31:0/2181424627 wait complete. 2024-04-24T23:19:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f36380700 1 Processor -- start 2024-04-24T23:19:36.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f36380700 1 -- start start 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f36380700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f30102e20 0x7f0f30113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f36380700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 0x7f0f30113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f36380700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301144d0 con 0x7f0f301037c0 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f36380700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3010dad0 con 0x7f0f30102e20 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f2f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 0x7f0f30113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f2f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 0x7f0f30113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39850/0 (socket says 172.21.15.31:39850) 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.583+0000 7f0f2f7fe700 1 -- 172.21.15.31:0/3376836353 learned_addr learned my addr 172.21.15.31:0/3376836353 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f2ffff700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f30102e20 0x7f0f30113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f2f7fe700 1 -- 172.21.15.31:0/3376836353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f30102e20 msgr2=0x7f0f30113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f2f7fe700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f30102e20 0x7f0f30113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f2f7fe700 1 -- 172.21.15.31:0/3376836353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24005040 con 0x7f0f301037c0 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f2f7fe700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 0x7f0f30113e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009180 tx=0x7f0f24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f2cff9700 1 -- 172.21.15.31:0/3376836353 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f2400e040 con 0x7f0f301037c0 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3010dd50 con 0x7f0f301037c0 2024-04-24T23:19:36.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f2cff9700 1 -- 172.21.15.31:0/3376836353 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f24013420 con 0x7f0f301037c0 2024-04-24T23:19:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f2cff9700 1 -- 172.21.15.31:0/3376836353 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f2401c610 con 0x7f0f301037c0 2024-04-24T23:19:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.584+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3010e290 con 0x7f0f301037c0 2024-04-24T23:19:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.585+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f30100160 con 0x7f0f301037c0 2024-04-24T23:19:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.586+0000 7f0f2cff9700 1 -- 172.21.15.31:0/3376836353 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0f24013a00 con 0x7f0f301037c0 2024-04-24T23:19:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.586+0000 7f0f2cff9700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f0f1806e270 0x7f0f18070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.586+0000 7f0f2cff9700 1 -- 172.21.15.31:0/3376836353 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0f2408ee50 con 0x7f0f301037c0 2024-04-24T23:19:36.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.586+0000 7f0f2ffff700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f0f1806e270 0x7f0f18070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.587+0000 7f0f2ffff700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f0f1806e270 0x7f0f18070720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009fd0 tx=0x7f0f2000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:36.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.592+0000 7f0f2cff9700 1 -- 172.21.15.31:0/3376836353 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0f2405bc60 con 0x7f0f301037c0 2024-04-24T23:19:36.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.824+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3010ef90 con 0x7f0f1806e270 2024-04-24T23:19:36.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.825+0000 7f0f2cff9700 1 -- 172.21.15.31:0/3376836353 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0f3010ef90 con 0x7f0f1806e270 2024-04-24T23:19:36.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f0f1806e270 msgr2=0x7f0f18070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f0f1806e270 0x7f0f18070720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009fd0 tx=0x7f0f2000b040 comp rx=0 tx=0).stop 2024-04-24T23:19:36.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 msgr2=0x7f0f30113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 0x7f0f30113e80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009180 tx=0x7f0f24009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:36.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 shutdown_connections 2024-04-24T23:19:36.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f0f1806e270 0x7f0f18070720 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f30102e20 0x7f0f30113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 --2- 172.21.15.31:0/3376836353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f301037c0 0x7f0f30113e80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 >> 172.21.15.31:0/3376836353 conn(0x7f0f30072380 msgr2=0x7f0f300faba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:36.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 shutdown_connections 2024-04-24T23:19:36.830 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.827+0000 7f0f36380700 1 -- 172.21.15.31:0/3376836353 wait complete. 2024-04-24T23:19:36.842 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 podman[122874]: 2024-04-24 23:19:36.840778285 +0000 UTC m=+0.404111079 container init d23570262bdf206feba52269ac075b7d0d1ffb31e5711392e64b758f70e78b33 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-04-24T23:19:36.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:36 smithi040 ceph-mon[83047]: Health check failed: Degraded data redundancy: 40/225 objects degraded (17.778%), 18 pgs degraded (PG_DEGRADED) 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.937+0000 7f459575b700 1 -- 172.21.15.31:0/1432099054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 msgr2=0x7f4590109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.937+0000 7f459575b700 1 --2- 172.21.15.31:0/1432099054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 0x7f4590109350 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4580004940 tx=0x7f4580009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.937+0000 7f459575b700 1 -- 172.21.15.31:0/1432099054 shutdown_connections 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.937+0000 7f459575b700 1 --2- 172.21.15.31:0/1432099054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 0x7f4590109350 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.937+0000 7f459575b700 1 --2- 172.21.15.31:0/1432099054 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45900fc790 0x7f45900fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.937+0000 7f459575b700 1 -- 172.21.15.31:0/1432099054 >> 172.21.15.31:0/1432099054 conn(0x7f4590071d00 msgr2=0x7f4590072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.937+0000 7f459575b700 1 -- 172.21.15.31:0/1432099054 shutdown_connections 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.937+0000 7f459575b700 1 -- 172.21.15.31:0/1432099054 wait complete. 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f459575b700 1 Processor -- start 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f459575b700 1 -- start start 2024-04-24T23:19:36.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f459575b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45900fc790 0x7f4590115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f459575b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 0x7f4590116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f459575b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45901169c0 con 0x7f45900fd0a0 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f459575b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459010fd90 con 0x7f45900fc790 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f458e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 0x7f4590116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f458e7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 0x7f4590116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39876/0 (socket says 172.21.15.31:39876) 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f458e7fc700 1 -- 172.21.15.31:0/3343243490 learned_addr learned my addr 172.21.15.31:0/3343243490 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f458effd700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45900fc790 0x7f4590115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f458e7fc700 1 -- 172.21.15.31:0/3343243490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45900fc790 msgr2=0x7f4590115e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f458e7fc700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45900fc790 0x7f4590115e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.938+0000 7f458e7fc700 1 -- 172.21.15.31:0/3343243490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4580005040 con 0x7f45900fd0a0 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.939+0000 7f458e7fc700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 0x7f4590116370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f458000b3b0 tx=0x7f4580009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:36.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.939+0000 7f45777fe700 1 -- 172.21.15.31:0/3343243490 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458000e040 con 0x7f45900fd0a0 2024-04-24T23:19:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.939+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4590110010 con 0x7f45900fd0a0 2024-04-24T23:19:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.939+0000 7f45777fe700 1 -- 172.21.15.31:0/3343243490 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4580013420 con 0x7f45900fd0a0 2024-04-24T23:19:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.939+0000 7f45777fe700 1 -- 172.21.15.31:0/3343243490 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f458001c630 con 0x7f45900fd0a0 2024-04-24T23:19:36.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.939+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4590110550 con 0x7f45900fd0a0 2024-04-24T23:19:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.940+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45900fc330 con 0x7f45900fd0a0 2024-04-24T23:19:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.940+0000 7f45777fe700 1 -- 172.21.15.31:0/3343243490 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4580013590 con 0x7f45900fd0a0 2024-04-24T23:19:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.941+0000 7f45777fe700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f457806e260 0x7f4578070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.941+0000 7f45777fe700 1 -- 172.21.15.31:0/3343243490 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f458008ee30 con 0x7f45900fd0a0 2024-04-24T23:19:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.941+0000 7f458effd700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f457806e260 0x7f4578070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:36.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.942+0000 7f458effd700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f457806e260 0x7f4578070710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4584004f40 tx=0x7f458400b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:36.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:36.946+0000 7f45777fe700 1 -- 172.21.15.31:0/3343243490 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f458005bc30 con 0x7f45900fd0a0 2024-04-24T23:19:37.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.166+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f459010c170 con 0x7f457806e260 2024-04-24T23:19:37.168 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 podman[122874]: 2024-04-24 23:19:36.846341176 +0000 UTC m=+0.409673966 container start d23570262bdf206feba52269ac075b7d0d1ffb31e5711392e64b758f70e78b33 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-24T23:19:37.168 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:36 smithi031 podman[122874]: 2024-04-24 23:19:36.965627263 +0000 UTC m=+0.528960067 container attach d23570262bdf206feba52269ac075b7d0d1ffb31e5711392e64b758f70e78b33 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:19:37.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.176+0000 7f45777fe700 1 -- 172.21.15.31:0/3343243490 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f459010c170 con 0x7f457806e260 2024-04-24T23:19:37.178 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (2m) 14s ago 14m 26.2M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (25s) 14s ago 14m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (20s) 16s ago 13m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (2m) 14s ago 14m 74.9M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (8m) 14s ago 8m 138M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (8m) 14s ago 8m 14.1M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (8m) 16s ago 8m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (8m) 16s ago 8m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (4m) 14s ago 15m 495M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (3m) 16s ago 13m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (92s) 14s ago 15m 41.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (66s) 16s ago 13m 32.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (3m) 14s ago 14m 16.9M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (3m) 16s ago 13m 14.7M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:19:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (12m) 14s ago 12m 223M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 2463ed4dbf7f 2024-04-24T23:19:37.180 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (12m) 14s ago 12m 239M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:19:37.180 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (11m) 14s ago 11m 199M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:19:37.180 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (11m) 16s ago 11m 278M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:19:37.180 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (10m) 16s ago 10m 246M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:19:37.180 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (10m) 16s ago 10m 225M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:19:37.180 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 14s ago 13m 58.6M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:19:37.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f457806e260 msgr2=0x7f4578070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f457806e260 0x7f4578070710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4584004f40 tx=0x7f458400b040 comp rx=0 tx=0).stop 2024-04-24T23:19:37.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 msgr2=0x7f4590116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 0x7f4590116370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f458000b3b0 tx=0x7f4580009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:37.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 shutdown_connections 2024-04-24T23:19:37.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f457806e260 0x7f4578070710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f45900fc790 0x7f4590115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 --2- 172.21.15.31:0/3343243490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45900fd0a0 0x7f4590116370 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 >> 172.21.15.31:0/3343243490 conn(0x7f4590071d00 msgr2=0x7f45900f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:37.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 shutdown_connections 2024-04-24T23:19:37.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.179+0000 7f459575b700 1 -- 172.21.15.31:0/3343243490 wait complete. 2024-04-24T23:19:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.286+0000 7f36cfbed700 1 -- 172.21.15.31:0/4010273793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c81037a0 msgr2=0x7f36c80fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.286+0000 7f36cfbed700 1 --2- 172.21.15.31:0/4010273793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c81037a0 0x7f36c80fce10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f36b8004940 tx=0x7f36b8009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.286+0000 7f36cfbed700 1 -- 172.21.15.31:0/4010273793 shutdown_connections 2024-04-24T23:19:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.286+0000 7f36cfbed700 1 --2- 172.21.15.31:0/4010273793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c81037a0 0x7f36c80fce10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.286+0000 7f36cfbed700 1 --2- 172.21.15.31:0/4010273793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c8102e00 0x7f36c81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.286+0000 7f36cfbed700 1 -- 172.21.15.31:0/4010273793 >> 172.21.15.31:0/4010273793 conn(0x7f36c80f89b0 msgr2=0x7f36c80fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cfbed700 1 -- 172.21.15.31:0/4010273793 shutdown_connections 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cfbed700 1 -- 172.21.15.31:0/4010273793 wait complete. 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cfbed700 1 Processor -- start 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cfbed700 1 -- start start 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cfbed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c8102e00 0x7f36c8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cfbed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c81037a0 0x7f36c80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cfbed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c80721b0 con 0x7f36c8102e00 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cfbed700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c8070ce0 con 0x7f36c81037a0 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.287+0000 7f36cd989700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c8102e00 0x7f36c8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cd989700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c8102e00 0x7f36c8070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39900/0 (socket says 172.21.15.31:39900) 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cd989700 1 -- 172.21.15.31:0/870204215 learned_addr learned my addr 172.21.15.31:0/870204215 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cd188700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c81037a0 0x7f36c80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cd989700 1 -- 172.21.15.31:0/870204215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c81037a0 msgr2=0x7f36c80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cd989700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c81037a0 0x7f36c80707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cd989700 1 -- 172.21.15.31:0/870204215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b8005040 con 0x7f36c8102e00 2024-04-24T23:19:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cd188700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c81037a0 0x7f36c80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:19:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cd989700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c8102e00 0x7f36c8070260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36c4002a50 tx=0x7f36c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36be7fc700 1 -- 172.21.15.31:0/870204215 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36c4007cc0 con 0x7f36c8102e00 2024-04-24T23:19:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36be7fc700 1 -- 172.21.15.31:0/870204215 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36c400f040 con 0x7f36c8102e00 2024-04-24T23:19:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c8070fc0 con 0x7f36c8102e00 2024-04-24T23:19:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36be7fc700 1 -- 172.21.15.31:0/870204215 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36c4009640 con 0x7f36c8102e00 2024-04-24T23:19:37.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.288+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c811a170 con 0x7f36c8102e00 2024-04-24T23:19:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.289+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c81052d0 con 0x7f36c8102e00 2024-04-24T23:19:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.290+0000 7f36be7fc700 1 -- 172.21.15.31:0/870204215 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f36c40097a0 con 0x7f36c8102e00 2024-04-24T23:19:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.291+0000 7f36be7fc700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f36b406e270 0x7f36b4070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.291+0000 7f36be7fc700 1 -- 172.21.15.31:0/870204215 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f36c408da80 con 0x7f36c8102e00 2024-04-24T23:19:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.291+0000 7f36cd188700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f36b406e270 0x7f36b4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:37.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.291+0000 7f36cd188700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f36b406e270 0x7f36b4070720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f36b800b3b0 tx=0x7f36b8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:37.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.296+0000 7f36be7fc700 1 -- 172.21.15.31:0/870204215 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f36c4056cb0 con 0x7f36c8102e00 2024-04-24T23:19:37.499 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate[122913]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T23:19:37.499 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate[122913]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T23:19:37.499 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 bash[122874]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T23:19:37.499 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 bash[122874]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 bash[122874]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 bash[122874]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 bash[122874]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 bash[122874]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 bash[122874]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate[122913]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate[122913]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate[122913]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate[122913]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate[122913]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-24T23:19:37.500 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 podman[122874]: 2024-04-24 23:19:37.342476668 +0000 UTC m=+0.905809447 container died d23570262bdf206feba52269ac075b7d0d1ffb31e5711392e64b758f70e78b33 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T23:19:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.604+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f36c8071740 con 0x7f36c8102e00 2024-04-24T23:19:37.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.604+0000 7f36be7fc700 1 -- 172.21.15.31:0/870204215 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f36c405a370 con 0x7f36c8102e00 2024-04-24T23:19:37.606 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:19:37.607 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:19:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:19:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:19:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:19:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 9, 2024-04-24T23:19:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T23:19:37.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:19:37.608 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f36b406e270 msgr2=0x7f36b4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f36b406e270 0x7f36b4070720 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f36b800b3b0 tx=0x7f36b8009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c8102e00 msgr2=0x7f36c8070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c8102e00 0x7f36c8070260 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36c4002a50 tx=0x7f36c40046b0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 shutdown_connections 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36c8102e00 0x7f36c8070260 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f36b406e270 0x7f36b4070720 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 --2- 172.21.15.31:0/870204215 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f36c81037a0 0x7f36c80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 >> 172.21.15.31:0/870204215 conn(0x7f36c80f89b0 msgr2=0x7f36c81021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 shutdown_connections 2024-04-24T23:19:37.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.607+0000 7f36cfbed700 1 -- 172.21.15.31:0/870204215 wait complete. 2024-04-24T23:19:37.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:37 smithi040 ceph-mon[83047]: pgmap v46: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:37.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:37 smithi040 ceph-mon[83047]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:37.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.712+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/69373026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4102df0 msgr2=0x7fb7f41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.712+0000 7fb7fa8c5700 1 --2- 172.21.15.31:0/69373026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4102df0 0x7fb7f41031c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4004940 tx=0x7fb7e4009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:37.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.712+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/69373026 shutdown_connections 2024-04-24T23:19:37.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.712+0000 7fb7fa8c5700 1 --2- 172.21.15.31:0/69373026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f4103790 0x7fb7f40fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.712+0000 7fb7fa8c5700 1 --2- 172.21.15.31:0/69373026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4102df0 0x7fb7f41031c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.712+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/69373026 >> 172.21.15.31:0/69373026 conn(0x7fb7f40f8980 msgr2=0x7fb7f40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:37.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.712+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/69373026 shutdown_connections 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.712+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/69373026 wait complete. 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7fa8c5700 1 Processor -- start 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7fa8c5700 1 -- start start 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7fa8c5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f4102df0 0x7fb7f4113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7fa8c5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4103790 0x7fb7f41140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7fa8c5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f4114710 con 0x7fb7f4103790 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7fa8c5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f410db20 con 0x7fb7f4102df0 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4103790 0x7fb7f41140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7f37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4103790 0x7fb7f41140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39918/0 (socket says 172.21.15.31:39918) 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7f37fe700 1 -- 172.21.15.31:0/4046606103 learned_addr learned my addr 172.21.15.31:0/4046606103 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7f3fff700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f4102df0 0x7fb7f4113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:37.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7f37fe700 1 -- 172.21.15.31:0/4046606103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f4102df0 msgr2=0x7fb7f4113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7f37fe700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f4102df0 0x7fb7f4113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.713+0000 7fb7f37fe700 1 -- 172.21.15.31:0/4046606103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e4005040 con 0x7fb7f4103790 2024-04-24T23:19:37.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.714+0000 7fb7f37fe700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4103790 0x7fb7f41140c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8002a50 tx=0x7fb7e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:37.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.714+0000 7fb7f0ff9700 1 -- 172.21.15.31:0/4046606103 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e8007cc0 con 0x7fb7f4103790 2024-04-24T23:19:37.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.714+0000 7fb7f0ff9700 1 -- 172.21.15.31:0/4046606103 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7e800f040 con 0x7fb7f4103790 2024-04-24T23:19:37.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.714+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f410de00 con 0x7fb7f4103790 2024-04-24T23:19:37.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.714+0000 7fb7f0ff9700 1 -- 172.21.15.31:0/4046606103 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e800d9c0 con 0x7fb7f4103790 2024-04-24T23:19:37.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.714+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f410e3c0 con 0x7fb7f4103790 2024-04-24T23:19:37.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.715+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f4100100 con 0x7fb7f4103790 2024-04-24T23:19:37.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.715+0000 7fb7f0ff9700 1 -- 172.21.15.31:0/4046606103 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fb7e8009730 con 0x7fb7f4103790 2024-04-24T23:19:37.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.716+0000 7fb7f0ff9700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb7dc06e260 0x7fb7dc070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:37.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.716+0000 7fb7f0ff9700 1 -- 172.21.15.31:0/4046606103 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb7e808d950 con 0x7fb7f4103790 2024-04-24T23:19:37.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.716+0000 7fb7f3fff700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb7dc06e260 0x7fb7dc070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:37.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.716+0000 7fb7f3fff700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb7dc06e260 0x7fb7dc070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b3b0 tx=0x7fb7e4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:37.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.721+0000 7fb7f0ff9700 1 -- 172.21.15.31:0/4046606103 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb7e8056b80 con 0x7fb7f4103790 2024-04-24T23:19:37.937 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-mon[111404]: pgmap v46: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:37.937 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:37 smithi031 ceph-mon[111404]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:37.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.978+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb7f410f1d0 con 0x7fb7f4103790 2024-04-24T23:19:37.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.979+0000 7fb7f0ff9700 1 -- 172.21.15.31:0/4046606103 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1878 (secure 0 0 0) 0x7fb7e805a240 con 0x7fb7f4103790 2024-04-24T23:19:37.981 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:19:37.981 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:19:37.981 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:19:37.981 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:19:37.981 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:19:37.981 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:19:37.982 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:19:37.983 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:19:37.984 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:19:37.984 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:19:37.984 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:19:37.984 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:19:37.984 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:19:37.984 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:19:37.984 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb7dc06e260 msgr2=0x7fb7dc070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb7dc06e260 0x7fb7dc070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400b3b0 tx=0x7fb7e4009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4103790 msgr2=0x7fb7f41140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4103790 0x7fb7f41140c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8002a50 tx=0x7fb7e8004600 comp rx=0 tx=0).stop 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 shutdown_connections 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb7dc06e260 0x7fb7dc070710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7f4102df0 0x7fb7f4113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 --2- 172.21.15.31:0/4046606103 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7f4103790 0x7fb7f41140c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.981+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 >> 172.21.15.31:0/4046606103 conn(0x7fb7f40f8980 msgr2=0x7fb7f41073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.982+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 shutdown_connections 2024-04-24T23:19:37.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:37.982+0000 7fb7fa8c5700 1 -- 172.21.15.31:0/4046606103 wait complete. 2024-04-24T23:19:37.986 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:19:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 -- 172.21.15.31:0/98286710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048102e50 msgr2=0x7f8048103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 --2- 172.21.15.31:0/98286710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048102e50 0x7f8048103220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8038004940 tx=0x7f8038009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 -- 172.21.15.31:0/98286710 shutdown_connections 2024-04-24T23:19:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 --2- 172.21.15.31:0/98286710 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8048065640 0x7f8048065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 --2- 172.21.15.31:0/98286710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048102e50 0x7f8048103220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 -- 172.21.15.31:0/98286710 >> 172.21.15.31:0/98286710 conn(0x7f8048071d00 msgr2=0x7f8048072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 -- 172.21.15.31:0/98286710 shutdown_connections 2024-04-24T23:19:38.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 -- 172.21.15.31:0/98286710 wait complete. 2024-04-24T23:19:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.097+0000 7f804d189700 1 Processor -- start 2024-04-24T23:19:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f804d189700 1 -- start start 2024-04-24T23:19:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f804d189700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048065640 0x7f80480fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f804d189700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8048102e50 0x7f80480fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f804d189700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80480fc880 con 0x7f8048065640 2024-04-24T23:19:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f804d189700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80480fc9c0 con 0x7f8048102e50 2024-04-24T23:19:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f8046d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048065640 0x7f80480fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f8046d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048065640 0x7f80480fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39936/0 (socket says 172.21.15.31:39936) 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f8046d9d700 1 -- 172.21.15.31:0/1381707866 learned_addr learned my addr 172.21.15.31:0/1381707866 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f804659c700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8048102e50 0x7f80480fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f8046d9d700 1 -- 172.21.15.31:0/1381707866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8048102e50 msgr2=0x7f80480fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f8046d9d700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8048102e50 0x7f80480fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f8046d9d700 1 -- 172.21.15.31:0/1381707866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038005040 con 0x7f8048065640 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.098+0000 7f8046d9d700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048065640 0x7f80480fbdb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f803800b8b0 tx=0x7f8038009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.099+0000 7f802f7fe700 1 -- 172.21.15.31:0/1381707866 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f803800e040 con 0x7f8048065640 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.099+0000 7f802f7fe700 1 -- 172.21.15.31:0/1381707866 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8038004290 con 0x7f8048065640 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.099+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80480fcc10 con 0x7f8048065640 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.099+0000 7f802f7fe700 1 -- 172.21.15.31:0/1381707866 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f803801c3f0 con 0x7f8048065640 2024-04-24T23:19:38.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.099+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804811a2b0 con 0x7f8048065640 2024-04-24T23:19:38.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.100+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8048100100 con 0x7f8048065640 2024-04-24T23:19:38.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.100+0000 7f802f7fe700 1 -- 172.21.15.31:0/1381707866 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f8038013420 con 0x7f8048065640 2024-04-24T23:19:38.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.101+0000 7f802f7fe700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f803006e260 0x7f8030070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.101+0000 7f802f7fe700 1 -- 172.21.15.31:0/1381707866 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f803808ef60 con 0x7f8048065640 2024-04-24T23:19:38.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.101+0000 7f804659c700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f803006e260 0x7f8030070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.101+0000 7f804659c700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f803006e260 0x7f8030070710 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f80480fd6f0 tx=0x7f803c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.105+0000 7f802f7fe700 1 -- 172.21.15.31:0/1381707866 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f80380c3740 con 0x7f8048065640 2024-04-24T23:19:38.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:37 smithi031 podman[123083]: 2024-04-24 23:19:37.936527438 +0000 UTC m=+0.585207417 container remove d23570262bdf206feba52269ac075b7d0d1ffb31e5711392e64b758f70e78b33 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-24T23:19:38.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:38 smithi031 podman[123168]: 2024-04-24 23:19:38.059298519 +0000 UTC m=+0.018676148 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:19:38.250 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:38 smithi031 podman[123168]: 2024-04-24 23:19:38.179466299 +0000 UTC m=+0.138843918 container create d216357cdbb6e7e1b1463f83376b7d6e5fd8abdca3116e23b190bb462512cc62 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-24T23:19:38.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.343+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f804804bed0 con 0x7f803006e260 2024-04-24T23:19:38.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.344+0000 7f802f7fe700 1 -- 172.21.15.31:0/1381707866 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f804804bed0 con 0x7f803006e260 2024-04-24T23:19:38.346 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:19:38.347 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:19:38.347 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:19:38.347 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:19:38.347 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:19:38.347 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:19:38.347 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:19:38.348 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:19:38.348 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:19:38.348 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "6/21 daemons upgraded", 2024-04-24T23:19:38.348 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T23:19:38.348 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:19:38.348 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:19:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f803006e260 msgr2=0x7f8030070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f803006e260 0x7f8030070710 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f80480fd6f0 tx=0x7f803c00b040 comp rx=0 tx=0).stop 2024-04-24T23:19:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048065640 msgr2=0x7f80480fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048065640 0x7f80480fbdb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f803800b8b0 tx=0x7f8038009e30 comp rx=0 tx=0).stop 2024-04-24T23:19:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 shutdown_connections 2024-04-24T23:19:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8048065640 0x7f80480fbdb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f803006e260 0x7f8030070710 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 --2- 172.21.15.31:0/1381707866 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8048102e50 0x7f80480fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 >> 172.21.15.31:0/1381707866 conn(0x7f8048071d00 msgr2=0x7f80480fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 shutdown_connections 2024-04-24T23:19:38.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.346+0000 7f804d189700 1 -- 172.21.15.31:0/1381707866 wait complete. 2024-04-24T23:19:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.453+0000 7ff0473c6700 1 -- 172.21.15.31:0/3363008770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 msgr2=0x7ff0400fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.453+0000 7ff0473c6700 1 --2- 172.21.15.31:0/3363008770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 0x7ff0400fba70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff030004a50 tx=0x7ff03000af40 comp rx=0 tx=0).stop 2024-04-24T23:19:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.453+0000 7ff0473c6700 1 -- 172.21.15.31:0/3363008770 shutdown_connections 2024-04-24T23:19:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.453+0000 7ff0473c6700 1 --2- 172.21.15.31:0/3363008770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 0x7ff0400fba70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.453+0000 7ff0473c6700 1 --2- 172.21.15.31:0/3363008770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0401014d0 0x7ff0401018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.453+0000 7ff0473c6700 1 -- 172.21.15.31:0/3363008770 >> 172.21.15.31:0/3363008770 conn(0x7ff040071460 msgr2=0x7ff040071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.453+0000 7ff0473c6700 1 -- 172.21.15.31:0/3363008770 shutdown_connections 2024-04-24T23:19:38.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.453+0000 7ff0473c6700 1 -- 172.21.15.31:0/3363008770 wait complete. 2024-04-24T23:19:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff0473c6700 1 Processor -- start 2024-04-24T23:19:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff0473c6700 1 -- start start 2024-04-24T23:19:38.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff0473c6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 0x7ff040113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff0473c6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0401014d0 0x7ff040114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff0473c6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0401146b0 con 0x7ff0400fb600 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff0473c6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04010daa0 con 0x7ff0401014d0 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff045162700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 0x7ff040113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff045162700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 0x7ff040113b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39964/0 (socket says 172.21.15.31:39964) 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff045162700 1 -- 172.21.15.31:0/4008262259 learned_addr learned my addr 172.21.15.31:0/4008262259 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.454+0000 7ff044961700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0401014d0 0x7ff040114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff045162700 1 -- 172.21.15.31:0/4008262259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0401014d0 msgr2=0x7ff040114060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff045162700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0401014d0 0x7ff040114060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff045162700 1 -- 172.21.15.31:0/4008262259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030005040 con 0x7ff0400fb600 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff045162700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 0x7ff040113b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00c830 tx=0x7ff03c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff035ffb700 1 -- 172.21.15.31:0/4008262259 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff03c0106f0 con 0x7ff0400fb600 2024-04-24T23:19:38.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff035ffb700 1 -- 172.21.15.31:0/4008262259 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff03c010d20 con 0x7ff0400fb600 2024-04-24T23:19:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04010dd80 con 0x7ff0400fb600 2024-04-24T23:19:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff035ffb700 1 -- 172.21.15.31:0/4008262259 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff03c0113a0 con 0x7ff0400fb600 2024-04-24T23:19:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.455+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04010e340 con 0x7ff0400fb600 2024-04-24T23:19:38.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.456+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0400fcbb0 con 0x7ff0400fb600 2024-04-24T23:19:38.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.456+0000 7ff035ffb700 1 -- 172.21.15.31:0/4008262259 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff03c012400 con 0x7ff0400fb600 2024-04-24T23:19:38.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.457+0000 7ff035ffb700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff02c06e260 0x7ff02c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:19:38.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.457+0000 7ff035ffb700 1 -- 172.21.15.31:0/4008262259 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff03c08dae0 con 0x7ff0400fb600 2024-04-24T23:19:38.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.457+0000 7ff044961700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff02c06e260 0x7ff02c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:19:38.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.457+0000 7ff044961700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff02c06e260 0x7ff02c070710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff04010ec60 tx=0x7ff03000af40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:19:38.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.462+0000 7ff035ffb700 1 -- 172.21.15.31:0/4008262259 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff03c05a9f0 con 0x7ff0400fb600 2024-04-24T23:19:38.579 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:38 smithi031 podman[123168]: 2024-04-24 23:19:38.398641838 +0000 UTC m=+0.358019477 container init d216357cdbb6e7e1b1463f83376b7d6e5fd8abdca3116e23b190bb462512cc62 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-24T23:19:38.579 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:38 smithi031 podman[123168]: 2024-04-24 23:19:38.401952937 +0000 UTC m=+0.361330562 container start d216357cdbb6e7e1b1463f83376b7d6e5fd8abdca3116e23b190bb462512cc62 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T23:19:38.580 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-osd[123209]: -- 172.21.15.31:0/2530008882 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559cd5305680 con 0x559cd52cf000 2024-04-24T23:19:38.580 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:38 smithi031 bash[123168]: d216357cdbb6e7e1b1463f83376b7d6e5fd8abdca3116e23b190bb462512cc62 2024-04-24T23:19:38.580 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:38 smithi031 systemd[1]: Started Ceph osd.0 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:19:38.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.774+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff04005f5c0 con 0x7ff0400fb600 2024-04-24T23:19:38.777 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 40/225 objects degraded (17.778%), 18 pgs degraded 2024-04-24T23:19:38.777 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-24T23:19:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: osd.0 (root=default,host=smithi031) is down 2024-04-24T23:19:38.777 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 40/225 objects degraded (17.778%), 18 pgs degraded 2024-04-24T23:19:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-24T23:19:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-24T23:19:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-24T23:19:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-24T23:19:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-24T23:19:38.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.775+0000 7ff035ffb700 1 -- 172.21.15.31:0/4008262259 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1271 (secure 0 0 0) 0x7ff03c05a580 con 0x7ff0400fb600 2024-04-24T23:19:38.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff02c06e260 msgr2=0x7ff02c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff02c06e260 0x7ff02c070710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff04010ec60 tx=0x7ff03000af40 comp rx=0 tx=0).stop 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 msgr2=0x7ff040113b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 0x7ff040113b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00c830 tx=0x7ff03c004470 comp rx=0 tx=0).stop 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 shutdown_connections 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0400fb600 0x7ff040113b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff02c06e260 0x7ff02c070710 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 --2- 172.21.15.31:0/4008262259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff0401014d0 0x7ff040114060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 >> 172.21.15.31:0/4008262259 conn(0x7ff040071460 msgr2=0x7ff0400f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 shutdown_connections 2024-04-24T23:19:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:19:38.777+0000 7ff0473c6700 1 -- 172.21.15.31:0/4008262259 wait complete. 2024-04-24T23:19:38.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:38 smithi040 ceph-mon[83047]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:38.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:38 smithi040 ceph-mon[83047]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:38.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:38 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/870204215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:38.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:38 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4046606103' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:19:38.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:38 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:38.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:38 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:38.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:38 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:38.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-mon[111404]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:38.945 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-mon[111404]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:38.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/870204215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:38.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4046606103' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:19:38.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:38.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:38.946 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:39.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:38 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[123202]: 2024-04-24T23:19:38.943+0000 7f01108ac7c0 -1 Falling back to public interface 2024-04-24T23:19:39.644 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:39 smithi031 ceph-mon[111404]: pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:39.644 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:39 smithi031 ceph-mon[111404]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:39.644 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:39 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4008262259' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:19:39.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:39 smithi040 ceph-mon[83047]: pgmap v47: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:39.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:39 smithi040 ceph-mon[83047]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:19:39.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:39 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4008262259' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:19:41.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:41 smithi040 ceph-mon[83047]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:41.973 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:41 smithi031 ceph-mon[111404]: pgmap v48: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:43.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:43 smithi040 ceph-mon[83047]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:43.936 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:43 smithi031 ceph-mon[111404]: pgmap v49: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:44.249 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:43 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[123202]: 2024-04-24T23:19:43.935+0000 7f01108ac7c0 -1 osd.0 49 log_to_monitors true 2024-04-24T23:19:44.999 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:19:44 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[123202]: 2024-04-24T23:19:44.790+0000 7f0101d04700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:19:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:44 smithi031 ceph-mon[111404]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:44.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:45.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:45.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:44 smithi031 ceph-mon[111404]: from='osd.0 [v2:172.21.15.31:6802/4093691970,v1:172.21.15.31:6803/4093691970]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:19:45.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:45.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:44 smithi040 ceph-mon[83047]: pgmap v50: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 40/225 objects degraded (17.778%) 2024-04-24T23:19:45.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:45.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:45.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:44 smithi040 ceph-mon[83047]: from='osd.0 [v2:172.21.15.31:6802/4093691970,v1:172.21.15.31:6803/4093691970]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-24T23:19:45.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:45.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:45.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:45 smithi031 ceph-mon[111404]: from='osd.0 [v2:172.21.15.31:6802/4093691970,v1:172.21.15.31:6803/4093691970]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:19:45.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:45 smithi031 ceph-mon[111404]: osdmap e52: 6 total, 5 up, 6 in 2024-04-24T23:19:45.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:45 smithi031 ceph-mon[111404]: from='osd.0 [v2:172.21.15.31:6802/4093691970,v1:172.21.15.31:6803/4093691970]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:19:46.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:45 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:46.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:45 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:46.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:45 smithi040 ceph-mon[83047]: from='osd.0 [v2:172.21.15.31:6802/4093691970,v1:172.21.15.31:6803/4093691970]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-24T23:19:46.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:45 smithi040 ceph-mon[83047]: osdmap e52: 6 total, 5 up, 6 in 2024-04-24T23:19:46.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:45 smithi040 ceph-mon[83047]: from='osd.0 [v2:172.21.15.31:6802/4093691970,v1:172.21.15.31:6803/4093691970]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:19:46.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:45 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:46.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:45 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:46.920 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:46 smithi031 ceph-mon[111404]: pgmap v52: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 40/225 objects degraded (17.778%) 2024-04-24T23:19:46.920 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:46 smithi031 ceph-mon[111404]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:19:46.920 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:46 smithi031 ceph-mon[111404]: osd.0 [v2:172.21.15.31:6802/4093691970,v1:172.21.15.31:6803/4093691970] boot 2024-04-24T23:19:46.920 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:46 smithi031 ceph-mon[111404]: osdmap e53: 6 total, 6 up, 6 in 2024-04-24T23:19:46.920 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:46 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:19:46.920 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:46 smithi031 ceph-mon[111404]: mgrmap e39: smithi031.rrnmwr(active, since 92s), standbys: smithi040.qtdwuq 2024-04-24T23:19:47.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:46 smithi040 ceph-mon[83047]: pgmap v52: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s; 40/225 objects degraded (17.778%) 2024-04-24T23:19:47.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:46 smithi040 ceph-mon[83047]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:19:47.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:46 smithi040 ceph-mon[83047]: osd.0 [v2:172.21.15.31:6802/4093691970,v1:172.21.15.31:6803/4093691970] boot 2024-04-24T23:19:47.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:46 smithi040 ceph-mon[83047]: osdmap e53: 6 total, 6 up, 6 in 2024-04-24T23:19:47.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:46 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-24T23:19:47.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:46 smithi040 ceph-mon[83047]: mgrmap e39: smithi031.rrnmwr(active, since 92s), standbys: smithi040.qtdwuq 2024-04-24T23:19:48.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:47 smithi031 ceph-mon[111404]: osdmap e54: 6 total, 6 up, 6 in 2024-04-24T23:19:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:47 smithi040 ceph-mon[83047]: osdmap e54: 6 total, 6 up, 6 in 2024-04-24T23:19:48.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: pgmap v55: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 40/225 objects degraded (17.778%) 2024-04-24T23:19:48.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:48.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:48.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:49.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:49.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:49.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:49.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:49.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:49.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:49.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T23:19:49.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: pgmap v55: 65 pgs: 18 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 40/225 objects degraded (17.778%) 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:19:49.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T23:19:50.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:49 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T23:19:50.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:49 smithi040 ceph-mon[83047]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-24T23:19:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:49 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T23:19:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:49 smithi031 ceph-mon[111404]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-24T23:19:51.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:50 smithi040 ceph-mon[83047]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 6/225 objects degraded (2.667%); 46 B/s, 0 objects/s recovering 2024-04-24T23:19:51.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:50 smithi040 ceph-mon[83047]: Health check update: Degraded data redundancy: 6/225 objects degraded (2.667%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T23:19:51.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:50 smithi031 ceph-mon[111404]: pgmap v56: 65 pgs: 3 active+undersized+degraded, 6 active+undersized, 56 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 6/225 objects degraded (2.667%); 46 B/s, 0 objects/s recovering 2024-04-24T23:19:51.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:50 smithi031 ceph-mon[111404]: Health check update: Degraded data redundancy: 6/225 objects degraded (2.667%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T23:19:52.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:51 smithi040 ceph-mon[83047]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/225 objects degraded (2.667%), 3 pgs degraded) 2024-04-24T23:19:52.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:51 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:19:52.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:51 smithi031 ceph-mon[111404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 6/225 objects degraded (2.667%), 3 pgs degraded) 2024-04-24T23:19:52.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:51 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:19:53.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:52 smithi040 ceph-mon[83047]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 40 B/s, 0 objects/s recovering 2024-04-24T23:19:53.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:52 smithi031 ceph-mon[111404]: pgmap v57: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 40 B/s, 0 objects/s recovering 2024-04-24T23:19:55.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:54 smithi040 ceph-mon[83047]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-04-24T23:19:55.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:54 smithi031 ceph-mon[111404]: pgmap v58: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 35 B/s, 0 objects/s recovering 2024-04-24T23:19:57.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:56 smithi040 ceph-mon[83047]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-04-24T23:19:57.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:56 smithi031 ceph-mon[111404]: pgmap v59: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28 B/s, 0 objects/s recovering 2024-04-24T23:19:59.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:58 smithi040 ceph-mon[83047]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 25 B/s, 0 objects/s recovering 2024-04-24T23:19:59.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:59.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:59.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:19:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:59.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:58 smithi031 ceph-mon[111404]: pgmap v60: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 25 B/s, 0 objects/s recovering 2024-04-24T23:19:59.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:19:59.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:19:59.173 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:19:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:01.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:00 smithi040 ceph-mon[83047]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-04-24T23:20:01.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:00 smithi040 ceph-mon[83047]: overall HEALTH_OK 2024-04-24T23:20:01.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:00 smithi031 ceph-mon[111404]: pgmap v61: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 23 B/s, 0 objects/s recovering 2024-04-24T23:20:01.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:00 smithi031 ceph-mon[111404]: overall HEALTH_OK 2024-04-24T23:20:03.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:02 smithi040 ceph-mon[83047]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:03.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:02 smithi031 ceph-mon[111404]: pgmap v62: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:04.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:03 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T23:20:04.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:03 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T23:20:04.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:04 smithi031 ceph-mon[111404]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:05.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:04 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T23:20:05.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:04 smithi031 ceph-mon[111404]: Upgrade: osd.1 is safe to restart 2024-04-24T23:20:05.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:04 smithi040 ceph-mon[83047]: pgmap v63: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:05.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:04 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-24T23:20:05.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:04 smithi040 ceph-mon[83047]: Upgrade: osd.1 is safe to restart 2024-04-24T23:20:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:06 smithi031 ceph-mon[111404]: Upgrade: Updating osd.1 2024-04-24T23:20:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:06 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:06 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:20:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:06 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:06.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:06 smithi031 ceph-mon[111404]: Deploying daemon osd.1 on smithi031 2024-04-24T23:20:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:06 smithi040 ceph-mon[83047]: Upgrade: Updating osd.1 2024-04-24T23:20:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:06 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:06 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-24T23:20:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:06 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:06.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:06 smithi040 ceph-mon[83047]: Deploying daemon osd.1 on smithi031 2024-04-24T23:20:07.615 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:07 smithi031 ceph-mon[111404]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:07 smithi040 ceph-mon[83047]: pgmap v64: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:08.749 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:08 smithi031 systemd[1]: Stopping Ceph osd.1 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 -- 172.21.15.31:0/484943067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc101d90 msgr2=0x7fadcc102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 --2- 172.21.15.31:0/484943067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc101d90 0x7fadcc102160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fadbc004940 tx=0x7fadbc009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 -- 172.21.15.31:0/484943067 shutdown_connections 2024-04-24T23:20:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 --2- 172.21.15.31:0/484943067 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadcc0fbd40 0x7fadcc0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 --2- 172.21.15.31:0/484943067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc101d90 0x7fadcc102160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 -- 172.21.15.31:0/484943067 >> 172.21.15.31:0/484943067 conn(0x7fadcc071d00 msgr2=0x7fadcc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:08.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 -- 172.21.15.31:0/484943067 shutdown_connections 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 -- 172.21.15.31:0/484943067 wait complete. 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 Processor -- start 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.896+0000 7fadd0b22700 1 -- start start 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadd0b22700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc0fbd40 0x7fadcc106b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadd0b22700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadcc101d90 0x7fadcc1070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadd0b22700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc107630 con 0x7fadcc0fbd40 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadd0b22700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc107770 con 0x7fadcc101d90 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc0fbd40 0x7fadcc106b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadca59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc0fbd40 0x7fadcc106b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49536/0 (socket says 172.21.15.31:49536) 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadca59c700 1 -- 172.21.15.31:0/2308811304 learned_addr learned my addr 172.21.15.31:0/2308811304 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadc9d9b700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadcc101d90 0x7fadcc1070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:08.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadca59c700 1 -- 172.21.15.31:0/2308811304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadcc101d90 msgr2=0x7fadcc1070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadca59c700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadcc101d90 0x7fadcc1070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadca59c700 1 -- 172.21.15.31:0/2308811304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc005040 con 0x7fadcc0fbd40 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadc9d9b700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadcc101d90 0x7fadcc1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadca59c700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc0fbd40 0x7fadcc106b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009150 tx=0x7fadbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadbaffd700 1 -- 172.21.15.31:0/2308811304 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadbc00e040 con 0x7fadcc0fbd40 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadbaffd700 1 -- 172.21.15.31:0/2308811304 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadbc00bbc0 con 0x7fadcc0fbd40 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc1079c0 con 0x7fadcc0fbd40 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadbaffd700 1 -- 172.21.15.31:0/2308811304 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadbc01c630 con 0x7fadcc0fbd40 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.897+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc11a2b0 con 0x7fadcc0fbd40 2024-04-24T23:20:08.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.898+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc0feff0 con 0x7fadcc0fbd40 2024-04-24T23:20:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.899+0000 7fadbaffd700 1 -- 172.21.15.31:0/2308811304 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fadbc00bd30 con 0x7fadcc0fbd40 2024-04-24T23:20:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.899+0000 7fadbaffd700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fadb406e150 0x7fadb4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.899+0000 7fadbaffd700 1 -- 172.21.15.31:0/2308811304 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fadbc08ec40 con 0x7fadcc0fbd40 2024-04-24T23:20:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.899+0000 7fadc9d9b700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fadb406e150 0x7fadb4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.900+0000 7fadc9d9b700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fadb406e150 0x7fadb4070600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009fd0 tx=0x7fadc0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:08.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:08.904+0000 7fadbaffd700 1 -- 172.21.15.31:0/2308811304 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fadbc058130 con 0x7fadcc0fbd40 2024-04-24T23:20:09.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.132+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadcc109f60 con 0x7fadb406e150 2024-04-24T23:20:09.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.133+0000 7fadbaffd700 1 -- 172.21.15.31:0/2308811304 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fadcc109f60 con 0x7fadb406e150 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.135+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fadb406e150 msgr2=0x7fadb4070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.135+0000 7fadd0b22700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fadb406e150 0x7fadb4070600 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009fd0 tx=0x7fadc0009ee0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.135+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc0fbd40 msgr2=0x7fadcc106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.135+0000 7fadd0b22700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc0fbd40 0x7fadcc106b60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fadbc009150 tx=0x7fadbc009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.136+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 shutdown_connections 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.136+0000 7fadd0b22700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadcc0fbd40 0x7fadcc106b60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.136+0000 7fadd0b22700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fadb406e150 0x7fadb4070600 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.136+0000 7fadd0b22700 1 --2- 172.21.15.31:0/2308811304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fadcc101d90 0x7fadcc1070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.136+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 >> 172.21.15.31:0/2308811304 conn(0x7fadcc071d00 msgr2=0x7fadcc0fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.136+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 shutdown_connections 2024-04-24T23:20:09.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.136+0000 7fadd0b22700 1 -- 172.21.15.31:0/2308811304 wait complete. 2024-04-24T23:20:09.149 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:20:09.149 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:08 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[53346]: 2024-04-24T23:20:08.979+0000 7f51e3546700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:09.149 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:08 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[53346]: 2024-04-24T23:20:08.979+0000 7f51e3546700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-24T23:20:09.149 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:08 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[53346]: 2024-04-24T23:20:08.979+0000 7f51e3546700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:20:09.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/108933882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 msgr2=0x7ff3ac0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 --2- 172.21.15.31:0/108933882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 0x7ff3ac0fc1b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff394004940 tx=0x7ff394009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:09.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/108933882 shutdown_connections 2024-04-24T23:20:09.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 --2- 172.21.15.31:0/108933882 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 0x7ff3ac0fc1b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 --2- 172.21.15.31:0/108933882 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac101d90 0x7ff3ac102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/108933882 >> 172.21.15.31:0/108933882 conn(0x7ff3ac071d00 msgr2=0x7ff3ac072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:09.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/108933882 shutdown_connections 2024-04-24T23:20:09.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/108933882 wait complete. 2024-04-24T23:20:09.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.242+0000 7ff3b1dc2700 1 Processor -- start 2024-04-24T23:20:09.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3b1dc2700 1 -- start start 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3b1dc2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 0x7ff3ac10eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3b1dc2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac101d90 0x7ff3ac10f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3b1dc2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac10f6e0 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3b1dc2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac10f820 con 0x7ff3ac101d90 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 0x7ff3ac10eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 0x7ff3ac10eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49546/0 (socket says 172.21.15.31:49546) 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3ab7fe700 1 -- 172.21.15.31:0/1015617607 learned_addr learned my addr 172.21.15.31:0/1015617607 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3aaffd700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac101d90 0x7ff3ac10f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3ab7fe700 1 -- 172.21.15.31:0/1015617607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac101d90 msgr2=0x7ff3ac10f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3ab7fe700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac101d90 0x7ff3ac10f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3ab7fe700 1 -- 172.21.15.31:0/1015617607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394005040 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.243+0000 7ff3ab7fe700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 0x7ff3ac10eb50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400cac0 tx=0x7ff3a4007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.244+0000 7ff3a3fff700 1 -- 172.21.15.31:0/1015617607 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3a400aa50 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.244+0000 7ff3a3fff700 1 -- 172.21.15.31:0/1015617607 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3a400abb0 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.244+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac117bc0 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.244+0000 7ff3a3fff700 1 -- 172.21.15.31:0/1015617607 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3a40115e0 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.244+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac118120 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.245+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac0fd430 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.245+0000 7ff3a3fff700 1 -- 172.21.15.31:0/1015617607 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff3a400ad20 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.245+0000 7ff3a3fff700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff39806e140 0x7ff3980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.246+0000 7ff3a3fff700 1 -- 172.21.15.31:0/1015617607 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff3a408e210 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.246+0000 7ff3aaffd700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff39806e140 0x7ff3980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.246+0000 7ff3aaffd700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff39806e140 0x7ff3980705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b3b0 tx=0x7ff394009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.250+0000 7ff3a3fff700 1 -- 172.21.15.31:0/1015617607 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff3a40575e0 con 0x7ff3ac0fbd40 2024-04-24T23:20:09.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.487+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3ac04bf70 con 0x7ff39806e140 2024-04-24T23:20:09.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.489+0000 7ff3a3fff700 1 -- 172.21.15.31:0/1015617607 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff39400b650 con 0x7ff39806e140 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff39806e140 msgr2=0x7ff3980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff39806e140 0x7ff3980705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff39400b3b0 tx=0x7ff394009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 msgr2=0x7ff3ac10eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 0x7ff3ac10eb50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400cac0 tx=0x7ff3a4007ee0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 shutdown_connections 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3ac0fbd40 0x7ff3ac10eb50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff39806e140 0x7ff3980705f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 --2- 172.21.15.31:0/1015617607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff3ac101d90 0x7ff3ac10f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 >> 172.21.15.31:0/1015617607 conn(0x7ff3ac071d00 msgr2=0x7ff3ac0fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 shutdown_connections 2024-04-24T23:20:09.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.491+0000 7ff3b1dc2700 1 -- 172.21.15.31:0/1015617607 wait complete. 2024-04-24T23:20:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 -- 172.21.15.31:0/963436864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4201026b0 msgr2=0x7fe4200fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 --2- 172.21.15.31:0/963436864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4201026b0 0x7fe4200fbd20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe414004940 tx=0x7fe414009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 -- 172.21.15.31:0/963436864 shutdown_connections 2024-04-24T23:20:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 --2- 172.21.15.31:0/963436864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4201026b0 0x7fe4200fbd20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 --2- 172.21.15.31:0/963436864 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe420101d10 0x7fe4201020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 -- 172.21.15.31:0/963436864 >> 172.21.15.31:0/963436864 conn(0x7fe4200f7860 msgr2=0x7fe4200f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 -- 172.21.15.31:0/963436864 shutdown_connections 2024-04-24T23:20:09.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 -- 172.21.15.31:0/963436864 wait complete. 2024-04-24T23:20:09.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 Processor -- start 2024-04-24T23:20:09.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.596+0000 7fe426855700 1 -- start start 2024-04-24T23:20:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe426855700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe420101d10 0x7fe4200fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe426855700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4201026b0 0x7fe4200fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe426855700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200ff8e0 con 0x7fe420101d10 2024-04-24T23:20:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe426855700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200fe410 con 0x7fe4201026b0 2024-04-24T23:20:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe420101d10 0x7fe4200fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe420101d10 0x7fe4200fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49570/0 (socket says 172.21.15.31:49570) 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41ffff700 1 -- 172.21.15.31:0/604918413 learned_addr learned my addr 172.21.15.31:0/604918413 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41f7fe700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4201026b0 0x7fe4200fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41ffff700 1 -- 172.21.15.31:0/604918413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4201026b0 msgr2=0x7fe4200fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41ffff700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4201026b0 0x7fe4200fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41ffff700 1 -- 172.21.15.31:0/604918413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414005040 con 0x7fe420101d10 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41f7fe700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4201026b0 0x7fe4200fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41ffff700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe420101d10 0x7fe4200fd990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe410002a50 tx=0x7fe4100046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41cff9700 1 -- 172.21.15.31:0/604918413 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe410007cc0 con 0x7fe420101d10 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41cff9700 1 -- 172.21.15.31:0/604918413 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe41000f040 con 0x7fe420101d10 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4200fe6f0 con 0x7fe420101d10 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.597+0000 7fe41cff9700 1 -- 172.21.15.31:0/604918413 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe410009640 con 0x7fe420101d10 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.598+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42011a170 con 0x7fe420101d10 2024-04-24T23:20:09.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.599+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4200fd460 con 0x7fe420101d10 2024-04-24T23:20:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.599+0000 7fe41cff9700 1 -- 172.21.15.31:0/604918413 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe4100097c0 con 0x7fe420101d10 2024-04-24T23:20:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.600+0000 7fe41cff9700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe40806e140 0x7fe4080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.600+0000 7fe41f7fe700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe40806e140 0x7fe4080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.600+0000 7fe41cff9700 1 -- 172.21.15.31:0/604918413 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe41008d870 con 0x7fe420101d10 2024-04-24T23:20:09.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.600+0000 7fe41f7fe700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe40806e140 0x7fe4080705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe414009fd0 tx=0x7fe414009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.604+0000 7fe41cff9700 1 -- 172.21.15.31:0/604918413 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe410056c40 con 0x7fe420101d10 2024-04-24T23:20:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:09 smithi031 ceph-mon[111404]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:09 smithi031 ceph-mon[111404]: osd.1 marked itself down and dead 2024-04-24T23:20:09.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.825+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4200f9640 con 0x7fe40806e140 2024-04-24T23:20:09.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.835+0000 7fe41cff9700 1 -- 172.21.15.31:0/604918413 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe4200f9640 con 0x7fe40806e140 2024-04-24T23:20:09.837 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (3m) 25s ago 14m 26.2M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (58s) 25s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (53s) 49s ago 14m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (3m) 25s ago 14m 79.0M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (9m) 25s ago 9m 138M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (9m) 25s ago 9m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (9m) 49s ago 9m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (9m) 49s ago 9m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (5m) 25s ago 16m 501M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (4m) 49s ago 14m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (2m) 25s ago 16m 43.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (99s) 49s ago 13m 32.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (4m) 25s ago 15m 16.5M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:20:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (3m) 49s ago 14m 14.7M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:20:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (31s) 25s ago 13m 11.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:20:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (12m) 25s ago 12m 238M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 86e73c1b8653 2024-04-24T23:20:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (12m) 25s ago 12m 198M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:20:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (11m) 49s ago 11m 278M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:20:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (11m) 49s ago 11m 246M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:20:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (10m) 49s ago 10m 225M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:20:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 25s ago 14m 58.7M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:20:09.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe40806e140 msgr2=0x7fe4080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe40806e140 0x7fe4080705f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe414009fd0 tx=0x7fe414009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe420101d10 msgr2=0x7fe4200fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe420101d10 0x7fe4200fd990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe410002a50 tx=0x7fe4100046b0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 shutdown_connections 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe420101d10 0x7fe4200fd990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe40806e140 0x7fe4080705f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 --2- 172.21.15.31:0/604918413 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4201026b0 0x7fe4200fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.837+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 >> 172.21.15.31:0/604918413 conn(0x7fe4200f7860 msgr2=0x7fe4200f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.838+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 shutdown_connections 2024-04-24T23:20:09.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.838+0000 7fe426855700 1 -- 172.21.15.31:0/604918413 wait complete. 2024-04-24T23:20:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:09 smithi040 ceph-mon[83047]: pgmap v65: 65 pgs: 65 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:09 smithi040 ceph-mon[83047]: osd.1 marked itself down and dead 2024-04-24T23:20:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.941+0000 7f3b22945700 1 -- 172.21.15.31:0/2638118872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fb5a0 msgr2=0x7f3b1c0fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.941+0000 7f3b22945700 1 --2- 172.21.15.31:0/2638118872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fb5a0 0x7f3b1c0fb970 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c004940 tx=0x7f3b0c009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.941+0000 7f3b22945700 1 -- 172.21.15.31:0/2638118872 shutdown_connections 2024-04-24T23:20:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.941+0000 7f3b22945700 1 --2- 172.21.15.31:0/2638118872 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1c0fbf40 0x7f3b1c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.941+0000 7f3b22945700 1 --2- 172.21.15.31:0/2638118872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fb5a0 0x7f3b1c0fb970 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.941+0000 7f3b22945700 1 -- 172.21.15.31:0/2638118872 >> 172.21.15.31:0/2638118872 conn(0x7f3b1c073350 msgr2=0x7f3b1c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:09.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.941+0000 7f3b22945700 1 -- 172.21.15.31:0/2638118872 shutdown_connections 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.942+0000 7f3b22945700 1 -- 172.21.15.31:0/2638118872 wait complete. 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.942+0000 7f3b22945700 1 Processor -- start 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.942+0000 7f3b22945700 1 -- start start 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.942+0000 7f3b22945700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1c0fb5a0 0x7f3b1c113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.942+0000 7f3b22945700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fbf40 0x7f3b1c113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.942+0000 7f3b22945700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1144f0 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.942+0000 7f3b22945700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c10daf0 con 0x7f3b1c0fb5a0 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.942+0000 7f3b1b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fbf40 0x7f3b1c113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b1b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fbf40 0x7f3b1c113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49586/0 (socket says 172.21.15.31:49586) 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b1b7fe700 1 -- 172.21.15.31:0/4245833670 learned_addr learned my addr 172.21.15.31:0/4245833670 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b1bfff700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1c0fb5a0 0x7f3b1c113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b1b7fe700 1 -- 172.21.15.31:0/4245833670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1c0fb5a0 msgr2=0x7f3b1c113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b1b7fe700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1c0fb5a0 0x7f3b1c113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:09.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b1b7fe700 1 -- 172.21.15.31:0/4245833670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c005040 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b1b7fe700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fbf40 0x7f3b1c113ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3b100044e0 tx=0x7f3b1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b18ff9700 1 -- 172.21.15.31:0/4245833670 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b10007cc0 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b18ff9700 1 -- 172.21.15.31:0/4245833670 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b1000f040 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c10dd70 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b18ff9700 1 -- 172.21.15.31:0/4245833670 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b1000ca50 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.943+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c10e2b0 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.944+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c0fe670 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.945+0000 7f3b18ff9700 1 -- 172.21.15.31:0/4245833670 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3b1000cbb0 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.945+0000 7f3b18ff9700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3b0406e210 0x7f3b040706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:09.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.945+0000 7f3b1bfff700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3b0406e210 0x7f3b040706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:09.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.945+0000 7f3b18ff9700 1 -- 172.21.15.31:0/4245833670 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f3b1008d450 con 0x7f3b1c0fbf40 2024-04-24T23:20:09.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.946+0000 7f3b1bfff700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3b0406e210 0x7f3b040706c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009b30 tx=0x7f3b0c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:09.952 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:09.950+0000 7f3b18ff9700 1 -- 172.21.15.31:0/4245833670 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3b1005a3e0 con 0x7f3b1c0fbf40 2024-04-24T23:20:10.249 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:09 smithi031 podman[128421]: 2024-04-24 23:20:09.996292501 +0000 UTC m=+1.144172795 container died 86e73c1b86534deb44214ac44dc2ab96f009bfa315aea483fa27df4fef30f8c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T23:20:10.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.250+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3b1c109f40 con 0x7f3b1c0fbf40 2024-04-24T23:20:10.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.250+0000 7f3b18ff9700 1 -- 172.21.15.31:0/4245833670 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f3b1000ce60 con 0x7f3b1c0fbf40 2024-04-24T23:20:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:20:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:20:10.253 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 8, 2024-04-24T23:20:10.254 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 5 2024-04-24T23:20:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:20:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:20:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3b0406e210 msgr2=0x7f3b040706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3b0406e210 0x7f3b040706c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009b30 tx=0x7f3b0c009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fbf40 msgr2=0x7f3b1c113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fbf40 0x7f3b1c113ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3b100044e0 tx=0x7f3b1000b040 comp rx=0 tx=0).stop 2024-04-24T23:20:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 shutdown_connections 2024-04-24T23:20:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3b0406e210 0x7f3b040706c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3b1c0fb5a0 0x7f3b1c113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 --2- 172.21.15.31:0/4245833670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b1c0fbf40 0x7f3b1c113ea0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 >> 172.21.15.31:0/4245833670 conn(0x7f3b1c073350 msgr2=0x7f3b1c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 shutdown_connections 2024-04-24T23:20:10.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.253+0000 7f3b22945700 1 -- 172.21.15.31:0/4245833670 wait complete. 2024-04-24T23:20:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.359+0000 7f074212f700 1 -- 172.21.15.31:0/3275349972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c101d90 msgr2=0x7f073c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.359+0000 7f074212f700 1 --2- 172.21.15.31:0/3275349972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c101d90 0x7f073c102160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f072c004940 tx=0x7f072c009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:10.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.359+0000 7f074212f700 1 -- 172.21.15.31:0/3275349972 shutdown_connections 2024-04-24T23:20:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.359+0000 7f074212f700 1 --2- 172.21.15.31:0/3275349972 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c0fbd40 0x7f073c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.359+0000 7f074212f700 1 --2- 172.21.15.31:0/3275349972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c101d90 0x7f073c102160 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.359+0000 7f074212f700 1 -- 172.21.15.31:0/3275349972 >> 172.21.15.31:0/3275349972 conn(0x7f073c071d00 msgr2=0x7f073c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.359+0000 7f074212f700 1 -- 172.21.15.31:0/3275349972 shutdown_connections 2024-04-24T23:20:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.359+0000 7f074212f700 1 -- 172.21.15.31:0/3275349972 wait complete. 2024-04-24T23:20:10.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f074212f700 1 Processor -- start 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f074212f700 1 -- start start 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f074212f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c0fbd40 0x7f073c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f074212f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c101d90 0x7f073c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f074212f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c103680 con 0x7f073c0fbd40 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f074212f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1037c0 con 0x7f073c101d90 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f073b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c0fbd40 0x7f073c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f073b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c0fbd40 0x7f073c102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49608/0 (socket says 172.21.15.31:49608) 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f073b7fe700 1 -- 172.21.15.31:0/3892944272 learned_addr learned my addr 172.21.15.31:0/3892944272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.360+0000 7f073affd700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c101d90 0x7f073c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f073b7fe700 1 -- 172.21.15.31:0/3892944272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c101d90 msgr2=0x7f073c103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f073b7fe700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c101d90 0x7f073c103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f073b7fe700 1 -- 172.21.15.31:0/3892944272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c005040 con 0x7f073c0fbd40 2024-04-24T23:20:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f073b7fe700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c0fbd40 0x7f073c102b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f072c009180 tx=0x7f072c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f0723fff700 1 -- 172.21.15.31:0/3892944272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072c00e040 con 0x7f073c0fbd40 2024-04-24T23:20:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f0723fff700 1 -- 172.21.15.31:0/3892944272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f072c00bbc0 con 0x7f073c0fbd40 2024-04-24T23:20:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c119e70 con 0x7f073c0fbd40 2024-04-24T23:20:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f0723fff700 1 -- 172.21.15.31:0/3892944272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f072c01c3f0 con 0x7f073c0fbd40 2024-04-24T23:20:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.361+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c11a350 con 0x7f073c0fbd40 2024-04-24T23:20:10.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.362+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c0fd430 con 0x7f073c0fbd40 2024-04-24T23:20:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.363+0000 7f0723fff700 1 -- 172.21.15.31:0/3892944272 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f072c00bd30 con 0x7f073c0fbd40 2024-04-24T23:20:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.363+0000 7f0723fff700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f072406e150 0x7f0724070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.363+0000 7f0723fff700 1 -- 172.21.15.31:0/3892944272 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f072c08ecf0 con 0x7f073c0fbd40 2024-04-24T23:20:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.363+0000 7f073affd700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f072406e150 0x7f0724070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:10.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.364+0000 7f073affd700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f072406e150 0x7f0724070600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f073c104460 tx=0x7f07300091a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:10.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.368+0000 7f0723fff700 1 -- 172.21.15.31:0/3892944272 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f072c058140 con 0x7f073c0fbd40 2024-04-24T23:20:10.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.672+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f073c0fd430 con 0x7f073c0fbd40 2024-04-24T23:20:10.675 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:10 smithi031 ceph-mon[111404]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:10.675 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:10 smithi031 ceph-mon[111404]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:20:10.676 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:10 smithi031 ceph-mon[111404]: osdmap e55: 6 total, 5 up, 6 in 2024-04-24T23:20:10.676 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:10 smithi031 ceph-mon[111404]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:10.676 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:10 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4245833670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:10.676 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:10 smithi031 podman[128421]: 2024-04-24 23:20:10.464331585 +0000 UTC m=+1.612211870 container cleanup 86e73c1b86534deb44214ac44dc2ab96f009bfa315aea483fa27df4fef30f8c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c) 2024-04-24T23:20:10.676 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:10 smithi031 bash[128421]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1 2024-04-24T23:20:10.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.673+0000 7f0723fff700 1 -- 172.21.15.31:0/3892944272 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f072c05b800 con 0x7f073c0fbd40 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:20:10.677 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:20:10.678 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:20:10.679 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:20:10.680 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:20:10.680 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:20:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f072406e150 msgr2=0x7f0724070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:10.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f072406e150 0x7f0724070600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f073c104460 tx=0x7f07300091a0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c0fbd40 msgr2=0x7f073c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c0fbd40 0x7f073c102b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f072c009180 tx=0x7f072c009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 shutdown_connections 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f073c0fbd40 0x7f073c102b20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f072406e150 0x7f0724070600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 --2- 172.21.15.31:0/3892944272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c101d90 0x7f073c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 >> 172.21.15.31:0/3892944272 conn(0x7f073c071d00 msgr2=0x7f073c0fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 shutdown_connections 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.676+0000 7f074212f700 1 -- 172.21.15.31:0/3892944272 wait complete. 2024-04-24T23:20:10.681 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:20:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.783+0000 7f16667b3700 1 -- 172.21.15.31:0/23185706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 msgr2=0x7f16600fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.783+0000 7f16667b3700 1 --2- 172.21.15.31:0/23185706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 0x7f16600fce00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1654004940 tx=0x7f1654009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.783+0000 7f16667b3700 1 -- 172.21.15.31:0/23185706 shutdown_connections 2024-04-24T23:20:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.783+0000 7f16667b3700 1 --2- 172.21.15.31:0/23185706 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 0x7f16600fce00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.783+0000 7f16667b3700 1 --2- 172.21.15.31:0/23185706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1660102df0 0x7f16601031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.783+0000 7f16667b3700 1 -- 172.21.15.31:0/23185706 >> 172.21.15.31:0/23185706 conn(0x7f16600f8980 msgr2=0x7f16600fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.783+0000 7f16667b3700 1 -- 172.21.15.31:0/23185706 shutdown_connections 2024-04-24T23:20:10.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.783+0000 7f16667b3700 1 -- 172.21.15.31:0/23185706 wait complete. 2024-04-24T23:20:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f16667b3700 1 Processor -- start 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f16667b3700 1 -- start start 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f16667b3700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1660102df0 0x7f1660113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f16667b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 0x7f16601140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f16667b3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1660114710 con 0x7f1660103790 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f16667b3700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166010db00 con 0x7f1660102df0 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f165f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 0x7f16601140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f165f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 0x7f16601140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49636/0 (socket says 172.21.15.31:49636) 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.784+0000 7f165f7fe700 1 -- 172.21.15.31:0/684655124 learned_addr learned my addr 172.21.15.31:0/684655124 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165ffff700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1660102df0 0x7f1660113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165f7fe700 1 -- 172.21.15.31:0/684655124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1660102df0 msgr2=0x7f1660113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165f7fe700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1660102df0 0x7f1660113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165f7fe700 1 -- 172.21.15.31:0/684655124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1654005040 con 0x7f1660103790 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165ffff700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1660102df0 0x7f1660113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:20:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165f7fe700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 0x7f16601140c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1654009180 tx=0x7f1654009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165cff9700 1 -- 172.21.15.31:0/684655124 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165400e040 con 0x7f1660103790 2024-04-24T23:20:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165cff9700 1 -- 172.21.15.31:0/684655124 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f165400bbc0 con 0x7f1660103790 2024-04-24T23:20:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166010dd80 con 0x7f1660103790 2024-04-24T23:20:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.785+0000 7f165cff9700 1 -- 172.21.15.31:0/684655124 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f165401c3f0 con 0x7f1660103790 2024-04-24T23:20:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.786+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166010e280 con 0x7f1660103790 2024-04-24T23:20:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.787+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16600fe540 con 0x7f1660103790 2024-04-24T23:20:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.787+0000 7f165cff9700 1 -- 172.21.15.31:0/684655124 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f165400bd30 con 0x7f1660103790 2024-04-24T23:20:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.788+0000 7f165cff9700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f164806e140 0x7f16480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.788+0000 7f165cff9700 1 -- 172.21.15.31:0/684655124 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f165408ed10 con 0x7f1660103790 2024-04-24T23:20:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.788+0000 7f165ffff700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f164806e140 0x7f16480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.788+0000 7f165ffff700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f164806e140 0x7f16480705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16500041d0 tx=0x7f1650009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:10.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:10.793+0000 7f165cff9700 1 -- 172.21.15.31:0/684655124 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f165405bd30 con 0x7f1660103790 2024-04-24T23:20:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:10 smithi040 ceph-mon[83047]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:10 smithi040 ceph-mon[83047]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:20:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:10 smithi040 ceph-mon[83047]: osdmap e55: 6 total, 5 up, 6 in 2024-04-24T23:20:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:10 smithi040 ceph-mon[83047]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:10.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:10 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4245833670' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:11.002 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:10 smithi031 podman[128530]: 2024-04-24 23:20:10.791730993 +0000 UTC m=+0.794091427 container remove 86e73c1b86534deb44214ac44dc2ab96f009bfa315aea483fa27df4fef30f8c2 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-24T23:20:11.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.094+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1660109ed0 con 0x7f164806e140 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.095+0000 7f165cff9700 1 -- 172.21.15.31:0/684655124 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1660109ed0 con 0x7f164806e140 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:20:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:20:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:20:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "7/21 daemons upgraded", 2024-04-24T23:20:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T23:20:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:20:11.099 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:20:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f164806e140 msgr2=0x7f16480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f164806e140 0x7f16480705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f16500041d0 tx=0x7f1650009040 comp rx=0 tx=0).stop 2024-04-24T23:20:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 msgr2=0x7f16601140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 0x7f16601140c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1654009180 tx=0x7f1654009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 shutdown_connections 2024-04-24T23:20:11.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f164806e140 0x7f16480705f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1660102df0 0x7f1660113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 --2- 172.21.15.31:0/684655124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1660103790 0x7f16601140c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 >> 172.21.15.31:0/684655124 conn(0x7f16600f8980 msgr2=0x7f16600f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 shutdown_connections 2024-04-24T23:20:11.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.097+0000 7f16667b3700 1 -- 172.21.15.31:0/684655124 wait complete. 2024-04-24T23:20:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.211+0000 7fe9f02a7700 1 -- 172.21.15.31:0/4129064907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 msgr2=0x7fe9e81020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.211+0000 7fe9f02a7700 1 --2- 172.21.15.31:0/4129064907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 0x7fe9e81020e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4004940 tx=0x7fe9e4009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.211+0000 7fe9f02a7700 1 -- 172.21.15.31:0/4129064907 shutdown_connections 2024-04-24T23:20:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.211+0000 7fe9f02a7700 1 --2- 172.21.15.31:0/4129064907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9e81026b0 0x7fe9e80fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.211+0000 7fe9f02a7700 1 --2- 172.21.15.31:0/4129064907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 0x7fe9e81020e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.211+0000 7fe9f02a7700 1 -- 172.21.15.31:0/4129064907 >> 172.21.15.31:0/4129064907 conn(0x7fe9e80f7860 msgr2=0x7fe9e80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:11.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.211+0000 7fe9f02a7700 1 -- 172.21.15.31:0/4129064907 shutdown_connections 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.211+0000 7fe9f02a7700 1 -- 172.21.15.31:0/4129064907 wait complete. 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9f02a7700 1 Processor -- start 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9f02a7700 1 -- start start 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9f02a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 0x7fe9e8106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9f02a7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9e81026b0 0x7fe9e81070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9f02a7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e8108ab0 con 0x7fe9e8101d10 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9f02a7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e8107630 con 0x7fe9e81026b0 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9ee043700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 0x7fe9e8106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9ee043700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 0x7fe9e8106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49656/0 (socket says 172.21.15.31:49656) 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.212+0000 7fe9ee043700 1 -- 172.21.15.31:0/2375721141 learned_addr learned my addr 172.21.15.31:0/2375721141 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9ed842700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9e81026b0 0x7fe9e81070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9ee043700 1 -- 172.21.15.31:0/2375721141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9e81026b0 msgr2=0x7fe9e81070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9ee043700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9e81026b0 0x7fe9e81070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9ee043700 1 -- 172.21.15.31:0/2375721141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4005040 con 0x7fe9e8101d10 2024-04-24T23:20:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9ee043700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 0x7fe9e8106bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009150 tx=0x7fe9e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9deffd700 1 -- 172.21.15.31:0/2375721141 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9e400e040 con 0x7fe9e8101d10 2024-04-24T23:20:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9deffd700 1 -- 172.21.15.31:0/2375721141 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9e4013420 con 0x7fe9e8101d10 2024-04-24T23:20:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e81078b0 con 0x7fe9e8101d10 2024-04-24T23:20:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9deffd700 1 -- 172.21.15.31:0/2375721141 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9e401c630 con 0x7fe9e8101d10 2024-04-24T23:20:11.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.213+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e811a0f0 con 0x7fe9e8101d10 2024-04-24T23:20:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.214+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e80ff020 con 0x7fe9e8101d10 2024-04-24T23:20:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.215+0000 7fe9deffd700 1 -- 172.21.15.31:0/2375721141 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe9e4013590 con 0x7fe9e8101d10 2024-04-24T23:20:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.215+0000 7fe9deffd700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe9d406e140 0x7fe9d40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.215+0000 7fe9deffd700 1 -- 172.21.15.31:0/2375721141 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe9e408ec30 con 0x7fe9e8101d10 2024-04-24T23:20:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.215+0000 7fe9ed842700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe9d406e140 0x7fe9d40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:11.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.216+0000 7fe9ed842700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe9d406e140 0x7fe9d40705f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8108290 tx=0x7fe9d800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.220+0000 7fe9deffd700 1 -- 172.21.15.31:0/2375721141 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe9e4058120 con 0x7fe9e8101d10 2024-04-24T23:20:11.430 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:11 smithi031 podman[128670]: 2024-04-24 23:20:11.08162143 +0000 UTC m=+0.023200486 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:11.430 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:11 smithi031 podman[128670]: 2024-04-24 23:20:11.201306265 +0000 UTC m=+0.142885293 container create c0ca69ad54cf2a4e2d4aa1177147a6da85cf006e5bfe104c0d4944a429332d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-24T23:20:11.430 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:11 smithi031 podman[128670]: 2024-04-24 23:20:11.42888224 +0000 UTC m=+0.370461270 container init c0ca69ad54cf2a4e2d4aa1177147a6da85cf006e5bfe104c0d4944a429332d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-24T23:20:11.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.528+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe9e8108040 con 0x7fe9e8101d10 2024-04-24T23:20:11.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.528+0000 7fe9deffd700 1 -- 172.21.15.31:0/2375721141 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+823 (secure 0 0 0) 0x7fe9e405b7e0 con 0x7fe9e8101d10 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 8 pgs peering 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: osd.1 (root=default,host=smithi031) is down 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 8 pgs peering 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.6 is stuck peering for 4m, current state peering, last acting [3,4] 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.9 is stuck peering for 4m, current state peering, last acting [4,0] 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.a is stuck peering for 4m, current state peering, last acting [4,3] 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.d is stuck peering for 4m, current state peering, last acting [3,2] 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: pg 3.0 is stuck peering for 4m, current state peering, last acting [2,4] 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: pg 3.2 is stuck peering for 4m, current state peering, last acting [5,3] 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: pg 3.4 is stuck peering for 4m, current state peering, last acting [2,3] 2024-04-24T23:20:11.532 INFO:teuthology.orchestra.run.smithi031.stdout: pg 3.19 is stuck peering for 4m, current state peering, last acting [4,3] 2024-04-24T23:20:11.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe9d406e140 msgr2=0x7fe9d40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe9d406e140 0x7fe9d40705f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8108290 tx=0x7fe9d800b040 comp rx=0 tx=0).stop 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 msgr2=0x7fe9e8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 0x7fe9e8106bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009150 tx=0x7fe9e4009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 shutdown_connections 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9e8101d10 0x7fe9e8106bb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe9d406e140 0x7fe9d40705f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 --2- 172.21.15.31:0/2375721141 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9e81026b0 0x7fe9e81070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 >> 172.21.15.31:0/2375721141 conn(0x7fe9e80f7860 msgr2=0x7fe9e8106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 shutdown_connections 2024-04-24T23:20:11.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:11.532+0000 7fe9f02a7700 1 -- 172.21.15.31:0/2375721141 wait complete. 2024-04-24T23:20:11.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:11 smithi031 ceph-mon[111404]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:11.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:11 smithi031 ceph-mon[111404]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:11.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:11 smithi031 ceph-mon[111404]: Health check failed: Reduced data availability: 8 pgs peering (PG_AVAILABILITY) 2024-04-24T23:20:11.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:11 smithi031 ceph-mon[111404]: osdmap e56: 6 total, 5 up, 6 in 2024-04-24T23:20:11.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:11 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3892944272' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:20:11.749 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:11 smithi031 podman[128670]: 2024-04-24 23:20:11.433129767 +0000 UTC m=+0.374708799 container start c0ca69ad54cf2a4e2d4aa1177147a6da85cf006e5bfe104c0d4944a429332d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-24T23:20:11.749 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:11 smithi031 podman[128670]: 2024-04-24 23:20:11.512180413 +0000 UTC m=+0.453759454 container attach c0ca69ad54cf2a4e2d4aa1177147a6da85cf006e5bfe104c0d4944a429332d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-24T23:20:11.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:11 smithi040 ceph-mon[83047]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:11.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:11 smithi040 ceph-mon[83047]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:11.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:11 smithi040 ceph-mon[83047]: Health check failed: Reduced data availability: 8 pgs peering (PG_AVAILABILITY) 2024-04-24T23:20:11.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:11 smithi040 ceph-mon[83047]: osdmap e56: 6 total, 5 up, 6 in 2024-04-24T23:20:11.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:11 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3892944272' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:20:11.999 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:11 smithi031 podman[128670]: 2024-04-24 23:20:11.747620049 +0000 UTC m=+0.689199093 container died c0ca69ad54cf2a4e2d4aa1177147a6da85cf006e5bfe104c0d4944a429332d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-24T23:20:12.753 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:12 smithi031 ceph-mon[111404]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:12.753 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:12 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/2375721141' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:20:12.753 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:12 smithi031 podman[128721]: 2024-04-24 23:20:12.458265694 +0000 UTC m=+0.705072636 container remove c0ca69ad54cf2a4e2d4aa1177147a6da85cf006e5bfe104c0d4944a429332d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-24T23:20:12.753 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:12 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-24T23:20:12.753 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:12 smithi031 systemd[1]: Stopped Ceph osd.1 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:20:12.753 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:12 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.1.service: Consumed 1min 11.838s CPU time. 2024-04-24T23:20:12.754 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:12 smithi031 systemd[1]: Starting Ceph osd.1 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:12.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:12 smithi040 ceph-mon[83047]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:12.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:12 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/2375721141' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:20:13.246 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:12 smithi031 podman[128825]: 2024-04-24 23:20:12.828285201 +0000 UTC m=+0.018648720 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:13.247 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:12 smithi031 podman[128825]: 2024-04-24 23:20:12.934572609 +0000 UTC m=+0.124936157 container create 3d21f386cb2ce675677486e80fe1e5714a66ac0b975a661380b873b641a565b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-24T23:20:13.247 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 podman[128825]: 2024-04-24 23:20:13.171094868 +0000 UTC m=+0.361458383 container init 3d21f386cb2ce675677486e80fe1e5714a66ac0b975a661380b873b641a565b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T23:20:13.247 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 podman[128825]: 2024-04-24 23:20:13.1746676 +0000 UTC m=+0.365031122 container start 3d21f386cb2ce675677486e80fe1e5714a66ac0b975a661380b873b641a565b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T23:20:13.247 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 podman[128825]: 2024-04-24 23:20:13.245451848 +0000 UTC m=+0.435815375 container attach 3d21f386cb2ce675677486e80fe1e5714a66ac0b975a661380b873b641a565b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T23:20:13.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:13 smithi031 ceph-mon[111404]: pgmap v69: 65 pgs: 34 peering, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:13.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:13 smithi040 ceph-mon[83047]: pgmap v69: 65 pgs: 34 peering, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:13.967 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate[128839]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T23:20:13.967 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate[128839]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 bash[128825]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 bash[128825]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 bash[128825]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 bash[128825]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 bash[128825]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 bash[128825]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 bash[128825]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate[128839]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate[128839]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate[128839]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate[128839]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate[128839]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-24T23:20:13.968 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:13 smithi031 podman[128825]: 2024-04-24 23:20:13.656543705 +0000 UTC m=+0.846907212 container died 3d21f386cb2ce675677486e80fe1e5714a66ac0b975a661380b873b641a565b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:20:14.587 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:14 smithi031 podman[128973]: 2024-04-24 23:20:14.316847242 +0000 UTC m=+0.655138713 container remove 3d21f386cb2ce675677486e80fe1e5714a66ac0b975a661380b873b641a565b1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-24T23:20:14.587 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:14 smithi031 podman[129037]: 2024-04-24 23:20:14.438100033 +0000 UTC m=+0.019037314 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:14.587 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:14 smithi031 podman[129037]: 2024-04-24 23:20:14.551233538 +0000 UTC m=+0.132170820 container create ad59872a4b025b23a43d958a5d3efd7ef4ffb48c80b357271487d3602b7279e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T23:20:14.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:14 smithi031 ceph-mon[111404]: pgmap v70: 65 pgs: 34 peering, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:14.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:14.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:20:14.897 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:14 smithi031 podman[129037]: 2024-04-24 23:20:14.796006334 +0000 UTC m=+0.376943622 container init ad59872a4b025b23a43d958a5d3efd7ef4ffb48c80b357271487d3602b7279e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:20:14.897 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:14 smithi031 podman[129037]: 2024-04-24 23:20:14.799504084 +0000 UTC m=+0.380441365 container start ad59872a4b025b23a43d958a5d3efd7ef4ffb48c80b357271487d3602b7279e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T23:20:14.898 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:14 smithi031 ceph-osd[129055]: -- 172.21.15.31:0/4145443502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af29787680 con 0x55af29750c00 2024-04-24T23:20:14.898 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:14 smithi031 bash[129037]: ad59872a4b025b23a43d958a5d3efd7ef4ffb48c80b357271487d3602b7279e0 2024-04-24T23:20:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:14 smithi040 ceph-mon[83047]: pgmap v70: 65 pgs: 34 peering, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2024-04-24T23:20:15.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:15.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:20:15.249 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:14 smithi031 systemd[1]: Started Ceph osd.1 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:20:15.749 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:15 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[129051]: 2024-04-24T23:20:15.342+0000 7f5fdc3007c0 -1 Falling back to public interface 2024-04-24T23:20:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:16.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:16.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:16.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:16 smithi031 ceph-mon[111404]: pgmap v71: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:16 smithi031 ceph-mon[111404]: Health check failed: Degraded data redundancy: 28/225 objects degraded (12.444%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T23:20:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:16 smithi031 ceph-mon[111404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 11 pgs peering) 2024-04-24T23:20:17.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:16 smithi040 ceph-mon[83047]: pgmap v71: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:17.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:16 smithi040 ceph-mon[83047]: Health check failed: Degraded data redundancy: 28/225 objects degraded (12.444%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T23:20:17.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:16 smithi040 ceph-mon[83047]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 11 pgs peering) 2024-04-24T23:20:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:18 smithi031 ceph-mon[111404]: pgmap v72: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:19.415 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:18 smithi040 ceph-mon[83047]: pgmap v72: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:20.999 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:20 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[129051]: 2024-04-24T23:20:20.501+0000 7f5fdc3007c0 -1 osd.1 54 log_to_monitors true 2024-04-24T23:20:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:21 smithi031 ceph-mon[111404]: pgmap v73: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:21 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:21 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:21 smithi031 ceph-mon[111404]: from='osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:20:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:21 smithi040 ceph-mon[83047]: pgmap v73: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:21.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:21 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:21.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:21 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:21.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:21 smithi040 ceph-mon[83047]: from='osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-24T23:20:22.489 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:20:22 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[129051]: 2024-04-24T23:20:22.137+0000 7f5fc9750700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:22.489 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:22 smithi031 ceph-mon[111404]: from='osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:20:22.489 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:22 smithi031 ceph-mon[111404]: osdmap e57: 6 total, 5 up, 6 in 2024-04-24T23:20:22.489 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:22 smithi031 ceph-mon[111404]: from='osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:20:22.489 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:22 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:22.489 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:22 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:22.489 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:22 smithi031 ceph-mon[111404]: from='osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972]' entity='osd.1' 2024-04-24T23:20:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:22 smithi040 ceph-mon[83047]: from='osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-24T23:20:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:22 smithi040 ceph-mon[83047]: osdmap e57: 6 total, 5 up, 6 in 2024-04-24T23:20:22.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:22 smithi040 ceph-mon[83047]: from='osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:20:22.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:22 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:22.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:22 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:22.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:22 smithi040 ceph-mon[83047]: from='osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972]' entity='osd.1' 2024-04-24T23:20:23.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:23 smithi031 ceph-mon[111404]: pgmap v75: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:23.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:23 smithi031 ceph-mon[111404]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:20:23.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:23 smithi031 ceph-mon[111404]: osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972] boot 2024-04-24T23:20:23.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:23 smithi031 ceph-mon[111404]: osdmap e58: 6 total, 6 up, 6 in 2024-04-24T23:20:23.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:23 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:20:23.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:23 smithi040 ceph-mon[83047]: pgmap v75: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:23.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:23 smithi040 ceph-mon[83047]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:20:23.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:23 smithi040 ceph-mon[83047]: osd.1 [v2:172.21.15.31:6810/1890497972,v1:172.21.15.31:6811/1890497972] boot 2024-04-24T23:20:23.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:23 smithi040 ceph-mon[83047]: osdmap e58: 6 total, 6 up, 6 in 2024-04-24T23:20:23.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:23 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-24T23:20:25.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:25 smithi040 ceph-mon[83047]: pgmap v77: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:25.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:25 smithi040 ceph-mon[83047]: osdmap e59: 6 total, 6 up, 6 in 2024-04-24T23:20:25.455 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:25 smithi031 ceph-mon[111404]: pgmap v77: 65 pgs: 12 active+undersized+degraded, 22 active+undersized, 31 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 28/225 objects degraded (12.444%) 2024-04-24T23:20:25.455 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:25 smithi031 ceph-mon[111404]: osdmap e59: 6 total, 6 up, 6 in 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T23:20:26.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T23:20:26.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:26 smithi040 ceph-mon[83047]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-24T23:20:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:20:26.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:26.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:26.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:26.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:26.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:26.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T23:20:26.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T23:20:26.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:26 smithi031 ceph-mon[111404]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-24T23:20:27.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:27 smithi040 ceph-mon[83047]: pgmap v79: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 15/225 objects degraded (6.667%) 2024-04-24T23:20:27.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:27 smithi040 ceph-mon[83047]: Health check update: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded (PG_DEGRADED) 2024-04-24T23:20:27.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:27 smithi031 ceph-mon[111404]: pgmap v79: 65 pgs: 6 active+undersized+degraded, 6 active+undersized, 53 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 15/225 objects degraded (6.667%) 2024-04-24T23:20:27.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:27 smithi031 ceph-mon[111404]: Health check update: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded (PG_DEGRADED) 2024-04-24T23:20:28.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:28 smithi040 ceph-mon[83047]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded) 2024-04-24T23:20:28.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:28 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:20:28.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:28 smithi031 ceph-mon[111404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded) 2024-04-24T23:20:28.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:28 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:20:29.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:29 smithi040 ceph-mon[83047]: pgmap v80: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:29.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:29.666 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:20:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:29 smithi031 ceph-mon[111404]: pgmap v80: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:29.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:20:31.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:31 smithi040 ceph-mon[83047]: pgmap v81: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:31.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:31 smithi031 ceph-mon[111404]: pgmap v81: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:33.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:33 smithi040 ceph-mon[83047]: pgmap v82: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:33.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:33 smithi031 ceph-mon[111404]: pgmap v82: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:35.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:34 smithi040 ceph-mon[83047]: pgmap v83: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:35.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:34 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:35.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:34 smithi031 ceph-mon[111404]: pgmap v83: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:35.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:34 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:37.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:36 smithi040 ceph-mon[83047]: pgmap v84: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:37.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:36 smithi031 ceph-mon[111404]: pgmap v84: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:39.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:38 smithi040 ceph-mon[83047]: pgmap v85: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:39.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:38 smithi031 ceph-mon[111404]: pgmap v85: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:41.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:40 smithi031 ceph-mon[111404]: pgmap v86: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:41.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:40 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T23:20:41.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:40 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T23:20:41.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:40 smithi031 ceph-mon[111404]: Upgrade: osd.2 is safe to restart 2024-04-24T23:20:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:40 smithi040 ceph-mon[83047]: pgmap v86: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:40 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T23:20:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:40 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-24T23:20:41.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:40 smithi040 ceph-mon[83047]: Upgrade: osd.2 is safe to restart 2024-04-24T23:20:41.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.646+0000 7f07ee1da700 1 -- 172.21.15.31:0/1616340000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8102df0 msgr2=0x7f07e81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.646+0000 7f07ee1da700 1 --2- 172.21.15.31:0/1616340000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8102df0 0x7f07e81031c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f07d8004940 tx=0x7f07d8009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.646+0000 7f07ee1da700 1 -- 172.21.15.31:0/1616340000 shutdown_connections 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.646+0000 7f07ee1da700 1 --2- 172.21.15.31:0/1616340000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e8103790 0x7f07e80fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.646+0000 7f07ee1da700 1 --2- 172.21.15.31:0/1616340000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8102df0 0x7f07e81031c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.646+0000 7f07ee1da700 1 -- 172.21.15.31:0/1616340000 >> 172.21.15.31:0/1616340000 conn(0x7f07e80f8980 msgr2=0x7f07e80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.646+0000 7f07ee1da700 1 -- 172.21.15.31:0/1616340000 shutdown_connections 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.646+0000 7f07ee1da700 1 -- 172.21.15.31:0/1616340000 wait complete. 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07ee1da700 1 Processor -- start 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07ee1da700 1 -- start start 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07ee1da700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e8102df0 0x7f07e8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07ee1da700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8103790 0x7f07e80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07ee1da700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8070dc0 con 0x7f07e8103790 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07ee1da700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8070f00 con 0x7f07e8102df0 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8103790 0x7f07e80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07e6ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8103790 0x7f07e80707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41562/0 (socket says 172.21.15.31:41562) 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.647+0000 7f07e6ffd700 1 -- 172.21.15.31:0/354466797 learned_addr learned my addr 172.21.15.31:0/354466797 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07e77fe700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e8102df0 0x7f07e8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07e6ffd700 1 -- 172.21.15.31:0/354466797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e8102df0 msgr2=0x7f07e8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07e6ffd700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e8102df0 0x7f07e8070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07e6ffd700 1 -- 172.21.15.31:0/354466797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8005040 con 0x7f07e8103790 2024-04-24T23:20:41.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07e6ffd700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8103790 0x7f07e80707a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b790 tx=0x7f07dc009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07cffff700 1 -- 172.21.15.31:0/354466797 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07dc00f6f0 con 0x7f07e8103790 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e8119e80 con 0x7f07e8103790 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07cffff700 1 -- 172.21.15.31:0/354466797 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07dc00fd20 con 0x7f07e8103790 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07cffff700 1 -- 172.21.15.31:0/354466797 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07dc0103a0 con 0x7f07e8103790 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.648+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e811a360 con 0x7f07e8103790 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.649+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e80fe540 con 0x7f07e8103790 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.650+0000 7f07cffff700 1 -- 172.21.15.31:0/354466797 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f07dc011400 con 0x7f07e8103790 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.650+0000 7f07cffff700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f07d006e150 0x7f07d0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.650+0000 7f07cffff700 1 -- 172.21.15.31:0/354466797 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f07dc08d910 con 0x7f07e8103790 2024-04-24T23:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.650+0000 7f07e77fe700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f07d006e150 0x7f07d0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.651+0000 7f07e77fe700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f07d006e150 0x7f07d0070600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07d800b3b0 tx=0x7f07d8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:41.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.655+0000 7f07cffff700 1 -- 172.21.15.31:0/354466797 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f07dc056cc0 con 0x7f07e8103790 2024-04-24T23:20:41.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.883+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f07e804bed0 con 0x7f07d006e150 2024-04-24T23:20:41.885 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.884+0000 7f07cffff700 1 -- 172.21.15.31:0/354466797 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f07e804bed0 con 0x7f07d006e150 2024-04-24T23:20:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f07d006e150 msgr2=0x7f07d0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f07d006e150 0x7f07d0070600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07d800b3b0 tx=0x7f07d8009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8103790 msgr2=0x7f07e80707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8103790 0x7f07e80707a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00b790 tx=0x7f07dc009220 comp rx=0 tx=0).stop 2024-04-24T23:20:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 shutdown_connections 2024-04-24T23:20:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f07d006e150 0x7f07d0070600 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07e8102df0 0x7f07e8070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 --2- 172.21.15.31:0/354466797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8103790 0x7f07e80707a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 >> 172.21.15.31:0/354466797 conn(0x7f07e80f8980 msgr2=0x7f07e80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:41.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 shutdown_connections 2024-04-24T23:20:41.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.886+0000 7f07ee1da700 1 -- 172.21.15.31:0/354466797 wait complete. 2024-04-24T23:20:41.899 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:20:41.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.991+0000 7faec76d1700 1 -- 172.21.15.31:0/541206092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 msgr2=0x7faec0106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.991+0000 7faec76d1700 1 --2- 172.21.15.31:0/541206092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 0x7faec0106de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faebc004940 tx=0x7faebc009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:41.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.991+0000 7faec76d1700 1 -- 172.21.15.31:0/541206092 shutdown_connections 2024-04-24T23:20:41.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.991+0000 7faec76d1700 1 --2- 172.21.15.31:0/541206092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 0x7faec0106de0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.991+0000 7faec76d1700 1 --2- 172.21.15.31:0/541206092 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faec00fde00 0x7faec00fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.991+0000 7faec76d1700 1 -- 172.21.15.31:0/541206092 >> 172.21.15.31:0/541206092 conn(0x7faec00f7870 msgr2=0x7faec00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:41.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.991+0000 7faec76d1700 1 -- 172.21.15.31:0/541206092 shutdown_connections 2024-04-24T23:20:41.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.991+0000 7faec76d1700 1 -- 172.21.15.31:0/541206092 wait complete. 2024-04-24T23:20:41.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec76d1700 1 Processor -- start 2024-04-24T23:20:41.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec76d1700 1 -- start start 2024-04-24T23:20:41.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec76d1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faec00fde00 0x7faec00fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec76d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 0x7faec00fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:41.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec76d1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00fdb80 con 0x7faec00fe7a0 2024-04-24T23:20:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec76d1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00fc6b0 con 0x7faec00fde00 2024-04-24T23:20:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec4c6c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 0x7faec00fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec4c6c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 0x7faec00fc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41592/0 (socket says 172.21.15.31:41592) 2024-04-24T23:20:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec4c6c700 1 -- 172.21.15.31:0/3092251527 learned_addr learned my addr 172.21.15.31:0/3092251527 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.992+0000 7faec546d700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faec00fde00 0x7faec00fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faec4c6c700 1 -- 172.21.15.31:0/3092251527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faec00fde00 msgr2=0x7faec00fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:41.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faec4c6c700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faec00fde00 0x7faec00fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:41.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faec4c6c700 1 -- 172.21.15.31:0/3092251527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc005040 con 0x7faec00fe7a0 2024-04-24T23:20:41.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faec4c6c700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 0x7faec00fc170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faebc009380 tx=0x7faebc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:41.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faeb5ffb700 1 -- 172.21.15.31:0/3092251527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faebc00e040 con 0x7faec00fe7a0 2024-04-24T23:20:41.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec00fc930 con 0x7faec00fe7a0 2024-04-24T23:20:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faeb5ffb700 1 -- 172.21.15.31:0/3092251527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faebc009af0 con 0x7faec00fe7a0 2024-04-24T23:20:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faeb5ffb700 1 -- 172.21.15.31:0/3092251527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faebc01c630 con 0x7faec00fe7a0 2024-04-24T23:20:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.993+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec011a060 con 0x7faec00fe7a0 2024-04-24T23:20:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.994+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec0066b30 con 0x7faec00fe7a0 2024-04-24T23:20:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.995+0000 7faeb5ffb700 1 -- 172.21.15.31:0/3092251527 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faebc00bbc0 con 0x7faec00fe7a0 2024-04-24T23:20:42.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.995+0000 7faeb5ffb700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faeac06e140 0x7faeac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.995+0000 7faeb5ffb700 1 -- 172.21.15.31:0/3092251527 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7faebc08ee10 con 0x7faec00fe7a0 2024-04-24T23:20:42.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.995+0000 7faec546d700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faeac06e140 0x7faeac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:41.996+0000 7faec546d700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faeac06e140 0x7faeac0705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faeb0004990 tx=0x7faeb0009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:42.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.000+0000 7faeb5ffb700 1 -- 172.21.15.31:0/3092251527 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faebc058260 con 0x7faec00fe7a0 2024-04-24T23:20:42.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.233+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faec00f94b0 con 0x7faeac06e140 2024-04-24T23:20:42.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.235+0000 7faeb5ffb700 1 -- 172.21.15.31:0/3092251527 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7faec00f94b0 con 0x7faeac06e140 2024-04-24T23:20:42.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.237+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faeac06e140 msgr2=0x7faeac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.237+0000 7faec76d1700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faeac06e140 0x7faeac0705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faeb0004990 tx=0x7faeb0009ee0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.237+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 msgr2=0x7faec00fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.237+0000 7faec76d1700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 0x7faec00fc170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7faebc009380 tx=0x7faebc009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.237+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 shutdown_connections 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.238+0000 7faec76d1700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7faeac06e140 0x7faeac0705f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.238+0000 7faec76d1700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faec00fde00 0x7faec00fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.238+0000 7faec76d1700 1 --2- 172.21.15.31:0/3092251527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faec00fe7a0 0x7faec00fc170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.238+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 >> 172.21.15.31:0/3092251527 conn(0x7faec00f7870 msgr2=0x7faec00f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.238+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 shutdown_connections 2024-04-24T23:20:42.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.238+0000 7faec76d1700 1 -- 172.21.15.31:0/3092251527 wait complete. 2024-04-24T23:20:42.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.345+0000 7f480d57e700 1 -- 172.21.15.31:0/1317261456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 msgr2=0x7f4808102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.345+0000 7f480d57e700 1 --2- 172.21.15.31:0/1317261456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 0x7f4808102160 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f47f8004940 tx=0x7f47f8009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.345+0000 7f480d57e700 1 -- 172.21.15.31:0/1317261456 shutdown_connections 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.345+0000 7f480d57e700 1 --2- 172.21.15.31:0/1317261456 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48080fbd40 0x7f48080fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.345+0000 7f480d57e700 1 --2- 172.21.15.31:0/1317261456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 0x7f4808102160 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.345+0000 7f480d57e700 1 -- 172.21.15.31:0/1317261456 >> 172.21.15.31:0/1317261456 conn(0x7f4808071d00 msgr2=0x7f4808072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.345+0000 7f480d57e700 1 -- 172.21.15.31:0/1317261456 shutdown_connections 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.345+0000 7f480d57e700 1 -- 172.21.15.31:0/1317261456 wait complete. 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f480d57e700 1 Processor -- start 2024-04-24T23:20:42.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f480d57e700 1 -- start start 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f480d57e700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48080fbd40 0x7f4808113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f480d57e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 0x7f4808114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f480d57e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081146e0 con 0x7f4808101d90 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f480d57e700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480810dad0 con 0x7f48080fbd40 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f48067fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 0x7f4808114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f48067fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 0x7f4808114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41610/0 (socket says 172.21.15.31:41610) 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.346+0000 7f48067fc700 1 -- 172.21.15.31:0/2357833923 learned_addr learned my addr 172.21.15.31:0/2357833923 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f4806ffd700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48080fbd40 0x7f4808113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f48067fc700 1 -- 172.21.15.31:0/2357833923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48080fbd40 msgr2=0x7f4808113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f48067fc700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48080fbd40 0x7f4808113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f48067fc700 1 -- 172.21.15.31:0/2357833923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8005040 con 0x7f4808101d90 2024-04-24T23:20:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f48067fc700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 0x7f4808114090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c830 tx=0x7f47f0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f47ff7fe700 1 -- 172.21.15.31:0/2357833923 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f00106f0 con 0x7f4808101d90 2024-04-24T23:20:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480810ddb0 con 0x7f4808101d90 2024-04-24T23:20:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f47ff7fe700 1 -- 172.21.15.31:0/2357833923 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47f0010d20 con 0x7f4808101d90 2024-04-24T23:20:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f47ff7fe700 1 -- 172.21.15.31:0/2357833923 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f00113a0 con 0x7f4808101d90 2024-04-24T23:20:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.347+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480810e370 con 0x7f4808101d90 2024-04-24T23:20:42.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.348+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48080fd430 con 0x7f4808101d90 2024-04-24T23:20:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.349+0000 7f47ff7fe700 1 -- 172.21.15.31:0/2357833923 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f47f0012400 con 0x7f4808101d90 2024-04-24T23:20:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.349+0000 7f47ff7fe700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f47f406e140 0x7f47f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.349+0000 7f47ff7fe700 1 -- 172.21.15.31:0/2357833923 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f47f008dac0 con 0x7f4808101d90 2024-04-24T23:20:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.349+0000 7f4806ffd700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f47f406e140 0x7f47f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.350+0000 7f4806ffd700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f47f406e140 0x7f47f40705f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47f8004560 tx=0x7f47f8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:42.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.354+0000 7f47ff7fe700 1 -- 172.21.15.31:0/2357833923 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f47f005a9a0 con 0x7f4808101d90 2024-04-24T23:20:42.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.573+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4808108e60 con 0x7f47f406e140 2024-04-24T23:20:42.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.584+0000 7f47ff7fe700 1 -- 172.21.15.31:0/2357833923 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4808108e60 con 0x7f47f406e140 2024-04-24T23:20:42.586 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:20:42.586 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (3m) 22s ago 15m 26.2M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:20:42.586 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (91s) 22s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:20:42.586 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (85s) 82s ago 14m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:20:42.586 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (3m) 22s ago 15m 79.4M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:20:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (9m) 22s ago 9m 138M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:20:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (9m) 22s ago 9m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:20:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (9m) 82s ago 9m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:20:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (9m) 82s ago 9m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:20:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (5m) 22s ago 16m 501M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:20:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (4m) 82s ago 14m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:20:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (2m) 22s ago 17m 44.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:20:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (2m) 82s ago 14m 32.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (4m) 22s ago 15m 17.0M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (4m) 82s ago 14m 14.7M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (64s) 22s ago 13m 86.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (28s) 22s ago 13m 12.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (12m) 22s ago 12m 195M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 07841aa03454 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (12m) 82s ago 12m 278M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (11m) 82s ago 11m 246M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (11m) 82s ago 11m 225M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:20:42.588 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (4m) 22s ago 15m 58.7M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:20:42.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.586+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f47f406e140 msgr2=0x7f47f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.586+0000 7f480d57e700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f47f406e140 0x7f47f40705f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f47f8004560 tx=0x7f47f8009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:42.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.586+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 msgr2=0x7f4808114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.586+0000 7f480d57e700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 0x7f4808114090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c830 tx=0x7f47f0004470 comp rx=0 tx=0).stop 2024-04-24T23:20:42.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.586+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 shutdown_connections 2024-04-24T23:20:42.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.586+0000 7f480d57e700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f47f406e140 0x7f47f40705f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.586+0000 7f480d57e700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f48080fbd40 0x7f4808113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.586+0000 7f480d57e700 1 --2- 172.21.15.31:0/2357833923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4808101d90 0x7f4808114090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.587+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 >> 172.21.15.31:0/2357833923 conn(0x7f4808071d00 msgr2=0x7f4808065610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:42.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.587+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 shutdown_connections 2024-04-24T23:20:42.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.587+0000 7f480d57e700 1 -- 172.21.15.31:0/2357833923 wait complete. 2024-04-24T23:20:42.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 -- 172.21.15.31:0/1782800565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 msgr2=0x7f663c0fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 --2- 172.21.15.31:0/1782800565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 0x7f663c0fe240 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f662c004940 tx=0x7f662c009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 -- 172.21.15.31:0/1782800565 shutdown_connections 2024-04-24T23:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 --2- 172.21.15.31:0/1782800565 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f663c0fe810 0x7f663c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 --2- 172.21.15.31:0/1782800565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 0x7f663c0fe240 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 -- 172.21.15.31:0/1782800565 >> 172.21.15.31:0/1782800565 conn(0x7f663c0f78e0 msgr2=0x7f663c0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:42.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 -- 172.21.15.31:0/1782800565 shutdown_connections 2024-04-24T23:20:42.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 -- 172.21.15.31:0/1782800565 wait complete. 2024-04-24T23:20:42.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.698+0000 7f6641cbf700 1 Processor -- start 2024-04-24T23:20:42.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f6641cbf700 1 -- start start 2024-04-24T23:20:42.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f6641cbf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 0x7f663c0fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f6641cbf700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f663c0fe810 0x7f663c0fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f6641cbf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0fdc60 con 0x7f663c0fde70 2024-04-24T23:20:42.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f6641cbf700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0fc720 con 0x7f663c0fe810 2024-04-24T23:20:42.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f663b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 0x7f663c0fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f663b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 0x7f663c0fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41626/0 (socket says 172.21.15.31:41626) 2024-04-24T23:20:42.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f663b7fe700 1 -- 172.21.15.31:0/3972636978 learned_addr learned my addr 172.21.15.31:0/3972636978 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:42.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f663affd700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f663c0fe810 0x7f663c0fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f663b7fe700 1 -- 172.21.15.31:0/3972636978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f663c0fe810 msgr2=0x7f663c0fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:42.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f663b7fe700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f663c0fe810 0x7f663c0fc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:42.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.699+0000 7f663b7fe700 1 -- 172.21.15.31:0/3972636978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c005040 con 0x7f663c0fde70 2024-04-24T23:20:42.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.700+0000 7f663b7fe700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 0x7f663c0fbca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f662c009380 tx=0x7f662c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:42.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.700+0000 7f6623fff700 1 -- 172.21.15.31:0/3972636978 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f662c00e040 con 0x7f663c0fde70 2024-04-24T23:20:42.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.700+0000 7f6623fff700 1 -- 172.21.15.31:0/3972636978 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f662c009af0 con 0x7f663c0fde70 2024-04-24T23:20:42.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.700+0000 7f6623fff700 1 -- 172.21.15.31:0/3972636978 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f662c01c3f0 con 0x7f663c0fde70 2024-04-24T23:20:42.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.700+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c0fc9a0 con 0x7f663c0fde70 2024-04-24T23:20:42.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.700+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c119fe0 con 0x7f663c0fde70 2024-04-24T23:20:42.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.701+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c066ad0 con 0x7f663c0fde70 2024-04-24T23:20:42.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.702+0000 7f6623fff700 1 -- 172.21.15.31:0/3972636978 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f662c013420 con 0x7f663c0fde70 2024-04-24T23:20:42.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.702+0000 7f6623fff700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f662406e140 0x7f66240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:42.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.702+0000 7f6623fff700 1 -- 172.21.15.31:0/3972636978 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f662c08eda0 con 0x7f663c0fde70 2024-04-24T23:20:42.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.702+0000 7f663affd700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f662406e140 0x7f66240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:42.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.703+0000 7f663affd700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f662406e140 0x7f66240705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f663c1040f0 tx=0x7f66300091a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:42.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:42.707+0000 7f6623fff700 1 -- 172.21.15.31:0/3972636978 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f662c058170 con 0x7f663c0fde70 2024-04-24T23:20:43.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.016+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f663c11a280 con 0x7f663c0fde70 2024-04-24T23:20:43.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.016+0000 7f6623fff700 1 -- 172.21.15.31:0/3972636978 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f662c05b830 con 0x7f663c0fde70 2024-04-24T23:20:43.018 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:20:43.018 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:20:43.018 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 8, 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T23:20:43.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:20:43.020 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:20:43.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.018+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f662406e140 msgr2=0x7f66240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.018+0000 7f6641cbf700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f662406e140 0x7f66240705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f663c1040f0 tx=0x7f66300091a0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.018+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 msgr2=0x7f663c0fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.018+0000 7f6641cbf700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 0x7f663c0fbca0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f662c009380 tx=0x7f662c009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.019+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 shutdown_connections 2024-04-24T23:20:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.019+0000 7f6641cbf700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f663c0fde70 0x7f663c0fbca0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.019+0000 7f6641cbf700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f662406e140 0x7f66240705f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.019+0000 7f6641cbf700 1 --2- 172.21.15.31:0/3972636978 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f663c0fe810 0x7f663c0fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.019+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 >> 172.21.15.31:0/3972636978 conn(0x7f663c0f78e0 msgr2=0x7f663c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.019+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 shutdown_connections 2024-04-24T23:20:43.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.019+0000 7f6641cbf700 1 -- 172.21.15.31:0/3972636978 wait complete. 2024-04-24T23:20:43.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.126+0000 7f5a2b274700 1 -- 172.21.15.31:0/796432682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241026b0 msgr2=0x7f5a240fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.126+0000 7f5a2b274700 1 --2- 172.21.15.31:0/796432682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241026b0 0x7f5a240fbd20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5a20004940 tx=0x7f5a20009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:43.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.126+0000 7f5a2b274700 1 -- 172.21.15.31:0/796432682 shutdown_connections 2024-04-24T23:20:43.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.126+0000 7f5a2b274700 1 --2- 172.21.15.31:0/796432682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a241026b0 0x7f5a240fbd20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.126+0000 7f5a2b274700 1 --2- 172.21.15.31:0/796432682 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a24101d10 0x7f5a241020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.126+0000 7f5a2b274700 1 -- 172.21.15.31:0/796432682 >> 172.21.15.31:0/796432682 conn(0x7f5a240f7860 msgr2=0x7f5a240f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:43.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.126+0000 7f5a2b274700 1 -- 172.21.15.31:0/796432682 shutdown_connections 2024-04-24T23:20:43.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.126+0000 7f5a2b274700 1 -- 172.21.15.31:0/796432682 wait complete. 2024-04-24T23:20:43.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a2b274700 1 Processor -- start 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a2b274700 1 -- start start 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a2b274700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a24101d10 0x7f5a240fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a2b274700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a241026b0 0x7f5a240fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a2b274700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240ff8e0 con 0x7f5a24101d10 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a2b274700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240fe410 con 0x7f5a241026b0 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a29010700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a24101d10 0x7f5a240fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a29010700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a24101d10 0x7f5a240fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41630/0 (socket says 172.21.15.31:41630) 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.127+0000 7f5a29010700 1 -- 172.21.15.31:0/1622708466 learned_addr learned my addr 172.21.15.31:0/1622708466 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a2880f700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a241026b0 0x7f5a240fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a29010700 1 -- 172.21.15.31:0/1622708466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a241026b0 msgr2=0x7f5a240fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a29010700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a241026b0 0x7f5a240fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a29010700 1 -- 172.21.15.31:0/1622708466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a20005040 con 0x7f5a24101d10 2024-04-24T23:20:43.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a29010700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a24101d10 0x7f5a240fd990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a18002a50 tx=0x7f5a180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a15ffb700 1 -- 172.21.15.31:0/1622708466 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a18007cc0 con 0x7f5a24101d10 2024-04-24T23:20:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a240fe6f0 con 0x7f5a24101d10 2024-04-24T23:20:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a15ffb700 1 -- 172.21.15.31:0/1622708466 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a1800f040 con 0x7f5a24101d10 2024-04-24T23:20:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a15ffb700 1 -- 172.21.15.31:0/1622708466 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a18009640 con 0x7f5a24101d10 2024-04-24T23:20:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.128+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2411a170 con 0x7f5a24101d10 2024-04-24T23:20:43.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.129+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a240fd460 con 0x7f5a24101d10 2024-04-24T23:20:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.130+0000 7f5a15ffb700 1 -- 172.21.15.31:0/1622708466 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5a180097a0 con 0x7f5a24101d10 2024-04-24T23:20:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.130+0000 7f5a15ffb700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5a1006e140 0x7f5a100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.130+0000 7f5a15ffb700 1 -- 172.21.15.31:0/1622708466 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5a1808d8d0 con 0x7f5a24101d10 2024-04-24T23:20:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.130+0000 7f5a2880f700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5a1006e140 0x7f5a100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.130+0000 7f5a2880f700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5a1006e140 0x7f5a100705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000b3b0 tx=0x7f5a20009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:43.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.135+0000 7f5a15ffb700 1 -- 172.21.15.31:0/1622708466 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5a18056ca0 con 0x7f5a24101d10 2024-04-24T23:20:43.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.391+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5a2411a360 con 0x7f5a24101d10 2024-04-24T23:20:43.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.391+0000 7f5a15ffb700 1 -- 172.21.15.31:0/1622708466 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f5a1805a360 con 0x7f5a24101d10 2024-04-24T23:20:43.395 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:20:43.395 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:20:43.395 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:20:43.395 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:20:43.396 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:20:43.397 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:20:43.398 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5a1006e140 msgr2=0x7f5a100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5a1006e140 0x7f5a100705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000b3b0 tx=0x7f5a20009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a24101d10 msgr2=0x7f5a240fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a24101d10 0x7f5a240fd990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a18002a50 tx=0x7f5a180046b0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 shutdown_connections 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a24101d10 0x7f5a240fd990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5a1006e140 0x7f5a100705f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 --2- 172.21.15.31:0/1622708466 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a241026b0 0x7f5a240fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.399 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 >> 172.21.15.31:0/1622708466 conn(0x7f5a240f7860 msgr2=0x7f5a240662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 shutdown_connections 2024-04-24T23:20:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.395+0000 7f5a2b274700 1 -- 172.21.15.31:0/1622708466 wait complete. 2024-04-24T23:20:43.400 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: pgmap v87: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: Upgrade: Updating osd.2 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: Deploying daemon osd.2 on smithi031 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:43.490 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:43 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3972636978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.503+0000 7f30393a2700 1 -- 172.21.15.31:0/117677153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30341035b0 msgr2=0x7f30340fcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.503+0000 7f30393a2700 1 --2- 172.21.15.31:0/117677153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30341035b0 0x7f30340fcc20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3028004940 tx=0x7f3028009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.503+0000 7f30393a2700 1 -- 172.21.15.31:0/117677153 shutdown_connections 2024-04-24T23:20:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.503+0000 7f30393a2700 1 --2- 172.21.15.31:0/117677153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30341035b0 0x7f30340fcc20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.503+0000 7f30393a2700 1 --2- 172.21.15.31:0/117677153 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3034102c10 0x7f3034102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.503+0000 7f30393a2700 1 -- 172.21.15.31:0/117677153 >> 172.21.15.31:0/117677153 conn(0x7f30340f8740 msgr2=0x7f30340fab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.503+0000 7f30393a2700 1 -- 172.21.15.31:0/117677153 shutdown_connections 2024-04-24T23:20:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.503+0000 7f30393a2700 1 -- 172.21.15.31:0/117677153 wait complete. 2024-04-24T23:20:43.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f30393a2700 1 Processor -- start 2024-04-24T23:20:43.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f30393a2700 1 -- start start 2024-04-24T23:20:43.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f30393a2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3034102c10 0x7f3034070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f30393a2700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30341035b0 0x7f30340707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f30393a2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034070d30 con 0x7f3034102c10 2024-04-24T23:20:43.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f30393a2700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034070e70 con 0x7f30341035b0 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f3033fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3034102c10 0x7f3034070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f3033fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3034102c10 0x7f3034070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41634/0 (socket says 172.21.15.31:41634) 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.504+0000 7f3033fff700 1 -- 172.21.15.31:0/2969254325 learned_addr learned my addr 172.21.15.31:0/2969254325 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f3033fff700 1 -- 172.21.15.31:0/2969254325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30341035b0 msgr2=0x7f30340707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f30337fe700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30341035b0 0x7f30340707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f3033fff700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30341035b0 0x7f30340707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f3033fff700 1 -- 172.21.15.31:0/2969254325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3028005040 con 0x7f3034102c10 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f3033fff700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3034102c10 0x7f3034070260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3024002a50 tx=0x7f30240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f3030ff9700 1 -- 172.21.15.31:0/2969254325 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3024007cc0 con 0x7f3034102c10 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f3030ff9700 1 -- 172.21.15.31:0/2969254325 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f302400f040 con 0x7f3034102c10 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3034119cc0 con 0x7f3034102c10 2024-04-24T23:20:43.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f3030ff9700 1 -- 172.21.15.31:0/2969254325 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3024009640 con 0x7f3034102c10 2024-04-24T23:20:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.505+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303411a140 con 0x7f3034102c10 2024-04-24T23:20:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.506+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30340fe360 con 0x7f3034102c10 2024-04-24T23:20:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.507+0000 7f3030ff9700 1 -- 172.21.15.31:0/2969254325 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f30240097a0 con 0x7f3034102c10 2024-04-24T23:20:43.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.507+0000 7f3030ff9700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f301c06e140 0x7f301c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.507+0000 7f3030ff9700 1 -- 172.21.15.31:0/2969254325 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f302408d840 con 0x7f3034102c10 2024-04-24T23:20:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.507+0000 7f30337fe700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f301c06e140 0x7f301c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.508+0000 7f30337fe700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f301c06e140 0x7f301c0705f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3028000c00 tx=0x7f3028009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:43.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.512+0000 7f3030ff9700 1 -- 172.21.15.31:0/2969254325 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3024056c90 con 0x7f3034102c10 2024-04-24T23:20:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: pgmap v87: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: Upgrade: Updating osd.2 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: Deploying daemon osd.2 on smithi031 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:43 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3972636978' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:20:43.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.747+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3034071950 con 0x7f301c06e140 2024-04-24T23:20:43.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.748+0000 7f3030ff9700 1 -- 172.21.15.31:0/2969254325 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3034071950 con 0x7f301c06e140 2024-04-24T23:20:43.750 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:20:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:20:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:20:43.750 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "8/21 daemons upgraded", 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:20:43.751 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:20:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f301c06e140 msgr2=0x7f301c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f301c06e140 0x7f301c0705f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3028000c00 tx=0x7f3028009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3034102c10 msgr2=0x7f3034070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3034102c10 0x7f3034070260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3024002a50 tx=0x7f30240046b0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 shutdown_connections 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3034102c10 0x7f3034070260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f301c06e140 0x7f301c0705f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 --2- 172.21.15.31:0/2969254325 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f30341035b0 0x7f30340707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 >> 172.21.15.31:0/2969254325 conn(0x7f30340f8740 msgr2=0x7f30340f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 shutdown_connections 2024-04-24T23:20:43.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.751+0000 7f30393a2700 1 -- 172.21.15.31:0/2969254325 wait complete. 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 -- 172.21.15.31:0/3489597065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78102df0 msgr2=0x7feb781031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.862 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 --2- 172.21.15.31:0/3489597065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78102df0 0x7feb781031c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feb74004880 tx=0x7feb74009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 -- 172.21.15.31:0/3489597065 shutdown_connections 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 --2- 172.21.15.31:0/3489597065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb78103790 0x7feb780fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 --2- 172.21.15.31:0/3489597065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78102df0 0x7feb781031c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 -- 172.21.15.31:0/3489597065 >> 172.21.15.31:0/3489597065 conn(0x7feb780f8980 msgr2=0x7feb780fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 -- 172.21.15.31:0/3489597065 shutdown_connections 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 -- 172.21.15.31:0/3489597065 wait complete. 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 Processor -- start 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.858+0000 7feb80150700 1 -- start start 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb80150700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb78102df0 0x7feb78070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb80150700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78103790 0x7feb780707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb80150700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb78070d30 con 0x7feb78103790 2024-04-24T23:20:43.863 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb80150700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb78070e70 con 0x7feb78102df0 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7d6eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78103790 0x7feb780707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7d6eb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78103790 0x7feb780707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41662/0 (socket says 172.21.15.31:41662) 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7d6eb700 1 -- 172.21.15.31:0/3308160241 learned_addr learned my addr 172.21.15.31:0/3308160241 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7deec700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb78102df0 0x7feb78070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7d6eb700 1 -- 172.21.15.31:0/3308160241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb78102df0 msgr2=0x7feb78070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7d6eb700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb78102df0 0x7feb78070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7d6eb700 1 -- 172.21.15.31:0/3308160241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb74005040 con 0x7feb78103790 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7deec700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb78102df0 0x7feb78070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb7d6eb700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78103790 0x7feb780707a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feb6800eb40 tx=0x7feb6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb6e7fc700 1 -- 172.21.15.31:0/3308160241 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb68007cc0 con 0x7feb78103790 2024-04-24T23:20:43.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.859+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb780710c0 con 0x7feb78103790 2024-04-24T23:20:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.860+0000 7feb6e7fc700 1 -- 172.21.15.31:0/3308160241 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb6800f040 con 0x7feb78103790 2024-04-24T23:20:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.860+0000 7feb6e7fc700 1 -- 172.21.15.31:0/3308160241 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb68009620 con 0x7feb78103790 2024-04-24T23:20:43.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.860+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7811a310 con 0x7feb78103790 2024-04-24T23:20:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.861+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb78100100 con 0x7feb78103790 2024-04-24T23:20:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.861+0000 7feb6e7fc700 1 -- 172.21.15.31:0/3308160241 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feb68009780 con 0x7feb78103790 2024-04-24T23:20:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.861+0000 7feb6e7fc700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7feb6406e150 0x7feb64070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:20:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.862+0000 7feb6e7fc700 1 -- 172.21.15.31:0/3308160241 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7feb6808d5c0 con 0x7feb78103790 2024-04-24T23:20:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.862+0000 7feb7deec700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7feb6406e150 0x7feb64070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:20:43.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.862+0000 7feb7deec700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7feb6406e150 0x7feb64070600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb74009fd0 tx=0x7feb74009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:20:43.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:43.871+0000 7feb6e7fc700 1 -- 172.21.15.31:0/3308160241 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feb68056a40 con 0x7feb78103790 2024-04-24T23:20:44.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.181+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feb780fe540 con 0x7feb78103790 2024-04-24T23:20:44.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.181+0000 7feb6e7fc700 1 -- 172.21.15.31:0/3308160241 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7feb6805a100 con 0x7feb78103790 2024-04-24T23:20:44.183 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:20:44.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7feb6406e150 msgr2=0x7feb64070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7feb6406e150 0x7feb64070600 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb74009fd0 tx=0x7feb74009e30 comp rx=0 tx=0).stop 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78103790 msgr2=0x7feb780707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78103790 0x7feb780707a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feb6800eb40 tx=0x7feb6800b040 comp rx=0 tx=0).stop 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 shutdown_connections 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7feb6406e150 0x7feb64070600 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb78102df0 0x7feb78070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 --2- 172.21.15.31:0/3308160241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb78103790 0x7feb780707a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 >> 172.21.15.31:0/3308160241 conn(0x7feb780f8980 msgr2=0x7feb781073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 shutdown_connections 2024-04-24T23:20:44.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:20:44.184+0000 7feb80150700 1 -- 172.21.15.31:0/3308160241 wait complete. 2024-04-24T23:20:44.278 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:44 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1622708466' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:20:44.278 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:20:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:44 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1622708466' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:20:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:20:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:45 smithi031 ceph-mon[111404]: pgmap v88: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:45 smithi031 ceph-mon[111404]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:45.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:45 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3308160241' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:20:45.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:45 smithi040 ceph-mon[83047]: pgmap v88: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:45.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:45 smithi040 ceph-mon[83047]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:20:45.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:45 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3308160241' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:20:45.846 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:45 smithi031 systemd[1]: Stopping Ceph osd.2 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:46.196 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:45 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[60435]: 2024-04-24T23:20:45.843+0000 7f98752f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:20:46.196 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:45 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[60435]: 2024-04-24T23:20:45.843+0000 7f98752f3700 -1 osd.2 59 *** Got signal Terminated *** 2024-04-24T23:20:46.196 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:45 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[60435]: 2024-04-24T23:20:45.843+0000 7f98752f3700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:20:46.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:46 smithi031 ceph-mon[111404]: osd.2 marked itself down and dead 2024-04-24T23:20:46.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:46 smithi040 ceph-mon[83047]: osd.2 marked itself down and dead 2024-04-24T23:20:47.071 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:46 smithi031 podman[134428]: 2024-04-24 23:20:46.758139259 +0000 UTC m=+1.041466416 container died 07841aa034542c18dcfc3172920fdee7ed9d585ed3500e210b5c5e71fde9ba6e (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T23:20:47.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:47 smithi031 ceph-mon[111404]: pgmap v89: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:47.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:47 smithi031 ceph-mon[111404]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:20:47.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:47 smithi031 ceph-mon[111404]: osdmap e60: 6 total, 5 up, 6 in 2024-04-24T23:20:47.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:47 smithi031 ceph-mon[111404]: osd.2 marked itself dead as of e60 2024-04-24T23:20:47.480 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:47 smithi031 podman[134428]: 2024-04-24 23:20:47.261270967 +0000 UTC m=+1.544598168 container cleanup 07841aa034542c18dcfc3172920fdee7ed9d585ed3500e210b5c5e71fde9ba6e (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-24T23:20:47.481 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:47 smithi031 bash[134428]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2 2024-04-24T23:20:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:47 smithi040 ceph-mon[83047]: pgmap v89: 65 pgs: 65 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:47 smithi040 ceph-mon[83047]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:20:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:47 smithi040 ceph-mon[83047]: osdmap e60: 6 total, 5 up, 6 in 2024-04-24T23:20:47.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:47 smithi040 ceph-mon[83047]: osd.2 marked itself dead as of e60 2024-04-24T23:20:47.749 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:47 smithi031 podman[134447]: 2024-04-24 23:20:47.479113339 +0000 UTC m=+0.720363805 container remove 07841aa034542c18dcfc3172920fdee7ed9d585ed3500e210b5c5e71fde9ba6e (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:20:48.119 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:47 smithi031 podman[134541]: 2024-04-24 23:20:47.723640104 +0000 UTC m=+0.019195865 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:48.119 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:47 smithi031 podman[134541]: 2024-04-24 23:20:47.824084915 +0000 UTC m=+0.119640664 container create 0e7267ca841d323c23f53f097ab4156ac69e94386099cf178c72b06092b0e574 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:20:48.119 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:48 smithi031 podman[134541]: 2024-04-24 23:20:48.043454874 +0000 UTC m=+0.339010634 container init 0e7267ca841d323c23f53f097ab4156ac69e94386099cf178c72b06092b0e574 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T23:20:48.119 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:48 smithi031 podman[134541]: 2024-04-24 23:20:48.046857881 +0000 UTC m=+0.342413639 container start 0e7267ca841d323c23f53f097ab4156ac69e94386099cf178c72b06092b0e574 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:20:48.120 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:48 smithi031 podman[134541]: 2024-04-24 23:20:48.118096589 +0000 UTC m=+0.413652341 container attach 0e7267ca841d323c23f53f097ab4156ac69e94386099cf178c72b06092b0e574 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:20:48.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:48 smithi031 ceph-mon[111404]: osdmap e61: 6 total, 5 up, 6 in 2024-04-24T23:20:48.499 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:48 smithi031 podman[134541]: 2024-04-24 23:20:48.346857216 +0000 UTC m=+0.642412966 container died 0e7267ca841d323c23f53f097ab4156ac69e94386099cf178c72b06092b0e574 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:20:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:48 smithi040 ceph-mon[83047]: osdmap e61: 6 total, 5 up, 6 in 2024-04-24T23:20:49.159 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:48 smithi031 podman[134568]: 2024-04-24 23:20:48.905210091 +0000 UTC m=+0.552791304 container remove 0e7267ca841d323c23f53f097ab4156ac69e94386099cf178c72b06092b0e574 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T23:20:49.159 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:48 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-24T23:20:49.159 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:48 smithi031 systemd[1]: Stopped Ceph osd.2 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:20:49.159 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:48 smithi031 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.2.service: Consumed 58.433s CPU time. 2024-04-24T23:20:49.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:49 smithi031 ceph-mon[111404]: pgmap v92: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:49.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:49 smithi031 ceph-mon[111404]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-24T23:20:49.480 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:49 smithi031 systemd[1]: Starting Ceph osd.2 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:20:49.480 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:49 smithi031 podman[134670]: 2024-04-24 23:20:49.373058628 +0000 UTC m=+0.109476252 container create b4ff3b32d70d12c15a26a2743ca8deed4322e2ce85e6c1a065017ee9cf20b820 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-24T23:20:49.480 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:49 smithi031 podman[134670]: 2024-04-24 23:20:49.28260571 +0000 UTC m=+0.019023346 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:49.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:49 smithi040 ceph-mon[83047]: pgmap v92: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:20:49.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:49 smithi040 ceph-mon[83047]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-04-24T23:20:49.749 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:49 smithi031 podman[134670]: 2024-04-24 23:20:49.575810493 +0000 UTC m=+0.312228119 container init b4ff3b32d70d12c15a26a2743ca8deed4322e2ce85e6c1a065017ee9cf20b820 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-24T23:20:49.749 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:49 smithi031 podman[134670]: 2024-04-24 23:20:49.579100448 +0000 UTC m=+0.315518077 container start b4ff3b32d70d12c15a26a2743ca8deed4322e2ce85e6c1a065017ee9cf20b820 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-24T23:20:49.749 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:49 smithi031 podman[134670]: 2024-04-24 23:20:49.650541808 +0000 UTC m=+0.386959445 container attach b4ff3b32d70d12c15a26a2743ca8deed4322e2ce85e6c1a065017ee9cf20b820 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:20:50.355 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:50 smithi031 ceph-mon[111404]: Health check failed: Degraded data redundancy: 10/225 objects degraded (4.444%), 5 pgs degraded (PG_DEGRADED) 2024-04-24T23:20:50.355 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate[134685]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T23:20:50.355 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate[134685]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T23:20:50.355 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 bash[134670]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T23:20:50.355 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 bash[134670]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T23:20:50.355 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 bash[134670]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 bash[134670]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 bash[134670]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 bash[134670]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 bash[134670]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate[134685]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate[134685]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate[134685]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate[134685]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate[134685]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-24T23:20:50.356 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 podman[134670]: 2024-04-24 23:20:50.066288226 +0000 UTC m=+0.802705850 container died b4ff3b32d70d12c15a26a2743ca8deed4322e2ce85e6c1a065017ee9cf20b820 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-24T23:20:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:50 smithi040 ceph-mon[83047]: Health check failed: Degraded data redundancy: 10/225 objects degraded (4.444%), 5 pgs degraded (PG_DEGRADED) 2024-04-24T23:20:50.945 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 podman[134808]: 2024-04-24 23:20:50.621244772 +0000 UTC m=+0.549488932 container remove b4ff3b32d70d12c15a26a2743ca8deed4322e2ce85e6c1a065017ee9cf20b820 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:20:50.946 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 podman[134875]: 2024-04-24 23:20:50.742993507 +0000 UTC m=+0.018823750 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:20:50.946 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:50 smithi031 podman[134875]: 2024-04-24 23:20:50.847106108 +0000 UTC m=+0.122936377 container create 271a0764ef5cdd1e84a0026a687ce1005370868875dc8225bdd31748ae201d87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:20:51.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:51 smithi031 podman[134875]: 2024-04-24 23:20:51.066632342 +0000 UTC m=+0.342462566 container init 271a0764ef5cdd1e84a0026a687ce1005370868875dc8225bdd31748ae201d87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:20:51.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:51 smithi031 podman[134875]: 2024-04-24 23:20:51.069752748 +0000 UTC m=+0.345582975 container start 271a0764ef5cdd1e84a0026a687ce1005370868875dc8225bdd31748ae201d87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-24T23:20:51.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:51 smithi031 ceph-osd[134894]: -- 172.21.15.31:0/3676240693 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cb2903680 con 0x560cb28cd000 2024-04-24T23:20:51.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:51 smithi031 bash[134875]: 271a0764ef5cdd1e84a0026a687ce1005370868875dc8225bdd31748ae201d87 2024-04-24T23:20:51.249 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:51 smithi031 systemd[1]: Started Ceph osd.2 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:20:51.612 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:51 smithi031 ceph-mon[111404]: pgmap v93: 65 pgs: 5 active+undersized+degraded, 7 peering, 3 stale+active+clean, 7 active+undersized, 43 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 10/225 objects degraded (4.444%) 2024-04-24T23:20:51.612 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:51 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:51.612 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:51 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:51.612 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:51 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:51 smithi040 ceph-mon[83047]: pgmap v93: 65 pgs: 5 active+undersized+degraded, 7 peering, 3 stale+active+clean, 7 active+undersized, 43 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 10/225 objects degraded (4.444%) 2024-04-24T23:20:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:51 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:51 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:51.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:51 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:20:51.999 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:51 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[134890]: 2024-04-24T23:20:51.609+0000 7fc9497487c0 -1 Falling back to public interface 2024-04-24T23:20:52.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:52 smithi040 ceph-mon[83047]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-04-24T23:20:52.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:52 smithi031 ceph-mon[111404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-04-24T23:20:53.585 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:53 smithi031 ceph-mon[111404]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/225 objects degraded (12.889%) 2024-04-24T23:20:53.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:53 smithi040 ceph-mon[83047]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/225 objects degraded (12.889%) 2024-04-24T23:20:55.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:55 smithi031 ceph-mon[111404]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/225 objects degraded (12.889%) 2024-04-24T23:20:55.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:55 smithi040 ceph-mon[83047]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/225 objects degraded (12.889%) 2024-04-24T23:20:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:56 smithi031 ceph-mon[111404]: from='osd.2 [v2:172.21.15.31:6818/1599239694,v1:172.21.15.31:6819/1599239694]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:20:56.499 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:56 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[134890]: 2024-04-24T23:20:56.245+0000 7fc9497487c0 -1 osd.2 60 log_to_monitors true 2024-04-24T23:20:56.499 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:20:56 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[134890]: 2024-04-24T23:20:56.342+0000 7fc93aba0700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:20:56.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:56 smithi040 ceph-mon[83047]: from='osd.2 [v2:172.21.15.31:6818/1599239694,v1:172.21.15.31:6819/1599239694]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-24T23:20:57.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:57 smithi040 ceph-mon[83047]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/225 objects degraded (12.889%) 2024-04-24T23:20:57.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:57 smithi040 ceph-mon[83047]: from='osd.2 [v2:172.21.15.31:6818/1599239694,v1:172.21.15.31:6819/1599239694]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:20:57.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:57 smithi040 ceph-mon[83047]: osdmap e62: 6 total, 5 up, 6 in 2024-04-24T23:20:57.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:57 smithi040 ceph-mon[83047]: from='osd.2 [v2:172.21.15.31:6818/1599239694,v1:172.21.15.31:6819/1599239694]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:20:57.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:57 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:57.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:57 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:57 smithi031 ceph-mon[111404]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 205 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 29/225 objects degraded (12.889%) 2024-04-24T23:20:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:57 smithi031 ceph-mon[111404]: from='osd.2 [v2:172.21.15.31:6818/1599239694,v1:172.21.15.31:6819/1599239694]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-24T23:20:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:57 smithi031 ceph-mon[111404]: osdmap e62: 6 total, 5 up, 6 in 2024-04-24T23:20:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:57 smithi031 ceph-mon[111404]: from='osd.2 [v2:172.21.15.31:6818/1599239694,v1:172.21.15.31:6819/1599239694]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-04-24T23:20:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:57 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:57.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:57 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:58.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:58 smithi031 ceph-mon[111404]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:20:58.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:58 smithi031 ceph-mon[111404]: osd.2 [v2:172.21.15.31:6818/1599239694,v1:172.21.15.31:6819/1599239694] boot 2024-04-24T23:20:58.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:58 smithi031 ceph-mon[111404]: osdmap e63: 6 total, 6 up, 6 in 2024-04-24T23:20:58.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:20:58.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:58.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:58.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:58 smithi031 ceph-mon[111404]: osdmap e64: 6 total, 6 up, 6 in 2024-04-24T23:20:58.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:58 smithi040 ceph-mon[83047]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:20:58.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:58 smithi040 ceph-mon[83047]: osd.2 [v2:172.21.15.31:6818/1599239694,v1:172.21.15.31:6819/1599239694] boot 2024-04-24T23:20:58.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:58 smithi040 ceph-mon[83047]: osdmap e63: 6 total, 6 up, 6 in 2024-04-24T23:20:58.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-24T23:20:58.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:58.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:58.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:58 smithi040 ceph-mon[83047]: osdmap e64: 6 total, 6 up, 6 in 2024-04-24T23:20:59.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:59 smithi040 ceph-mon[83047]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29/225 objects degraded (12.889%) 2024-04-24T23:20:59.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:20:59.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:20:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:00.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:59 smithi031 ceph-mon[111404]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 29/225 objects degraded (12.889%) 2024-04-24T23:21:00.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:00.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:20:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:00.734 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:00 smithi031 ceph-mon[111404]: Health check update: Degraded data redundancy: 29/225 objects degraded (12.889%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T23:21:00.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:00 smithi040 ceph-mon[83047]: Health check update: Degraded data redundancy: 29/225 objects degraded (12.889%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T23:21:01.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: pgmap v101: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 20/225 objects degraded (8.889%) 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:01.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:01 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T23:21:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: pgmap v101: 65 pgs: 8 active+undersized+degraded, 9 active+undersized, 48 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 20/225 objects degraded (8.889%) 2024-04-24T23:21:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:01.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:02.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:01 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T23:21:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:02 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T23:21:02.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:02 smithi040 ceph-mon[83047]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-04-24T23:21:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:02 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T23:21:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:02 smithi031 ceph-mon[111404]: Upgrade: unsafe to stop osd(s) at this time (6 PGs are or would become offline) 2024-04-24T23:21:03.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:03 smithi031 ceph-mon[111404]: pgmap v102: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:03.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:03 smithi031 ceph-mon[111404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/225 objects degraded (8.889%), 8 pgs degraded) 2024-04-24T23:21:03.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:03 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:21:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:03 smithi040 ceph-mon[83047]: pgmap v102: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:03 smithi040 ceph-mon[83047]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 20/225 objects degraded (8.889%), 8 pgs degraded) 2024-04-24T23:21:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:03 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:21:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:04 smithi040 ceph-mon[83047]: pgmap v103: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:05.020 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:04 smithi031 ceph-mon[111404]: pgmap v103: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:07.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:06 smithi040 ceph-mon[83047]: pgmap v104: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:07.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:06 smithi031 ceph-mon[111404]: pgmap v104: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:09.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:08 smithi040 ceph-mon[83047]: pgmap v105: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:09.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:08 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:08 smithi031 ceph-mon[111404]: pgmap v105: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:08 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:10 smithi040 ceph-mon[83047]: pgmap v106: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:11.193 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:10 smithi031 ceph-mon[111404]: pgmap v106: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:13.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:12 smithi040 ceph-mon[83047]: pgmap v107: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:13.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:12 smithi031 ceph-mon[111404]: pgmap v107: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:14.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.302+0000 7f6293cde700 1 -- 172.21.15.31:0/933510601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 msgr2=0x7f628c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.302+0000 7f6293cde700 1 --2- 172.21.15.31:0/933510601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 0x7f628c0fbd20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f627c004940 tx=0x7f627c009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.302+0000 7f6293cde700 1 -- 172.21.15.31:0/933510601 shutdown_connections 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.302+0000 7f6293cde700 1 --2- 172.21.15.31:0/933510601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 0x7f628c0fbd20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.302+0000 7f6293cde700 1 --2- 172.21.15.31:0/933510601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c101d10 0x7f628c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.302+0000 7f6293cde700 1 -- 172.21.15.31:0/933510601 >> 172.21.15.31:0/933510601 conn(0x7f628c0f7860 msgr2=0x7f628c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.302+0000 7f6293cde700 1 -- 172.21.15.31:0/933510601 shutdown_connections 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.302+0000 7f6293cde700 1 -- 172.21.15.31:0/933510601 wait complete. 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6293cde700 1 Processor -- start 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6293cde700 1 -- start start 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6293cde700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c101d10 0x7f628c0fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6293cde700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 0x7f628c0fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.307 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6293cde700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0ff8e0 con 0x7f628c1026b0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6293cde700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c0fe410 con 0x7f628c101d10 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6291279700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 0x7f628c0fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6291a7a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c101d10 0x7f628c0fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6291279700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 0x7f628c0fded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47552/0 (socket says 172.21.15.31:47552) 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6291279700 1 -- 172.21.15.31:0/1996888057 learned_addr learned my addr 172.21.15.31:0/1996888057 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6291279700 1 -- 172.21.15.31:0/1996888057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c101d10 msgr2=0x7f628c0fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6291279700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c101d10 0x7f628c0fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.303+0000 7f6291279700 1 -- 172.21.15.31:0/1996888057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c005040 con 0x7f628c1026b0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.304+0000 7f6291279700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 0x7f628c0fded0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f627c000c00 tx=0x7f627c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.304+0000 7f62827fc700 1 -- 172.21.15.31:0/1996888057 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627c021070 con 0x7f628c1026b0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.304+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0fe690 con 0x7f628c1026b0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.304+0000 7f62827fc700 1 -- 172.21.15.31:0/1996888057 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f627c013c00 con 0x7f628c1026b0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.304+0000 7f62827fc700 1 -- 172.21.15.31:0/1996888057 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f627c00e070 con 0x7f628c1026b0 2024-04-24T23:21:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.304+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c119fe0 con 0x7f628c1026b0 2024-04-24T23:21:14.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.305+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c101230 con 0x7f628c1026b0 2024-04-24T23:21:14.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.305+0000 7f62827fc700 1 -- 172.21.15.31:0/1996888057 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f627c01ca60 con 0x7f628c1026b0 2024-04-24T23:21:14.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.306+0000 7f62827fc700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f627806e140 0x7f62780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.306+0000 7f62827fc700 1 -- 172.21.15.31:0/1996888057 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f627c08ee80 con 0x7f628c1026b0 2024-04-24T23:21:14.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.306+0000 7f6291a7a700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f627806e140 0x7f62780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.310 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.307+0000 7f6291a7a700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f627806e140 0x7f62780705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6288009b80 tx=0x7f628800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:14.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.311+0000 7f62827fc700 1 -- 172.21.15.31:0/1996888057 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f627c0582d0 con 0x7f628c1026b0 2024-04-24T23:21:14.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.543+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f628c108df0 con 0x7f627806e140 2024-04-24T23:21:14.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.545+0000 7f62827fc700 1 -- 172.21.15.31:0/1996888057 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f628c108df0 con 0x7f627806e140 2024-04-24T23:21:14.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f627806e140 msgr2=0x7f62780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f627806e140 0x7f62780705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6288009b80 tx=0x7f628800b040 comp rx=0 tx=0).stop 2024-04-24T23:21:14.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 msgr2=0x7f628c0fded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 0x7f628c0fded0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f627c000c00 tx=0x7f627c009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:14.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 shutdown_connections 2024-04-24T23:21:14.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f627806e140 0x7f62780705f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f628c101d10 0x7f628c0fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 --2- 172.21.15.31:0/1996888057 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f628c1026b0 0x7f628c0fded0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 >> 172.21.15.31:0/1996888057 conn(0x7f628c0f7860 msgr2=0x7f628c0f9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:14.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 shutdown_connections 2024-04-24T23:21:14.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.547+0000 7f6293cde700 1 -- 172.21.15.31:0/1996888057 wait complete. 2024-04-24T23:21:14.560 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1376194493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec0102f60 msgr2=0x7f4ec0103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 --2- 172.21.15.31:0/1376194493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec0102f60 0x7f4ec0103330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0004940 tx=0x7f4eb0009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1376194493 shutdown_connections 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 --2- 172.21.15.31:0/1376194493 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec00fb520 0x7f4ec00fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 --2- 172.21.15.31:0/1376194493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec0102f60 0x7f4ec0103330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1376194493 >> 172.21.15.31:0/1376194493 conn(0x7f4ec00714d0 msgr2=0x7f4ec00718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1376194493 shutdown_connections 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1376194493 wait complete. 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 Processor -- start 2024-04-24T23:21:14.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.654+0000 7f4ec5aed700 1 -- start start 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ec5aed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec00fb520 0x7f4ec0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ec5aed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec0102f60 0x7f4ec0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ec5aed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0114510 con 0x7f4ec00fb520 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ec5aed700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec010db30 con 0x7f4ec0102f60 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ebf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec00fb520 0x7f4ec0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ebf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec00fb520 0x7f4ec0113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47568/0 (socket says 172.21.15.31:47568) 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ebf7fe700 1 -- 172.21.15.31:0/1568183137 learned_addr learned my addr 172.21.15.31:0/1568183137 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ebeffd700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec0102f60 0x7f4ec0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ebf7fe700 1 -- 172.21.15.31:0/1568183137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec0102f60 msgr2=0x7f4ec0113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ebf7fe700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec0102f60 0x7f4ec0113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ebf7fe700 1 -- 172.21.15.31:0/1568183137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb0005040 con 0x7f4ec00fb520 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ebf7fe700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec00fb520 0x7f4ec0113980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b8b0 tx=0x7f4eb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.655+0000 7f4ea7fff700 1 -- 172.21.15.31:0/1568183137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eb000e040 con 0x7f4ec00fb520 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.656+0000 7f4ea7fff700 1 -- 172.21.15.31:0/1568183137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4eb0004290 con 0x7f4ec00fb520 2024-04-24T23:21:14.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.656+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec010ddb0 con 0x7f4ec00fb520 2024-04-24T23:21:14.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.656+0000 7f4ea7fff700 1 -- 172.21.15.31:0/1568183137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eb001c3f0 con 0x7f4ec00fb520 2024-04-24T23:21:14.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.656+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec010e2f0 con 0x7f4ec00fb520 2024-04-24T23:21:14.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.656+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec0100250 con 0x7f4ec00fb520 2024-04-24T23:21:14.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.657+0000 7f4ea7fff700 1 -- 172.21.15.31:0/1568183137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4eb00137b0 con 0x7f4ec00fb520 2024-04-24T23:21:14.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.657+0000 7f4ea7fff700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4ea806e140 0x7f4ea80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:14.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.658+0000 7f4ea7fff700 1 -- 172.21.15.31:0/1568183137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4eb008ed20 con 0x7f4ec00fb520 2024-04-24T23:21:14.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.658+0000 7f4ebeffd700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4ea806e140 0x7f4ea80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:14.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.658+0000 7f4ebeffd700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4ea806e140 0x7f4ea80705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4007fd0 tx=0x7f4eb4009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:14.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.662+0000 7f4ea7fff700 1 -- 172.21.15.31:0/1568183137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4eb0058210 con 0x7f4ec00fb520 2024-04-24T23:21:14.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.889+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ec0109fe0 con 0x7f4ea806e140 2024-04-24T23:21:14.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:14 smithi031 ceph-mon[111404]: pgmap v108: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:14.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:14.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:14.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:14 smithi031 ceph-mon[111404]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:14.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.891+0000 7f4ea7fff700 1 -- 172.21.15.31:0/1568183137 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4ec0109fe0 con 0x7f4ea806e140 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4ea806e140 msgr2=0x7f4ea80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4ea806e140 0x7f4ea80705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4007fd0 tx=0x7f4eb4009040 comp rx=0 tx=0).stop 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec00fb520 msgr2=0x7f4ec0113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec00fb520 0x7f4ec0113980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4eb000b8b0 tx=0x7f4eb0009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 shutdown_connections 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec00fb520 0x7f4ec0113980 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4ea806e140 0x7f4ea80705f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 --2- 172.21.15.31:0/1568183137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4ec0102f60 0x7f4ec0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 >> 172.21.15.31:0/1568183137 conn(0x7f4ec00714d0 msgr2=0x7f4ec00fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:14.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 shutdown_connections 2024-04-24T23:21:14.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.893+0000 7f4ec5aed700 1 -- 172.21.15.31:0/1568183137 wait complete. 2024-04-24T23:21:15.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.998+0000 7f2289dc9700 1 -- 172.21.15.31:0/2478492143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 msgr2=0x7f22841031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.998+0000 7f2289dc9700 1 --2- 172.21.15.31:0/2478492143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 0x7f22841031f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2274004940 tx=0x7f2274009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:15.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.998+0000 7f2289dc9700 1 -- 172.21.15.31:0/2478492143 shutdown_connections 2024-04-24T23:21:15.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.998+0000 7f2289dc9700 1 --2- 172.21.15.31:0/2478492143 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22841037c0 0x7f22840fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.998+0000 7f2289dc9700 1 --2- 172.21.15.31:0/2478492143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 0x7f22841031f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.998+0000 7f2289dc9700 1 -- 172.21.15.31:0/2478492143 >> 172.21.15.31:0/2478492143 conn(0x7f2284072380 msgr2=0x7f2284072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:15.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.998+0000 7f2289dc9700 1 -- 172.21.15.31:0/2478492143 shutdown_connections 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.998+0000 7f2289dc9700 1 -- 172.21.15.31:0/2478492143 wait complete. 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f2289dc9700 1 Processor -- start 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f2289dc9700 1 -- start start 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f2289dc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 0x7f2284113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f2289dc9700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22841037c0 0x7f2284113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f2289dc9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22841144a0 con 0x7f2284102e20 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f2289dc9700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228410daa0 con 0x7f22841037c0 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f22837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 0x7f2284113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f22837fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 0x7f2284113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47574/0 (socket says 172.21.15.31:47574) 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f22837fe700 1 -- 172.21.15.31:0/2104826282 learned_addr learned my addr 172.21.15.31:0/2104826282 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:14.999+0000 7f227bfff700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22841037c0 0x7f2284113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f22837fe700 1 -- 172.21.15.31:0/2104826282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22841037c0 msgr2=0x7f2284113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f22837fe700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22841037c0 0x7f2284113e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f22837fe700 1 -- 172.21.15.31:0/2104826282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274005040 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f22837fe700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 0x7f2284113910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2274009180 tx=0x7f2274009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f2280ff9700 1 -- 172.21.15.31:0/2104826282 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f227400e040 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f2280ff9700 1 -- 172.21.15.31:0/2104826282 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2274013420 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228410dd20 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f2280ff9700 1 -- 172.21.15.31:0/2104826282 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f227401c610 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.000+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228410e260 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.001+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2284100160 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.002+0000 7f2280ff9700 1 -- 172.21.15.31:0/2104826282 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2274013a00 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.002+0000 7f2280ff9700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.002+0000 7f2280ff9700 1 -- 172.21.15.31:0/2104826282 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f227408ebc0 con 0x7f2284102e20 2024-04-24T23:21:15.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.002+0000 7f227bfff700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.003+0000 7f227bfff700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f228410ec60 tx=0x7f226c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.007+0000 7f2280ff9700 1 -- 172.21.15.31:0/2104826282 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f227405bbf0 con 0x7f2284102e20 2024-04-24T23:21:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:14 smithi040 ceph-mon[83047]: pgmap v108: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:15.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:14 smithi040 ceph-mon[83047]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:15.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.223+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f228410ef60 con 0x7f227006e140 2024-04-24T23:21:15.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.234+0000 7f2280ff9700 1 -- 172.21.15.31:0/2104826282 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f228410ef60 con 0x7f227006e140 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (4m) 18s ago 15m 25.6M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (2m) 18s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (118s) 114s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (4m) 18s ago 15m 79.9M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (10m) 18s ago 10m 138M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (10m) 18s ago 10m 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (10m) 114s ago 10m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (10m) 114s ago 10m 15.7M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (6m) 18s ago 17m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (5m) 114s ago 15m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (3m) 18s ago 17m 45.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:21:15.237 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (2m) 114s ago 14m 32.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (5m) 18s ago 16m 16.7M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (5m) 114s ago 15m 14.7M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (97s) 18s ago 14m 87.6M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (60s) 18s ago 13m 101M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (24s) 18s ago 13m 11.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 271a0764ef5c 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (12m) 114s ago 12m 278M 2583M 17.2.7-902-gc61472c8 0de407c1b3af c3fe722ceca4 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (12m) 114s ago 12m 246M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (11m) 114s ago 11m 225M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:21:15.238 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (4m) 18s ago 15m 59.9M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:21:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 msgr2=0x7f22700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f228410ec60 tx=0x7f226c00b040 comp rx=0 tx=0).stop 2024-04-24T23:21:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 msgr2=0x7f2284113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 0x7f2284113910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2274009180 tx=0x7f2274009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 shutdown_connections 2024-04-24T23:21:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2284102e20 0x7f2284113910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 --2- 172.21.15.31:0/2104826282 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22841037c0 0x7f2284113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 >> 172.21.15.31:0/2104826282 conn(0x7f2284072380 msgr2=0x7f22840fac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:15.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 shutdown_connections 2024-04-24T23:21:15.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.236+0000 7f2289dc9700 1 -- 172.21.15.31:0/2104826282 wait complete. 2024-04-24T23:21:15.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.339+0000 7fa6130ed700 1 -- 172.21.15.31:0/1541683858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 msgr2=0x7fa60c0fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.339+0000 7fa6130ed700 1 --2- 172.21.15.31:0/1541683858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 0x7fa60c0fb3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa600004940 tx=0x7fa600009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.339+0000 7fa6130ed700 1 -- 172.21.15.31:0/1541683858 shutdown_connections 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.339+0000 7fa6130ed700 1 --2- 172.21.15.31:0/1541683858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 0x7fa60c0fb3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.339+0000 7fa6130ed700 1 --2- 172.21.15.31:0/1541683858 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa60c102e20 0x7fa60c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.339+0000 7fa6130ed700 1 -- 172.21.15.31:0/1541683858 >> 172.21.15.31:0/1541683858 conn(0x7fa60c072380 msgr2=0x7fa60c072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.339+0000 7fa6130ed700 1 -- 172.21.15.31:0/1541683858 shutdown_connections 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.339+0000 7fa6130ed700 1 -- 172.21.15.31:0/1541683858 wait complete. 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.340+0000 7fa6130ed700 1 Processor -- start 2024-04-24T23:21:15.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.340+0000 7fa6130ed700 1 -- start start 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.340+0000 7fa6130ed700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa60c102e20 0x7fa60c113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.340+0000 7fa6130ed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 0x7fa60c113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.340+0000 7fa6130ed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c1144d0 con 0x7fa60c1037c0 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.340+0000 7fa6130ed700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c10dad0 con 0x7fa60c102e20 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa60bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 0x7fa60c113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa60bfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 0x7fa60c113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47588/0 (socket says 172.21.15.31:47588) 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa60bfff700 1 -- 172.21.15.31:0/3631220117 learned_addr learned my addr 172.21.15.31:0/3631220117 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa610e89700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa60c102e20 0x7fa60c113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa60bfff700 1 -- 172.21.15.31:0/3631220117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa60c102e20 msgr2=0x7fa60c113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa60bfff700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa60c102e20 0x7fa60c113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa60bfff700 1 -- 172.21.15.31:0/3631220117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa600005040 con 0x7fa60c1037c0 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa610e89700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa60c102e20 0x7fa60c113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa60bfff700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 0x7fa60c113e80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa600009180 tx=0x7fa600009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa6097fa700 1 -- 172.21.15.31:0/3631220117 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa60000e040 con 0x7fa60c1037c0 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa6097fa700 1 -- 172.21.15.31:0/3631220117 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa60000bbc0 con 0x7fa60c1037c0 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60c10dd50 con 0x7fa60c1037c0 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa6097fa700 1 -- 172.21.15.31:0/3631220117 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa60001c3f0 con 0x7fa60c1037c0 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.341+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60c10e290 con 0x7fa60c1037c0 2024-04-24T23:21:15.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.342+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60c100160 con 0x7fa60c1037c0 2024-04-24T23:21:15.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.343+0000 7fa6097fa700 1 -- 172.21.15.31:0/3631220117 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa60000bd30 con 0x7fa60c1037c0 2024-04-24T23:21:15.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.343+0000 7fa6097fa700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa5f406e210 0x7fa5f40706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.343+0000 7fa6097fa700 1 -- 172.21.15.31:0/3631220117 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa60008edf0 con 0x7fa60c1037c0 2024-04-24T23:21:15.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.344+0000 7fa610e89700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa5f406e210 0x7fa5f40706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.344+0000 7fa610e89700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa5f406e210 0x7fa5f40706c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009b80 tx=0x7fa5fc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.348+0000 7fa6097fa700 1 -- 172.21.15.31:0/3631220117 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa60005bd50 con 0x7fa60c1037c0 2024-04-24T23:21:15.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.648+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa60c10ef90 con 0x7fa60c1037c0 2024-04-24T23:21:15.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.649+0000 7fa6097fa700 1 -- 172.21.15.31:0/3631220117 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fa60005b8e0 con 0x7fa60c1037c0 2024-04-24T23:21:15.651 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:21:15.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:21:15.652 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:21:15.653 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 7, 2024-04-24T23:21:15.653 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 7 2024-04-24T23:21:15.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:21:15.653 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa5f406e210 msgr2=0x7fa5f40706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa5f406e210 0x7fa5f40706c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009b80 tx=0x7fa5fc00b040 comp rx=0 tx=0).stop 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 msgr2=0x7fa60c113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 0x7fa60c113e80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa600009180 tx=0x7fa600009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 shutdown_connections 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa5f406e210 0x7fa5f40706c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa60c102e20 0x7fa60c113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 --2- 172.21.15.31:0/3631220117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa60c1037c0 0x7fa60c113e80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 >> 172.21.15.31:0/3631220117 conn(0x7fa60c072380 msgr2=0x7fa60c0fabb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 shutdown_connections 2024-04-24T23:21:15.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.651+0000 7fa6130ed700 1 -- 172.21.15.31:0/3631220117 wait complete. 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 -- 172.21.15.31:0/1311056965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350103790 msgr2=0x7f43500fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 --2- 172.21.15.31:0/1311056965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350103790 0x7f43500fce00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4344004940 tx=0x7f4344009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 -- 172.21.15.31:0/1311056965 shutdown_connections 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 --2- 172.21.15.31:0/1311056965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350103790 0x7f43500fce00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 --2- 172.21.15.31:0/1311056965 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4350102df0 0x7f43501031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 -- 172.21.15.31:0/1311056965 >> 172.21.15.31:0/1311056965 conn(0x7f43500f8980 msgr2=0x7f43500fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 -- 172.21.15.31:0/1311056965 shutdown_connections 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 -- 172.21.15.31:0/1311056965 wait complete. 2024-04-24T23:21:15.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.756+0000 7f435689d700 1 Processor -- start 2024-04-24T23:21:15.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f435689d700 1 -- start start 2024-04-24T23:21:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f435689d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350102df0 0x7f4350070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f435689d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4350103790 0x7f43500707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f435689d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350070d30 con 0x7f4350102df0 2024-04-24T23:21:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f435689d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350070e70 con 0x7f4350103790 2024-04-24T23:21:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f434ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350102df0 0x7f4350070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f434ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350102df0 0x7f4350070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47614/0 (socket says 172.21.15.31:47614) 2024-04-24T23:21:15.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f434ffff700 1 -- 172.21.15.31:0/4125821761 learned_addr learned my addr 172.21.15.31:0/4125821761 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f434f7fe700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4350103790 0x7f43500707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f434ffff700 1 -- 172.21.15.31:0/4125821761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4350103790 msgr2=0x7f43500707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f434ffff700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4350103790 0x7f43500707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f434ffff700 1 -- 172.21.15.31:0/4125821761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344005040 con 0x7f4350102df0 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.757+0000 7f434ffff700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350102df0 0x7f4350070260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4340002a50 tx=0x7f43400046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.758+0000 7f434cff9700 1 -- 172.21.15.31:0/4125821761 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4340007cc0 con 0x7f4350102df0 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.758+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4350119e70 con 0x7f4350102df0 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.758+0000 7f434cff9700 1 -- 172.21.15.31:0/4125821761 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f434000f040 con 0x7f4350102df0 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.758+0000 7f434cff9700 1 -- 172.21.15.31:0/4125821761 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f434000da30 con 0x7f4350102df0 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.758+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435011a290 con 0x7f4350102df0 2024-04-24T23:21:15.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.759+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43500fe540 con 0x7f4350102df0 2024-04-24T23:21:15.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.759+0000 7f434cff9700 1 -- 172.21.15.31:0/4125821761 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f434000db90 con 0x7f4350102df0 2024-04-24T23:21:15.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.760+0000 7f434cff9700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f433806e140 0x7f43380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:15.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.760+0000 7f434cff9700 1 -- 172.21.15.31:0/4125821761 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f434008d720 con 0x7f4350102df0 2024-04-24T23:21:15.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.760+0000 7f434f7fe700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f433806e140 0x7f43380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:15.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.760+0000 7f434f7fe700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f433806e140 0x7f43380705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4344004890 tx=0x7f4344009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:15.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:15.765+0000 7f434cff9700 1 -- 172.21.15.31:0/4125821761 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4340056b70 con 0x7f4350102df0 2024-04-24T23:21:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:15 smithi031 ceph-mon[111404]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:15 smithi031 ceph-mon[111404]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:15.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:15 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3631220117' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.017+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f435004bed0 con 0x7f4350102df0 2024-04-24T23:21:16.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.018+0000 7f434cff9700 1 -- 172.21.15.31:0/4125821761 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f434005a230 con 0x7f4350102df0 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:21:16.021 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:21:16.022 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:21:16.023 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f433806e140 msgr2=0x7f43380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f433806e140 0x7f43380705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4344004890 tx=0x7f4344009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350102df0 msgr2=0x7f4350070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350102df0 0x7f4350070260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4340002a50 tx=0x7f43400046b0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 shutdown_connections 2024-04-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4350102df0 0x7f4350070260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f433806e140 0x7f43380705f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 --2- 172.21.15.31:0/4125821761 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4350103790 0x7f43500707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 >> 172.21.15.31:0/4125821761 conn(0x7f43500f8980 msgr2=0x7f43500f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 shutdown_connections 2024-04-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.021+0000 7f435689d700 1 -- 172.21.15.31:0/4125821761 wait complete. 2024-04-24T23:21:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:21:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.125+0000 7f6918227700 1 -- 172.21.15.31:0/1835994813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69101026a0 msgr2=0x7f69100fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.125+0000 7f6918227700 1 --2- 172.21.15.31:0/1835994813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69101026a0 0x7f69100fbd10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6900004940 tx=0x7f6900009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.125+0000 7f6918227700 1 -- 172.21.15.31:0/1835994813 shutdown_connections 2024-04-24T23:21:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.125+0000 7f6918227700 1 --2- 172.21.15.31:0/1835994813 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69101026a0 0x7f69100fbd10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.125+0000 7f6918227700 1 --2- 172.21.15.31:0/1835994813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6910101d00 0x7f69101020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.125+0000 7f6918227700 1 -- 172.21.15.31:0/1835994813 >> 172.21.15.31:0/1835994813 conn(0x7f69100f7870 msgr2=0x7f69100f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.125+0000 7f6918227700 1 -- 172.21.15.31:0/1835994813 shutdown_connections 2024-04-24T23:21:16.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.125+0000 7f6918227700 1 -- 172.21.15.31:0/1835994813 wait complete. 2024-04-24T23:21:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6918227700 1 Processor -- start 2024-04-24T23:21:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6918227700 1 -- start start 2024-04-24T23:21:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6918227700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6910101d00 0x7f6910113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6918227700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69101026a0 0x7f6910113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6918227700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6910114490 con 0x7f6910101d00 2024-04-24T23:21:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6918227700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691010da90 con 0x7f69101026a0 2024-04-24T23:21:16.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6915fc3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6910101d00 0x7f6910113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6915fc3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6910101d00 0x7f6910113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47640/0 (socket says 172.21.15.31:47640) 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f6915fc3700 1 -- 172.21.15.31:0/4034963000 learned_addr learned my addr 172.21.15.31:0/4034963000 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.126+0000 7f69157c2700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69101026a0 0x7f6910113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f6915fc3700 1 -- 172.21.15.31:0/4034963000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69101026a0 msgr2=0x7f6910113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f6915fc3700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69101026a0 0x7f6910113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f6915fc3700 1 -- 172.21.15.31:0/4034963000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6900005040 con 0x7f6910101d00 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f6915fc3700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6910101d00 0x7f6910113900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f690c002a80 tx=0x7f690c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f69067fc700 1 -- 172.21.15.31:0/4034963000 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f690c007cc0 con 0x7f6910101d00 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f69067fc700 1 -- 172.21.15.31:0/4034963000 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f690c00f040 con 0x7f6910101d00 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691010dd70 con 0x7f6910101d00 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f69067fc700 1 -- 172.21.15.31:0/4034963000 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f690c00dac0 con 0x7f6910101d00 2024-04-24T23:21:16.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.127+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691010e300 con 0x7f6910101d00 2024-04-24T23:21:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.128+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69100fd450 con 0x7f6910101d00 2024-04-24T23:21:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.129+0000 7f69067fc700 1 -- 172.21.15.31:0/4034963000 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f690c00dc20 con 0x7f6910101d00 2024-04-24T23:21:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.129+0000 7f69067fc700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f68fc06e140 0x7f68fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.129+0000 7f69067fc700 1 -- 172.21.15.31:0/4034963000 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f690c08d730 con 0x7f6910101d00 2024-04-24T23:21:16.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.129+0000 7f69157c2700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f68fc06e140 0x7f68fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:16.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.130+0000 7f69157c2700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f68fc06e140 0x7f68fc0705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f691010ed70 tx=0x7f6900009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:16.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.135+0000 7f69067fc700 1 -- 172.21.15.31:0/4034963000 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f690c05a630 con 0x7f6910101d00 2024-04-24T23:21:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:15 smithi040 ceph-mon[83047]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:15 smithi040 ceph-mon[83047]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:15 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3631220117' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:16.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.361+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6910108e30 con 0x7f68fc06e140 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.362+0000 7f69067fc700 1 -- 172.21.15.31:0/4034963000 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6910108e30 con 0x7f68fc06e140 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:21:16.365 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:21:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "9/21 daemons upgraded", 2024-04-24T23:21:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T23:21:16.366 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:21:16.366 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:21:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.365+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f68fc06e140 msgr2=0x7f68fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.365+0000 7f6918227700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f68fc06e140 0x7f68fc0705f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f691010ed70 tx=0x7f6900009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.365+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6910101d00 msgr2=0x7f6910113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.365+0000 7f6918227700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6910101d00 0x7f6910113900 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f690c002a80 tx=0x7f690c0046b0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.366+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 shutdown_connections 2024-04-24T23:21:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.366+0000 7f6918227700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6910101d00 0x7f6910113900 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.366+0000 7f6918227700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f68fc06e140 0x7f68fc0705f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.366+0000 7f6918227700 1 --2- 172.21.15.31:0/4034963000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f69101026a0 0x7f6910113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.366+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 >> 172.21.15.31:0/4034963000 conn(0x7f69100f7870 msgr2=0x7f69100f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.366+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 shutdown_connections 2024-04-24T23:21:16.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.366+0000 7f6918227700 1 -- 172.21.15.31:0/4034963000 wait complete. 2024-04-24T23:21:16.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 -- 172.21.15.31:0/3802304009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 msgr2=0x7fb9380fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 --2- 172.21.15.31:0/3802304009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 0x7fb9380fcdf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb92c004940 tx=0x7fb92c009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 -- 172.21.15.31:0/3802304009 shutdown_connections 2024-04-24T23:21:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 --2- 172.21.15.31:0/3802304009 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 0x7fb9380fcdf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 --2- 172.21.15.31:0/3802304009 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb938102de0 0x7fb9381031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 -- 172.21.15.31:0/3802304009 >> 172.21.15.31:0/3802304009 conn(0x7fb9380f8990 msgr2=0x7fb9380fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 -- 172.21.15.31:0/3802304009 shutdown_connections 2024-04-24T23:21:16.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 -- 172.21.15.31:0/3802304009 wait complete. 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.472+0000 7fb93e968700 1 Processor -- start 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb93e968700 1 -- start start 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb93e968700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb938102de0 0x7fb938070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb93e968700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 0x7fb9380707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb93e968700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380721b0 con 0x7fb938103780 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb93e968700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb938070ce0 con 0x7fb938102de0 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb9377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 0x7fb9380707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb9377fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 0x7fb9380707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47656/0 (socket says 172.21.15.31:47656) 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb9377fe700 1 -- 172.21.15.31:0/1057105225 learned_addr learned my addr 172.21.15.31:0/1057105225 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:16.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb937fff700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb938102de0 0x7fb938070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb9377fe700 1 -- 172.21.15.31:0/1057105225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb938102de0 msgr2=0x7fb938070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb9377fe700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb938102de0 0x7fb938070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.473+0000 7fb9377fe700 1 -- 172.21.15.31:0/1057105225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c005040 con 0x7fb938103780 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.474+0000 7fb9377fe700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 0x7fb9380707a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009100 tx=0x7fb92c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.474+0000 7fb934ff9700 1 -- 172.21.15.31:0/1057105225 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb92c00e040 con 0x7fb938103780 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.474+0000 7fb934ff9700 1 -- 172.21.15.31:0/1057105225 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb92c013420 con 0x7fb938103780 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.474+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb938070f60 con 0x7fb938103780 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.474+0000 7fb934ff9700 1 -- 172.21.15.31:0/1057105225 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb92c01c630 con 0x7fb938103780 2024-04-24T23:21:16.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.474+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93811a050 con 0x7fb938103780 2024-04-24T23:21:16.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.475+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9380fe530 con 0x7fb938103780 2024-04-24T23:21:16.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.475+0000 7fb934ff9700 1 -- 172.21.15.31:0/1057105225 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb92c013a00 con 0x7fb938103780 2024-04-24T23:21:16.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.476+0000 7fb934ff9700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb92006e140 0x7fb9200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:16.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.476+0000 7fb934ff9700 1 -- 172.21.15.31:0/1057105225 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb92c08ebb0 con 0x7fb938103780 2024-04-24T23:21:16.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.476+0000 7fb937fff700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb92006e140 0x7fb9200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:16.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.476+0000 7fb937fff700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb92006e140 0x7fb9200705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb928009fd0 tx=0x7fb92800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:16.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.481+0000 7fb934ff9700 1 -- 172.21.15.31:0/1057105225 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb92c095080 con 0x7fb938103780 2024-04-24T23:21:16.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.774+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb93804c710 con 0x7fb938103780 2024-04-24T23:21:16.776 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.775+0000 7fb934ff9700 1 -- 172.21.15.31:0/1057105225 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb92c058000 con 0x7fb938103780 2024-04-24T23:21:16.777 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb92006e140 msgr2=0x7fb9200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb92006e140 0x7fb9200705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb928009fd0 tx=0x7fb92800b040 comp rx=0 tx=0).stop 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 msgr2=0x7fb9380707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 0x7fb9380707a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009100 tx=0x7fb92c009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 shutdown_connections 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb92006e140 0x7fb9200705f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb938102de0 0x7fb938070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 --2- 172.21.15.31:0/1057105225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb938103780 0x7fb9380707a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 >> 172.21.15.31:0/1057105225 conn(0x7fb9380f8990 msgr2=0x7fb9380f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 shutdown_connections 2024-04-24T23:21:16.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:16.777+0000 7fb93e968700 1 -- 172.21.15.31:0/1057105225 wait complete. 2024-04-24T23:21:17.024 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:16 smithi040 ceph-mon[83047]: pgmap v109: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:17.024 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:16 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4125821761' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:21:17.024 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:16 smithi040 ceph-mon[83047]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:17.025 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T23:21:17.025 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:16 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T23:21:17.025 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:16 smithi040 ceph-mon[83047]: Upgrade: osd.3 is safe to restart 2024-04-24T23:21:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:16 smithi031 ceph-mon[111404]: pgmap v109: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:16 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4125821761' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:21:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:16 smithi031 ceph-mon[111404]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T23:21:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:16 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-24T23:21:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:16 smithi031 ceph-mon[111404]: Upgrade: osd.3 is safe to restart 2024-04-24T23:21:18.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:17 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1057105225' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:21:18.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:17 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1057105225' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:21:19.255 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:19 smithi040 ceph-mon[83047]: pgmap v110: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:19.255 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:19 smithi040 ceph-mon[83047]: Upgrade: Updating osd.3 2024-04-24T23:21:19.255 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:19 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:19.256 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:19 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:21:19.256 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:19 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:19.256 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:19 smithi040 ceph-mon[83047]: Deploying daemon osd.3 on smithi040 2024-04-24T23:21:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:19 smithi031 ceph-mon[111404]: pgmap v110: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:19 smithi031 ceph-mon[111404]: Upgrade: Updating osd.3 2024-04-24T23:21:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:19 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:19 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-24T23:21:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:19 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:19.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:19 smithi031 ceph-mon[111404]: Deploying daemon osd.3 on smithi040 2024-04-24T23:21:21.370 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:21 smithi040 ceph-mon[83047]: pgmap v111: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:21 smithi031 ceph-mon[111404]: pgmap v111: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:21.662 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:21 smithi040 systemd[1]: Stopping Ceph osd.3 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:21:22.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:22 smithi040 ceph-mon[83047]: osd.3 marked itself down and dead 2024-04-24T23:21:22.163 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:21 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[38653]: 2024-04-24T23:21:21.807+0000 7f0a21452700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:21:22.163 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:21 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[38653]: 2024-04-24T23:21:21.807+0000 7f0a21452700 -1 osd.3 64 *** Got signal Terminated *** 2024-04-24T23:21:22.163 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:21 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[38653]: 2024-04-24T23:21:21.807+0000 7f0a21452700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:21:22.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:22 smithi031 ceph-mon[111404]: osd.3 marked itself down and dead 2024-04-24T23:21:22.662 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:22 smithi040 podman[91979]: 2024-04-24 23:21:22.391686123 +0000 UTC m=+0.691691214 container died c3fe722ceca42a081631a27043a46b3bf0173604b0bc40479c40f9f4cbe19d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:21:23.035 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:23 smithi040 ceph-mon[83047]: pgmap v112: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:23.035 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:23 smithi040 ceph-mon[83047]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:21:23.035 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:23 smithi040 ceph-mon[83047]: osdmap e65: 6 total, 5 up, 6 in 2024-04-24T23:21:23.036 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:22 smithi040 podman[91979]: 2024-04-24 23:21:22.891017012 +0000 UTC m=+1.191022101 container cleanup c3fe722ceca42a081631a27043a46b3bf0173604b0bc40479c40f9f4cbe19d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:21:23.036 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:22 smithi040 bash[91979]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3 2024-04-24T23:21:23.412 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:23 smithi040 podman[91997]: 2024-04-24 23:21:23.243238828 +0000 UTC m=+0.854022099 container remove c3fe722ceca42a081631a27043a46b3bf0173604b0bc40479c40f9f4cbe19d25 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T23:21:23.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:23 smithi031 ceph-mon[111404]: pgmap v112: 65 pgs: 65 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:23.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:23 smithi031 ceph-mon[111404]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:21:23.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:23 smithi031 ceph-mon[111404]: osdmap e65: 6 total, 5 up, 6 in 2024-04-24T23:21:23.830 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:23 smithi040 podman[92089]: 2024-04-24 23:21:23.470630783 +0000 UTC m=+0.018512358 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:21:23.831 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:23 smithi040 podman[92089]: 2024-04-24 23:21:23.593943943 +0000 UTC m=+0.141825516 container create 27200cb07bd62a61acd41f819a61c3af8413432233e581cd2f9aa8c355be84f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-24T23:21:24.139 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:24 smithi040 ceph-mon[83047]: osdmap e66: 6 total, 5 up, 6 in 2024-04-24T23:21:24.140 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:23 smithi040 podman[92089]: 2024-04-24 23:21:23.829962685 +0000 UTC m=+0.377844262 container init 27200cb07bd62a61acd41f819a61c3af8413432233e581cd2f9aa8c355be84f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:21:24.140 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:23 smithi040 podman[92089]: 2024-04-24 23:21:23.833428964 +0000 UTC m=+0.381310544 container start 27200cb07bd62a61acd41f819a61c3af8413432233e581cd2f9aa8c355be84f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-04-24T23:21:24.140 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:23 smithi040 podman[92089]: 2024-04-24 23:21:23.904892633 +0000 UTC m=+0.452774223 container attach 27200cb07bd62a61acd41f819a61c3af8413432233e581cd2f9aa8c355be84f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T23:21:24.140 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:24 smithi040 podman[92089]: 2024-04-24 23:21:24.139638329 +0000 UTC m=+0.687519893 container died 27200cb07bd62a61acd41f819a61c3af8413432233e581cd2f9aa8c355be84f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-24T23:21:24.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:24 smithi031 ceph-mon[111404]: osdmap e66: 6 total, 5 up, 6 in 2024-04-24T23:21:24.878 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:24 smithi040 podman[92115]: 2024-04-24 23:21:24.625102156 +0000 UTC m=+0.474405730 container remove 27200cb07bd62a61acd41f819a61c3af8413432233e581cd2f9aa8c355be84f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-24T23:21:24.878 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:24 smithi040 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-24T23:21:24.878 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:24 smithi040 systemd[1]: Stopped Ceph osd.3 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:21:24.878 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:24 smithi040 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 20.535s CPU time. 2024-04-24T23:21:25.152 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:25 smithi040 ceph-mon[83047]: pgmap v115: 65 pgs: 16 stale+active+clean, 49 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:25.152 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:24 smithi040 systemd[1]: Starting Ceph osd.3 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:21:25.152 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 podman[92216]: 2024-04-24 23:21:24.998841538 +0000 UTC m=+0.018548380 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:21:25.153 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 podman[92216]: 2024-04-24 23:21:25.151562948 +0000 UTC m=+0.171269768 container create 282733f280162bd275f7c81c3ccf27361330f749ef1462ba4fb2209ce38835f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-24T23:21:25.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:25 smithi031 ceph-mon[111404]: pgmap v115: 65 pgs: 16 stale+active+clean, 49 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:25.912 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 podman[92216]: 2024-04-24 23:21:25.49752155 +0000 UTC m=+0.517228366 container init 282733f280162bd275f7c81c3ccf27361330f749ef1462ba4fb2209ce38835f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-24T23:21:25.912 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 podman[92216]: 2024-04-24 23:21:25.501088952 +0000 UTC m=+0.520795770 container start 282733f280162bd275f7c81c3ccf27361330f749ef1462ba4fb2209ce38835f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:21:25.912 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 podman[92216]: 2024-04-24 23:21:25.579131835 +0000 UTC m=+0.598838651 container attach 282733f280162bd275f7c81c3ccf27361330f749ef1462ba4fb2209ce38835f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T23:21:26.292 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:26 smithi040 ceph-mon[83047]: Health check failed: Degraded data redundancy: 36/225 objects degraded (16.000%), 11 pgs degraded (PG_DEGRADED) 2024-04-24T23:21:26.292 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate[92232]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T23:21:26.292 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate[92232]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T23:21:26.292 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 bash[92216]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 bash[92216]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 bash[92216]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 bash[92216]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 bash[92216]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 bash[92216]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 bash[92216]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate[92232]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate[92232]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate[92232]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate[92232]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate[92232]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-24T23:21:26.293 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:25 smithi040 podman[92216]: 2024-04-24 23:21:25.986664985 +0000 UTC m=+1.006371818 container died 282733f280162bd275f7c81c3ccf27361330f749ef1462ba4fb2209ce38835f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-24T23:21:26.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:26 smithi031 ceph-mon[111404]: Health check failed: Degraded data redundancy: 36/225 objects degraded (16.000%), 11 pgs degraded (PG_DEGRADED) 2024-04-24T23:21:26.835 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:26 smithi040 podman[92351]: 2024-04-24 23:21:26.575005712 +0000 UTC m=+0.583338099 container remove 282733f280162bd275f7c81c3ccf27361330f749ef1462ba4fb2209ce38835f6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T23:21:26.835 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:26 smithi040 podman[92412]: 2024-04-24 23:21:26.698799783 +0000 UTC m=+0.018892044 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:21:26.835 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:26 smithi040 podman[92412]: 2024-04-24 23:21:26.8343788 +0000 UTC m=+0.154471103 container create 97c6f05632f661a4a5d5f82372778f5cb7ece81cc42c5a8d0b826026bf930d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:21:27.102 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:27 smithi040 podman[92412]: 2024-04-24 23:21:27.044894755 +0000 UTC m=+0.364987013 container init 97c6f05632f661a4a5d5f82372778f5cb7ece81cc42c5a8d0b826026bf930d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-04-24T23:21:27.102 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:27 smithi040 podman[92412]: 2024-04-24 23:21:27.047852234 +0000 UTC m=+0.367944492 container start 97c6f05632f661a4a5d5f82372778f5cb7ece81cc42c5a8d0b826026bf930d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:21:27.103 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:27 smithi040 ceph-osd[92430]: -- 172.21.15.40:0/872085876 <== mon.1 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d190c1680 con 0x556d1908ac00 2024-04-24T23:21:27.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:27 smithi040 ceph-mon[83047]: pgmap v116: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 14 active+undersized, 37 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 36/225 objects degraded (16.000%) 2024-04-24T23:21:27.413 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:27 smithi040 bash[92412]: 97c6f05632f661a4a5d5f82372778f5cb7ece81cc42c5a8d0b826026bf930d9f 2024-04-24T23:21:27.413 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:27 smithi040 systemd[1]: Started Ceph osd.3 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:21:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:27 smithi031 ceph-mon[111404]: pgmap v116: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 14 active+undersized, 37 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 36/225 objects degraded (16.000%) 2024-04-24T23:21:27.912 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:27 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[92426]: 2024-04-24T23:21:27.585+0000 7fb00d17d7c0 -1 Falling back to public interface 2024-04-24T23:21:28.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:28.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:28.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:28.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:29 smithi031 ceph-mon[111404]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:29 smithi040 ceph-mon[83047]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:31.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:31 smithi031 ceph-mon[111404]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:31.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:31 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:31.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:31 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:31.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:31 smithi040 ceph-mon[83047]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:31.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:31 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:31.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:31 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:32.912 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:32 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[92426]: 2024-04-24T23:21:32.880+0000 7fb00d17d7c0 -1 osd.3 64 log_to_monitors true 2024-04-24T23:21:32.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:32 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:32.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:32 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:32.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:32 smithi040 ceph-mon[83047]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:32.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:32 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:32.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:32 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:32.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:32 smithi031 ceph-mon[111404]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:33.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:33 smithi040 ceph-mon[83047]: from='osd.3 [v2:172.21.15.40:6800/4020191995,v1:172.21.15.40:6801/4020191995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:21:33.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:33 smithi040 ceph-mon[83047]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:21:33.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:33 smithi031 ceph-mon[111404]: from='osd.3 [v2:172.21.15.40:6800/4020191995,v1:172.21.15.40:6801/4020191995]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:21:33.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:33 smithi031 ceph-mon[111404]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-24T23:21:34.912 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:21:34 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[92426]: 2024-04-24T23:21:34.602+0000 7faffa5cd700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:21:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:34 smithi040 ceph-mon[83047]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:21:34.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:34 smithi040 ceph-mon[83047]: osdmap e67: 6 total, 5 up, 6 in 2024-04-24T23:21:34.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:34 smithi040 ceph-mon[83047]: from='osd.3 [v2:172.21.15.40:6800/4020191995,v1:172.21.15.40:6801/4020191995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:21:34.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:34 smithi040 ceph-mon[83047]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:21:34.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:34 smithi040 ceph-mon[83047]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:34.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:34 smithi040 ceph-mon[83047]: from='osd.3 ' entity='osd.3' 2024-04-24T23:21:34.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:34 smithi031 ceph-mon[111404]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-24T23:21:34.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:34 smithi031 ceph-mon[111404]: osdmap e67: 6 total, 5 up, 6 in 2024-04-24T23:21:34.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:34 smithi031 ceph-mon[111404]: from='osd.3 [v2:172.21.15.40:6800/4020191995,v1:172.21.15.40:6801/4020191995]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:21:34.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:34 smithi031 ceph-mon[111404]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:21:34.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:34 smithi031 ceph-mon[111404]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:34.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:34 smithi031 ceph-mon[111404]: from='osd.3 ' entity='osd.3' 2024-04-24T23:21:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: Health check update: Degraded data redundancy: 54/225 objects degraded (24.000%), 18 pgs degraded (PG_DEGRADED) 2024-04-24T23:21:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:35.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: osd.3 [v2:172.21.15.40:6800/4020191995,v1:172.21.15.40:6801/4020191995] boot 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: osdmap e68: 6 total, 6 up, 6 in 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:36.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:35 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T23:21:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: Health check update: Degraded data redundancy: 54/225 objects degraded (24.000%), 18 pgs degraded (PG_DEGRADED) 2024-04-24T23:21:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:36.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: osd.3 [v2:172.21.15.40:6800/4020191995,v1:172.21.15.40:6801/4020191995] boot 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: osdmap e68: 6 total, 6 up, 6 in 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:36.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:35 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T23:21:37.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:37 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T23:21:37.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:37 smithi040 ceph-mon[83047]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-24T23:21:37.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:37 smithi040 ceph-mon[83047]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:37.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:37 smithi040 ceph-mon[83047]: osdmap e69: 6 total, 6 up, 6 in 2024-04-24T23:21:37.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:37 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T23:21:37.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:37 smithi031 ceph-mon[111404]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-24T23:21:37.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:37 smithi031 ceph-mon[111404]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 54/225 objects degraded (24.000%) 2024-04-24T23:21:37.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:37 smithi031 ceph-mon[111404]: osdmap e69: 6 total, 6 up, 6 in 2024-04-24T23:21:39.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:39 smithi040 ceph-mon[83047]: pgmap v125: 65 pgs: 10 peering, 12 active+undersized+degraded, 16 active+undersized, 27 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37/225 objects degraded (16.444%) 2024-04-24T23:21:39.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:39 smithi031 ceph-mon[111404]: pgmap v125: 65 pgs: 10 peering, 12 active+undersized+degraded, 16 active+undersized, 27 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37/225 objects degraded (16.444%) 2024-04-24T23:21:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:40 smithi040 ceph-mon[83047]: Health check update: Degraded data redundancy: 37/225 objects degraded (16.444%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T23:21:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:40 smithi031 ceph-mon[111404]: Health check update: Degraded data redundancy: 37/225 objects degraded (16.444%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T23:21:41.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:41 smithi040 ceph-mon[83047]: pgmap v126: 65 pgs: 10 peering, 1 active+undersized+degraded, 3 active+undersized, 51 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3/225 objects degraded (1.333%) 2024-04-24T23:21:41.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:41 smithi031 ceph-mon[111404]: pgmap v126: 65 pgs: 10 peering, 1 active+undersized+degraded, 3 active+undersized, 51 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3/225 objects degraded (1.333%) 2024-04-24T23:21:42.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:42 smithi031 ceph-mon[111404]: pgmap v127: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:42.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:42 smithi031 ceph-mon[111404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/225 objects degraded (1.333%), 1 pg degraded) 2024-04-24T23:21:42.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:42 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:21:43.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:42 smithi040 ceph-mon[83047]: pgmap v127: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:43.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:42 smithi040 ceph-mon[83047]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/225 objects degraded (1.333%), 1 pg degraded) 2024-04-24T23:21:43.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:42 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:21:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:44 smithi040 ceph-mon[83047]: pgmap v128: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:45.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:44 smithi031 ceph-mon[111404]: pgmap v128: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:45.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:45.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.896+0000 7f9385248700 1 -- 172.21.15.31:0/3319428459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 msgr2=0x7f93800fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.896+0000 7f9385248700 1 --2- 172.21.15.31:0/3319428459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 0x7f93800fc1b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9368004940 tx=0x7f9368009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.896+0000 7f9385248700 1 -- 172.21.15.31:0/3319428459 shutdown_connections 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.896+0000 7f9385248700 1 --2- 172.21.15.31:0/3319428459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 0x7f93800fc1b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.896+0000 7f9385248700 1 --2- 172.21.15.31:0/3319428459 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9380101d90 0x7f9380102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.896+0000 7f9385248700 1 -- 172.21.15.31:0/3319428459 >> 172.21.15.31:0/3319428459 conn(0x7f9380071d00 msgr2=0x7f9380072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.896+0000 7f9385248700 1 -- 172.21.15.31:0/3319428459 shutdown_connections 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.896+0000 7f9385248700 1 -- 172.21.15.31:0/3319428459 wait complete. 2024-04-24T23:21:46.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f9385248700 1 Processor -- start 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f9385248700 1 -- start start 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f9385248700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 0x7f9380102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f9385248700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9380101d90 0x7f9380103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f9385248700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93801035f0 con 0x7f93800fbd40 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f9385248700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9380103730 con 0x7f9380101d90 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f937ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 0x7f9380102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f937ed9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 0x7f9380102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43058/0 (socket says 172.21.15.31:43058) 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f937ed9d700 1 -- 172.21.15.31:0/4148890756 learned_addr learned my addr 172.21.15.31:0/4148890756 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.897+0000 7f937e59c700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9380101d90 0x7f9380103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f937ed9d700 1 -- 172.21.15.31:0/4148890756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9380101d90 msgr2=0x7f9380103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f937ed9d700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9380101d90 0x7f9380103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f937ed9d700 1 -- 172.21.15.31:0/4148890756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368005040 con 0x7f93800fbd40 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f937ed9d700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 0x7f9380102b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f937000c830 tx=0x7f9370004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:46.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f93777fe700 1 -- 172.21.15.31:0/4148890756 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93700106f0 con 0x7f93800fbd40 2024-04-24T23:21:46.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9380119eb0 con 0x7f93800fbd40 2024-04-24T23:21:46.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f93777fe700 1 -- 172.21.15.31:0/4148890756 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9370010d20 con 0x7f93800fbd40 2024-04-24T23:21:46.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f93777fe700 1 -- 172.21.15.31:0/4148890756 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93700113a0 con 0x7f93800fbd40 2024-04-24T23:21:46.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.898+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938011a330 con 0x7f93800fbd40 2024-04-24T23:21:46.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.899+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93800fd430 con 0x7f93800fbd40 2024-04-24T23:21:46.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.900+0000 7f93777fe700 1 -- 172.21.15.31:0/4148890756 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9370012400 con 0x7f93800fbd40 2024-04-24T23:21:46.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.900+0000 7f93777fe700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f936c06e140 0x7f936c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:46.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.900+0000 7f93777fe700 1 -- 172.21.15.31:0/4148890756 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f937008da20 con 0x7f93800fbd40 2024-04-24T23:21:46.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.900+0000 7f937e59c700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f936c06e140 0x7f936c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:46.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.901+0000 7f937e59c700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f936c06e140 0x7f936c0705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9380104460 tx=0x7f9368009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:46.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:46.905+0000 7f93777fe700 1 -- 172.21.15.31:0/4148890756 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9370056e70 con 0x7f93800fbd40 2024-04-24T23:21:47.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.132+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9380104210 con 0x7f936c06e140 2024-04-24T23:21:47.134 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:46 smithi031 ceph-mon[111404]: pgmap v129: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:47.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.133+0000 7f93777fe700 1 -- 172.21.15.31:0/4148890756 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9380104210 con 0x7f936c06e140 2024-04-24T23:21:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f936c06e140 msgr2=0x7f936c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f936c06e140 0x7f936c0705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9380104460 tx=0x7f9368009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 msgr2=0x7f9380102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 0x7f9380102b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f937000c830 tx=0x7f9370004470 comp rx=0 tx=0).stop 2024-04-24T23:21:47.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 shutdown_connections 2024-04-24T23:21:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800fbd40 0x7f9380102b20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f936c06e140 0x7f936c0705f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 --2- 172.21.15.31:0/4148890756 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9380101d90 0x7f9380103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.139+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 >> 172.21.15.31:0/4148890756 conn(0x7f9380071d00 msgr2=0x7f93800fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.140+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 shutdown_connections 2024-04-24T23:21:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.140+0000 7f9385248700 1 -- 172.21.15.31:0/4148890756 wait complete. 2024-04-24T23:21:47.152 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:21:47.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:46 smithi040 ceph-mon[83047]: pgmap v129: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 -- 172.21.15.31:0/352979312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708103780 msgr2=0x7f67080fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 --2- 172.21.15.31:0/352979312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708103780 0x7f67080fcdf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f66fc004940 tx=0x7f66fc009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 -- 172.21.15.31:0/352979312 shutdown_connections 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 --2- 172.21.15.31:0/352979312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708103780 0x7f67080fcdf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 --2- 172.21.15.31:0/352979312 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6708102de0 0x7f67081031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 -- 172.21.15.31:0/352979312 >> 172.21.15.31:0/352979312 conn(0x7f67080f8990 msgr2=0x7f67080fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 -- 172.21.15.31:0/352979312 shutdown_connections 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 -- 172.21.15.31:0/352979312 wait complete. 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 Processor -- start 2024-04-24T23:21:47.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.245+0000 7f670ccfd700 1 -- start start 2024-04-24T23:21:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670ccfd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708102de0 0x7f6708070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670ccfd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6708103780 0x7f67080707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670ccfd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708072240 con 0x7f6708102de0 2024-04-24T23:21:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670ccfd700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6708070ce0 con 0x7f6708103780 2024-04-24T23:21:47.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708102de0 0x7f6708070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708102de0 0x7f6708070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43082/0 (socket says 172.21.15.31:43082) 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670659c700 1 -- 172.21.15.31:0/3087783697 learned_addr learned my addr 172.21.15.31:0/3087783697 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f6705d9b700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6708103780 0x7f67080707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670659c700 1 -- 172.21.15.31:0/3087783697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6708103780 msgr2=0x7f67080707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670659c700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6708103780 0x7f67080707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670659c700 1 -- 172.21.15.31:0/3087783697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc005040 con 0x7f6708102de0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f670659c700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708102de0 0x7f6708070260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f66f800bbb0 tx=0x7f66f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.246+0000 7f66f6ffd700 1 -- 172.21.15.31:0/3087783697 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66f80092e0 con 0x7f6708102de0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.247+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6708070fc0 con 0x7f6708102de0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.247+0000 7f66f6ffd700 1 -- 172.21.15.31:0/3087783697 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66f800cb20 con 0x7f6708102de0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.247+0000 7f66f6ffd700 1 -- 172.21.15.31:0/3087783697 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66f8015d70 con 0x7f6708102de0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.247+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670811a1b0 con 0x7f6708102de0 2024-04-24T23:21:47.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.248+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67081000f0 con 0x7f6708102de0 2024-04-24T23:21:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.248+0000 7f66f6ffd700 1 -- 172.21.15.31:0/3087783697 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f66f8012070 con 0x7f6708102de0 2024-04-24T23:21:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.248+0000 7f66f6ffd700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f66f006e140 0x7f66f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.249+0000 7f66f6ffd700 1 -- 172.21.15.31:0/3087783697 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f66f808d820 con 0x7f6708102de0 2024-04-24T23:21:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.249+0000 7f6705d9b700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f66f006e140 0x7f66f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.249+0000 7f6705d9b700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f66f006e140 0x7f66f00705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66fc004890 tx=0x7f66fc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.253+0000 7f66f6ffd700 1 -- 172.21.15.31:0/3087783697 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f66f8056bf0 con 0x7f6708102de0 2024-04-24T23:21:47.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.477+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6708109f30 con 0x7f66f006e140 2024-04-24T23:21:47.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.479+0000 7f66f6ffd700 1 -- 172.21.15.31:0/3087783697 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6708109f30 con 0x7f66f006e140 2024-04-24T23:21:47.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.480+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f66f006e140 msgr2=0x7f66f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.480+0000 7f670ccfd700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f66f006e140 0x7f66f00705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66fc004890 tx=0x7f66fc009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:47.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.480+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708102de0 msgr2=0x7f6708070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.480+0000 7f670ccfd700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708102de0 0x7f6708070260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f66f800bbb0 tx=0x7f66f8009840 comp rx=0 tx=0).stop 2024-04-24T23:21:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.481+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 shutdown_connections 2024-04-24T23:21:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.481+0000 7f670ccfd700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6708102de0 0x7f6708070260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.481+0000 7f670ccfd700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f66f006e140 0x7f66f00705f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.481+0000 7f670ccfd700 1 --2- 172.21.15.31:0/3087783697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6708103780 0x7f67080707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.481+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 >> 172.21.15.31:0/3087783697 conn(0x7f67080f8990 msgr2=0x7f67080fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.481+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 shutdown_connections 2024-04-24T23:21:47.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.481+0000 7f670ccfd700 1 -- 172.21.15.31:0/3087783697 wait complete. 2024-04-24T23:21:47.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.585+0000 7f73c4ba1700 1 -- 172.21.15.31:0/862705662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0103600 msgr2=0x7f73c00fb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.585+0000 7f73c4ba1700 1 --2- 172.21.15.31:0/862705662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0103600 0x7f73c00fb200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f73b4004940 tx=0x7f73b4009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.585+0000 7f73c4ba1700 1 -- 172.21.15.31:0/862705662 shutdown_connections 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.585+0000 7f73c4ba1700 1 --2- 172.21.15.31:0/862705662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0103600 0x7f73c00fb200 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.585+0000 7f73c4ba1700 1 --2- 172.21.15.31:0/862705662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0102c60 0x7f73c0103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.585+0000 7f73c4ba1700 1 -- 172.21.15.31:0/862705662 >> 172.21.15.31:0/862705662 conn(0x7f73c0072160 msgr2=0x7f73c0072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.585+0000 7f73c4ba1700 1 -- 172.21.15.31:0/862705662 shutdown_connections 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.585+0000 7f73c4ba1700 1 -- 172.21.15.31:0/862705662 wait complete. 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73c4ba1700 1 Processor -- start 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73c4ba1700 1 -- start start 2024-04-24T23:21:47.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73c4ba1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0102c60 0x7f73c0113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73c4ba1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0103600 0x7f73c0113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73c4ba1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c01145d0 con 0x7f73c0102c60 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73c4ba1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c010d9c0 con 0x7f73c0103600 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0102c60 0x7f73c0113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73bf7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0102c60 0x7f73c0113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43100/0 (socket says 172.21.15.31:43100) 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.586+0000 7f73bf7fe700 1 -- 172.21.15.31:0/100874171 learned_addr learned my addr 172.21.15.31:0/100874171 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73beffd700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0103600 0x7f73c0113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73bf7fe700 1 -- 172.21.15.31:0/100874171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0103600 msgr2=0x7f73c0113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73bf7fe700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0103600 0x7f73c0113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73bf7fe700 1 -- 172.21.15.31:0/100874171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b4005040 con 0x7f73c0102c60 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73bf7fe700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0102c60 0x7f73c0113a40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f73b0002a80 tx=0x7f73b0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73a7fff700 1 -- 172.21.15.31:0/100874171 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73b0007cc0 con 0x7f73c0102c60 2024-04-24T23:21:47.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73a7fff700 1 -- 172.21.15.31:0/100874171 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73b000f040 con 0x7f73c0102c60 2024-04-24T23:21:47.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73a7fff700 1 -- 172.21.15.31:0/100874171 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73b000da30 con 0x7f73c0102c60 2024-04-24T23:21:47.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c010dca0 con 0x7f73c0102c60 2024-04-24T23:21:47.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.587+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c010e1b0 con 0x7f73c0102c60 2024-04-24T23:21:47.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.588+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c00fe520 con 0x7f73c0102c60 2024-04-24T23:21:47.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.589+0000 7f73a7fff700 1 -- 172.21.15.31:0/100874171 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f73b000db90 con 0x7f73c0102c60 2024-04-24T23:21:47.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.589+0000 7f73a7fff700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f73a806e140 0x7f73a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.589+0000 7f73a7fff700 1 -- 172.21.15.31:0/100874171 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f73b008d870 con 0x7f73c0102c60 2024-04-24T23:21:47.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.589+0000 7f73beffd700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f73a806e140 0x7f73a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.590+0000 7f73beffd700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f73a806e140 0x7f73a80705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f73b4004890 tx=0x7f73b4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.594+0000 7f73a7fff700 1 -- 172.21.15.31:0/100874171 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f73b005a770 con 0x7f73c0102c60 2024-04-24T23:21:47.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.812+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f73c0109dd0 con 0x7f73a806e140 2024-04-24T23:21:47.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.822+0000 7f73a7fff700 1 -- 172.21.15.31:0/100874171 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f73c0109dd0 con 0x7f73a806e140 2024-04-24T23:21:47.824 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:21:47.824 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (4m) 51s ago 16m 25.6M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:21:47.824 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (2m) 51s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (2m) 17s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (4m) 51s ago 16m 79.9M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (10m) 51s ago 10m 138M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (11m) 51s ago 11m 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (11m) 17s ago 11m 14.9M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (10m) 17s ago 10m 16.3M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (6m) 51s ago 17m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (5m) 17s ago 15m 452M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (3m) 51s ago 18m 45.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (3m) 17s ago 15m 35.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (5m) 51s ago 16m 16.7M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (5m) 17s ago 15m 14.5M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (2m) 51s ago 14m 87.6M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (93s) 51s ago 14m 101M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:21:47.825 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (57s) 51s ago 13m 11.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 271a0764ef5c 2024-04-24T23:21:47.826 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (21s) 17s ago 13m 12.0M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 97c6f05632f6 2024-04-24T23:21:47.826 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (12m) 17s ago 12m 246M 2583M 17.2.7-902-gc61472c8 0de407c1b3af 727ed09fe1c3 2024-04-24T23:21:47.826 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (12m) 17s ago 12m 225M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:21:47.826 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 51s ago 16m 59.9M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:21:47.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f73a806e140 msgr2=0x7f73a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.826 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f73a806e140 0x7f73a80705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f73b4004890 tx=0x7f73b4009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0102c60 msgr2=0x7f73c0113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0102c60 0x7f73c0113a40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f73b0002a80 tx=0x7f73b0004600 comp rx=0 tx=0).stop 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 shutdown_connections 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f73c0102c60 0x7f73c0113a40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f73a806e140 0x7f73a80705f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 --2- 172.21.15.31:0/100874171 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73c0103600 0x7f73c0113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 >> 172.21.15.31:0/100874171 conn(0x7f73c0072160 msgr2=0x7f73c00f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 shutdown_connections 2024-04-24T23:21:47.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.825+0000 7f73c4ba1700 1 -- 172.21.15.31:0/100874171 wait complete. 2024-04-24T23:21:47.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 -- 172.21.15.31:0/2207492117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 msgr2=0x7f1fe0103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 --2- 172.21.15.31:0/2207492117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 0x7f1fe0103220 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8004940 tx=0x7f1fd8009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 -- 172.21.15.31:0/2207492117 shutdown_connections 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 --2- 172.21.15.31:0/2207492117 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe0065640 0x7f1fe0065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 --2- 172.21.15.31:0/2207492117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 0x7f1fe0103220 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 -- 172.21.15.31:0/2207492117 >> 172.21.15.31:0/2207492117 conn(0x7f1fe0071d00 msgr2=0x7f1fe0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 -- 172.21.15.31:0/2207492117 shutdown_connections 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 -- 172.21.15.31:0/2207492117 wait complete. 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.929+0000 7f1fe820c700 1 Processor -- start 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe820c700 1 -- start start 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe820c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe0065640 0x7f1fe0115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe820c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 0x7f1fe01162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe820c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe0116920 con 0x7f1fe0102e50 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe820c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe010fcf0 con 0x7f1fe0065640 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe57a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 0x7f1fe01162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe57a7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 0x7f1fe01162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43118/0 (socket says 172.21.15.31:43118) 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe57a7700 1 -- 172.21.15.31:0/1712769272 learned_addr learned my addr 172.21.15.31:0/1712769272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe5fa8700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe0065640 0x7f1fe0115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe57a7700 1 -- 172.21.15.31:0/1712769272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe0065640 msgr2=0x7f1fe0115d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe57a7700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe0065640 0x7f1fe0115d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe57a7700 1 -- 172.21.15.31:0/1712769272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8005040 con 0x7f1fe0102e50 2024-04-24T23:21:47.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.930+0000 7f1fe57a7700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 0x7f1fe01162d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400c8c0 tx=0x7f1fd40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.931+0000 7f1fde7fc700 1 -- 172.21.15.31:0/1712769272 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd400a760 con 0x7f1fe0102e50 2024-04-24T23:21:47.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.931+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe010ffd0 con 0x7f1fe0102e50 2024-04-24T23:21:47.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.931+0000 7f1fde7fc700 1 -- 172.21.15.31:0/1712769272 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fd400ad90 con 0x7f1fe0102e50 2024-04-24T23:21:47.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.931+0000 7f1fde7fc700 1 -- 172.21.15.31:0/1712769272 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd40113a0 con 0x7f1fe0102e50 2024-04-24T23:21:47.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.931+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe0110590 con 0x7f1fe0102e50 2024-04-24T23:21:47.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.932+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe0107260 con 0x7f1fe0102e50 2024-04-24T23:21:47.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.932+0000 7f1fde7fc700 1 -- 172.21.15.31:0/1712769272 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1fd4012400 con 0x7f1fe0102e50 2024-04-24T23:21:47.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.932+0000 7f1fde7fc700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1fc806e140 0x7f1fc80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:47.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.933+0000 7f1fde7fc700 1 -- 172.21.15.31:0/1712769272 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1fd408da00 con 0x7f1fe0102e50 2024-04-24T23:21:47.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.933+0000 7f1fe5fa8700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1fc806e140 0x7f1fc80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:47.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.933+0000 7f1fe5fa8700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1fc806e140 0x7f1fc80705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8004f90 tx=0x7f1fd8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:47.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:47.937+0000 7f1fde7fc700 1 -- 172.21.15.31:0/1712769272 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1fd405aa20 con 0x7f1fe0102e50 2024-04-24T23:21:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:47 smithi040 ceph-mon[83047]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:48.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:47 smithi040 ceph-mon[83047]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:48.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.237+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1fe00fe540 con 0x7f1fe0102e50 2024-04-24T23:21:48.239 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:47 smithi031 ceph-mon[111404]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:48.239 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:47 smithi031 ceph-mon[111404]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:48.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.237+0000 7f1fde7fc700 1 -- 172.21.15.31:0/1712769272 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f1fd405a5b0 con 0x7f1fe0102e50 2024-04-24T23:21:48.239 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:21:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:21:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:21:48.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6, 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 8 2024-04-24T23:21:48.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:21:48.241 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:21:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1fc806e140 msgr2=0x7f1fc80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1fc806e140 0x7f1fc80705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8004f90 tx=0x7f1fd8009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 msgr2=0x7f1fe01162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 0x7f1fe01162d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400c8c0 tx=0x7f1fd40044c0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 shutdown_connections 2024-04-24T23:21:48.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1fc806e140 0x7f1fc80705f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fe0065640 0x7f1fe0115d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 --2- 172.21.15.31:0/1712769272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fe0102e50 0x7f1fe01162d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.239+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 >> 172.21.15.31:0/1712769272 conn(0x7f1fe0071d00 msgr2=0x7f1fe00fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:48.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.240+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 shutdown_connections 2024-04-24T23:21:48.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.240+0000 7f1fe820c700 1 -- 172.21.15.31:0/1712769272 wait complete. 2024-04-24T23:21:48.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 -- 172.21.15.31:0/1292867602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 msgr2=0x7ffb30109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 --2- 172.21.15.31:0/1292867602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 0x7ffb30109170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c004940 tx=0x7ffb1c009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 -- 172.21.15.31:0/1292867602 shutdown_connections 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 --2- 172.21.15.31:0/1292867602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 0x7ffb30109170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 --2- 172.21.15.31:0/1292867602 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb30070280 0x7ffb30070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 -- 172.21.15.31:0/1292867602 >> 172.21.15.31:0/1292867602 conn(0x7ffb300f8990 msgr2=0x7ffb300fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 -- 172.21.15.31:0/1292867602 shutdown_connections 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 -- 172.21.15.31:0/1292867602 wait complete. 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 Processor -- start 2024-04-24T23:21:48.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.344+0000 7ffb36bdb700 1 -- start start 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb36bdb700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb30070280 0x7ffb30115bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb36bdb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 0x7ffb30116130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb36bdb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb30116780 con 0x7ffb30070b90 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb36bdb700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3010fd60 con 0x7ffb30070280 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 0x7ffb30116130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb2ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 0x7ffb30116130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43134/0 (socket says 172.21.15.31:43134) 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb2ffff700 1 -- 172.21.15.31:0/3136684374 learned_addr learned my addr 172.21.15.31:0/3136684374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb34977700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb30070280 0x7ffb30115bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb2ffff700 1 -- 172.21.15.31:0/3136684374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb30070280 msgr2=0x7ffb30115bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb2ffff700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb30070280 0x7ffb30115bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb2ffff700 1 -- 172.21.15.31:0/3136684374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c005040 con 0x7ffb30070b90 2024-04-24T23:21:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb34977700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb30070280 0x7ffb30115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:21:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb2ffff700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 0x7ffb30116130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00b8b0 tx=0x7ffb1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.345+0000 7ffb2d7fa700 1 -- 172.21.15.31:0/3136684374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb1c00e040 con 0x7ffb30070b90 2024-04-24T23:21:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.346+0000 7ffb2d7fa700 1 -- 172.21.15.31:0/3136684374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb1c004290 con 0x7ffb30070b90 2024-04-24T23:21:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.346+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3010ffe0 con 0x7ffb30070b90 2024-04-24T23:21:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.346+0000 7ffb2d7fa700 1 -- 172.21.15.31:0/3136684374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb1c01c3f0 con 0x7ffb30070b90 2024-04-24T23:21:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.346+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb30110520 con 0x7ffb30070b90 2024-04-24T23:21:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.346+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb300719c0 con 0x7ffb30070b90 2024-04-24T23:21:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.347+0000 7ffb2d7fa700 1 -- 172.21.15.31:0/3136684374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffb1c0137b0 con 0x7ffb30070b90 2024-04-24T23:21:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.348+0000 7ffb2d7fa700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffb2006e140 0x7ffb200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.348+0000 7ffb2d7fa700 1 -- 172.21.15.31:0/3136684374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ffb1c08eea0 con 0x7ffb30070b90 2024-04-24T23:21:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.348+0000 7ffb34977700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffb2006e140 0x7ffb200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.348+0000 7ffb34977700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffb2006e140 0x7ffb200705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb24007fd0 tx=0x7ffb24009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:48.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.352+0000 7ffb2d7fa700 1 -- 172.21.15.31:0/3136684374 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ffb1c05bd80 con 0x7ffb30070b90 2024-04-24T23:21:48.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.609+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffb30111270 con 0x7ffb30070b90 2024-04-24T23:21:48.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.609+0000 7ffb2d7fa700 1 -- 172.21.15.31:0/3136684374 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1878 (secure 0 0 0) 0x7ffb1c05b910 con 0x7ffb30070b90 2024-04-24T23:21:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:21:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:21:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:21:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:21:48.611 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:21:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:21:48.611 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:21:48.612 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:21:48.613 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:21:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:21:48.614 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:21:48.614 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:21:48.614 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:21:48.614 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:21:48.614 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffb2006e140 msgr2=0x7ffb200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffb2006e140 0x7ffb200705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffb24007fd0 tx=0x7ffb24009040 comp rx=0 tx=0).stop 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 msgr2=0x7ffb30116130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 0x7ffb30116130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00b8b0 tx=0x7ffb1c009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 shutdown_connections 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffb2006e140 0x7ffb200705f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb30070280 0x7ffb30115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 --2- 172.21.15.31:0/3136684374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb30070b90 0x7ffb30116130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.611+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 >> 172.21.15.31:0/3136684374 conn(0x7ffb300f8990 msgr2=0x7ffb300f9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.612+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 shutdown_connections 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.612+0000 7ffb36bdb700 1 -- 172.21.15.31:0/3136684374 wait complete. 2024-04-24T23:21:48.615 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3443503785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 msgr2=0x7f1c580fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 --2- 172.21.15.31:0/3443503785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 0x7f1c580fcdd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c004940 tx=0x7f1c4c009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3443503785 shutdown_connections 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 --2- 172.21.15.31:0/3443503785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 0x7f1c580fcdd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 --2- 172.21.15.31:0/3443503785 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c58102dc0 0x7f1c58103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3443503785 >> 172.21.15.31:0/3443503785 conn(0x7f1c580f8930 msgr2=0x7f1c580fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3443503785 shutdown_connections 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3443503785 wait complete. 2024-04-24T23:21:48.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 Processor -- start 2024-04-24T23:21:48.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.718+0000 7f1c5fa6a700 1 -- start start 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5fa6a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c58102dc0 0x7f1c58113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5fa6a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 0x7f1c58113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5fa6a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c581144e0 con 0x7f1c58103760 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5fa6a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5810dae0 con 0x7f1c58102dc0 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d005700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 0x7f1c58113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d005700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 0x7f1c58113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46536/0 (socket says 172.21.15.31:46536) 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d005700 1 -- 172.21.15.31:0/3555168142 learned_addr learned my addr 172.21.15.31:0/3555168142 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d806700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c58102dc0 0x7f1c58113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d005700 1 -- 172.21.15.31:0/3555168142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c58102dc0 msgr2=0x7f1c58113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d005700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c58102dc0 0x7f1c58113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d005700 1 -- 172.21.15.31:0/3555168142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c005040 con 0x7f1c58103760 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d806700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c58102dc0 0x7f1c58113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5d005700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 0x7f1c58113e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009100 tx=0x7f1c4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:48.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c567fc700 1 -- 172.21.15.31:0/3555168142 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c4c00e040 con 0x7f1c58103760 2024-04-24T23:21:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c567fc700 1 -- 172.21.15.31:0/3555168142 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c4c013420 con 0x7f1c58103760 2024-04-24T23:21:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c567fc700 1 -- 172.21.15.31:0/3555168142 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c4c01c610 con 0x7f1c58103760 2024-04-24T23:21:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.719+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5810dd60 con 0x7f1c58103760 2024-04-24T23:21:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.720+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5810e320 con 0x7f1c58103760 2024-04-24T23:21:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.721+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c580fe510 con 0x7f1c58103760 2024-04-24T23:21:48.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.721+0000 7f1c567fc700 1 -- 172.21.15.31:0/3555168142 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1c4c013590 con 0x7f1c58103760 2024-04-24T23:21:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.721+0000 7f1c567fc700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1c4006e140 0x7f1c400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.722+0000 7f1c567fc700 1 -- 172.21.15.31:0/3555168142 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1c4c08ec40 con 0x7f1c58103760 2024-04-24T23:21:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.722+0000 7f1c5d806700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1c4006e140 0x7f1c400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:48.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.722+0000 7f1c5d806700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1c4006e140 0x7f1c400705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c48004c80 tx=0x7f1c48009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:48.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.726+0000 7f1c567fc700 1 -- 172.21.15.31:0/3555168142 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1c4c05bc60 con 0x7f1c58103760 2024-04-24T23:21:48.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.967+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c58109ea0 con 0x7f1c4006e140 2024-04-24T23:21:48.968 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:48 smithi031 ceph-mon[111404]: pgmap v130: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:48.968 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:48 smithi031 ceph-mon[111404]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:48.969 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:48 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1712769272' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:48.969 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:48 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3136684374' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:21:48.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.968+0000 7f1c567fc700 1 -- 172.21.15.31:0/3555168142 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f1c58109ea0 con 0x7f1c4006e140 2024-04-24T23:21:48.969 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:21:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:21:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:21:48.969 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "10/21 daemons upgraded", 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:21:48.970 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:21:48.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1c4006e140 msgr2=0x7f1c400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1c4006e140 0x7f1c400705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c48004c80 tx=0x7f1c48009040 comp rx=0 tx=0).stop 2024-04-24T23:21:48.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 msgr2=0x7f1c58113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:48.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 0x7f1c58113e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009100 tx=0x7f1c4c009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:48.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 shutdown_connections 2024-04-24T23:21:48.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f1c4006e140 0x7f1c400705f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1c58102dc0 0x7f1c58113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 --2- 172.21.15.31:0/3555168142 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1c58103760 0x7f1c58113e90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:48.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 >> 172.21.15.31:0/3555168142 conn(0x7f1c580f8930 msgr2=0x7f1c580f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:48.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 shutdown_connections 2024-04-24T23:21:48.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:48.970+0000 7f1c5fa6a700 1 -- 172.21.15.31:0/3555168142 wait complete. 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.075+0000 7f41cc946700 1 -- 172.21.15.31:0/2342351236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 msgr2=0x7f41c80fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.075+0000 7f41cc946700 1 --2- 172.21.15.31:0/2342351236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 0x7f41c80fbc80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f41b8004940 tx=0x7f41b8009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.075+0000 7f41cc946700 1 -- 172.21.15.31:0/2342351236 shutdown_connections 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.075+0000 7f41cc946700 1 --2- 172.21.15.31:0/2342351236 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 0x7f41c80fbc80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.075+0000 7f41cc946700 1 --2- 172.21.15.31:0/2342351236 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41c8101c70 0x7f41c8102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.075+0000 7f41cc946700 1 -- 172.21.15.31:0/2342351236 >> 172.21.15.31:0/2342351236 conn(0x7f41c8072b60 msgr2=0x7f41c8072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41cc946700 1 -- 172.21.15.31:0/2342351236 shutdown_connections 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41cc946700 1 -- 172.21.15.31:0/2342351236 wait complete. 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41cc946700 1 Processor -- start 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41cc946700 1 -- start start 2024-04-24T23:21:49.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41cc946700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41c8101c70 0x7f41c810ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41cc946700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 0x7f41c810f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41cc946700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c810f6d0 con 0x7f41c8102610 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41cc946700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c810f810 con 0x7f41c8101c70 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.076+0000 7f41c5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 0x7f41c810f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41c5d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 0x7f41c810f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46550/0 (socket says 172.21.15.31:46550) 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41c5d9b700 1 -- 172.21.15.31:0/3899102517 learned_addr learned my addr 172.21.15.31:0/3899102517 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41c659c700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41c8101c70 0x7f41c810ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41c5d9b700 1 -- 172.21.15.31:0/3899102517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41c8101c70 msgr2=0x7f41c810ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41c5d9b700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41c8101c70 0x7f41c810ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41c5d9b700 1 -- 172.21.15.31:0/3899102517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b8005040 con 0x7f41c8102610 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41c5d9b700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 0x7f41c810f190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009100 tx=0x7f41b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41beffd700 1 -- 172.21.15.31:0/3899102517 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b800e040 con 0x7f41c8102610 2024-04-24T23:21:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41beffd700 1 -- 172.21.15.31:0/3899102517 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41b8013420 con 0x7f41c8102610 2024-04-24T23:21:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c8117b60 con 0x7f41c8102610 2024-04-24T23:21:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41beffd700 1 -- 172.21.15.31:0/3899102517 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b801c610 con 0x7f41c8102610 2024-04-24T23:21:49.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.077+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c8118120 con 0x7f41c8102610 2024-04-24T23:21:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.078+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c80fd3c0 con 0x7f41c8102610 2024-04-24T23:21:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.079+0000 7f41beffd700 1 -- 172.21.15.31:0/3899102517 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f41b8013590 con 0x7f41c8102610 2024-04-24T23:21:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.079+0000 7f41beffd700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f41b406e140 0x7f41b40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:21:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.080+0000 7f41beffd700 1 -- 172.21.15.31:0/3899102517 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f41b808edf0 con 0x7f41c8102610 2024-04-24T23:21:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.080+0000 7f41c659c700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f41b406e140 0x7f41b40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:21:49.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.080+0000 7f41c659c700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f41b406e140 0x7f41b40705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41b0006fd0 tx=0x7f41b0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:21:49.085 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.084+0000 7f41beffd700 1 -- 172.21.15.31:0/3899102517 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f41b80581a0 con 0x7f41c8102610 2024-04-24T23:21:49.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:48 smithi040 ceph-mon[83047]: pgmap v130: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:49.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:48 smithi040 ceph-mon[83047]: from='client.34324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:49.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:48 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1712769272' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:21:49.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:48 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3136684374' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:21:49.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.380+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f41c805f5c0 con 0x7f41c8102610 2024-04-24T23:21:49.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.380+0000 7f41beffd700 1 -- 172.21.15.31:0/3899102517 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f41b805b860 con 0x7f41c8102610 2024-04-24T23:21:49.382 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:21:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f41b406e140 msgr2=0x7f41b40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f41b406e140 0x7f41b40705f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f41b0006fd0 tx=0x7f41b0008040 comp rx=0 tx=0).stop 2024-04-24T23:21:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 msgr2=0x7f41c810f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:21:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 0x7f41c810f190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009100 tx=0x7f41b8009e30 comp rx=0 tx=0).stop 2024-04-24T23:21:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 shutdown_connections 2024-04-24T23:21:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f41b406e140 0x7f41b40705f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f41c8101c70 0x7f41c810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 --2- 172.21.15.31:0/3899102517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c8102610 0x7f41c810f190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:21:49.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 >> 172.21.15.31:0/3899102517 conn(0x7f41c8072b60 msgr2=0x7f41c80fa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:21:49.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.382+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 shutdown_connections 2024-04-24T23:21:49.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:21:49.383+0000 7f41cc946700 1 -- 172.21.15.31:0/3899102517 wait complete. 2024-04-24T23:21:50.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:49 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:50.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:49 smithi040 ceph-mon[83047]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:50.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:49 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3899102517' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:21:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:49 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:49 smithi031 ceph-mon[111404]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:21:50.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:49 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3899102517' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:21:51.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:50 smithi040 ceph-mon[83047]: pgmap v131: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:51.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:50 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T23:21:51.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:50 smithi031 ceph-mon[111404]: pgmap v131: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:51.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:50 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T23:21:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:51 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T23:21:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:51 smithi040 ceph-mon[83047]: Upgrade: osd.4 is safe to restart 2024-04-24T23:21:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:51 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-24T23:21:52.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:51 smithi031 ceph-mon[111404]: Upgrade: osd.4 is safe to restart 2024-04-24T23:21:53.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:53 smithi040 ceph-mon[83047]: pgmap v132: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:53.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:53 smithi040 ceph-mon[83047]: Upgrade: Updating osd.4 2024-04-24T23:21:53.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:53.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:21:53.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:53.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:53 smithi040 ceph-mon[83047]: Deploying daemon osd.4 on smithi040 2024-04-24T23:21:53.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:53 smithi031 ceph-mon[111404]: pgmap v132: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:53.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:53 smithi031 ceph-mon[111404]: Upgrade: Updating osd.4 2024-04-24T23:21:53.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:53.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-24T23:21:53.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:21:53.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:53 smithi031 ceph-mon[111404]: Deploying daemon osd.4 on smithi040 2024-04-24T23:21:55.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:55 smithi040 ceph-mon[83047]: pgmap v133: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:55.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:55 smithi031 ceph-mon[111404]: pgmap v133: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:55.736 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:55 smithi040 systemd[1]: Stopping Ceph osd.4 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:21:56.162 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[45250]: 2024-04-24T23:21:55.740+0000 7efc7cdbf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:21:56.162 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[45250]: 2024-04-24T23:21:55.740+0000 7efc7cdbf700 -1 osd.4 69 *** Got signal Terminated *** 2024-04-24T23:21:56.162 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[45250]: 2024-04-24T23:21:55.740+0000 7efc7cdbf700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:21:56.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:56 smithi031 ceph-mon[111404]: osd.4 marked itself down and dead 2024-04-24T23:21:56.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:56 smithi040 ceph-mon[83047]: osd.4 marked itself down and dead 2024-04-24T23:21:57.067 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:56 smithi040 podman[97249]: 2024-04-24 23:21:56.747098819 +0000 UTC m=+1.123776447 container died 727ed09fe1c37e7ab16fb4f2f107759588901204967a95b4067910e74d8d557b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-24T23:21:57.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:57 smithi040 ceph-mon[83047]: pgmap v134: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:57.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:57 smithi040 ceph-mon[83047]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:21:57.384 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:57 smithi040 ceph-mon[83047]: osdmap e70: 6 total, 5 up, 6 in 2024-04-24T23:21:57.384 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:57 smithi040 podman[97249]: 2024-04-24 23:21:57.173572479 +0000 UTC m=+1.550250087 container cleanup 727ed09fe1c37e7ab16fb4f2f107759588901204967a95b4067910e74d8d557b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, ceph=True) 2024-04-24T23:21:57.384 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:57 smithi040 bash[97249]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4 2024-04-24T23:21:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:57 smithi031 ceph-mon[111404]: pgmap v134: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:57 smithi031 ceph-mon[111404]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:21:57.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:57 smithi031 ceph-mon[111404]: osdmap e70: 6 total, 5 up, 6 in 2024-04-24T23:21:57.662 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:57 smithi040 podman[97279]: 2024-04-24 23:21:57.383525337 +0000 UTC m=+0.636428971 container remove 727ed09fe1c37e7ab16fb4f2f107759588901204967a95b4067910e74d8d557b (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=quincy-c61472c) 2024-04-24T23:21:58.162 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:57 smithi040 podman[97370]: 2024-04-24 23:21:57.702114856 +0000 UTC m=+0.018414646 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:21:58.162 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:57 smithi040 podman[97370]: 2024-04-24 23:21:57.809344715 +0000 UTC m=+0.125644529 container create d2c1ccc0ed4a7f4a75c56d3da4c7426b5be957fe06a1eeffe5ba0aab22c55b9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:21:58.471 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:58 smithi040 ceph-mon[83047]: osdmap e71: 6 total, 5 up, 6 in 2024-04-24T23:21:58.472 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:58 smithi040 podman[97370]: 2024-04-24 23:21:58.162292686 +0000 UTC m=+0.478592469 container init d2c1ccc0ed4a7f4a75c56d3da4c7426b5be957fe06a1eeffe5ba0aab22c55b9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-24T23:21:58.472 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:58 smithi040 podman[97370]: 2024-04-24 23:21:58.166152094 +0000 UTC m=+0.482451866 container start d2c1ccc0ed4a7f4a75c56d3da4c7426b5be957fe06a1eeffe5ba0aab22c55b9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-24T23:21:58.472 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:58 smithi040 podman[97370]: 2024-04-24 23:21:58.253593487 +0000 UTC m=+0.569893272 container attach d2c1ccc0ed4a7f4a75c56d3da4c7426b5be957fe06a1eeffe5ba0aab22c55b9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-24T23:21:58.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:58 smithi031 ceph-mon[111404]: osdmap e71: 6 total, 5 up, 6 in 2024-04-24T23:21:58.783 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:58 smithi040 podman[97370]: 2024-04-24 23:21:58.471044601 +0000 UTC m=+0.787344369 container died d2c1ccc0ed4a7f4a75c56d3da4c7426b5be957fe06a1eeffe5ba0aab22c55b9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-24T23:21:59.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:59 smithi040 ceph-mon[83047]: pgmap v137: 65 pgs: 10 peering, 4 stale+active+clean, 51 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:59.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:59 smithi040 ceph-mon[83047]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T23:21:59.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:59.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:21:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:59.413 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 podman[97396]: 2024-04-24 23:21:59.083038421 +0000 UTC m=+0.606174336 container remove d2c1ccc0ed4a7f4a75c56d3da4c7426b5be957fe06a1eeffe5ba0aab22c55b9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-24T23:21:59.413 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-24T23:21:59.413 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 systemd[1]: Stopped Ceph osd.4 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:21:59.413 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.4.service: Consumed 1min 8.456s CPU time. 2024-04-24T23:21:59.413 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 systemd[1]: Starting Ceph osd.4 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:21:59.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:59 smithi031 ceph-mon[111404]: pgmap v137: 65 pgs: 10 peering, 4 stale+active+clean, 51 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:21:59.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:59 smithi031 ceph-mon[111404]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-24T23:21:59.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:21:59.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:21:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:21:59.812 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 podman[97497]: 2024-04-24 23:21:59.434541871 +0000 UTC m=+0.018494210 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:21:59.812 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 podman[97497]: 2024-04-24 23:21:59.559189036 +0000 UTC m=+0.143141367 container create 5ae62ae99ca00a90ab3d890cb3cf786313ee36e83bb0f11f22bd6f660534e155 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:22:00.162 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 podman[97497]: 2024-04-24 23:21:59.81186801 +0000 UTC m=+0.395820341 container init 5ae62ae99ca00a90ab3d890cb3cf786313ee36e83bb0f11f22bd6f660534e155 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-24T23:22:00.163 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 podman[97497]: 2024-04-24 23:21:59.815231871 +0000 UTC m=+0.399184211 container start 5ae62ae99ca00a90ab3d890cb3cf786313ee36e83bb0f11f22bd6f660534e155 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-24T23:22:00.163 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:21:59 smithi040 podman[97497]: 2024-04-24 23:21:59.903335837 +0000 UTC m=+0.487288178 container attach 5ae62ae99ca00a90ab3d890cb3cf786313ee36e83bb0f11f22bd6f660534e155 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-24T23:22:00.623 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:00 smithi040 ceph-mon[83047]: Health check failed: Degraded data redundancy: 25/225 objects degraded (11.111%), 7 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate[97512]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate[97512]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 bash[97497]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 bash[97497]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 bash[97497]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 bash[97497]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 bash[97497]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 bash[97497]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 bash[97497]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-24T23:22:00.624 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate[97512]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-24T23:22:00.625 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate[97512]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-24T23:22:00.625 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate[97512]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-24T23:22:00.625 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate[97512]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T23:22:00.625 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate[97512]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-24T23:22:00.625 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 podman[97497]: 2024-04-24 23:22:00.301117665 +0000 UTC m=+0.885069997 container died 5ae62ae99ca00a90ab3d890cb3cf786313ee36e83bb0f11f22bd6f660534e155 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:22:00.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:00 smithi031 ceph-mon[111404]: Health check failed: Degraded data redundancy: 25/225 objects degraded (11.111%), 7 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:01.282 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:00 smithi040 podman[97637]: 2024-04-24 23:22:00.949116456 +0000 UTC m=+0.638288088 container remove 5ae62ae99ca00a90ab3d890cb3cf786313ee36e83bb0f11f22bd6f660534e155 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-24T23:22:01.282 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:01 smithi040 podman[97701]: 2024-04-24 23:22:01.068059907 +0000 UTC m=+0.018709599 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:22:01.282 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:01 smithi040 podman[97701]: 2024-04-24 23:22:01.183454499 +0000 UTC m=+0.134104236 container create 85d9bab7b352c0753c8453792a8db8e1aee84756a7d60bd65af779f108d58c9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T23:22:01.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:01 smithi040 ceph-mon[83047]: pgmap v138: 65 pgs: 7 active+undersized+degraded, 10 peering, 2 stale+active+clean, 9 active+undersized, 37 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 25/225 objects degraded (11.111%) 2024-04-24T23:22:01.662 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:01 smithi040 podman[97701]: 2024-04-24 23:22:01.402181404 +0000 UTC m=+0.352831097 container init 85d9bab7b352c0753c8453792a8db8e1aee84756a7d60bd65af779f108d58c9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-24T23:22:01.662 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:01 smithi040 podman[97701]: 2024-04-24 23:22:01.405412834 +0000 UTC m=+0.356062534 container start 85d9bab7b352c0753c8453792a8db8e1aee84756a7d60bd65af779f108d58c9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-24T23:22:01.662 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:01 smithi040 ceph-osd[97721]: -- 172.21.15.40:0/1262616046 <== mon.1 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f6949cd680 con 0x55f694996c00 2024-04-24T23:22:01.662 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:01 smithi040 bash[97701]: 85d9bab7b352c0753c8453792a8db8e1aee84756a7d60bd65af779f108d58c9b 2024-04-24T23:22:01.662 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:01 smithi040 systemd[1]: Started Ceph osd.4 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:22:01.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:01 smithi031 ceph-mon[111404]: pgmap v138: 65 pgs: 7 active+undersized+degraded, 10 peering, 2 stale+active+clean, 9 active+undersized, 37 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 25/225 objects degraded (11.111%) 2024-04-24T23:22:02.366 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:01 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:22:01.965+0000 7f82bb31a7c0 -1 Falling back to public interface 2024-04-24T23:22:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:02 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:02 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:02.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:02 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:02 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:02 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:02.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:02 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:03.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:03 smithi031 ceph-mon[111404]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/225 objects degraded (17.333%) 2024-04-24T23:22:03.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:03 smithi031 ceph-mon[111404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T23:22:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:03 smithi040 ceph-mon[83047]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/225 objects degraded (17.333%) 2024-04-24T23:22:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:03 smithi040 ceph-mon[83047]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-24T23:22:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:05 smithi040 ceph-mon[83047]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/225 objects degraded (17.333%) 2024-04-24T23:22:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:05.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:05 smithi031 ceph-mon[111404]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/225 objects degraded (17.333%) 2024-04-24T23:22:05.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:05.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:06.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:06 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:06.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:06 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:06.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:06 smithi031 ceph-mon[111404]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/225 objects degraded (17.333%) 2024-04-24T23:22:07.094 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:06 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:07.094 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:06 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:07.094 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:06 smithi040 ceph-mon[83047]: pgmap v141: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 39/225 objects degraded (17.333%) 2024-04-24T23:22:07.412 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:07 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:22:07.092+0000 7f82bb31a7c0 -1 osd.4 69 log_to_monitors true 2024-04-24T23:22:07.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:07 smithi031 ceph-mon[111404]: from='osd.4 [v2:172.21.15.40:6808/3032298464,v1:172.21.15.40:6809/3032298464]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:22:07.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:07 smithi031 ceph-mon[111404]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:22:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:07 smithi040 ceph-mon[83047]: from='osd.4 [v2:172.21.15.40:6808/3032298464,v1:172.21.15.40:6809/3032298464]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:22:08.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:07 smithi040 ceph-mon[83047]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-24T23:22:08.163 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:22:07 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:22:07.765+0000 7f82ac772700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:22:08.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:08 smithi040 ceph-mon[83047]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:22:08.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:08 smithi040 ceph-mon[83047]: osdmap e72: 6 total, 5 up, 6 in 2024-04-24T23:22:08.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:08 smithi040 ceph-mon[83047]: from='osd.4 [v2:172.21.15.40:6808/3032298464,v1:172.21.15.40:6809/3032298464]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:22:08.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:08 smithi040 ceph-mon[83047]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:22:08.904 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:08 smithi040 ceph-mon[83047]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 39/225 objects degraded (17.333%) 2024-04-24T23:22:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:08 smithi031 ceph-mon[111404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-24T23:22:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:08 smithi031 ceph-mon[111404]: osdmap e72: 6 total, 5 up, 6 in 2024-04-24T23:22:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:08 smithi031 ceph-mon[111404]: from='osd.4 [v2:172.21.15.40:6808/3032298464,v1:172.21.15.40:6809/3032298464]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:22:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:08 smithi031 ceph-mon[111404]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:22:09.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:08 smithi031 ceph-mon[111404]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 39/225 objects degraded (17.333%) 2024-04-24T23:22:10.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:22:10.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: osd.4 [v2:172.21.15.40:6808/3032298464,v1:172.21.15.40:6809/3032298464] boot 2024-04-24T23:22:10.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: osdmap e73: 6 total, 6 up, 6 in 2024-04-24T23:22:10.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:22:10.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:10.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:10.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T23:22:10.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:09 smithi040 ceph-mon[83047]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: osd.4 [v2:172.21.15.40:6808/3032298464,v1:172.21.15.40:6809/3032298464] boot 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: osdmap e73: 6 total, 6 up, 6 in 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:10.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:10.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:10.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:10.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T23:22:10.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T23:22:10.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:09 smithi031 ceph-mon[111404]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-24T23:22:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:10 smithi040 ceph-mon[83047]: osdmap e74: 6 total, 6 up, 6 in 2024-04-24T23:22:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:10 smithi040 ceph-mon[83047]: pgmap v146: 65 pgs: 4 peering, 14 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 34/225 objects degraded (15.111%) 2024-04-24T23:22:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:10 smithi040 ceph-mon[83047]: Health check update: Degraded data redundancy: 39/225 objects degraded (17.333%), 15 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:11.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:10 smithi031 ceph-mon[111404]: osdmap e74: 6 total, 6 up, 6 in 2024-04-24T23:22:11.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:10 smithi031 ceph-mon[111404]: pgmap v146: 65 pgs: 4 peering, 14 active+undersized+degraded, 16 active+undersized, 31 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 34/225 objects degraded (15.111%) 2024-04-24T23:22:11.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:10 smithi031 ceph-mon[111404]: Health check update: Degraded data redundancy: 39/225 objects degraded (17.333%), 15 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:13.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:12 smithi040 ceph-mon[83047]: pgmap v147: 65 pgs: 2 active, 4 peering, 7 active+undersized+degraded, 9 active+undersized, 43 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/225 objects degraded (9.778%) 2024-04-24T23:22:13.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:12 smithi031 ceph-mon[111404]: pgmap v147: 65 pgs: 2 active, 4 peering, 7 active+undersized+degraded, 9 active+undersized, 43 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/225 objects degraded (9.778%) 2024-04-24T23:22:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:22:14.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:14 smithi031 ceph-mon[111404]: pgmap v148: 65 pgs: 2 active, 4 peering, 59 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:22:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:14 smithi040 ceph-mon[83047]: pgmap v148: 65 pgs: 2 active, 4 peering, 59 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:15 smithi040 ceph-mon[83047]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/225 objects degraded (9.778%), 7 pgs degraded) 2024-04-24T23:22:16.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:15 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:22:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:15 smithi031 ceph-mon[111404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/225 objects degraded (9.778%), 7 pgs degraded) 2024-04-24T23:22:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:15 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:22:17.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:16 smithi040 ceph-mon[83047]: pgmap v149: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:17.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:16 smithi031 ceph-mon[111404]: pgmap v149: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:19.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:18 smithi040 ceph-mon[83047]: pgmap v150: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:18 smithi031 ceph-mon[111404]: pgmap v150: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.501+0000 7ff4115b6700 1 -- 172.21.15.31:0/3692724855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 msgr2=0x7ff40c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.501+0000 7ff4115b6700 1 --2- 172.21.15.31:0/3692724855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 0x7ff40c106e70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff400004940 tx=0x7ff400009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.501+0000 7ff4115b6700 1 -- 172.21.15.31:0/3692724855 shutdown_connections 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.501+0000 7ff4115b6700 1 --2- 172.21.15.31:0/3692724855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 0x7ff40c106e70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.501+0000 7ff4115b6700 1 --2- 172.21.15.31:0/3692724855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c0fde00 0x7ff40c0fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.501+0000 7ff4115b6700 1 -- 172.21.15.31:0/3692724855 >> 172.21.15.31:0/3692724855 conn(0x7ff40c0f7860 msgr2=0x7ff40c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.501+0000 7ff4115b6700 1 -- 172.21.15.31:0/3692724855 shutdown_connections 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.501+0000 7ff4115b6700 1 -- 172.21.15.31:0/3692724855 wait complete. 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.503+0000 7ff4115b6700 1 Processor -- start 2024-04-24T23:22:19.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.503+0000 7ff4115b6700 1 -- start start 2024-04-24T23:22:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.503+0000 7ff4115b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c0fde00 0x7ff40c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.503+0000 7ff4115b6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 0x7ff40c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.503+0000 7ff4115b6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c114520 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.503+0000 7ff4115b6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40c10db40 con 0x7ff40c0fde00 2024-04-24T23:22:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.503+0000 7ff40a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 0x7ff40c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff40affd700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c0fde00 0x7ff40c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff40a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 0x7ff40c113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46404/0 (socket says 172.21.15.31:46404) 2024-04-24T23:22:19.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff40a7fc700 1 -- 172.21.15.31:0/2853300780 learned_addr learned my addr 172.21.15.31:0/2853300780 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff40a7fc700 1 -- 172.21.15.31:0/2853300780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c0fde00 msgr2=0x7ff40c113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff40a7fc700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c0fde00 0x7ff40c113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff40a7fc700 1 -- 172.21.15.31:0/2853300780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff400005040 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff40a7fc700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 0x7ff40c113ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff400009380 tx=0x7ff400009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:19.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff3f37fe700 1 -- 172.21.15.31:0/2853300780 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40000e040 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40c10ddc0 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff3f37fe700 1 -- 172.21.15.31:0/2853300780 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff400009af0 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff3f37fe700 1 -- 172.21.15.31:0/2853300780 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff40001c3f0 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.504+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40c10e300 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.505+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40c066ad0 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.508 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.506+0000 7ff3f37fe700 1 -- 172.21.15.31:0/2853300780 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff400013420 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.506+0000 7ff3f37fe700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff3f406e140 0x7ff3f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.507+0000 7ff3f37fe700 1 -- 172.21.15.31:0/2853300780 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff40008ed20 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.507+0000 7ff40affd700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff3f406e140 0x7ff3f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:19.509 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.507+0000 7ff40affd700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff3f406e140 0x7ff3f40705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc004f40 tx=0x7ff3fc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:19.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.512+0000 7ff3f37fe700 1 -- 172.21.15.31:0/2853300780 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff400058170 con 0x7ff40c0fe7a0 2024-04-24T23:22:19.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.738+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff40c109e90 con 0x7ff3f406e140 2024-04-24T23:22:19.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.739+0000 7ff3f37fe700 1 -- 172.21.15.31:0/2853300780 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff40c109e90 con 0x7ff3f406e140 2024-04-24T23:22:19.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff3f406e140 msgr2=0x7ff3f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff3f406e140 0x7ff3f40705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc004f40 tx=0x7ff3fc00b040 comp rx=0 tx=0).stop 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 msgr2=0x7ff40c113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 0x7ff40c113ed0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff400009380 tx=0x7ff400009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 shutdown_connections 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff3f406e140 0x7ff3f40705f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff40c0fde00 0x7ff40c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 --2- 172.21.15.31:0/2853300780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff40c0fe7a0 0x7ff40c113ed0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 >> 172.21.15.31:0/2853300780 conn(0x7ff40c0f7860 msgr2=0x7ff40c0f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 shutdown_connections 2024-04-24T23:22:19.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.741+0000 7ff4115b6700 1 -- 172.21.15.31:0/2853300780 wait complete. 2024-04-24T23:22:19.753 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:22:19.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.847+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/3763059138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b64101d90 msgr2=0x7f4b64102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.847+0000 7f4b6bcfa700 1 --2- 172.21.15.31:0/3763059138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b64101d90 0x7f4b64102160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009ce0 tx=0x7f4b5c004d10 comp rx=0 tx=0).stop 2024-04-24T23:22:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.847+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/3763059138 shutdown_connections 2024-04-24T23:22:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.847+0000 7f4b6bcfa700 1 --2- 172.21.15.31:0/3763059138 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b640fbd40 0x7f4b640fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.847+0000 7f4b6bcfa700 1 --2- 172.21.15.31:0/3763059138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b64101d90 0x7f4b64102160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.847+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/3763059138 >> 172.21.15.31:0/3763059138 conn(0x7f4b64071d00 msgr2=0x7f4b64072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.848+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/3763059138 shutdown_connections 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.848+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/3763059138 wait complete. 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.848+0000 7f4b6bcfa700 1 Processor -- start 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.848+0000 7f4b6bcfa700 1 -- start start 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.848+0000 7f4b6bcfa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b640fbd40 0x7f4b64113d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.848+0000 7f4b6bcfa700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b64101d90 0x7f4b64114290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.848+0000 7f4b6bcfa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641148e0 con 0x7f4b640fbd40 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.848+0000 7f4b6bcfa700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6410dcb0 con 0x7f4b64101d90 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b69a96700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b640fbd40 0x7f4b64113d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b69a96700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b640fbd40 0x7f4b64113d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46422/0 (socket says 172.21.15.31:46422) 2024-04-24T23:22:19.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b69a96700 1 -- 172.21.15.31:0/1388569497 learned_addr learned my addr 172.21.15.31:0/1388569497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b69295700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b64101d90 0x7f4b64114290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b69a96700 1 -- 172.21.15.31:0/1388569497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b64101d90 msgr2=0x7f4b64114290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b69a96700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b64101d90 0x7f4b64114290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b69a96700 1 -- 172.21.15.31:0/1388569497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c009940 con 0x7f4b640fbd40 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b69a96700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b640fbd40 0x7f4b64113d50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005010 tx=0x7f4b5c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b627fc700 1 -- 172.21.15.31:0/1388569497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b5c00e040 con 0x7f4b640fbd40 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b627fc700 1 -- 172.21.15.31:0/1388569497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b5c008da0 con 0x7f4b640fbd40 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b627fc700 1 -- 172.21.15.31:0/1388569497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b5c01c780 con 0x7f4b640fbd40 2024-04-24T23:22:19.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.849+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6410df30 con 0x7f4b640fbd40 2024-04-24T23:22:19.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.850+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6410e430 con 0x7f4b640fbd40 2024-04-24T23:22:19.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.851+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b640fd430 con 0x7f4b640fbd40 2024-04-24T23:22:19.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.851+0000 7f4b627fc700 1 -- 172.21.15.31:0/1388569497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4b5c012b70 con 0x7f4b640fbd40 2024-04-24T23:22:19.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.851+0000 7f4b627fc700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4c06e150 0x7f4b4c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:19.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.852+0000 7f4b627fc700 1 -- 172.21.15.31:0/1388569497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4b5c08f610 con 0x7f4b640fbd40 2024-04-24T23:22:19.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.852+0000 7f4b69295700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4c06e150 0x7f4b4c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:19.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.852+0000 7f4b69295700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4c06e150 0x7f4b4c070600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4b6410f5f0 tx=0x7f4b58009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:19.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:19.856+0000 7f4b627fc700 1 -- 172.21.15.31:0/1388569497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4b5c05c590 con 0x7f4b640fbd40 2024-04-24T23:22:20.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.081+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b6404bed0 con 0x7f4b4c06e150 2024-04-24T23:22:20.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.082+0000 7f4b627fc700 1 -- 172.21.15.31:0/1388569497 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4b6404bed0 con 0x7f4b4c06e150 2024-04-24T23:22:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4c06e150 msgr2=0x7f4b4c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4c06e150 0x7f4b4c070600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4b6410f5f0 tx=0x7f4b58009040 comp rx=0 tx=0).stop 2024-04-24T23:22:20.086 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b640fbd40 msgr2=0x7f4b64113d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b640fbd40 0x7f4b64113d50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c005010 tx=0x7f4b5c004d10 comp rx=0 tx=0).stop 2024-04-24T23:22:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 shutdown_connections 2024-04-24T23:22:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b640fbd40 0x7f4b64113d50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4c06e150 0x7f4b4c070600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 --2- 172.21.15.31:0/1388569497 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b64101d90 0x7f4b64114290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 >> 172.21.15.31:0/1388569497 conn(0x7f4b64071d00 msgr2=0x7f4b640655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 shutdown_connections 2024-04-24T23:22:20.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.085+0000 7f4b6bcfa700 1 -- 172.21.15.31:0/1388569497 wait complete. 2024-04-24T23:22:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.195+0000 7f289ce51700 1 -- 172.21.15.31:0/3272112098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fe830 msgr2=0x7f2898106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.195+0000 7f289ce51700 1 --2- 172.21.15.31:0/3272112098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fe830 0x7f2898106e70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2888004940 tx=0x7f2888009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.195+0000 7f289ce51700 1 -- 172.21.15.31:0/3272112098 shutdown_connections 2024-04-24T23:22:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.195+0000 7f289ce51700 1 --2- 172.21.15.31:0/3272112098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fe830 0x7f2898106e70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.195+0000 7f289ce51700 1 --2- 172.21.15.31:0/3272112098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28980fde90 0x7f28980fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.195+0000 7f289ce51700 1 -- 172.21.15.31:0/3272112098 >> 172.21.15.31:0/3272112098 conn(0x7f28980f7940 msgr2=0x7f28980f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.195+0000 7f289ce51700 1 -- 172.21.15.31:0/3272112098 shutdown_connections 2024-04-24T23:22:20.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.195+0000 7f289ce51700 1 -- 172.21.15.31:0/3272112098 wait complete. 2024-04-24T23:22:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.196+0000 7f289ce51700 1 Processor -- start 2024-04-24T23:22:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.196+0000 7f289ce51700 1 -- start start 2024-04-24T23:22:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.196+0000 7f289ce51700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fde90 0x7f28981139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.196+0000 7f289ce51700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28980fe830 0x7f2898113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.196+0000 7f289ce51700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2898114530 con 0x7f28980fde90 2024-04-24T23:22:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.196+0000 7f289ce51700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289810db30 con 0x7f28980fe830 2024-04-24T23:22:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f289659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fde90 0x7f28981139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:20.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f289659c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fde90 0x7f28981139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46450/0 (socket says 172.21.15.31:46450) 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f289659c700 1 -- 172.21.15.31:0/1719576429 learned_addr learned my addr 172.21.15.31:0/1719576429 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f2895d9b700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28980fe830 0x7f2898113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f289659c700 1 -- 172.21.15.31:0/1719576429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28980fe830 msgr2=0x7f2898113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f289659c700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28980fe830 0x7f2898113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f289659c700 1 -- 172.21.15.31:0/1719576429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2888005040 con 0x7f28980fde90 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f289659c700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fde90 0x7f28981139a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f288c002a50 tx=0x7f288c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f2886ffd700 1 -- 172.21.15.31:0/1719576429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f288c007cc0 con 0x7f28980fde90 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f2886ffd700 1 -- 172.21.15.31:0/1719576429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f288c00f040 con 0x7f28980fde90 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.197+0000 7f2886ffd700 1 -- 172.21.15.31:0/1719576429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f288c00da30 con 0x7f28980fde90 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.198+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289810de10 con 0x7f28980fde90 2024-04-24T23:22:20.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.198+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289810e350 con 0x7f28980fde90 2024-04-24T23:22:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.199+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2898101820 con 0x7f28980fde90 2024-04-24T23:22:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.199+0000 7f2886ffd700 1 -- 172.21.15.31:0/1719576429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f288c00dbb0 con 0x7f28980fde90 2024-04-24T23:22:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.200+0000 7f2886ffd700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f288006e140 0x7f28800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.200+0000 7f2886ffd700 1 -- 172.21.15.31:0/1719576429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f288c08d790 con 0x7f28980fde90 2024-04-24T23:22:20.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.200+0000 7f2895d9b700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f288006e140 0x7f28800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:20.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.200+0000 7f2895d9b700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f288006e140 0x7f28800705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f289810ecf0 tx=0x7f2888009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:20.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.204+0000 7f2886ffd700 1 -- 172.21.15.31:0/1719576429 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f288c05a710 con 0x7f28980fde90 2024-04-24T23:22:20.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.419+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f289810eff0 con 0x7f288006e140 2024-04-24T23:22:20.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.429+0000 7f2886ffd700 1 -- 172.21.15.31:0/1719576429 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f289810eff0 con 0x7f288006e140 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (5m) 83s ago 17m 25.6M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (3m) 83s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (3m) 15s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 83s ago 16m 79.9M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (11m) 83s ago 11m 138M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (11m) 83s ago 11m 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af 5993e9eff5b2 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (11m) 15s ago 11m 15.1M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (11m) 15s ago 11m 16.4M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (7m) 83s ago 18m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (6m) 15s ago 16m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:22:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (4m) 83s ago 18m 45.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (3m) 15s ago 16m 35.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (6m) 83s ago 17m 16.7M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (6m) 15s ago 16m 14.9M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (2m) 83s ago 15m 87.6M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (2m) 83s ago 14m 101M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (89s) 83s ago 14m 11.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 271a0764ef5c 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (53s) 15s ago 13m 114M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 97c6f05632f6 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (19s) 15s ago 13m 14.8M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 85d9bab7b352 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (13m) 15s ago 13m 226M 2583M 17.2.7-902-gc61472c8 0de407c1b3af ea87e2bf6ab9 2024-04-24T23:22:20.432 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 83s ago 16m 59.9M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:22:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f288006e140 msgr2=0x7f28800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f288006e140 0x7f28800705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f289810ecf0 tx=0x7f2888009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fde90 msgr2=0x7f28981139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fde90 0x7f28981139a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f288c002a50 tx=0x7f288c0046b0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 shutdown_connections 2024-04-24T23:22:20.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28980fde90 0x7f28981139a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f288006e140 0x7f28800705f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 --2- 172.21.15.31:0/1719576429 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f28980fe830 0x7f2898113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 >> 172.21.15.31:0/1719576429 conn(0x7f28980f7940 msgr2=0x7f28980ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:20.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.431+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 shutdown_connections 2024-04-24T23:22:20.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.432+0000 7f289ce51700 1 -- 172.21.15.31:0/1719576429 wait complete. 2024-04-24T23:22:20.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.536+0000 7fc2a162a700 1 -- 172.21.15.31:0/2187500453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 msgr2=0x7fc29c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.536+0000 7fc2a162a700 1 --2- 172.21.15.31:0/2187500453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 0x7fc29c1020d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009ce0 tx=0x7fc28c004d10 comp rx=0 tx=0).stop 2024-04-24T23:22:20.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.536+0000 7fc2a162a700 1 -- 172.21.15.31:0/2187500453 shutdown_connections 2024-04-24T23:22:20.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.536+0000 7fc2a162a700 1 --2- 172.21.15.31:0/2187500453 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc29c1026a0 0x7fc29c0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.536+0000 7fc2a162a700 1 --2- 172.21.15.31:0/2187500453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 0x7fc29c1020d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.536+0000 7fc2a162a700 1 -- 172.21.15.31:0/2187500453 >> 172.21.15.31:0/2187500453 conn(0x7fc29c0f7870 msgr2=0x7fc29c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:20.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.536+0000 7fc2a162a700 1 -- 172.21.15.31:0/2187500453 shutdown_connections 2024-04-24T23:22:20.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.536+0000 7fc2a162a700 1 -- 172.21.15.31:0/2187500453 wait complete. 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc2a162a700 1 Processor -- start 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc2a162a700 1 -- start start 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc2a162a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 0x7fc29c112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc2a162a700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc29c1026a0 0x7fc29c112f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc2a162a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c113620 con 0x7fc29c101d00 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc2a162a700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c10cb90 con 0x7fc29c1026a0 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc29affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 0x7fc29c112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc29affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 0x7fc29c112a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46468/0 (socket says 172.21.15.31:46468) 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc29affd700 1 -- 172.21.15.31:0/695849507 learned_addr learned my addr 172.21.15.31:0/695849507 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc29a7fc700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc29c1026a0 0x7fc29c112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc29affd700 1 -- 172.21.15.31:0/695849507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc29c1026a0 msgr2=0x7fc29c112f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc29affd700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc29c1026a0 0x7fc29c112f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc29affd700 1 -- 172.21.15.31:0/695849507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c009940 con 0x7fc29c101d00 2024-04-24T23:22:20.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.537+0000 7fc29affd700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 0x7fc29c112a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00be90 tx=0x7fc28c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:20.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.538+0000 7fc2837fe700 1 -- 172.21.15.31:0/695849507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28c00e040 con 0x7fc29c101d00 2024-04-24T23:22:20.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.538+0000 7fc2837fe700 1 -- 172.21.15.31:0/695849507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc28c008ea0 con 0x7fc29c101d00 2024-04-24T23:22:20.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.538+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c10ce10 con 0x7fc29c101d00 2024-04-24T23:22:20.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.538+0000 7fc2837fe700 1 -- 172.21.15.31:0/695849507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc28c01c810 con 0x7fc29c101d00 2024-04-24T23:22:20.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.538+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c10d3d0 con 0x7fc29c101d00 2024-04-24T23:22:20.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.539+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c0fd450 con 0x7fc29c101d00 2024-04-24T23:22:20.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.540+0000 7fc2837fe700 1 -- 172.21.15.31:0/695849507 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc28c012c00 con 0x7fc29c101d00 2024-04-24T23:22:20.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.540+0000 7fc2837fe700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc28406e140 0x7fc2840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.540+0000 7fc2837fe700 1 -- 172.21.15.31:0/695849507 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc28c08f700 con 0x7fc29c101d00 2024-04-24T23:22:20.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.540+0000 7fc29a7fc700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc28406e140 0x7fc2840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:20.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.540+0000 7fc29a7fc700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc28406e140 0x7fc2840705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc290006fd0 tx=0x7fc290008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:20.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.545+0000 7fc2837fe700 1 -- 172.21.15.31:0/695849507 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc28c05c600 con 0x7fc29c101d00 2024-04-24T23:22:20.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.844+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc29c04c710 con 0x7fc29c101d00 2024-04-24T23:22:20.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.844+0000 7fc2837fe700 1 -- 172.21.15.31:0/695849507 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fc28c004e10 con 0x7fc29c101d00 2024-04-24T23:22:20.846 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:22:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:22:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:22:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:22:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:22:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:22:20.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:22:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:22:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-24T23:22:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 5 2024-04-24T23:22:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:22:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:22:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-24T23:22:20.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:22:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:22:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5, 2024-04-24T23:22:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 9 2024-04-24T23:22:20.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:22:20.849 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:22:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc28406e140 msgr2=0x7fc2840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc28406e140 0x7fc2840705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc290006fd0 tx=0x7fc290008040 comp rx=0 tx=0).stop 2024-04-24T23:22:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 msgr2=0x7fc29c112a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 0x7fc29c112a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00be90 tx=0x7fc28c004d10 comp rx=0 tx=0).stop 2024-04-24T23:22:20.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 shutdown_connections 2024-04-24T23:22:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc29c101d00 0x7fc29c112a00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc28406e140 0x7fc2840705f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 --2- 172.21.15.31:0/695849507 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc29c1026a0 0x7fc29c112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 >> 172.21.15.31:0/695849507 conn(0x7fc29c0f7870 msgr2=0x7fc29c0f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 shutdown_connections 2024-04-24T23:22:20.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.847+0000 7fc2a162a700 1 -- 172.21.15.31:0/695849507 wait complete. 2024-04-24T23:22:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.951+0000 7f5efe72f700 1 -- 172.21.15.31:0/1157578007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 msgr2=0x7f5ef81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.951+0000 7f5efe72f700 1 --2- 172.21.15.31:0/1157578007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 0x7f5ef81031b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8004940 tx=0x7f5ee8009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.951+0000 7f5efe72f700 1 -- 172.21.15.31:0/1157578007 shutdown_connections 2024-04-24T23:22:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.951+0000 7f5efe72f700 1 --2- 172.21.15.31:0/1157578007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef8103780 0x7f5ef80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.951+0000 7f5efe72f700 1 --2- 172.21.15.31:0/1157578007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 0x7f5ef81031b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.951+0000 7f5efe72f700 1 -- 172.21.15.31:0/1157578007 >> 172.21.15.31:0/1157578007 conn(0x7f5ef80f8990 msgr2=0x7f5ef80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.951+0000 7f5efe72f700 1 -- 172.21.15.31:0/1157578007 shutdown_connections 2024-04-24T23:22:20.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.951+0000 7f5efe72f700 1 -- 172.21.15.31:0/1157578007 wait complete. 2024-04-24T23:22:20.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5efe72f700 1 Processor -- start 2024-04-24T23:22:20.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5efe72f700 1 -- start start 2024-04-24T23:22:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5efe72f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 0x7f5ef8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5efe72f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef8103780 0x7f5ef80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5efe72f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8072160 con 0x7f5ef8102de0 2024-04-24T23:22:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5efe72f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef8070ce0 con 0x7f5ef8103780 2024-04-24T23:22:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5ef7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 0x7f5ef8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:20.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5ef7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 0x7f5ef8070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46486/0 (socket says 172.21.15.31:46486) 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5ef7fff700 1 -- 172.21.15.31:0/3079295668 learned_addr learned my addr 172.21.15.31:0/3079295668 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5ef7fff700 1 -- 172.21.15.31:0/3079295668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef8103780 msgr2=0x7f5ef80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5ef7fff700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef8103780 0x7f5ef80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.952+0000 7f5ef7fff700 1 -- 172.21.15.31:0/3079295668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8005040 con 0x7f5ef8102de0 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.953+0000 7f5ef7fff700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 0x7f5ef8070260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b3b0 tx=0x7f5ee8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.953+0000 7f5ef4ff9700 1 -- 172.21.15.31:0/3079295668 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee800e040 con 0x7f5ef8102de0 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.953+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef8070f60 con 0x7f5ef8102de0 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.953+0000 7f5ef4ff9700 1 -- 172.21.15.31:0/3079295668 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ee800bbc0 con 0x7f5ef8102de0 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.953+0000 7f5ef4ff9700 1 -- 172.21.15.31:0/3079295668 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ee801c3f0 con 0x7f5ef8102de0 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.953+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef811a110 con 0x7f5ef8102de0 2024-04-24T23:22:20.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.954+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef81000f0 con 0x7f5ef8102de0 2024-04-24T23:22:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.954+0000 7f5ef4ff9700 1 -- 172.21.15.31:0/3079295668 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5ee800bd30 con 0x7f5ef8102de0 2024-04-24T23:22:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.955+0000 7f5ef4ff9700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ee406e140 0x7f5ee40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.955+0000 7f5ef4ff9700 1 -- 172.21.15.31:0/3079295668 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f5ee808ed50 con 0x7f5ef8102de0 2024-04-24T23:22:20.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.955+0000 7f5ef77fe700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ee406e140 0x7f5ee40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:20.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.955+0000 7f5ef77fe700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ee406e140 0x7f5ee40705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8071940 tx=0x7f5ee0007d40 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:20.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:20.960+0000 7f5ef4ff9700 1 -- 172.21.15.31:0/3079295668 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5ee8058120 con 0x7f5ef8102de0 2024-04-24T23:22:21.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.214+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5ef8071750 con 0x7f5ef8102de0 2024-04-24T23:22:21.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.215+0000 7f5ef4ff9700 1 -- 172.21.15.31:0/3079295668 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f5ee805b7e0 con 0x7f5ef8102de0 2024-04-24T23:22:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:e15 2024-04-24T23:22:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:22:21.216 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:22:21.217 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:22:21.218 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:14478} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/1388844653,v1:172.21.15.31:6827/1388844653] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:22:21.219 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:22:21.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ee406e140 msgr2=0x7f5ee40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ee406e140 0x7f5ee40705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8071940 tx=0x7f5ee0007d40 comp rx=0 tx=0).stop 2024-04-24T23:22:21.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 msgr2=0x7f5ef8070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 0x7f5ef8070260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b3b0 tx=0x7f5ee8009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:21.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 shutdown_connections 2024-04-24T23:22:21.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ef8102de0 0x7f5ef8070260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ee406e140 0x7f5ee40705f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 --2- 172.21.15.31:0/3079295668 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5ef8103780 0x7f5ef80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 >> 172.21.15.31:0/3079295668 conn(0x7f5ef80f8990 msgr2=0x7f5ef8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:21.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 shutdown_connections 2024-04-24T23:22:21.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.217+0000 7f5efe72f700 1 -- 172.21.15.31:0/3079295668 wait complete. 2024-04-24T23:22:21.221 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 15 2024-04-24T23:22:21.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 -- 172.21.15.31:0/3812127098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070c10 msgr2=0x7fda501090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 --2- 172.21.15.31:0/3812127098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070c10 0x7fda501090f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda44004970 tx=0x7fda44009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:21.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 -- 172.21.15.31:0/3812127098 shutdown_connections 2024-04-24T23:22:21.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 --2- 172.21.15.31:0/3812127098 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070c10 0x7fda501090f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 --2- 172.21.15.31:0/3812127098 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50070300 0x7fda500706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 -- 172.21.15.31:0/3812127098 >> 172.21.15.31:0/3812127098 conn(0x7fda500f8950 msgr2=0x7fda500fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:21.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 -- 172.21.15.31:0/3812127098 shutdown_connections 2024-04-24T23:22:21.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 -- 172.21.15.31:0/3812127098 wait complete. 2024-04-24T23:22:21.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.323+0000 7fda562a8700 1 Processor -- start 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda562a8700 1 -- start start 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda562a8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070300 0x7fda50115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda562a8700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50070c10 0x7fda50116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda562a8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda50116880 con 0x7fda50070300 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda562a8700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5010fc50 con 0x7fda50070c10 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070300 0x7fda50115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070300 0x7fda50115cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46508/0 (socket says 172.21.15.31:46508) 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4ffff700 1 -- 172.21.15.31:0/2384320424 learned_addr learned my addr 172.21.15.31:0/2384320424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4f7fe700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50070c10 0x7fda50116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4ffff700 1 -- 172.21.15.31:0/2384320424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50070c10 msgr2=0x7fda50116230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4ffff700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50070c10 0x7fda50116230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4ffff700 1 -- 172.21.15.31:0/2384320424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda44005040 con 0x7fda50070300 2024-04-24T23:22:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4f7fe700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50070c10 0x7fda50116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.324+0000 7fda4ffff700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070300 0x7fda50115cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda4000c830 tx=0x7fda40004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.325+0000 7fda4cff9700 1 -- 172.21.15.31:0/2384320424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda4000a760 con 0x7fda50070300 2024-04-24T23:22:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.325+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5010ff30 con 0x7fda50070300 2024-04-24T23:22:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.325+0000 7fda4cff9700 1 -- 172.21.15.31:0/2384320424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda4000ad90 con 0x7fda50070300 2024-04-24T23:22:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.325+0000 7fda4cff9700 1 -- 172.21.15.31:0/2384320424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda400113a0 con 0x7fda50070300 2024-04-24T23:22:21.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.325+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda50110440 con 0x7fda50070300 2024-04-24T23:22:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.326+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda50071a40 con 0x7fda50070300 2024-04-24T23:22:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.326+0000 7fda4cff9700 1 -- 172.21.15.31:0/2384320424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fda4000a8c0 con 0x7fda50070300 2024-04-24T23:22:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.327+0000 7fda4cff9700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fda3806e140 0x7fda380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.327+0000 7fda4cff9700 1 -- 172.21.15.31:0/2384320424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fda4008dae0 con 0x7fda50070300 2024-04-24T23:22:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.327+0000 7fda4f7fe700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fda3806e140 0x7fda380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:21.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.327+0000 7fda4f7fe700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fda3806e140 0x7fda380705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fda44004f50 tx=0x7fda44009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:21.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.332+0000 7fda4cff9700 1 -- 172.21.15.31:0/2384320424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fda4005aa60 con 0x7fda50070300 2024-04-24T23:22:21.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:21 smithi031 ceph-mon[111404]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:21 smithi031 ceph-mon[111404]: pgmap v151: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:21 smithi031 ceph-mon[111404]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:21 smithi031 ceph-mon[111404]: from='client.34356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:21.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:21 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/695849507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:21.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.562+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda5004bf20 con 0x7fda3806e140 2024-04-24T23:22:21.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.564+0000 7fda4cff9700 1 -- 172.21.15.31:0/2384320424 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fda5004bf20 con 0x7fda3806e140 2024-04-24T23:22:21.565 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:22:21.565 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:22:21.565 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:22:21.565 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:22:21.565 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:22:21.565 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:22:21.565 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:22:21.566 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:22:21.566 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:22:21.566 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "11/21 daemons upgraded", 2024-04-24T23:22:21.566 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-04-24T23:22:21.566 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:22:21.566 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:22:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fda3806e140 msgr2=0x7fda380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fda3806e140 0x7fda380705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fda44004f50 tx=0x7fda44009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070300 msgr2=0x7fda50115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070300 0x7fda50115cf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda4000c830 tx=0x7fda40004470 comp rx=0 tx=0).stop 2024-04-24T23:22:21.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 shutdown_connections 2024-04-24T23:22:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50070300 0x7fda50115cf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fda3806e140 0x7fda380705f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 --2- 172.21.15.31:0/2384320424 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda50070c10 0x7fda50116230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 >> 172.21.15.31:0/2384320424 conn(0x7fda500f8950 msgr2=0x7fda500f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 shutdown_connections 2024-04-24T23:22:21.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.566+0000 7fda562a8700 1 -- 172.21.15.31:0/2384320424 wait complete. 2024-04-24T23:22:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:21 smithi040 ceph-mon[83047]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:21 smithi040 ceph-mon[83047]: pgmap v151: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:21 smithi040 ceph-mon[83047]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:21 smithi040 ceph-mon[83047]: from='client.34356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:21.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:21 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/695849507' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.670+0000 7f77c2096700 1 -- 172.21.15.31:0/2424682005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc101d10 msgr2=0x7f77bc1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.670+0000 7f77c2096700 1 --2- 172.21.15.31:0/2424682005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc101d10 0x7f77bc1020e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f77ac004940 tx=0x7f77ac009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.670+0000 7f77c2096700 1 -- 172.21.15.31:0/2424682005 shutdown_connections 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.670+0000 7f77c2096700 1 --2- 172.21.15.31:0/2424682005 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc1026b0 0x7f77bc0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.670+0000 7f77c2096700 1 --2- 172.21.15.31:0/2424682005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc101d10 0x7f77bc1020e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.670+0000 7f77c2096700 1 -- 172.21.15.31:0/2424682005 >> 172.21.15.31:0/2424682005 conn(0x7f77bc0f7860 msgr2=0x7f77bc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.670+0000 7f77c2096700 1 -- 172.21.15.31:0/2424682005 shutdown_connections 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.670+0000 7f77c2096700 1 -- 172.21.15.31:0/2424682005 wait complete. 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77c2096700 1 Processor -- start 2024-04-24T23:22:21.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77c2096700 1 -- start start 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77c2096700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc101d10 0x7f77bc1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77c2096700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc1026b0 0x7f77bc113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77c2096700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc114530 con 0x7f77bc1026b0 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77c2096700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc10db50 con 0x7f77bc101d10 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc1026b0 0x7f77bc113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77b2ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc1026b0 0x7f77bc113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46526/0 (socket says 172.21.15.31:46526) 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.671+0000 7f77b2ffd700 1 -- 172.21.15.31:0/685663431 learned_addr learned my addr 172.21.15.31:0/685663431 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77bb7fe700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc101d10 0x7f77bc1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:21.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77b2ffd700 1 -- 172.21.15.31:0/685663431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc101d10 msgr2=0x7f77bc1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77b2ffd700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc101d10 0x7f77bc1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77b2ffd700 1 -- 172.21.15.31:0/685663431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac005040 con 0x7f77bc1026b0 2024-04-24T23:22:21.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77b2ffd700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc1026b0 0x7f77bc113ee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77a4002a50 tx=0x7f77a4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:21.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77b8ff9700 1 -- 172.21.15.31:0/685663431 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a4007cc0 con 0x7f77bc1026b0 2024-04-24T23:22:21.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77b8ff9700 1 -- 172.21.15.31:0/685663431 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77a400f040 con 0x7f77bc1026b0 2024-04-24T23:22:21.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77b8ff9700 1 -- 172.21.15.31:0/685663431 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a400d9c0 con 0x7f77bc1026b0 2024-04-24T23:22:21.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc10de30 con 0x7f77bc1026b0 2024-04-24T23:22:21.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.672+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc10e3f0 con 0x7f77bc1026b0 2024-04-24T23:22:21.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.673+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc0fd460 con 0x7f77bc1026b0 2024-04-24T23:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.674+0000 7f77b8ff9700 1 -- 172.21.15.31:0/685663431 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f77a4009730 con 0x7f77bc1026b0 2024-04-24T23:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.674+0000 7f77b8ff9700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f77a806e140 0x7f77a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:21.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.674+0000 7f77b8ff9700 1 -- 172.21.15.31:0/685663431 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f77a408d750 con 0x7f77bc1026b0 2024-04-24T23:22:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.675+0000 7f77bb7fe700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f77a806e140 0x7f77a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.675+0000 7f77bb7fe700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f77a806e140 0x7f77a80705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b3b0 tx=0x7f77ac009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:21.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.679+0000 7f77b8ff9700 1 -- 172.21.15.31:0/685663431 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f77a4056ba0 con 0x7f77bc1026b0 2024-04-24T23:22:21.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.970+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f77bc10e660 con 0x7f77bc1026b0 2024-04-24T23:22:21.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.971+0000 7f77b8ff9700 1 -- 172.21.15.31:0/685663431 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f77a405a260 con 0x7f77bc1026b0 2024-04-24T23:22:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:22:21.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f77a806e140 msgr2=0x7f77a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f77a806e140 0x7f77a80705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b3b0 tx=0x7f77ac009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:21.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc1026b0 msgr2=0x7f77bc113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc1026b0 0x7f77bc113ee0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f77a4002a50 tx=0x7f77a4004600 comp rx=0 tx=0).stop 2024-04-24T23:22:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 shutdown_connections 2024-04-24T23:22:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f77a806e140 0x7f77a80705f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77bc101d10 0x7f77bc1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 --2- 172.21.15.31:0/685663431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77bc1026b0 0x7f77bc113ee0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.973+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 >> 172.21.15.31:0/685663431 conn(0x7f77bc0f7860 msgr2=0x7f77bc0662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:21.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.974+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 shutdown_connections 2024-04-24T23:22:21.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:21.974+0000 7f77c2096700 1 -- 172.21.15.31:0/685663431 wait complete. 2024-04-24T23:22:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:22 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3079295668' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:22:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:22 smithi031 ceph-mon[111404]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:22.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:22 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/685663431' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:22:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:22 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3079295668' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:22:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:22 smithi040 ceph-mon[83047]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:22.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:22 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/685663431' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:22:23.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:23 smithi031 ceph-mon[111404]: pgmap v152: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:23.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:23 smithi040 ceph-mon[83047]: pgmap v152: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:25.111 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:24 smithi040 ceph-mon[83047]: pgmap v153: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:25.111 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:24 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:25.111 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:24 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T23:22:25.112 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:24 smithi040 ceph-mon[83047]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T23:22:25.112 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:24 smithi040 ceph-mon[83047]: Upgrade: osd.5 is safe to restart 2024-04-24T23:22:25.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:24 smithi031 ceph-mon[111404]: pgmap v153: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:25.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:24 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:25.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:24 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T23:22:25.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:24 smithi031 ceph-mon[111404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-24T23:22:25.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:24 smithi031 ceph-mon[111404]: Upgrade: osd.5 is safe to restart 2024-04-24T23:22:27.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:27 smithi040 ceph-mon[83047]: pgmap v154: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:27.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:27 smithi040 ceph-mon[83047]: Upgrade: Updating osd.5 2024-04-24T23:22:27.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:27 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:27.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:27 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:22:27.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:27 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:27.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:27 smithi040 ceph-mon[83047]: Deploying daemon osd.5 on smithi040 2024-04-24T23:22:27.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:27 smithi031 ceph-mon[111404]: pgmap v154: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:27 smithi031 ceph-mon[111404]: Upgrade: Updating osd.5 2024-04-24T23:22:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:27 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:27 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-24T23:22:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:27 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:27.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:27 smithi031 ceph-mon[111404]: Deploying daemon osd.5 on smithi040 2024-04-24T23:22:29.426 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:29 smithi040 ceph-mon[83047]: pgmap v155: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:29.426 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:29.426 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:22:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:29 smithi031 ceph-mon[111404]: pgmap v155: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:29.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:22:29.876 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:29 smithi040 systemd[1]: Stopping Ceph osd.5 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:22:30.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:29 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[52137]: 2024-04-24T23:22:29.887+0000 7fd4b7c99700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:22:30.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:29 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[52137]: 2024-04-24T23:22:29.887+0000 7fd4b7c99700 -1 osd.5 74 *** Got signal Terminated *** 2024-04-24T23:22:30.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:29 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[52137]: 2024-04-24T23:22:29.887+0000 7fd4b7c99700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:22:30.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:30 smithi031 ceph-mon[111404]: osd.5 marked itself down and dead 2024-04-24T23:22:30.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:30 smithi040 ceph-mon[83047]: osd.5 marked itself down and dead 2024-04-24T23:22:31.022 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:30 smithi040 podman[102114]: 2024-04-24 23:22:30.761958433 +0000 UTC m=+0.999860977 container died ea87e2bf6ab9a0c1398b2197e3e3e33bc461624a35cee53436c26522d8881d5d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-24T23:22:31.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:31 smithi040 ceph-mon[83047]: pgmap v156: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:31.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:31 smithi040 ceph-mon[83047]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:22:31.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:31 smithi040 ceph-mon[83047]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-24T23:22:31.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:31 smithi040 ceph-mon[83047]: osdmap e75: 6 total, 5 up, 6 in 2024-04-24T23:22:31.413 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:31 smithi040 podman[102114]: 2024-04-24 23:22:31.11280401 +0000 UTC m=+1.350706594 container cleanup ea87e2bf6ab9a0c1398b2197e3e3e33bc461624a35cee53436c26522d8881d5d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-c61472c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T23:22:31.413 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:31 smithi040 bash[102114]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5 2024-04-24T23:22:31.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:31 smithi031 ceph-mon[111404]: pgmap v156: 65 pgs: 65 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-24T23:22:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:31 smithi031 ceph-mon[111404]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-24T23:22:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:31 smithi031 ceph-mon[111404]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-24T23:22:31.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:31 smithi031 ceph-mon[111404]: osdmap e75: 6 total, 5 up, 6 in 2024-04-24T23:22:31.750 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:31 smithi040 podman[102141]: 2024-04-24 23:22:31.414123513 +0000 UTC m=+0.655297586 container remove ea87e2bf6ab9a0c1398b2197e3e3e33bc461624a35cee53436c26522d8881d5d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=quincy-c61472c) 2024-04-24T23:22:32.076 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:31 smithi040 podman[102233]: 2024-04-24 23:22:31.653048154 +0000 UTC m=+0.019137368 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:22:32.077 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:31 smithi040 podman[102233]: 2024-04-24 23:22:31.765296113 +0000 UTC m=+0.131385342 container create 212f7d982da0e251c86c2725f0dd415d66b92a4e3c7b9c3879e9a93f428e535c (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:22:32.077 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:31 smithi040 podman[102233]: 2024-04-24 23:22:31.992665754 +0000 UTC m=+0.358754947 container init 212f7d982da0e251c86c2725f0dd415d66b92a4e3c7b9c3879e9a93f428e535c (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-24T23:22:32.077 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:31 smithi040 podman[102233]: 2024-04-24 23:22:31.996079431 +0000 UTC m=+0.362168618 container start 212f7d982da0e251c86c2725f0dd415d66b92a4e3c7b9c3879e9a93f428e535c (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T23:22:32.077 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:32 smithi040 podman[102233]: 2024-04-24 23:22:32.075966765 +0000 UTC m=+0.442055960 container attach 212f7d982da0e251c86c2725f0dd415d66b92a4e3c7b9c3879e9a93f428e535c (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-24T23:22:32.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:32 smithi040 ceph-mon[83047]: osdmap e76: 6 total, 5 up, 6 in 2024-04-24T23:22:32.412 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:32 smithi040 podman[102233]: 2024-04-24 23:22:32.301923128 +0000 UTC m=+0.668012326 container died 212f7d982da0e251c86c2725f0dd415d66b92a4e3c7b9c3879e9a93f428e535c (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T23:22:32.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:32 smithi031 ceph-mon[111404]: osdmap e76: 6 total, 5 up, 6 in 2024-04-24T23:22:33.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:32 smithi040 podman[102259]: 2024-04-24 23:22:32.854601617 +0000 UTC m=+0.543846104 container remove 212f7d982da0e251c86c2725f0dd415d66b92a4e3c7b9c3879e9a93f428e535c (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-24T23:22:33.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:32 smithi040 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-24T23:22:33.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:32 smithi040 systemd[1]: Stopped Ceph osd.5 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:22:33.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:32 smithi040 systemd[1]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.5.service: Consumed 1min 7.857s CPU time. 2024-04-24T23:22:33.162 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:33 smithi040 systemd[1]: Starting Ceph osd.5 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:22:33.445 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:33 smithi040 ceph-mon[83047]: pgmap v159: 65 pgs: 3 active+undersized+degraded, 9 stale+active+clean, 5 active+undersized, 48 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4/225 objects degraded (1.778%) 2024-04-24T23:22:33.445 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:33 smithi040 ceph-mon[83047]: Health check failed: Degraded data redundancy: 4/225 objects degraded (1.778%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:33.445 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:33 smithi040 podman[102362]: 2024-04-24 23:22:33.217863637 +0000 UTC m=+0.018783517 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:22:33.445 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:33 smithi040 podman[102362]: 2024-04-24 23:22:33.330752115 +0000 UTC m=+0.131671967 container create 764e8b71b0c2e8bfe594bf1aa637b3da46cc8a74852079f108be95561b1f0e21 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:22:33.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:33 smithi031 ceph-mon[111404]: pgmap v159: 65 pgs: 3 active+undersized+degraded, 9 stale+active+clean, 5 active+undersized, 48 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4/225 objects degraded (1.778%) 2024-04-24T23:22:33.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:33 smithi031 ceph-mon[111404]: Health check failed: Degraded data redundancy: 4/225 objects degraded (1.778%), 3 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:33.912 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:33 smithi040 podman[102362]: 2024-04-24 23:22:33.558496245 +0000 UTC m=+0.359416101 container init 764e8b71b0c2e8bfe594bf1aa637b3da46cc8a74852079f108be95561b1f0e21 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9) 2024-04-24T23:22:33.912 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:33 smithi040 podman[102362]: 2024-04-24 23:22:33.562118116 +0000 UTC m=+0.363037978 container start 764e8b71b0c2e8bfe594bf1aa637b3da46cc8a74852079f108be95561b1f0e21 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-24T23:22:33.912 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:33 smithi040 podman[102362]: 2024-04-24 23:22:33.658477433 +0000 UTC m=+0.459397298 container attach 764e8b71b0c2e8bfe594bf1aa637b3da46cc8a74852079f108be95561b1f0e21 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-24T23:22:34.337 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate[102378]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T23:22:34.337 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate[102378]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T23:22:34.337 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 bash[102362]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T23:22:34.337 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 bash[102362]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-24T23:22:34.337 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 bash[102362]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 bash[102362]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 bash[102362]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 bash[102362]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 bash[102362]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate[102378]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate[102378]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate[102378]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate[102378]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate[102378]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-24T23:22:34.338 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 podman[102362]: 2024-04-24 23:22:34.050076169 +0000 UTC m=+0.850996032 container died 764e8b71b0c2e8bfe594bf1aa637b3da46cc8a74852079f108be95561b1f0e21 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-24T23:22:34.912 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 podman[102497]: 2024-04-24 23:22:34.612157103 +0000 UTC m=+0.555488589 container remove 764e8b71b0c2e8bfe594bf1aa637b3da46cc8a74852079f108be95561b1f0e21 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-24T23:22:34.912 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 podman[102559]: 2024-04-24 23:22:34.733793837 +0000 UTC m=+0.017967535 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:22:34.912 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:34 smithi040 podman[102559]: 2024-04-24 23:22:34.838202164 +0000 UTC m=+0.122375870 container create 32b6ff45fea8ba1cd0783c226a0ce54feb2639ccfd0691c6b6a14677e867b6ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:22:35.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:35 smithi040 ceph-mon[83047]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 2 stale+active+clean, 13 active+undersized, 38 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:35.412 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:35 smithi040 podman[102559]: 2024-04-24 23:22:35.073945345 +0000 UTC m=+0.358119010 container init 32b6ff45fea8ba1cd0783c226a0ce54feb2639ccfd0691c6b6a14677e867b6ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-24T23:22:35.412 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:35 smithi040 podman[102559]: 2024-04-24 23:22:35.077233435 +0000 UTC m=+0.361407100 container start 32b6ff45fea8ba1cd0783c226a0ce54feb2639ccfd0691c6b6a14677e867b6ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-24T23:22:35.412 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:35 smithi040 ceph-osd[102578]: -- 172.21.15.40:0/1275345265 <== mon.1 v2:172.21.15.40:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2b6d49680 con 0x55b2b6d13000 2024-04-24T23:22:35.412 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:35 smithi040 bash[102559]: 32b6ff45fea8ba1cd0783c226a0ce54feb2639ccfd0691c6b6a14677e867b6ba 2024-04-24T23:22:35.413 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:35 smithi040 systemd[1]: Started Ceph osd.5 for ca7b26e4-028e-11ef-bc93-c7b262605968. 2024-04-24T23:22:35.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:35 smithi031 ceph-mon[111404]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 2 stale+active+clean, 13 active+undersized, 38 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:35.912 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:35 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:22:35.619+0000 7f8ae54767c0 -1 Falling back to public interface 2024-04-24T23:22:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:36 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:36 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:36.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:36 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:36 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:36 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:36.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:36 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:37.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:37 smithi040 ceph-mon[83047]: pgmap v161: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:37.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:37 smithi031 ceph-mon[111404]: pgmap v161: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:39.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:39 smithi031 ceph-mon[111404]: pgmap v162: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:39.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:39.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:39 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:39.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:39 smithi040 ceph-mon[83047]: pgmap v162: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:39.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:39.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:39 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:40.662 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:40 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:22:40.288+0000 7f8ae54767c0 -1 osd.5 74 log_to_monitors true 2024-04-24T23:22:40.662 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:22:40 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:22:40.396+0000 7f8ad68ce700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T23:22:40.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:40 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:40.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:40 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:40.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:40 smithi040 ceph-mon[83047]: Health check update: Degraded data redundancy: 35/225 objects degraded (15.556%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:40.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:40 smithi040 ceph-mon[83047]: from='osd.5 [v2:172.21.15.40:6816/3395166888,v1:172.21.15.40:6817/3395166888]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:22:40.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:40 smithi040 ceph-mon[83047]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:22:40.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:40 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:40.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:40 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:40.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:40 smithi031 ceph-mon[111404]: Health check update: Degraded data redundancy: 35/225 objects degraded (15.556%), 12 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:40.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:40 smithi031 ceph-mon[111404]: from='osd.5 [v2:172.21.15.40:6816/3395166888,v1:172.21.15.40:6817/3395166888]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:22:40.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:40 smithi031 ceph-mon[111404]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-24T23:22:41.554 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:41 smithi040 ceph-mon[83047]: pgmap v163: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:41.554 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:41 smithi040 ceph-mon[83047]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:22:41.555 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:41 smithi040 ceph-mon[83047]: osdmap e77: 6 total, 5 up, 6 in 2024-04-24T23:22:41.555 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:41 smithi040 ceph-mon[83047]: from='osd.5 [v2:172.21.15.40:6816/3395166888,v1:172.21.15.40:6817/3395166888]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:22:41.555 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:41 smithi040 ceph-mon[83047]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:22:41.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:41 smithi031 ceph-mon[111404]: pgmap v163: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 205 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:41.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:41 smithi031 ceph-mon[111404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-24T23:22:41.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:41 smithi031 ceph-mon[111404]: osdmap e77: 6 total, 5 up, 6 in 2024-04-24T23:22:41.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:41 smithi031 ceph-mon[111404]: from='osd.5 [v2:172.21.15.40:6816/3395166888,v1:172.21.15.40:6817/3395166888]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:22:41.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:41 smithi031 ceph-mon[111404]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi040", "root=default"]}]: dispatch 2024-04-24T23:22:42.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:42 smithi040 ceph-mon[83047]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:22:42.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:42 smithi040 ceph-mon[83047]: osd.5 [v2:172.21.15.40:6816/3395166888,v1:172.21.15.40:6817/3395166888] boot 2024-04-24T23:22:42.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:42 smithi040 ceph-mon[83047]: osdmap e78: 6 total, 6 up, 6 in 2024-04-24T23:22:42.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:42 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:22:42.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:42 smithi031 ceph-mon[111404]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-24T23:22:42.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:42 smithi031 ceph-mon[111404]: osd.5 [v2:172.21.15.40:6816/3395166888,v1:172.21.15.40:6817/3395166888] boot 2024-04-24T23:22:42.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:42 smithi031 ceph-mon[111404]: osdmap e78: 6 total, 6 up, 6 in 2024-04-24T23:22:42.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:42 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-24T23:22:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: pgmap v166: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: osdmap e79: 6 total, 6 up, 6 in 2024-04-24T23:22:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:43.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-24T23:22:43.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-24T23:22:43.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-24T23:22:43.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-24T23:22:43.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-24T23:22:43.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-24T23:22:43.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-24T23:22:43.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-24T23:22:43.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-24T23:22:43.664 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: pgmap v166: 65 pgs: 12 active+undersized+degraded, 16 active+undersized, 37 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 35/225 objects degraded (15.556%) 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: osdmap e79: 6 total, 6 up, 6 in 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:43.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-24T23:22:43.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-24T23:22:43.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-24T23:22:43.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-24T23:22:43.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-24T23:22:43.751 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-24T23:22:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:44 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all osd 2024-04-24T23:22:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:44 smithi040 ceph-mon[83047]: Upgrade: Setting require_osd_release to 19 squid 2024-04-24T23:22:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:22:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:44 smithi040 ceph-mon[83047]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-24T23:22:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-24T23:22:44.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:44 smithi040 ceph-mon[83047]: osdmap e80: 6 total, 6 up, 6 in 2024-04-24T23:22:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:44 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all osd 2024-04-24T23:22:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:44 smithi031 ceph-mon[111404]: Upgrade: Setting require_osd_release to 19 squid 2024-04-24T23:22:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:22:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:44 smithi031 ceph-mon[111404]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-24T23:22:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-24T23:22:44.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:44 smithi031 ceph-mon[111404]: osdmap e80: 6 total, 6 up, 6 in 2024-04-24T23:22:45.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:45 smithi040 ceph-mon[83047]: pgmap v168: 65 pgs: 5 active+undersized+degraded, 13 active+undersized, 47 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 13/225 objects degraded (5.778%) 2024-04-24T23:22:45.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:45 smithi040 ceph-mon[83047]: Health check update: Degraded data redundancy: 13/225 objects degraded (5.778%), 5 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:45.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:45 smithi031 ceph-mon[111404]: pgmap v168: 65 pgs: 5 active+undersized+degraded, 13 active+undersized, 47 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 13/225 objects degraded (5.778%) 2024-04-24T23:22:45.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:45 smithi031 ceph-mon[111404]: Health check update: Degraded data redundancy: 13/225 objects degraded (5.778%), 5 pgs degraded (PG_DEGRADED) 2024-04-24T23:22:47.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:47 smithi031 ceph-mon[111404]: pgmap v170: 65 pgs: 1 active+undersized+degraded, 3 active+undersized, 61 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 2/225 objects degraded (0.889%) 2024-04-24T23:22:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:47 smithi031 ceph-mon[111404]: Upgrade: Updating mds.cephfs.smithi031.xxejxr 2024-04-24T23:22:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:47 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:47 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.xxejxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:22:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:47 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:47.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:47 smithi031 ceph-mon[111404]: Deploying daemon mds.cephfs.smithi031.xxejxr on smithi031 2024-04-24T23:22:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:47 smithi040 ceph-mon[83047]: pgmap v170: 65 pgs: 1 active+undersized+degraded, 3 active+undersized, 61 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 2/225 objects degraded (0.889%) 2024-04-24T23:22:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:47 smithi040 ceph-mon[83047]: Upgrade: Updating mds.cephfs.smithi031.xxejxr 2024-04-24T23:22:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:47 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:47 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.xxejxr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:22:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:47 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:22:47.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:47 smithi040 ceph-mon[83047]: Deploying daemon mds.cephfs.smithi031.xxejxr on smithi031 2024-04-24T23:22:48.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:48 smithi031 ceph-mon[111404]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/225 objects degraded (0.889%), 1 pg degraded) 2024-04-24T23:22:48.446 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:48 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:22:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:48 smithi040 ceph-mon[83047]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/225 objects degraded (0.889%), 1 pg degraded) 2024-04-24T23:22:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:48 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:22:49.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:49 smithi031 ceph-mon[111404]: pgmap v171: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:49.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:49 smithi040 ceph-mon[83047]: pgmap v171: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:50.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:50 smithi031 ceph-mon[111404]: osdmap e81: 6 total, 6 up, 6 in 2024-04-24T23:22:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:50 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 2 up:standby 2024-04-24T23:22:50.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:50 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:22:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:50 smithi040 ceph-mon[83047]: osdmap e81: 6 total, 6 up, 6 in 2024-04-24T23:22:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:50 smithi040 ceph-mon[83047]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 2 up:standby 2024-04-24T23:22:50.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:50 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:22:51.463 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:51 smithi031 ceph-mon[111404]: pgmap v173: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:51 smithi040 ceph-mon[83047]: pgmap v173: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.085+0000 7f7de54e7700 1 -- 172.21.15.31:0/3113342348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0101cf0 msgr2=0x7f7de01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.085+0000 7f7de54e7700 1 --2- 172.21.15.31:0/3113342348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0101cf0 0x7f7de01020c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0004940 tx=0x7f7dd0009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.085+0000 7f7de54e7700 1 -- 172.21.15.31:0/3113342348 shutdown_connections 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.085+0000 7f7de54e7700 1 --2- 172.21.15.31:0/3113342348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0102690 0x7f7de00fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.085+0000 7f7de54e7700 1 --2- 172.21.15.31:0/3113342348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0101cf0 0x7f7de01020c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.085+0000 7f7de54e7700 1 -- 172.21.15.31:0/3113342348 >> 172.21.15.31:0/3113342348 conn(0x7f7de00f78a0 msgr2=0x7f7de00f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.085+0000 7f7de54e7700 1 -- 172.21.15.31:0/3113342348 shutdown_connections 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.085+0000 7f7de54e7700 1 -- 172.21.15.31:0/3113342348 wait complete. 2024-04-24T23:22:52.089 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.086+0000 7f7de54e7700 1 Processor -- start 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.086+0000 7f7de54e7700 1 -- start start 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.086+0000 7f7de54e7700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0101cf0 0x7f7de0113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.086+0000 7f7de54e7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0102690 0x7f7de01140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.086+0000 7f7de54e7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0114720 con 0x7f7de0102690 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.086+0000 7f7de54e7700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de010db10 con 0x7f7de0101cf0 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7dde7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0102690 0x7f7de01140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7dde7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0102690 0x7f7de01140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41608/0 (socket says 172.21.15.31:41608) 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7dde7fc700 1 -- 172.21.15.31:0/3688923029 learned_addr learned my addr 172.21.15.31:0/3688923029 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7ddeffd700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0101cf0 0x7f7de0113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7dde7fc700 1 -- 172.21.15.31:0/3688923029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0101cf0 msgr2=0x7f7de0113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7dde7fc700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0101cf0 0x7f7de0113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7dde7fc700 1 -- 172.21.15.31:0/3688923029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd0005040 con 0x7f7de0102690 2024-04-24T23:22:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7dde7fc700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0102690 0x7f7de01140d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4002a50 tx=0x7f7dd4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7dc77fe700 1 -- 172.21.15.31:0/3688923029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dd4007cc0 con 0x7f7de0102690 2024-04-24T23:22:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.087+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de010dd90 con 0x7f7de0102690 2024-04-24T23:22:52.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.088+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de010e350 con 0x7f7de0102690 2024-04-24T23:22:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.088+0000 7f7dc77fe700 1 -- 172.21.15.31:0/3688923029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dd400f040 con 0x7f7de0102690 2024-04-24T23:22:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.088+0000 7f7dc77fe700 1 -- 172.21.15.31:0/3688923029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dd400d9c0 con 0x7f7de0102690 2024-04-24T23:22:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.089+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dcc0052b0 con 0x7f7de0102690 2024-04-24T23:22:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.089+0000 7f7dc77fe700 1 -- 172.21.15.31:0/3688923029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7dd400db20 con 0x7f7de0102690 2024-04-24T23:22:52.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.090+0000 7f7dc77fe700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7dc806e140 0x7f7dc80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.090+0000 7f7ddeffd700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7dc806e140 0x7f7dc80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.090+0000 7f7dc77fe700 1 -- 172.21.15.31:0/3688923029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f7dd408d690 con 0x7f7de0102690 2024-04-24T23:22:52.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.090+0000 7f7ddeffd700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7dc806e140 0x7f7dc80705f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0004f90 tx=0x7f7dd0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.095+0000 7f7dc77fe700 1 -- 172.21.15.31:0/3688923029 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7dd405a610 con 0x7f7de0102690 2024-04-24T23:22:52.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.336+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dcc000bc0 con 0x7f7dc806e140 2024-04-24T23:22:52.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.338+0000 7f7dc77fe700 1 -- 172.21.15.31:0/3688923029 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7dcc000bc0 con 0x7f7dc806e140 2024-04-24T23:22:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7dc806e140 msgr2=0x7f7dc80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7dc806e140 0x7f7dc80705f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0004f90 tx=0x7f7dd0009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:52.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0102690 msgr2=0x7f7de01140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0102690 0x7f7de01140d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4002a50 tx=0x7f7dd4004600 comp rx=0 tx=0).stop 2024-04-24T23:22:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 shutdown_connections 2024-04-24T23:22:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7dc806e140 0x7f7dc80705f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7de0101cf0 0x7f7de0113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 --2- 172.21.15.31:0/3688923029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7de0102690 0x7f7de01140d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 >> 172.21.15.31:0/3688923029 conn(0x7f7de00f78a0 msgr2=0x7f7de00f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 shutdown_connections 2024-04-24T23:22:52.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.340+0000 7f7de54e7700 1 -- 172.21.15.31:0/3688923029 wait complete. 2024-04-24T23:22:52.354 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.453+0000 7f2eac18c700 1 -- 172.21.15.31:0/665497566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4103780 msgr2=0x7f2ea40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.453+0000 7f2eac18c700 1 --2- 172.21.15.31:0/665497566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4103780 0x7f2ea40fcdf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e98004940 tx=0x7f2e98009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.453+0000 7f2eac18c700 1 -- 172.21.15.31:0/665497566 shutdown_connections 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.453+0000 7f2eac18c700 1 --2- 172.21.15.31:0/665497566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4103780 0x7f2ea40fcdf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.453+0000 7f2eac18c700 1 --2- 172.21.15.31:0/665497566 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ea4102de0 0x7f2ea41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.453+0000 7f2eac18c700 1 -- 172.21.15.31:0/665497566 >> 172.21.15.31:0/665497566 conn(0x7f2ea40f8990 msgr2=0x7f2ea40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.453+0000 7f2eac18c700 1 -- 172.21.15.31:0/665497566 shutdown_connections 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.453+0000 7f2eac18c700 1 -- 172.21.15.31:0/665497566 wait complete. 2024-04-24T23:22:52.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2eac18c700 1 Processor -- start 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2eac18c700 1 -- start start 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2eac18c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4102de0 0x7f2ea41138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2eac18c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ea4103780 0x7f2ea4113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2eac18c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea4114460 con 0x7f2ea4102de0 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2eac18c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea410da60 con 0x7f2ea4103780 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2ea9f28700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4102de0 0x7f2ea41138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2ea9f28700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4102de0 0x7f2ea41138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41626/0 (socket says 172.21.15.31:41626) 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2ea9f28700 1 -- 172.21.15.31:0/2795759476 learned_addr learned my addr 172.21.15.31:0/2795759476 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2ea9727700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ea4103780 0x7f2ea4113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2ea9f28700 1 -- 172.21.15.31:0/2795759476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ea4103780 msgr2=0x7f2ea4113e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2ea9f28700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ea4103780 0x7f2ea4113e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2ea9f28700 1 -- 172.21.15.31:0/2795759476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98005040 con 0x7f2ea4102de0 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.454+0000 7f2ea9f28700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4102de0 0x7f2ea41138d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000eb40 tx=0x7f2ea000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.455+0000 7f2e967fc700 1 -- 172.21.15.31:0/2795759476 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ea0007cc0 con 0x7f2ea4102de0 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.455+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea410dce0 con 0x7f2ea4102de0 2024-04-24T23:22:52.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.455+0000 7f2e967fc700 1 -- 172.21.15.31:0/2795759476 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ea000f040 con 0x7f2ea4102de0 2024-04-24T23:22:52.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.455+0000 7f2e967fc700 1 -- 172.21.15.31:0/2795759476 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ea000ca10 con 0x7f2ea4102de0 2024-04-24T23:22:52.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.455+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea410e1f0 con 0x7f2ea4102de0 2024-04-24T23:22:52.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.456+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea40fe530 con 0x7f2ea4102de0 2024-04-24T23:22:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.456+0000 7f2e967fc700 1 -- 172.21.15.31:0/2795759476 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2ea000cc80 con 0x7f2ea4102de0 2024-04-24T23:22:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.457+0000 7f2e967fc700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2e9006e140 0x7f2e900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.457+0000 7f2e967fc700 1 -- 172.21.15.31:0/2795759476 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2ea008d6a0 con 0x7f2ea4102de0 2024-04-24T23:22:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.457+0000 7f2ea9727700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2e9006e140 0x7f2e900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.457+0000 7f2ea9727700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2e9006e140 0x7f2e900705f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2e98004890 tx=0x7f2e98009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.461+0000 7f2e967fc700 1 -- 172.21.15.31:0/2795759476 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2ea005a650 con 0x7f2ea4102de0 2024-04-24T23:22:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.696+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ea4109f10 con 0x7f2e9006e140 2024-04-24T23:22:52.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.697+0000 7f2e967fc700 1 -- 172.21.15.31:0/2795759476 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2ea4109f10 con 0x7f2e9006e140 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2e9006e140 msgr2=0x7f2e900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2e9006e140 0x7f2e900705f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2e98004890 tx=0x7f2e98009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4102de0 msgr2=0x7f2ea41138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4102de0 0x7f2ea41138d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000eb40 tx=0x7f2ea000b040 comp rx=0 tx=0).stop 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 shutdown_connections 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ea4102de0 0x7f2ea41138d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2e9006e140 0x7f2e900705f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 --2- 172.21.15.31:0/2795759476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ea4103780 0x7f2ea4113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.699+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 >> 172.21.15.31:0/2795759476 conn(0x7f2ea40f8990 msgr2=0x7f2ea40f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.700+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 shutdown_connections 2024-04-24T23:22:52.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.700+0000 7f2eac18c700 1 -- 172.21.15.31:0/2795759476 wait complete. 2024-04-24T23:22:52.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.806+0000 7fcc084b1700 1 -- 172.21.15.31:0/4290230532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 msgr2=0x7fcc000fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.806+0000 7fcc084b1700 1 --2- 172.21.15.31:0/4290230532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 0x7fcc000fcdf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc004940 tx=0x7fcbfc009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:52.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.806+0000 7fcc084b1700 1 -- 172.21.15.31:0/4290230532 shutdown_connections 2024-04-24T23:22:52.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.806+0000 7fcc084b1700 1 --2- 172.21.15.31:0/4290230532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 0x7fcc000fcdf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.806+0000 7fcc084b1700 1 --2- 172.21.15.31:0/4290230532 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc00102de0 0x7fcc001031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.806+0000 7fcc084b1700 1 -- 172.21.15.31:0/4290230532 >> 172.21.15.31:0/4290230532 conn(0x7fcc000f8990 msgr2=0x7fcc000fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.806+0000 7fcc084b1700 1 -- 172.21.15.31:0/4290230532 shutdown_connections 2024-04-24T23:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.806+0000 7fcc084b1700 1 -- 172.21.15.31:0/4290230532 wait complete. 2024-04-24T23:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.807+0000 7fcc084b1700 1 Processor -- start 2024-04-24T23:22:52.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.807+0000 7fcc084b1700 1 -- start start 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.807+0000 7fcc084b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc00102de0 0x7fcc00070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.807+0000 7fcc084b1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 0x7fcc000707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.807+0000 7fcc084b1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc000721f0 con 0x7fcc00103780 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.807+0000 7fcc084b1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc00070ce0 con 0x7fcc00102de0 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.807+0000 7fcc05a4c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 0x7fcc000707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc05a4c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 0x7fcc000707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41668/0 (socket says 172.21.15.31:41668) 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc05a4c700 1 -- 172.21.15.31:0/859868606 learned_addr learned my addr 172.21.15.31:0/859868606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc0624d700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc00102de0 0x7fcc00070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc05a4c700 1 -- 172.21.15.31:0/859868606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc00102de0 msgr2=0x7fcc00070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc05a4c700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc00102de0 0x7fcc00070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc05a4c700 1 -- 172.21.15.31:0/859868606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc005040 con 0x7fcc00103780 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc05a4c700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 0x7fcc000707a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009100 tx=0x7fcbfc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcbf6ffd700 1 -- 172.21.15.31:0/859868606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc00e040 con 0x7fcc00103780 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcbf6ffd700 1 -- 172.21.15.31:0/859868606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbfc013420 con 0x7fcc00103780 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc00070f60 con 0x7fcc00103780 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcbf6ffd700 1 -- 172.21.15.31:0/859868606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc01c610 con 0x7fcc00103780 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.808+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0011a170 con 0x7fcc00103780 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.809+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc001000f0 con 0x7fcc00103780 2024-04-24T23:22:52.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.810+0000 7fcbf6ffd700 1 -- 172.21.15.31:0/859868606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcbfc013590 con 0x7fcc00103780 2024-04-24T23:22:52.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.810+0000 7fcbf6ffd700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcbec06e140 0x7fcbec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:52.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.810+0000 7fcbf6ffd700 1 -- 172.21.15.31:0/859868606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcbfc08ed50 con 0x7fcc00103780 2024-04-24T23:22:52.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.810+0000 7fcc0624d700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcbec06e140 0x7fcbec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:52.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.811+0000 7fcc0624d700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcbec06e140 0x7fcbec0705f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0006fd0 tx=0x7fcbf0008040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:52.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:52.815+0000 7fcbf6ffd700 1 -- 172.21.15.31:0/859868606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcbfc0581a0 con 0x7fcc00103780 2024-04-24T23:22:53.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.033+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc00071790 con 0x7fcbec06e140 2024-04-24T23:22:53.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.043+0000 7fcbf6ffd700 1 -- 172.21.15.31:0/859868606 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcc00071790 con 0x7fcbec06e140 2024-04-24T23:22:53.044 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:22:53.044 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 116s ago 17m 25.6M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:22:53.044 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (3m) 116s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (3m) 14s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 116s ago 17m 79.9M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (12m) 116s ago 12m 138M - 17.2.7-902-gc61472c8 0de407c1b3af 344b48981d63 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 starting - - - - 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (12m) 14s ago 12m 15.2M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (12m) 14s ago 11m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (8m) 116s ago 19m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (6m) 14s ago 16m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (4m) 116s ago 19m 45.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (4m) 14s ago 16m 36.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (6m) 116s ago 17m 16.7M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (6m) 14s ago 16m 14.8M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:22:53.045 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 116s ago 15m 87.6M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:22:53.046 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (2m) 116s ago 15m 101M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:22:53.046 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (2m) 116s ago 14m 11.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 271a0764ef5c 2024-04-24T23:22:53.046 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (86s) 14s ago 14m 116M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 97c6f05632f6 2024-04-24T23:22:53.046 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (51s) 14s ago 14m 97.7M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 85d9bab7b352 2024-04-24T23:22:53.046 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (18s) 14s ago 13m 12.6M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 32b6ff45fea8 2024-04-24T23:22:53.046 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 116s ago 17m 59.9M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcbec06e140 msgr2=0x7fcbec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcbec06e140 0x7fcbec0705f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0006fd0 tx=0x7fcbf0008040 comp rx=0 tx=0).stop 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 msgr2=0x7fcc000707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 0x7fcc000707a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc009100 tx=0x7fcbfc009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 shutdown_connections 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcbec06e140 0x7fcbec0705f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcc00102de0 0x7fcc00070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 --2- 172.21.15.31:0/859868606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc00103780 0x7fcc000707a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 >> 172.21.15.31:0/859868606 conn(0x7fcc000f8990 msgr2=0x7fcc00107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 shutdown_connections 2024-04-24T23:22:53.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.045+0000 7fcc084b1700 1 -- 172.21.15.31:0/859868606 wait complete. 2024-04-24T23:22:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.154+0000 7f1563f81700 1 -- 172.21.15.31:0/119797050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 msgr2=0x7f155c0fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.154+0000 7f1563f81700 1 --2- 172.21.15.31:0/119797050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 0x7f155c0fce00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1550004940 tx=0x7f1550009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.154+0000 7f1563f81700 1 -- 172.21.15.31:0/119797050 shutdown_connections 2024-04-24T23:22:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.154+0000 7f1563f81700 1 --2- 172.21.15.31:0/119797050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 0x7f155c0fce00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.154+0000 7f1563f81700 1 --2- 172.21.15.31:0/119797050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f155c102df0 0x7f155c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.154+0000 7f1563f81700 1 -- 172.21.15.31:0/119797050 >> 172.21.15.31:0/119797050 conn(0x7f155c0f8980 msgr2=0x7f155c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.154+0000 7f1563f81700 1 -- 172.21.15.31:0/119797050 shutdown_connections 2024-04-24T23:22:53.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.154+0000 7f1563f81700 1 -- 172.21.15.31:0/119797050 wait complete. 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f1563f81700 1 Processor -- start 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f1563f81700 1 -- start start 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f1563f81700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f155c102df0 0x7f155c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f1563f81700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 0x7f155c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f1563f81700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c070d30 con 0x7f155c103790 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f1563f81700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c070e70 con 0x7f155c102df0 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f156151c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 0x7f155c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f156151c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 0x7f155c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41686/0 (socket says 172.21.15.31:41686) 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f156151c700 1 -- 172.21.15.31:0/1686285544 learned_addr learned my addr 172.21.15.31:0/1686285544 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:53.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.155+0000 7f1561d1d700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f155c102df0 0x7f155c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f156151c700 1 -- 172.21.15.31:0/1686285544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f155c102df0 msgr2=0x7f155c070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f156151c700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f155c102df0 0x7f155c070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f156151c700 1 -- 172.21.15.31:0/1686285544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1550005040 con 0x7f155c103790 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f156151c700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 0x7f155c0707a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1550009100 tx=0x7f1550009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f154e7fc700 1 -- 172.21.15.31:0/1686285544 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f155000e040 con 0x7f155c103790 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f154e7fc700 1 -- 172.21.15.31:0/1686285544 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1550009af0 con 0x7f155c103790 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155c0710c0 con 0x7f155c103790 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f154e7fc700 1 -- 172.21.15.31:0/1686285544 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f155001c3f0 con 0x7f155c103790 2024-04-24T23:22:53.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.156+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c11a2c0 con 0x7f155c103790 2024-04-24T23:22:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.157+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155c0fe540 con 0x7f155c103790 2024-04-24T23:22:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.158+0000 7f154e7fc700 1 -- 172.21.15.31:0/1686285544 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f155000baf0 con 0x7f155c103790 2024-04-24T23:22:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.158+0000 7f154e7fc700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f154806e140 0x7f15480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.158+0000 7f154e7fc700 1 -- 172.21.15.31:0/1686285544 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f155008ed10 con 0x7f155c103790 2024-04-24T23:22:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.158+0000 7f1561d1d700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f154806e140 0x7f15480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.159+0000 7f1561d1d700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f154806e140 0x7f15480705f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f15580041d0 tx=0x7f15580043c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.163+0000 7f154e7fc700 1 -- 172.21.15.31:0/1686285544 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1550058160 con 0x7f155c103790 2024-04-24T23:22:53.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.477+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f155c109f00 con 0x7f155c103790 2024-04-24T23:22:53.479 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:53 smithi031 ceph-mon[111404]: pgmap v174: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:53.479 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:53 smithi031 ceph-mon[111404]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:53.479 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:53.479 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:53.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:53 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.478+0000 7f154e7fc700 1 -- 172.21.15.31:0/1686285544 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f155005b820 con 0x7f155c103790 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:22:53.480 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 11 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:22:53.481 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f154806e140 msgr2=0x7f15480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f154806e140 0x7f15480705f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f15580041d0 tx=0x7f15580043c0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 msgr2=0x7f155c0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 0x7f155c0707a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1550009100 tx=0x7f1550009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 shutdown_connections 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f154806e140 0x7f15480705f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f155c102df0 0x7f155c070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 --2- 172.21.15.31:0/1686285544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f155c103790 0x7f155c0707a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 >> 172.21.15.31:0/1686285544 conn(0x7f155c0f8980 msgr2=0x7f155c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.481+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 shutdown_connections 2024-04-24T23:22:53.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.482+0000 7f1563f81700 1 -- 172.21.15.31:0/1686285544 wait complete. 2024-04-24T23:22:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.599+0000 7f228d388700 1 -- 172.21.15.31:0/2917190788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fe810 msgr2=0x7f2288106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.599+0000 7f228d388700 1 --2- 172.21.15.31:0/2917190788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fe810 0x7f2288106e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f227c004940 tx=0x7f227c009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.599+0000 7f228d388700 1 -- 172.21.15.31:0/2917190788 shutdown_connections 2024-04-24T23:22:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.599+0000 7f228d388700 1 --2- 172.21.15.31:0/2917190788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fe810 0x7f2288106e50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.599+0000 7f228d388700 1 --2- 172.21.15.31:0/2917190788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22880fde70 0x7f22880fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.599+0000 7f228d388700 1 -- 172.21.15.31:0/2917190788 >> 172.21.15.31:0/2917190788 conn(0x7f22880f78e0 msgr2=0x7f22880f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.599+0000 7f228d388700 1 -- 172.21.15.31:0/2917190788 shutdown_connections 2024-04-24T23:22:53.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.599+0000 7f228d388700 1 -- 172.21.15.31:0/2917190788 wait complete. 2024-04-24T23:22:53.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.600+0000 7f228d388700 1 Processor -- start 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.600+0000 7f228d388700 1 -- start start 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.600+0000 7f228d388700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fde70 0x7f2288113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.600+0000 7f228d388700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22880fe810 0x7f2288113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.600+0000 7f228d388700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22881144d0 con 0x7f22880fde70 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.600+0000 7f228d388700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228810dad0 con 0x7f22880fe810 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f2286ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fde70 0x7f2288113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f2286ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fde70 0x7f2288113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41710/0 (socket says 172.21.15.31:41710) 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f2286ffd700 1 -- 172.21.15.31:0/3683084030 learned_addr learned my addr 172.21.15.31:0/3683084030 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f22867fc700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22880fe810 0x7f2288113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f2286ffd700 1 -- 172.21.15.31:0/3683084030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22880fe810 msgr2=0x7f2288113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f2286ffd700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22880fe810 0x7f2288113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f2286ffd700 1 -- 172.21.15.31:0/3683084030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c005040 con 0x7f22880fde70 2024-04-24T23:22:53.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f22867fc700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22880fe810 0x7f2288113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:22:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.601+0000 7f2286ffd700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fde70 0x7f2288113940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2278002a80 tx=0x7f22780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.602+0000 7f226f7fe700 1 -- 172.21.15.31:0/3683084030 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2278007cc0 con 0x7f22880fde70 2024-04-24T23:22:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.602+0000 7f226f7fe700 1 -- 172.21.15.31:0/3683084030 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f227800f040 con 0x7f22880fde70 2024-04-24T23:22:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.602+0000 7f226f7fe700 1 -- 172.21.15.31:0/3683084030 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f227800db20 con 0x7f22880fde70 2024-04-24T23:22:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.602+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228810ddb0 con 0x7f22880fde70 2024-04-24T23:22:53.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.602+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228810e2a0 con 0x7f22880fde70 2024-04-24T23:22:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.603+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2288101860 con 0x7f22880fde70 2024-04-24T23:22:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.604+0000 7f226f7fe700 1 -- 172.21.15.31:0/3683084030 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f227800dc80 con 0x7f22880fde70 2024-04-24T23:22:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.604+0000 7f226f7fe700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.604+0000 7f226f7fe700 1 -- 172.21.15.31:0/3683084030 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f227808d3d0 con 0x7f22880fde70 2024-04-24T23:22:53.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.604+0000 7f22867fc700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.605+0000 7f22867fc700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f227c00b3b0 tx=0x7f227c009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.609+0000 7f226f7fe700 1 -- 172.21.15.31:0/3683084030 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f227805a350 con 0x7f22880fde70 2024-04-24T23:22:53.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:53 smithi040 ceph-mon[83047]: pgmap v174: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:53.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:53 smithi040 ceph-mon[83047]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:53.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:53.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:53.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:53 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:22:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.866+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f228810ef90 con 0x7f22880fde70 2024-04-24T23:22:53.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.866+0000 7f226f7fe700 1 -- 172.21.15.31:0/3683084030 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1879 (secure 0 0 0) 0x7f2278059ee0 con 0x7f22880fde70 2024-04-24T23:22:53.868 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-04-24T23:22:53.868 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:22:53.868 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:22:53.868 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:22:53.868 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:22:53.868 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 15 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:10:58.131406+0000 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:22:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 46 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14464} 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:22:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:22:53.871 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{0:14464} state up:active seq 5 join_fscid=1 addr [v2:172.21.15.31:6828/1027887614,v1:172.21.15.31:6829/1027887614] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:22:53.871 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:22:53.871 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:22:53.871 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:22:53.871 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:22:53.871 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:24285} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:22:53.871 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:22:53.871 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:34386} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.868+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 msgr2=0x7f22700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.868+0000 7f228d388700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f227c00b3b0 tx=0x7f227c009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.868+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fde70 msgr2=0x7f2288113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.868+0000 7f228d388700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fde70 0x7f2288113940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2278002a80 tx=0x7f22780046b0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.869+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 shutdown_connections 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.869+0000 7f228d388700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22880fde70 0x7f2288113940 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.869+0000 7f228d388700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f227006e140 0x7f22700705f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.869+0000 7f228d388700 1 --2- 172.21.15.31:0/3683084030 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22880fe810 0x7f2288113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.869+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 >> 172.21.15.31:0/3683084030 conn(0x7f22880f78e0 msgr2=0x7f22880ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:53.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.869+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 shutdown_connections 2024-04-24T23:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.869+0000 7f228d388700 1 -- 172.21.15.31:0/3683084030 wait complete. 2024-04-24T23:22:53.873 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-04-24T23:22:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.978+0000 7ff12dac0700 1 -- 172.21.15.31:0/264801136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1280fc6b0 msgr2=0x7ff1280fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.978+0000 7ff12dac0700 1 --2- 172.21.15.31:0/264801136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1280fc6b0 0x7ff1280fcb20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff11c004940 tx=0x7ff11c009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:53.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.978+0000 7ff12dac0700 1 -- 172.21.15.31:0/264801136 shutdown_connections 2024-04-24T23:22:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.978+0000 7ff12dac0700 1 --2- 172.21.15.31:0/264801136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1280fc6b0 0x7ff1280fcb20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.978+0000 7ff12dac0700 1 --2- 172.21.15.31:0/264801136 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff128101d90 0x7ff128102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.978+0000 7ff12dac0700 1 -- 172.21.15.31:0/264801136 >> 172.21.15.31:0/264801136 conn(0x7ff128071460 msgr2=0x7ff128071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.979+0000 7ff12dac0700 1 -- 172.21.15.31:0/264801136 shutdown_connections 2024-04-24T23:22:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.979+0000 7ff12dac0700 1 -- 172.21.15.31:0/264801136 wait complete. 2024-04-24T23:22:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.979+0000 7ff12dac0700 1 Processor -- start 2024-04-24T23:22:53.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.979+0000 7ff12dac0700 1 -- start start 2024-04-24T23:22:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.979+0000 7ff12dac0700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1280fc6b0 0x7ff12810eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.979+0000 7ff12dac0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128101d90 0x7ff12810f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.979+0000 7ff12dac0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12810f5a0 con 0x7ff128101d90 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.979+0000 7ff12dac0700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12810f6e0 con 0x7ff1280fc6b0 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff126ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128101d90 0x7ff12810f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff126ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128101d90 0x7ff12810f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41734/0 (socket says 172.21.15.31:41734) 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff126ffd700 1 -- 172.21.15.31:0/1947476044 learned_addr learned my addr 172.21.15.31:0/1947476044 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff1277fe700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1280fc6b0 0x7ff12810eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff126ffd700 1 -- 172.21.15.31:0/1947476044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1280fc6b0 msgr2=0x7ff12810eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff126ffd700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1280fc6b0 0x7ff12810eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff126ffd700 1 -- 172.21.15.31:0/1947476044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c005040 con 0x7ff128101d90 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff1277fe700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1280fc6b0 0x7ff12810eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff126ffd700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128101d90 0x7ff12810f060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00b3b0 tx=0x7ff11c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff10ffff700 1 -- 172.21.15.31:0/1947476044 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11c00e040 con 0x7ff128101d90 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff10ffff700 1 -- 172.21.15.31:0/1947476044 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff11c00bbc0 con 0x7ff128101d90 2024-04-24T23:22:53.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12810f960 con 0x7ff128101d90 2024-04-24T23:22:53.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff10ffff700 1 -- 172.21.15.31:0/1947476044 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11c01c3f0 con 0x7ff128101d90 2024-04-24T23:22:53.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.980+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff128117e10 con 0x7ff128101d90 2024-04-24T23:22:53.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.981+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1280ff070 con 0x7ff128101d90 2024-04-24T23:22:53.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.982+0000 7ff10ffff700 1 -- 172.21.15.31:0/1947476044 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff11c00bd30 con 0x7ff128101d90 2024-04-24T23:22:53.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.982+0000 7ff10ffff700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff11006e140 0x7ff1100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:53.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.982+0000 7ff10ffff700 1 -- 172.21.15.31:0/1947476044 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff11c08ed50 con 0x7ff128101d90 2024-04-24T23:22:53.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.982+0000 7ff1277fe700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff11006e140 0x7ff1100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:53.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.983+0000 7ff1277fe700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff11006e140 0x7ff1100705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff118007fd0 tx=0x7ff118009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:53.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:53.987+0000 7ff10ffff700 1 -- 172.21.15.31:0/1947476044 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff11c058120 con 0x7ff128101d90 2024-04-24T23:22:54.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.219+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1280fbbb0 con 0x7ff11006e140 2024-04-24T23:22:54.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.220+0000 7ff10ffff700 1 -- 172.21.15.31:0/1947476044 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff1280fbbb0 con 0x7ff11006e140 2024-04-24T23:22:54.221 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:22:54.221 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:22:54.221 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:22:54.221 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: "osd", 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "12/21 daemons upgraded", 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:22:54.222 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.222+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff11006e140 msgr2=0x7ff1100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.222+0000 7ff12dac0700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff11006e140 0x7ff1100705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff118007fd0 tx=0x7ff118009040 comp rx=0 tx=0).stop 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.222+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128101d90 msgr2=0x7ff12810f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.222+0000 7ff12dac0700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128101d90 0x7ff12810f060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00b3b0 tx=0x7ff11c009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.222+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 shutdown_connections 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.222+0000 7ff12dac0700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff11006e140 0x7ff1100705f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.222+0000 7ff12dac0700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1280fc6b0 0x7ff12810eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.222+0000 7ff12dac0700 1 --2- 172.21.15.31:0/1947476044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff128101d90 0x7ff12810f060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.223+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 >> 172.21.15.31:0/1947476044 conn(0x7ff128071460 msgr2=0x7ff1280fa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:54.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.223+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 shutdown_connections 2024-04-24T23:22:54.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.223+0000 7ff12dac0700 1 -- 172.21.15.31:0/1947476044 wait complete. 2024-04-24T23:22:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.339+0000 7f140cc74700 1 -- 172.21.15.31:0/439357861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f140400bcb0 con 0x7f14080702a0 2024-04-24T23:22:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.340+0000 7f140feda700 1 -- 172.21.15.31:0/439357861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 msgr2=0x7f1408070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.340+0000 7f140feda700 1 --2- 172.21.15.31:0/439357861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 0x7f1408070670 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1404004970 tx=0x7f1404009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:54.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.340+0000 7f140feda700 1 -- 172.21.15.31:0/439357861 shutdown_connections 2024-04-24T23:22:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.340+0000 7f140feda700 1 --2- 172.21.15.31:0/439357861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1408070bb0 0x7f1408109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.340+0000 7f140feda700 1 --2- 172.21.15.31:0/439357861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 0x7f1408070670 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.340+0000 7f140feda700 1 -- 172.21.15.31:0/439357861 >> 172.21.15.31:0/439357861 conn(0x7f14080f89d0 msgr2=0x7f14080fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.340+0000 7f140feda700 1 -- 172.21.15.31:0/439357861 shutdown_connections 2024-04-24T23:22:54.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.340+0000 7f140feda700 1 -- 172.21.15.31:0/439357861 wait complete. 2024-04-24T23:22:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140feda700 1 Processor -- start 2024-04-24T23:22:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140feda700 1 -- start start 2024-04-24T23:22:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140feda700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 0x7f1408109120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140feda700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1408070bb0 0x7f14081052b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140feda700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1408105900 con 0x7f14080702a0 2024-04-24T23:22:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140feda700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1408105a70 con 0x7f1408070bb0 2024-04-24T23:22:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140dc76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 0x7f1408109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:54.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140dc76700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 0x7f1408109120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41744/0 (socket says 172.21.15.31:41744) 2024-04-24T23:22:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140dc76700 1 -- 172.21.15.31:0/1162542664 learned_addr learned my addr 172.21.15.31:0/1162542664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:22:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.341+0000 7f140d475700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1408070bb0 0x7f14081052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f140d475700 1 -- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 msgr2=0x7f1408109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f140d475700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 0x7f1408109120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f140d475700 1 -- 172.21.15.31:0/1162542664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404005040 con 0x7f1408070bb0 2024-04-24T23:22:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f140dc76700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 0x7f1408109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:22:54.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f140d475700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1408070bb0 0x7f14081052b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f13f8002a50 tx=0x7f13f8004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f13fe7fc700 1 -- 172.21.15.31:0/1162542664 <== mon.1 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13f8007cc0 con 0x7f1408070bb0 2024-04-24T23:22:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f13fe7fc700 1 -- 172.21.15.31:0/1162542664 <== mon.1 v2:172.21.15.40:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13f800f040 con 0x7f1408070bb0 2024-04-24T23:22:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f13fe7fc700 1 -- 172.21.15.31:0/1162542664 <== mon.1 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13f80094b0 con 0x7f1408070bb0 2024-04-24T23:22:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1408105d50 con 0x7f1408070bb0 2024-04-24T23:22:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.342+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14081228b0 con 0x7f1408070bb0 2024-04-24T23:22:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.343+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140806bd30 con 0x7f1408070bb0 2024-04-24T23:22:54.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.344+0000 7f13fe7fc700 1 -- 172.21.15.31:0/1162542664 <== mon.1 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f13f8009770 con 0x7f1408070bb0 2024-04-24T23:22:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.344+0000 7f13fe7fc700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f13f406e140 0x7f13f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:22:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.344+0000 7f13fe7fc700 1 -- 172.21.15.31:0/1162542664 <== mon.1 v2:172.21.15.40:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f13f808d8b0 con 0x7f1408070bb0 2024-04-24T23:22:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.345+0000 7f140dc76700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f13f406e140 0x7f13f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:22:54.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.345+0000 7f140dc76700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f13f406e140 0x7f13f40705f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f140400b020 tx=0x7f1404009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:22:54.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.349+0000 7f13fe7fc700 1 -- 172.21.15.31:0/1162542664 <== mon.1 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f13f8056be0 con 0x7f1408070bb0 2024-04-24T23:22:54.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:54 smithi031 ceph-mon[111404]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:54 smithi031 ceph-mon[111404]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:54 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] up:boot 2024-04-24T23:22:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:54 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:22:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:54 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:22:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:54 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1686285544' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:54 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:54.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:54 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3683084030' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:22:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:54 smithi040 ceph-mon[83047]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:54 smithi040 ceph-mon[83047]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:54 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] up:boot 2024-04-24T23:22:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:54 smithi040 ceph-mon[83047]: fsmap cephfs:1 {0=cephfs.smithi031.acqiox=up:active} 3 up:standby 2024-04-24T23:22:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:54 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:22:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:54 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1686285544' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:22:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:54 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:54.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:54 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3683084030' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:22:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.667+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1408106790 con 0x7f1408070bb0 2024-04-24T23:22:54.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.667+0000 7f13fe7fc700 1 -- 172.21.15.31:0/1162542664 <== mon.1 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f13f805a2a0 con 0x7f1408070bb0 2024-04-24T23:22:54.669 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:22:54.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f13f406e140 msgr2=0x7f13f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f13f406e140 0x7f13f40705f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f140400b020 tx=0x7f1404009e30 comp rx=0 tx=0).stop 2024-04-24T23:22:54.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1408070bb0 msgr2=0x7f14081052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:22:54.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1408070bb0 0x7f14081052b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f13f8002a50 tx=0x7f13f8004600 comp rx=0 tx=0).stop 2024-04-24T23:22:54.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 shutdown_connections 2024-04-24T23:22:54.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14080702a0 0x7f1408109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f13f406e140 0x7f13f40705f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 --2- 172.21.15.31:0/1162542664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1408070bb0 0x7f14081052b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:22:54.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.669+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 >> 172.21.15.31:0/1162542664 conn(0x7f14080f89d0 msgr2=0x7f14080f9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:22:54.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.670+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 shutdown_connections 2024-04-24T23:22:54.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:22:54.670+0000 7f140feda700 1 -- 172.21.15.31:0/1162542664 wait complete. 2024-04-24T23:22:55.603 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:55 smithi031 ceph-mon[111404]: pgmap v175: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:55.603 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:55 smithi031 ceph-mon[111404]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:55.603 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:55 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1162542664' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:22:55.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:55 smithi040 ceph-mon[83047]: pgmap v175: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:55.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:55 smithi040 ceph-mon[83047]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:22:55.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:55 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1162542664' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:22:57.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:57 smithi031 ceph-mon[111404]: pgmap v176: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:57.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:57 smithi040 ceph-mon[83047]: pgmap v176: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:59 smithi040 ceph-mon[83047]: pgmap v177: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:22:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:22:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:59 smithi031 ceph-mon[111404]: pgmap v177: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:22:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:59.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:59.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:22:59.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:22:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:00 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:00 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:00.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:00 smithi031 ceph-mon[111404]: pgmap v178: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:01.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:00 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:01.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:00 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:01.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:00 smithi040 ceph-mon[83047]: pgmap v178: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:02.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:02 smithi031 ceph-mon[111404]: pgmap v179: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:03.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:02 smithi040 ceph-mon[83047]: pgmap v179: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:04.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:04.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:05.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:05 smithi031 ceph-mon[111404]: pgmap v180: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:05.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:05.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.acqiox", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:23:05.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:05 smithi040 ceph-mon[83047]: pgmap v180: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.acqiox", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:23:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:06.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:06 smithi031 ceph-mon[111404]: Upgrade: Updating mds.cephfs.smithi031.acqiox 2024-04-24T23:23:06.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:06 smithi031 ceph-mon[111404]: Deploying daemon mds.cephfs.smithi031.acqiox on smithi031 2024-04-24T23:23:06.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:06 smithi040 ceph-mon[83047]: Upgrade: Updating mds.cephfs.smithi031.acqiox 2024-04-24T23:23:06.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:06 smithi040 ceph-mon[83047]: Deploying daemon mds.cephfs.smithi031.acqiox on smithi031 2024-04-24T23:23:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[111398]: 2024-04-24T23:23:07.318+0000 7fda7a362700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-mon[111404]: pgmap v181: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-mon[111404]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:23:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-mon[111404]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:07.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-mon[111404]: osdmap e82: 6 total, 6 up, 6 in 2024-04-24T23:23:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-mon[111404]: Standby daemon mds.cephfs.smithi040.dicevu assigned to filesystem cephfs as rank 0 2024-04-24T23:23:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-mon[111404]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:23:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-mon[111404]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T23:23:07.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:07 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi040.dicevu=up:replay} 2 up:standby 2024-04-24T23:23:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:07 smithi040 ceph-mon[83047]: pgmap v181: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:23:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:07 smithi040 ceph-mon[83047]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:23:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:07 smithi040 ceph-mon[83047]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:07 smithi040 ceph-mon[83047]: osdmap e82: 6 total, 6 up, 6 in 2024-04-24T23:23:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:07 smithi040 ceph-mon[83047]: Standby daemon mds.cephfs.smithi040.dicevu assigned to filesystem cephfs as rank 0 2024-04-24T23:23:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:07 smithi040 ceph-mon[83047]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:23:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:07 smithi040 ceph-mon[83047]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T23:23:07.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:07 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi040.dicevu=up:replay} 2 up:standby 2024-04-24T23:23:09.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:09 smithi031 ceph-mon[111404]: pgmap v183: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-04-24T23:23:09.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:09 smithi040 ceph-mon[83047]: pgmap v183: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-04-24T23:23:11.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:11 smithi031 ceph-mon[111404]: pgmap v184: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-04-24T23:23:11.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:11 smithi040 ceph-mon[83047]: pgmap v184: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-04-24T23:23:13.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:13 smithi031 ceph-mon[111404]: pgmap v185: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-24T23:23:13.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:13 smithi040 ceph-mon[83047]: pgmap v185: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-24T23:23:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:14 smithi040 ceph-mon[83047]: pgmap v186: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-04-24T23:23:15.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:15.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:15.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:14 smithi031 ceph-mon[111404]: pgmap v186: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5 op/s 2024-04-24T23:23:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:15 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.31:6828/3258483612,v1:172.21.15.31:6829/3258483612] up:boot 2024-04-24T23:23:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:15 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi040.dicevu=up:replay} 3 up:standby 2024-04-24T23:23:16.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:23:16.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:16.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:16.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:16.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:15 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.31:6828/3258483612,v1:172.21.15.31:6829/3258483612] up:boot 2024-04-24T23:23:16.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:15 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi040.dicevu=up:replay} 3 up:standby 2024-04-24T23:23:16.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:23:17.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:16 smithi031 ceph-mon[111404]: pgmap v187: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-24T23:23:17.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:16 smithi040 ceph-mon[83047]: pgmap v187: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-24T23:23:18.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:17 smithi031 ceph-mon[111404]: reconnect by client.14518 192.168.0.1:0/2806865259 after 0.000999987 2024-04-24T23:23:18.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:17 smithi031 ceph-mon[111404]: reconnect by client.24327 192.168.0.1:0/3971968853 after 0.000999987 2024-04-24T23:23:18.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:17 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:reconnect 2024-04-24T23:23:18.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:17 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi040.dicevu=up:reconnect} 3 up:standby 2024-04-24T23:23:18.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:17 smithi040 ceph-mon[83047]: reconnect by client.14518 192.168.0.1:0/2806865259 after 0.000999987 2024-04-24T23:23:18.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:17 smithi040 ceph-mon[83047]: reconnect by client.24327 192.168.0.1:0/3971968853 after 0.000999987 2024-04-24T23:23:18.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:17 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:reconnect 2024-04-24T23:23:18.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:17 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi040.dicevu=up:reconnect} 3 up:standby 2024-04-24T23:23:19.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:18 smithi031 ceph-mon[111404]: pgmap v188: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-24T23:23:19.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:18 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:rejoin 2024-04-24T23:23:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:18 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi040.dicevu=up:rejoin} 3 up:standby 2024-04-24T23:23:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:18 smithi031 ceph-mon[111404]: daemon mds.cephfs.smithi040.dicevu is now active in filesystem cephfs as rank 0 2024-04-24T23:23:19.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:18 smithi040 ceph-mon[83047]: pgmap v188: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-24T23:23:19.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:18 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:rejoin 2024-04-24T23:23:19.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:18 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi040.dicevu=up:rejoin} 3 up:standby 2024-04-24T23:23:19.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:18 smithi040 ceph-mon[83047]: daemon mds.cephfs.smithi040.dicevu is now active in filesystem cephfs as rank 0 2024-04-24T23:23:20.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:19 smithi031 ceph-mon[111404]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:23:20.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:19 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:23:20.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:19 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:active 2024-04-24T23:23:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:19 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi040.dicevu=up:active} 3 up:standby 2024-04-24T23:23:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:19 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:20.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:19 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:20.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:19 smithi040 ceph-mon[83047]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:23:20.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:19 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:23:20.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:19 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] up:active 2024-04-24T23:23:20.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:19 smithi040 ceph-mon[83047]: fsmap cephfs:1 {0=cephfs.smithi040.dicevu=up:active} 3 up:standby 2024-04-24T23:23:20.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:19 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:20.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:19 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:21 smithi040 ceph-mon[83047]: pgmap v189: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:23:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:21 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:21.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:21 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:21.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:21 smithi031 ceph-mon[111404]: pgmap v189: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:23:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:21 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:21.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:21 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:22.956 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:22 smithi031 ceph-mon[111404]: pgmap v190: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T23:23:23.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:22 smithi040 ceph-mon[83047]: pgmap v190: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 426 B/s wr, 6 op/s 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.783+0000 7f84e8575700 1 -- 172.21.15.31:0/1499232720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0101ce0 msgr2=0x7f84e01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.783+0000 7f84e8575700 1 --2- 172.21.15.31:0/1499232720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0101ce0 0x7f84e01020b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84d0004940 tx=0x7f84d0009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.783+0000 7f84e8575700 1 -- 172.21.15.31:0/1499232720 shutdown_connections 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.783+0000 7f84e8575700 1 --2- 172.21.15.31:0/1499232720 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84e0102680 0x7f84e00fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.783+0000 7f84e8575700 1 --2- 172.21.15.31:0/1499232720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0101ce0 0x7f84e01020b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.783+0000 7f84e8575700 1 -- 172.21.15.31:0/1499232720 >> 172.21.15.31:0/1499232720 conn(0x7f84e00f7850 msgr2=0x7f84e00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.783+0000 7f84e8575700 1 -- 172.21.15.31:0/1499232720 shutdown_connections 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.783+0000 7f84e8575700 1 -- 172.21.15.31:0/1499232720 wait complete. 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e8575700 1 Processor -- start 2024-04-24T23:23:24.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e8575700 1 -- start start 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e8575700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84e0101ce0 0x7f84e010ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e8575700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0102680 0x7f84e010f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e8575700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e010f730 con 0x7f84e0102680 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e8575700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e010f870 con 0x7f84e0101ce0 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e5b10700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0102680 0x7f84e010f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e5b10700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0102680 0x7f84e010f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45332/0 (socket says 172.21.15.31:45332) 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e5b10700 1 -- 172.21.15.31:0/325770656 learned_addr learned my addr 172.21.15.31:0/325770656 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.784+0000 7f84e6311700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84e0101ce0 0x7f84e010ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84e5b10700 1 -- 172.21.15.31:0/325770656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84e0101ce0 msgr2=0x7f84e010ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84e5b10700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84e0101ce0 0x7f84e010ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84e5b10700 1 -- 172.21.15.31:0/325770656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0005040 con 0x7f84e0102680 2024-04-24T23:23:24.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84e5b10700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0102680 0x7f84e010f1f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84dc002a50 tx=0x7f84dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:24.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84d6ffd700 1 -- 172.21.15.31:0/325770656 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84dc007cc0 con 0x7f84e0102680 2024-04-24T23:23:24.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e0117b60 con 0x7f84e0102680 2024-04-24T23:23:24.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84d6ffd700 1 -- 172.21.15.31:0/325770656 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84dc00f040 con 0x7f84e0102680 2024-04-24T23:23:24.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84d6ffd700 1 -- 172.21.15.31:0/325770656 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84dc00d9c0 con 0x7f84e0102680 2024-04-24T23:23:24.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.785+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e01180f0 con 0x7f84e0102680 2024-04-24T23:23:24.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.786+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e00fd430 con 0x7f84e0102680 2024-04-24T23:23:24.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.787+0000 7f84d6ffd700 1 -- 172.21.15.31:0/325770656 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f84dc009730 con 0x7f84e0102680 2024-04-24T23:23:24.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.787+0000 7f84d6ffd700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f84cc06e150 0x7f84cc070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:24.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.787+0000 7f84d6ffd700 1 -- 172.21.15.31:0/325770656 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f84dc08d7c0 con 0x7f84e0102680 2024-04-24T23:23:24.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.787+0000 7f84e6311700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f84cc06e150 0x7f84cc070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:24.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.788+0000 7f84e6311700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f84cc06e150 0x7f84cc070600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84d000b3b0 tx=0x7f84d0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:24.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:24.792+0000 7f84d6ffd700 1 -- 172.21.15.31:0/325770656 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f84dc056b40 con 0x7f84e0102680 2024-04-24T23:23:25.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.018+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84e004bed0 con 0x7f84cc06e150 2024-04-24T23:23:25.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.019+0000 7f84d6ffd700 1 -- 172.21.15.31:0/325770656 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f84e004bed0 con 0x7f84cc06e150 2024-04-24T23:23:25.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f84cc06e150 msgr2=0x7f84cc070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f84cc06e150 0x7f84cc070600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f84d000b3b0 tx=0x7f84d0009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0102680 msgr2=0x7f84e010f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0102680 0x7f84e010f1f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84dc002a50 tx=0x7f84dc004600 comp rx=0 tx=0).stop 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 shutdown_connections 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f84cc06e150 0x7f84cc070600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84e0101ce0 0x7f84e010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 --2- 172.21.15.31:0/325770656 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84e0102680 0x7f84e010f1f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 >> 172.21.15.31:0/325770656 conn(0x7f84e00f7850 msgr2=0x7f84e00f88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 shutdown_connections 2024-04-24T23:23:25.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.021+0000 7f84e8575700 1 -- 172.21.15.31:0/325770656 wait complete. 2024-04-24T23:23:25.033 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 -- 172.21.15.31:0/67928191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b541037a0 msgr2=0x7f4b540fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 --2- 172.21.15.31:0/67928191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b541037a0 0x7f4b540fce10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4b44004940 tx=0x7f4b44009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 -- 172.21.15.31:0/67928191 shutdown_connections 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 --2- 172.21.15.31:0/67928191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b541037a0 0x7f4b540fce10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 --2- 172.21.15.31:0/67928191 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b54102e00 0x7f4b541031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 -- 172.21.15.31:0/67928191 >> 172.21.15.31:0/67928191 conn(0x7f4b540f89d0 msgr2=0x7f4b540fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 -- 172.21.15.31:0/67928191 shutdown_connections 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 -- 172.21.15.31:0/67928191 wait complete. 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 Processor -- start 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.129+0000 7f4b5bb35700 1 -- start start 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b5bb35700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102e00 0x7f4b541138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b5bb35700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b541037a0 0x7f4b54113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b5bb35700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54114430 con 0x7f4b54102e00 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b5bb35700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5410da30 con 0x7f4b541037a0 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b598d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102e00 0x7f4b541138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b598d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102e00 0x7f4b541138a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45338/0 (socket says 172.21.15.31:45338) 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b598d1700 1 -- 172.21.15.31:0/4061053572 learned_addr learned my addr 172.21.15.31:0/4061053572 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b590d0700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b541037a0 0x7f4b54113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b598d1700 1 -- 172.21.15.31:0/4061053572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b541037a0 msgr2=0x7f4b54113de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b598d1700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b541037a0 0x7f4b54113de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b598d1700 1 -- 172.21.15.31:0/4061053572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b44005040 con 0x7f4b54102e00 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b598d1700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102e00 0x7f4b541138a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b50002a80 tx=0x7f4b500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/4061053572 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b50007cc0 con 0x7f4b54102e00 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/4061053572 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b5000f040 con 0x7f4b54102e00 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5410dd10 con 0x7f4b54102e00 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.130+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/4061053572 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b5000dac0 con 0x7f4b54102e00 2024-04-24T23:23:25.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.131+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5410e2a0 con 0x7f4b54102e00 2024-04-24T23:23:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.131+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b541052d0 con 0x7f4b54102e00 2024-04-24T23:23:25.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.132+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/4061053572 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4b5000dc20 con 0x7f4b54102e00 2024-04-24T23:23:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.132+0000 7f4b4a7fc700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4006e140 0x7f4b400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.132+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/4061053572 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4b5008d830 con 0x7f4b54102e00 2024-04-24T23:23:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.132+0000 7f4b590d0700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4006e140 0x7f4b400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.133+0000 7f4b590d0700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4006e140 0x7f4b400705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4b5410ed10 tx=0x7f4b44009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:25.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.137+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/4061053572 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4b5005a760 con 0x7f4b54102e00 2024-04-24T23:23:25.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.367+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b5410dea0 con 0x7f4b4006e140 2024-04-24T23:23:25.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.368+0000 7f4b4a7fc700 1 -- 172.21.15.31:0/4061053572 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4b5410dea0 con 0x7f4b4006e140 2024-04-24T23:23:25.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4006e140 msgr2=0x7f4b400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4006e140 0x7f4b400705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4b5410ed10 tx=0x7f4b44009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:25.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102e00 msgr2=0x7f4b541138a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102e00 0x7f4b541138a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b50002a80 tx=0x7f4b500046b0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 shutdown_connections 2024-04-24T23:23:25.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b54102e00 0x7f4b541138a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4b4006e140 0x7f4b400705f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 --2- 172.21.15.31:0/4061053572 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4b541037a0 0x7f4b54113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 >> 172.21.15.31:0/4061053572 conn(0x7f4b540f89d0 msgr2=0x7f4b540fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:25.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 shutdown_connections 2024-04-24T23:23:25.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.371+0000 7f4b5bb35700 1 -- 172.21.15.31:0/4061053572 wait complete. 2024-04-24T23:23:25.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.475+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/32684261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc071f30 msgr2=0x7fa2dc070300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.475+0000 7fa2e2f9f700 1 --2- 172.21.15.31:0/32684261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc071f30 0x7fa2dc070300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0004970 tx=0x7fa2d0009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:25.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.475+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/32684261 shutdown_connections 2024-04-24T23:23:25.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.475+0000 7fa2e2f9f700 1 --2- 172.21.15.31:0/32684261 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc070840 0x7fa2dc070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.475+0000 7fa2e2f9f700 1 --2- 172.21.15.31:0/32684261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc071f30 0x7fa2dc070300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.475+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/32684261 >> 172.21.15.31:0/32684261 conn(0x7fa2dc0f89b0 msgr2=0x7fa2dc0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:25.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.476+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/32684261 shutdown_connections 2024-04-24T23:23:25.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.476+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/32684261 wait complete. 2024-04-24T23:23:25.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.476+0000 7fa2e2f9f700 1 Processor -- start 2024-04-24T23:23:25.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.476+0000 7fa2e2f9f700 1 -- start start 2024-04-24T23:23:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.476+0000 7fa2e2f9f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc070840 0x7fa2dc11a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.476+0000 7fa2e2f9f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc071f30 0x7fa2dc11a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e2f9f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc11ad00 con 0x7fa2dc070840 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e2f9f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc1140d0 con 0x7fa2dc071f30 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e0d3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc070840 0x7fa2dc11a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e0d3b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc070840 0x7fa2dc11a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45352/0 (socket says 172.21.15.31:45352) 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e0d3b700 1 -- 172.21.15.31:0/1735180190 learned_addr learned my addr 172.21.15.31:0/1735180190 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2dbfff700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc071f30 0x7fa2dc11a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e0d3b700 1 -- 172.21.15.31:0/1735180190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc071f30 msgr2=0x7fa2dc11a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e0d3b700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc071f30 0x7fa2dc11a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e0d3b700 1 -- 172.21.15.31:0/1735180190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0005040 con 0x7fa2dc070840 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e0d3b700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc070840 0x7fa2dc11a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000c00 tx=0x7fa2d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2d97fa700 1 -- 172.21.15.31:0/1735180190 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d000e040 con 0x7fa2dc070840 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2d97fa700 1 -- 172.21.15.31:0/1735180190 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2d000bc10 con 0x7fa2dc070840 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc114350 con 0x7fa2dc070840 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2d97fa700 1 -- 172.21.15.31:0/1735180190 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d001c630 con 0x7fa2dc070840 2024-04-24T23:23:25.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.477+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc114890 con 0x7fa2dc070840 2024-04-24T23:23:25.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.478+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc071a40 con 0x7fa2dc070840 2024-04-24T23:23:25.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.479+0000 7fa2d97fa700 1 -- 172.21.15.31:0/1735180190 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa2d0013420 con 0x7fa2dc070840 2024-04-24T23:23:25.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.479+0000 7fa2d97fa700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa2cc06e140 0x7fa2cc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.479+0000 7fa2d97fa700 1 -- 172.21.15.31:0/1735180190 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa2d008ee80 con 0x7fa2dc070840 2024-04-24T23:23:25.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.479+0000 7fa2dbfff700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa2cc06e140 0x7fa2cc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.480+0000 7fa2dbfff700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa2cc06e140 0x7fa2cc0705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc115a10 tx=0x7fa2c8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:25.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.484+0000 7fa2d97fa700 1 -- 172.21.15.31:0/1735180190 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa2d005bdb0 con 0x7fa2dc070840 2024-04-24T23:23:25.602 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: pgmap v191: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s rd, 426 B/s wr, 5 op/s 2024-04-24T23:23:25.602 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:25.602 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:25.602 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:25.602 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:25.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:25.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:25.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:25.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:25.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:25.603 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:25 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:25.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.702+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2dc1105a0 con 0x7fa2cc06e140 2024-04-24T23:23:25.703 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: pgmap v191: 65 pgs: 65 active+clean; 205 MiB data, 846 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s rd, 426 B/s wr, 5 op/s 2024-04-24T23:23:25.703 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:25.704 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:25 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:25.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.712+0000 7fa2d97fa700 1 -- 172.21.15.31:0/1735180190 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa2c800f9b0 con 0x7fa2cc06e140 2024-04-24T23:23:25.713 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:23:25.713 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 6s ago 18m 25.6M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:23:25.713 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (4m) 6s ago 18m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:23:25.713 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (4m) 47s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:23:25.713 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (6m) 6s ago 17m 79.7M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:23:25.713 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (11s) 6s ago 12m 11.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 956ce7ff5df1 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (33s) 6s ago 12m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1d7ff0e06219 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (12m) 47s ago 12m 15.2M - 17.2.7-902-gc61472c8 0de407c1b3af e9ec359dcad9 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (12m) 47s ago 12m 16.6M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (8m) 6s ago 19m 507M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (7m) 47s ago 17m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (5m) 6s ago 19m 52.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (4m) 47s ago 17m 36.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 6s ago 18m 16.8M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (7m) 47s ago 17m 14.8M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 6s ago 16m 96.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (3m) 6s ago 15m 111M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (2m) 6s ago 15m 91.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 271a0764ef5c 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (119s) 47s ago 15m 116M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 97c6f05632f6 2024-04-24T23:23:25.714 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (84s) 47s ago 14m 97.7M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 85d9bab7b352 2024-04-24T23:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (50s) 47s ago 14m 12.6M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 32b6ff45fea8 2024-04-24T23:23:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 6s ago 17m 60.1M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:23:25.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa2cc06e140 msgr2=0x7fa2cc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa2cc06e140 0x7fa2cc0705f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc115a10 tx=0x7fa2c8009040 comp rx=0 tx=0).stop 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc070840 msgr2=0x7fa2dc11a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc070840 0x7fa2dc11a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0000c00 tx=0x7fa2d0009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 shutdown_connections 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2dc070840 0x7fa2dc11a170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa2cc06e140 0x7fa2cc0705f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 --2- 172.21.15.31:0/1735180190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2dc071f30 0x7fa2dc11a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 >> 172.21.15.31:0/1735180190 conn(0x7fa2dc0f89b0 msgr2=0x7fa2dc0f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 shutdown_connections 2024-04-24T23:23:25.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.714+0000 7fa2e2f9f700 1 -- 172.21.15.31:0/1735180190 wait complete. 2024-04-24T23:23:25.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.817+0000 7f5f13b8f700 1 -- 172.21.15.31:0/3986274200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fe810 msgr2=0x7f5f0c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.817+0000 7f5f13b8f700 1 --2- 172.21.15.31:0/3986274200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fe810 0x7f5f0c106e50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5efc004940 tx=0x7f5efc009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:25.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.817+0000 7f5f13b8f700 1 -- 172.21.15.31:0/3986274200 shutdown_connections 2024-04-24T23:23:25.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.817+0000 7f5f13b8f700 1 --2- 172.21.15.31:0/3986274200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fe810 0x7f5f0c106e50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.817+0000 7f5f13b8f700 1 --2- 172.21.15.31:0/3986274200 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f0c0fde70 0x7f5f0c0fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.817+0000 7f5f13b8f700 1 -- 172.21.15.31:0/3986274200 >> 172.21.15.31:0/3986274200 conn(0x7f5f0c0f7860 msgr2=0x7f5f0c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:25.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f13b8f700 1 -- 172.21.15.31:0/3986274200 shutdown_connections 2024-04-24T23:23:25.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f13b8f700 1 -- 172.21.15.31:0/3986274200 wait complete. 2024-04-24T23:23:25.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f13b8f700 1 Processor -- start 2024-04-24T23:23:25.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f13b8f700 1 -- start start 2024-04-24T23:23:25.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f13b8f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fde70 0x7f5f0c113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f13b8f700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f0c0fe810 0x7f5f0c113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f13b8f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c114560 con 0x7f5f0c0fde70 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f13b8f700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c10dad0 con 0x7f5f0c0fe810 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.818+0000 7f5f1192b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fde70 0x7f5f0c113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f1192b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fde70 0x7f5f0c113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45374/0 (socket says 172.21.15.31:45374) 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f1192b700 1 -- 172.21.15.31:0/389734903 learned_addr learned my addr 172.21.15.31:0/389734903 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f1112a700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f0c0fe810 0x7f5f0c113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f1192b700 1 -- 172.21.15.31:0/389734903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f0c0fe810 msgr2=0x7f5f0c113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f1192b700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f0c0fe810 0x7f5f0c113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f1192b700 1 -- 172.21.15.31:0/389734903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc005040 con 0x7f5f0c0fde70 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f1192b700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fde70 0x7f5f0c113940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800b790 tx=0x7f5f08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f027fc700 1 -- 172.21.15.31:0/389734903 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f0800f770 con 0x7f5f0c0fde70 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f027fc700 1 -- 172.21.15.31:0/389734903 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f0800fda0 con 0x7f5f0c0fde70 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c10ddb0 con 0x7f5f0c0fde70 2024-04-24T23:23:25.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f027fc700 1 -- 172.21.15.31:0/389734903 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f0800d310 con 0x7f5f0c0fde70 2024-04-24T23:23:25.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.819+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c10e2f0 con 0x7f5f0c0fde70 2024-04-24T23:23:25.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.820+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c066af0 con 0x7f5f0c0fde70 2024-04-24T23:23:25.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.821+0000 7f5f027fc700 1 -- 172.21.15.31:0/389734903 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5f08010420 con 0x7f5f0c0fde70 2024-04-24T23:23:25.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.821+0000 7f5f027fc700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ef806e140 0x7f5ef80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:25.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.821+0000 7f5f027fc700 1 -- 172.21.15.31:0/389734903 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5f0808db40 con 0x7f5f0c0fde70 2024-04-24T23:23:25.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.821+0000 7f5f1112a700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ef806e140 0x7f5ef80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:25.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.822+0000 7f5f1112a700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ef806e140 0x7f5ef80705f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c10ec90 tx=0x7f5efc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:25.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:25.826+0000 7f5f027fc700 1 -- 172.21.15.31:0/389734903 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5f0805aa70 con 0x7f5f0c0fde70 2024-04-24T23:23:26.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.121+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5f0c109f40 con 0x7f5f0c0fde70 2024-04-24T23:23:26.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.122+0000 7f5f027fc700 1 -- 172.21.15.31:0/389734903 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f5f0805a600 con 0x7f5f0c0fde70 2024-04-24T23:23:26.123 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:23:26.123 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:23:26.123 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:23:26.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:23:26.123 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 12 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:23:26.124 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ef806e140 msgr2=0x7f5ef80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ef806e140 0x7f5ef80705f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c10ec90 tx=0x7f5efc009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fde70 msgr2=0x7f5f0c113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fde70 0x7f5f0c113940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800b790 tx=0x7f5f08009250 comp rx=0 tx=0).stop 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 shutdown_connections 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0c0fde70 0x7f5f0c113940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f5ef806e140 0x7f5ef80705f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 --2- 172.21.15.31:0/389734903 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5f0c0fe810 0x7f5f0c113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 >> 172.21.15.31:0/389734903 conn(0x7f5f0c0f7860 msgr2=0x7f5f0c0f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 shutdown_connections 2024-04-24T23:23:26.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.124+0000 7f5f13b8f700 1 -- 172.21.15.31:0/389734903 wait complete. 2024-04-24T23:23:26.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.232+0000 7f9bca2b1700 1 -- 172.21.15.31:0/2532641133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fc790 msgr2=0x7f9bc40fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.232+0000 7f9bca2b1700 1 --2- 172.21.15.31:0/2532641133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fc790 0x7f9bc40fcb60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8004940 tx=0x7f9bb8009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:26.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.233+0000 7f9bca2b1700 1 -- 172.21.15.31:0/2532641133 shutdown_connections 2024-04-24T23:23:26.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.233+0000 7f9bca2b1700 1 --2- 172.21.15.31:0/2532641133 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40fd0a0 0x7f9bc4109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.233+0000 7f9bca2b1700 1 --2- 172.21.15.31:0/2532641133 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fc790 0x7f9bc40fcb60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.233+0000 7f9bca2b1700 1 -- 172.21.15.31:0/2532641133 >> 172.21.15.31:0/2532641133 conn(0x7f9bc4071d00 msgr2=0x7f9bc4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:26.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.233+0000 7f9bca2b1700 1 -- 172.21.15.31:0/2532641133 shutdown_connections 2024-04-24T23:23:26.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.233+0000 7f9bca2b1700 1 -- 172.21.15.31:0/2532641133 wait complete. 2024-04-24T23:23:26.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.233+0000 7f9bca2b1700 1 Processor -- start 2024-04-24T23:23:26.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.233+0000 7f9bca2b1700 1 -- start start 2024-04-24T23:23:26.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bca2b1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40fc790 0x7f9bc4115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bca2b1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fd0a0 0x7f9bc41162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bca2b1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4116930 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bca2b1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc410fd40 con 0x7f9bc40fc790 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fd0a0 0x7f9bc41162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc37fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fd0a0 0x7f9bc41162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45406/0 (socket says 172.21.15.31:45406) 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc3fff700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40fc790 0x7f9bc4115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc37fe700 1 -- 172.21.15.31:0/1704346870 learned_addr learned my addr 172.21.15.31:0/1704346870 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc37fe700 1 -- 172.21.15.31:0/1704346870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40fc790 msgr2=0x7f9bc4115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc37fe700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40fc790 0x7f9bc4115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc37fe700 1 -- 172.21.15.31:0/1704346870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb8005040 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc37fe700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fd0a0 0x7f9bc41162e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4002a50 tx=0x7f9bb40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc0ff9700 1 -- 172.21.15.31:0/1704346870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bb4007cc0 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc4110020 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc0ff9700 1 -- 172.21.15.31:0/1704346870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bb400f040 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bc0ff9700 1 -- 172.21.15.31:0/1704346870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bb400da30 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.234+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc4110560 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.235+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc40fc330 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.236+0000 7f9bc0ff9700 1 -- 172.21.15.31:0/1704346870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9bb400db90 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.236+0000 7f9bc0ff9700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9bac06e150 0x7f9bac070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.236+0000 7f9bc0ff9700 1 -- 172.21.15.31:0/1704346870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9bb408d7f0 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.236+0000 7f9bc3fff700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9bac06e150 0x7f9bac070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.237+0000 7f9bc3fff700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9bac06e150 0x7f9bac070600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800b3b0 tx=0x7f9bb8009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:26.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.241+0000 7f9bc0ff9700 1 -- 172.21.15.31:0/1704346870 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9bb4056bf0 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.495+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9bc410c100 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.496+0000 7f9bc0ff9700 1 -- 172.21.15.31:0/1704346870 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1891 (secure 0 0 0) 0x7f9bb405a2b0 con 0x7f9bc40fd0a0 2024-04-24T23:23:26.497 INFO:teuthology.orchestra.run.smithi031.stdout:e23 2024-04-24T23:23:26.497 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 23 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:23:18.976584+0000 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:23:26.498 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 82 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24285} 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:23:26.499 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 24285 members: 24285 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{0:24285} state up:active seq 191 join_fscid=1 addr [v2:172.21.15.40:6824/1413423535,v1:172.21.15.40:6825/1413423535] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:24291} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] compat {c=[1],r=[1],i=[7ff]}] 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{-1:34386} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:23:26.500 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/3258483612,v1:172.21.15.31:6829/3258483612] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:23:26.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9bac06e150 msgr2=0x7f9bac070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9bac06e150 0x7f9bac070600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800b3b0 tx=0x7f9bb8009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:26.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fd0a0 msgr2=0x7f9bc41162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fd0a0 0x7f9bc41162e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4002a50 tx=0x7f9bb40046b0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 shutdown_connections 2024-04-24T23:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f9bac06e150 0x7f9bac070600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40fc790 0x7f9bc4115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 --2- 172.21.15.31:0/1704346870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bc40fd0a0 0x7f9bc41162e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 >> 172.21.15.31:0/1704346870 conn(0x7f9bc4071d00 msgr2=0x7f9bc40f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 shutdown_connections 2024-04-24T23:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.499+0000 7f9bca2b1700 1 -- 172.21.15.31:0/1704346870 wait complete. 2024-04-24T23:23:26.502 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-24T23:23:26.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.608+0000 7fdf6e283700 1 -- 172.21.15.31:0/685705401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf680fbd40 msgr2=0x7fdf680fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.608+0000 7fdf6e283700 1 --2- 172.21.15.31:0/685705401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf680fbd40 0x7fdf680fc1b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf50004940 tx=0x7fdf50009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:26.609 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.608+0000 7fdf6e283700 1 -- 172.21.15.31:0/685705401 shutdown_connections 2024-04-24T23:23:26.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.608+0000 7fdf6e283700 1 --2- 172.21.15.31:0/685705401 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf680fbd40 0x7fdf680fc1b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.608+0000 7fdf6e283700 1 --2- 172.21.15.31:0/685705401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf68101d90 0x7fdf68102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.608+0000 7fdf6e283700 1 -- 172.21.15.31:0/685705401 >> 172.21.15.31:0/685705401 conn(0x7fdf68071d00 msgr2=0x7fdf68072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:26.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.608+0000 7fdf6e283700 1 -- 172.21.15.31:0/685705401 shutdown_connections 2024-04-24T23:23:26.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.608+0000 7fdf6e283700 1 -- 172.21.15.31:0/685705401 wait complete. 2024-04-24T23:23:26.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf6e283700 1 Processor -- start 2024-04-24T23:23:26.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf6e283700 1 -- start start 2024-04-24T23:23:26.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf6e283700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680fbd40 0x7fdf68113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf6e283700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf68101d90 0x7fdf68114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf6e283700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf681146b0 con 0x7fdf68101d90 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf6e283700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6810daa0 con 0x7fdf680fbd40 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf68101d90 0x7fdf68114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf677fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf68101d90 0x7fdf68114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45424/0 (socket says 172.21.15.31:45424) 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf677fe700 1 -- 172.21.15.31:0/2961810949 learned_addr learned my addr 172.21.15.31:0/2961810949 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.609+0000 7fdf67fff700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680fbd40 0x7fdf68113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf677fe700 1 -- 172.21.15.31:0/2961810949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680fbd40 msgr2=0x7fdf68113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf677fe700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680fbd40 0x7fdf68113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf677fe700 1 -- 172.21.15.31:0/2961810949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf50005040 con 0x7fdf68101d90 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf67fff700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680fbd40 0x7fdf68113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf677fe700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf68101d90 0x7fdf68114060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009b30 tx=0x7fdf50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf64ff9700 1 -- 172.21.15.31:0/2961810949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf50021070 con 0x7fdf68101d90 2024-04-24T23:23:26.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf64ff9700 1 -- 172.21.15.31:0/2961810949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf50013c00 con 0x7fdf68101d90 2024-04-24T23:23:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6810dd20 con 0x7fdf68101d90 2024-04-24T23:23:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf64ff9700 1 -- 172.21.15.31:0/2961810949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf5000e070 con 0x7fdf68101d90 2024-04-24T23:23:26.613 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.610+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6810e200 con 0x7fdf68101d90 2024-04-24T23:23:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.611+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf680fd430 con 0x7fdf68101d90 2024-04-24T23:23:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.612+0000 7fdf64ff9700 1 -- 172.21.15.31:0/2961810949 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdf5001cb60 con 0x7fdf68101d90 2024-04-24T23:23:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.612+0000 7fdf64ff9700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf5406e140 0x7fdf540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.612+0000 7fdf64ff9700 1 -- 172.21.15.31:0/2961810949 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdf5008efa0 con 0x7fdf68101d90 2024-04-24T23:23:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.612+0000 7fdf67fff700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf5406e140 0x7fdf540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.613+0000 7fdf67fff700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf5406e140 0x7fdf540705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf58004fd0 tx=0x7fdf5800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:26.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.617+0000 7fdf64ff9700 1 -- 172.21.15.31:0/2961810949 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdf5005bed0 con 0x7fdf68101d90 2024-04-24T23:23:26.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:26 smithi040 ceph-mon[83047]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:26.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:26 smithi040 ceph-mon[83047]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:26.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:26 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/389734903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:26.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:26.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.dicevu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:23:26.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:26 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:26.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:26 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1704346870' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:23:26.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:26 smithi031 ceph-mon[111404]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:26.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:26 smithi031 ceph-mon[111404]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:26 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/389734903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.dicevu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:23:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:26 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:26.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:26 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1704346870' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:23:26.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.845+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf681137f0 con 0x7fdf5406e140 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.846+0000 7fdf64ff9700 1 -- 172.21.15.31:0/2961810949 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdf681137f0 con 0x7fdf5406e140 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "osd", 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:23:26.848 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:23:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "14/21 daemons upgraded", 2024-04-24T23:23:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T23:23:26.849 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:23:26.849 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.848+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf5406e140 msgr2=0x7fdf540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.848+0000 7fdf6e283700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf5406e140 0x7fdf540705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdf58004fd0 tx=0x7fdf5800b040 comp rx=0 tx=0).stop 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.848+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf68101d90 msgr2=0x7fdf68114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.848+0000 7fdf6e283700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf68101d90 0x7fdf68114060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009b30 tx=0x7fdf50009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.849+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 shutdown_connections 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.849+0000 7fdf6e283700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf5406e140 0x7fdf540705f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.849+0000 7fdf6e283700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf680fbd40 0x7fdf68113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.849+0000 7fdf6e283700 1 --2- 172.21.15.31:0/2961810949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf68101d90 0x7fdf68114060 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.849+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 >> 172.21.15.31:0/2961810949 conn(0x7fdf68071d00 msgr2=0x7fdf68065620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.849+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 shutdown_connections 2024-04-24T23:23:26.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.849+0000 7fdf6e283700 1 -- 172.21.15.31:0/2961810949 wait complete. 2024-04-24T23:23:26.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.953+0000 7f94b1902700 1 -- 172.21.15.31:0/1843567662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 msgr2=0x7f94ac0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.953+0000 7f94b1902700 1 --2- 172.21.15.31:0/1843567662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 0x7f94ac0fcdf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f94a0004940 tx=0x7f94a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:26.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.953+0000 7f94b1902700 1 -- 172.21.15.31:0/1843567662 shutdown_connections 2024-04-24T23:23:26.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.953+0000 7f94b1902700 1 --2- 172.21.15.31:0/1843567662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 0x7f94ac0fcdf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.953+0000 7f94b1902700 1 --2- 172.21.15.31:0/1843567662 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ac102de0 0x7f94ac1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.953+0000 7f94b1902700 1 -- 172.21.15.31:0/1843567662 >> 172.21.15.31:0/1843567662 conn(0x7f94ac0f8990 msgr2=0x7f94ac0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:26.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.953+0000 7f94b1902700 1 -- 172.21.15.31:0/1843567662 shutdown_connections 2024-04-24T23:23:26.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.953+0000 7f94b1902700 1 -- 172.21.15.31:0/1843567662 wait complete. 2024-04-24T23:23:26.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.954+0000 7f94b1902700 1 Processor -- start 2024-04-24T23:23:26.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.954+0000 7f94b1902700 1 -- start start 2024-04-24T23:23:26.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94b1902700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ac102de0 0x7f94ac113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94b1902700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 0x7f94ac113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94b1902700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac114490 con 0x7f94ac103780 2024-04-24T23:23:26.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94b1902700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac10da90 con 0x7f94ac102de0 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94aa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 0x7f94ac113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94aa7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 0x7f94ac113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45444/0 (socket says 172.21.15.31:45444) 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94aa7fc700 1 -- 172.21.15.31:0/3651199346 learned_addr learned my addr 172.21.15.31:0/3651199346 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94aaffd700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ac102de0 0x7f94ac113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94aa7fc700 1 -- 172.21.15.31:0/3651199346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ac102de0 msgr2=0x7f94ac113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94aa7fc700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ac102de0 0x7f94ac113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94aa7fc700 1 -- 172.21.15.31:0/3651199346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0005040 con 0x7f94ac103780 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94aa7fc700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 0x7f94ac113e40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009100 tx=0x7f94a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94937fe700 1 -- 172.21.15.31:0/3651199346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a000e040 con 0x7f94ac103780 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94937fe700 1 -- 172.21.15.31:0/3651199346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94a0013420 con 0x7f94ac103780 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ac10dd10 con 0x7f94ac103780 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94937fe700 1 -- 172.21.15.31:0/3651199346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94a001c630 con 0x7f94ac103780 2024-04-24T23:23:26.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.955+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac10e2d0 con 0x7f94ac103780 2024-04-24T23:23:26.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.956+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0fe530 con 0x7f94ac103780 2024-04-24T23:23:26.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.957+0000 7f94937fe700 1 -- 172.21.15.31:0/3651199346 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f94a0013a00 con 0x7f94ac103780 2024-04-24T23:23:26.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.957+0000 7f94937fe700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f949406e140 0x7f94940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:26.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.958+0000 7f94aaffd700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f949406e140 0x7f94940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:26.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.958+0000 7f94937fe700 1 -- 172.21.15.31:0/3651199346 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f94a008eb20 con 0x7f94ac103780 2024-04-24T23:23:26.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.958+0000 7f94aaffd700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f949406e140 0x7f94940705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f949c009fd0 tx=0x7f949c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:26.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:26.963+0000 7f94937fe700 1 -- 172.21.15.31:0/3651199346 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f94a005baf0 con 0x7f94ac103780 2024-04-24T23:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.256+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f94ac109ec0 con 0x7f94ac103780 2024-04-24T23:23:27.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.256+0000 7f94937fe700 1 -- 172.21.15.31:0/3651199346 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f94a005b680 con 0x7f94ac103780 2024-04-24T23:23:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:23:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f949406e140 msgr2=0x7f94940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f949406e140 0x7f94940705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f949c009fd0 tx=0x7f949c00b040 comp rx=0 tx=0).stop 2024-04-24T23:23:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 msgr2=0x7f94ac113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:27.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 0x7f94ac113e40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f94a0009100 tx=0x7f94a0009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 shutdown_connections 2024-04-24T23:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f949406e140 0x7f94940705f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f94ac102de0 0x7f94ac113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 --2- 172.21.15.31:0/3651199346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94ac103780 0x7f94ac113e40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 >> 172.21.15.31:0/3651199346 conn(0x7f94ac0f8990 msgr2=0x7f94ac0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 shutdown_connections 2024-04-24T23:23:27.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:27.258+0000 7f94b1902700 1 -- 172.21.15.31:0/3651199346 wait complete. 2024-04-24T23:23:27.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:27 smithi040 ceph-mon[83047]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:27.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:27 smithi040 ceph-mon[83047]: pgmap v192: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 426 B/s wr, 4 op/s 2024-04-24T23:23:27.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:27 smithi040 ceph-mon[83047]: Upgrade: Updating mds.cephfs.smithi040.dicevu 2024-04-24T23:23:27.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:27 smithi040 ceph-mon[83047]: Deploying daemon mds.cephfs.smithi040.dicevu on smithi040 2024-04-24T23:23:27.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:27 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3651199346' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:23:27.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:27 smithi031 ceph-mon[111404]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:27.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:27 smithi031 ceph-mon[111404]: pgmap v192: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s rd, 426 B/s wr, 4 op/s 2024-04-24T23:23:27.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:27 smithi031 ceph-mon[111404]: Upgrade: Updating mds.cephfs.smithi040.dicevu 2024-04-24T23:23:27.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:27 smithi031 ceph-mon[111404]: Deploying daemon mds.cephfs.smithi040.dicevu on smithi040 2024-04-24T23:23:27.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:27 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3651199346' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:23:28.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:28 smithi040 ceph-mon[83047]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:28.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:28 smithi031 ceph-mon[111404]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:28.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:28 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[111398]: 2024-04-24T23:23:28.676+0000 7fda7a362700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: pgmap v193: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: osdmap e83: 6 total, 6 up, 6 in 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: Standby daemon mds.cephfs.smithi040.osunbp assigned to filesystem cephfs as rank 0 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi040.osunbp=up:replay} 2 up:standby 2024-04-24T23:23:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:29.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: pgmap v193: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.5 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-24T23:23:29.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:23:29.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:29.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: osdmap e83: 6 total, 6 up, 6 in 2024-04-24T23:23:29.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: Standby daemon mds.cephfs.smithi040.osunbp assigned to filesystem cephfs as rank 0 2024-04-24T23:23:29.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:23:29.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T23:23:29.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi040.osunbp=up:replay} 2 up:standby 2024-04-24T23:23:29.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:30.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:30 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:23:30.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:30 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:23:31.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:31 smithi040 ceph-mon[83047]: pgmap v195: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-04-24T23:23:31.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:31 smithi031 ceph-mon[111404]: pgmap v195: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 5.6 KiB/s wr, 7 op/s 2024-04-24T23:23:33.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:33 smithi040 ceph-mon[83047]: pgmap v196: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-04-24T23:23:33.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:33 smithi031 ceph-mon[111404]: pgmap v196: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 6.4 MiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-04-24T23:23:35.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:35 smithi040 ceph-mon[83047]: pgmap v197: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-04-24T23:23:35.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:35 smithi031 ceph-mon[111404]: pgmap v197: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-04-24T23:23:37.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:37 smithi040 ceph-mon[83047]: pgmap v198: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-24T23:23:37.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:37 smithi031 ceph-mon[111404]: pgmap v198: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-24T23:23:39.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:39 smithi040 ceph-mon[83047]: pgmap v199: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-24T23:23:39.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:39 smithi040 ceph-mon[83047]: reconnect by client.24327 192.168.0.1:0/3971968853 after 0.000999987 2024-04-24T23:23:39.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:39 smithi040 ceph-mon[83047]: reconnect by client.14518 192.168.0.1:0/2806865259 after 0.000999987 2024-04-24T23:23:39.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:39 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:reconnect 2024-04-24T23:23:39.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:39 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi040.osunbp=up:reconnect} 2 up:standby 2024-04-24T23:23:39.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:39 smithi031 ceph-mon[111404]: pgmap v199: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-24T23:23:39.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:39 smithi031 ceph-mon[111404]: reconnect by client.24327 192.168.0.1:0/3971968853 after 0.000999987 2024-04-24T23:23:39.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:39 smithi031 ceph-mon[111404]: reconnect by client.14518 192.168.0.1:0/2806865259 after 0.000999987 2024-04-24T23:23:39.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:39 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:reconnect 2024-04-24T23:23:39.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:39 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi040.osunbp=up:reconnect} 2 up:standby 2024-04-24T23:23:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:40 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:rejoin 2024-04-24T23:23:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:40 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.40:6824/3497670820,v1:172.21.15.40:6825/3497670820] up:boot 2024-04-24T23:23:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:40 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi040.osunbp=up:rejoin} 3 up:standby 2024-04-24T23:23:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:40 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:23:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:40 smithi040 ceph-mon[83047]: daemon mds.cephfs.smithi040.osunbp is now active in filesystem cephfs as rank 0 2024-04-24T23:23:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:40 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:40 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:40.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:40 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:40.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:40 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:rejoin 2024-04-24T23:23:40.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:40 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.40:6824/3497670820,v1:172.21.15.40:6825/3497670820] up:boot 2024-04-24T23:23:40.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:40 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi040.osunbp=up:rejoin} 3 up:standby 2024-04-24T23:23:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:40 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:23:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:40 smithi031 ceph-mon[111404]: daemon mds.cephfs.smithi040.osunbp is now active in filesystem cephfs as rank 0 2024-04-24T23:23:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:40 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:40 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:40.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:40 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:41.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:41 smithi040 ceph-mon[83047]: pgmap v200: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T23:23:41.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:41 smithi040 ceph-mon[83047]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:23:41.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:41 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:23:41.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:41 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:active 2024-04-24T23:23:41.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:41 smithi040 ceph-mon[83047]: fsmap cephfs:1 {0=cephfs.smithi040.osunbp=up:active} 3 up:standby 2024-04-24T23:23:41.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:41 smithi031 ceph-mon[111404]: pgmap v200: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-04-24T23:23:41.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:41 smithi031 ceph-mon[111404]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:23:41.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:41 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:23:41.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:41 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.40:6826/1172521773,v1:172.21.15.40:6827/1172521773] up:active 2024-04-24T23:23:41.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:41 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi040.osunbp=up:active} 3 up:standby 2024-04-24T23:23:43.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:43 smithi031 ceph-mon[111404]: pgmap v201: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T23:23:43.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:43.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:43.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:43 smithi040 ceph-mon[83047]: pgmap v201: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 6 op/s 2024-04-24T23:23:43.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:43.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:44 smithi040 ceph-mon[83047]: pgmap v202: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T23:23:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:45.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:45.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:45.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:45.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:44 smithi031 ceph-mon[111404]: pgmap v202: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-24T23:23:45.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:45.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:47.045 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:46 smithi040 ceph-mon[83047]: pgmap v203: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-24T23:23:47.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:46 smithi031 ceph-mon[111404]: pgmap v203: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 7.2 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:48.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:48 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:48.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:48.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:48 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:49.727 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:49 smithi040 ceph-mon[83047]: pgmap v204: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-24T23:23:49.728 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:49 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:49.728 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:49 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.osunbp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:23:49.728 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:49 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:49.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:49 smithi031 ceph-mon[111404]: pgmap v204: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 4.3 KiB/s wr, 6 op/s 2024-04-24T23:23:49.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:49 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:49.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:49 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi040.osunbp", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-24T23:23:49.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:49 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:23:50.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:50 smithi031 ceph-mon[111404]: Upgrade: Updating mds.cephfs.smithi040.osunbp 2024-04-24T23:23:50.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:50 smithi031 ceph-mon[111404]: Deploying daemon mds.cephfs.smithi040.osunbp on smithi040 2024-04-24T23:23:50.791 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:50 smithi040 ceph-mon[83047]: Upgrade: Updating mds.cephfs.smithi040.osunbp 2024-04-24T23:23:50.791 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:50 smithi040 ceph-mon[83047]: Deploying daemon mds.cephfs.smithi040.osunbp on smithi040 2024-04-24T23:23:51.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[111398]: 2024-04-24T23:23:51.321+0000 7fda7a362700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-mon[111404]: pgmap v205: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-24T23:23:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-mon[111404]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:23:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-mon[111404]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-mon[111404]: osdmap e84: 6 total, 6 up, 6 in 2024-04-24T23:23:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-mon[111404]: Standby daemon mds.cephfs.smithi031.xxejxr assigned to filesystem cephfs as rank 0 2024-04-24T23:23:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-mon[111404]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:23:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-mon[111404]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T23:23:51.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:51 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi031.xxejxr=up:replay} 2 up:standby 2024-04-24T23:23:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:51 smithi040 ceph-mon[83047]: pgmap v205: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-24T23:23:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:51 smithi040 ceph-mon[83047]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-24T23:23:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:51 smithi040 ceph-mon[83047]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-24T23:23:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:51 smithi040 ceph-mon[83047]: osdmap e84: 6 total, 6 up, 6 in 2024-04-24T23:23:51.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:51 smithi040 ceph-mon[83047]: Standby daemon mds.cephfs.smithi031.xxejxr assigned to filesystem cephfs as rank 0 2024-04-24T23:23:51.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:51 smithi040 ceph-mon[83047]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-24T23:23:51.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:51 smithi040 ceph-mon[83047]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-24T23:23:51.913 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:51 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi031.xxejxr=up:replay} 2 up:standby 2024-04-24T23:23:53.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:53 smithi031 ceph-mon[111404]: pgmap v207: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 412 KiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-04-24T23:23:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:53 smithi040 ceph-mon[83047]: pgmap v207: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 412 KiB/s rd, 5.1 KiB/s wr, 4 op/s 2024-04-24T23:23:55.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:55 smithi031 ceph-mon[111404]: pgmap v208: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 5.2 MiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-04-24T23:23:55.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:55 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:23:55.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:55 smithi040 ceph-mon[83047]: pgmap v208: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 5.2 MiB/s rd, 5.0 KiB/s wr, 4 op/s 2024-04-24T23:23:55.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:55 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:23:57.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.367+0000 7f15fbc37700 1 -- 172.21.15.31:0/1644203278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4101cd0 msgr2=0x7f15f41020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.367+0000 7f15fbc37700 1 --2- 172.21.15.31:0/1644203278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4101cd0 0x7f15f41020a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15f0004940 tx=0x7f15f0009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.367+0000 7f15fbc37700 1 -- 172.21.15.31:0/1644203278 shutdown_connections 2024-04-24T23:23:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.367+0000 7f15fbc37700 1 --2- 172.21.15.31:0/1644203278 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f4102670 0x7f15f40fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.367+0000 7f15fbc37700 1 --2- 172.21.15.31:0/1644203278 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4101cd0 0x7f15f41020a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.367+0000 7f15fbc37700 1 -- 172.21.15.31:0/1644203278 >> 172.21.15.31:0/1644203278 conn(0x7f15f4071460 msgr2=0x7f15f4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.367+0000 7f15fbc37700 1 -- 172.21.15.31:0/1644203278 shutdown_connections 2024-04-24T23:23:57.369 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.367+0000 7f15fbc37700 1 -- 172.21.15.31:0/1644203278 wait complete. 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.368+0000 7f15fbc37700 1 Processor -- start 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.368+0000 7f15fbc37700 1 -- start start 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15fbc37700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f4101cd0 0x7f15f4115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15fbc37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4102670 0x7f15f41162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15fbc37700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4116900 con 0x7f15f4102670 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15fbc37700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f410fcd0 con 0x7f15f4101cd0 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15f91d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4102670 0x7f15f41162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15f91d2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4102670 0x7f15f41162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51734/0 (socket says 172.21.15.31:51734) 2024-04-24T23:23:57.371 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15f91d2700 1 -- 172.21.15.31:0/2077667765 learned_addr learned my addr 172.21.15.31:0/2077667765 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15f99d3700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f4101cd0 0x7f15f4115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15f91d2700 1 -- 172.21.15.31:0/2077667765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f4101cd0 msgr2=0x7f15f4115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15f91d2700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f4101cd0 0x7f15f4115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15f91d2700 1 -- 172.21.15.31:0/2077667765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0005040 con 0x7f15f4102670 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.369+0000 7f15f91d2700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4102670 0x7f15f41162b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c810 tx=0x7f15e40043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.370+0000 7f15ea7fc700 1 -- 172.21.15.31:0/2077667765 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e4012810 con 0x7f15f4102670 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.370+0000 7f15ea7fc700 1 -- 172.21.15.31:0/2077667765 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15e4012e40 con 0x7f15f4102670 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.370+0000 7f15ea7fc700 1 -- 172.21.15.31:0/2077667765 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15e40102e0 con 0x7f15f4102670 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.370+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f410ffb0 con 0x7f15f4102670 2024-04-24T23:23:57.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.370+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f4110520 con 0x7f15f4102670 2024-04-24T23:23:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.371+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f4101130 con 0x7f15f4102670 2024-04-24T23:23:57.373 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.372+0000 7f15ea7fc700 1 -- 172.21.15.31:0/2077667765 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f15e4011380 con 0x7f15f4102670 2024-04-24T23:23:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.372+0000 7f15ea7fc700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f15e006e150 0x7f15e0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.372+0000 7f15ea7fc700 1 -- 172.21.15.31:0/2077667765 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f15e408ddf0 con 0x7f15f4102670 2024-04-24T23:23:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.372+0000 7f15f99d3700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f15e006e150 0x7f15e0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.374 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.373+0000 7f15f99d3700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f15e006e150 0x7f15e0070600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.377+0000 7f15ea7fc700 1 -- 172.21.15.31:0/2077667765 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f15e405abf0 con 0x7f15f4102670 2024-04-24T23:23:57.612 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.611+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15f4111460 con 0x7f15e006e150 2024-04-24T23:23:57.614 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.613+0000 7f15ea7fc700 1 -- 172.21.15.31:0/2077667765 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f15f4111460 con 0x7f15e006e150 2024-04-24T23:23:57.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f15e006e150 msgr2=0x7f15e0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f15e006e150 0x7f15e0070600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f15f0000c00 tx=0x7f15f0009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:57.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4102670 msgr2=0x7f15f41162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4102670 0x7f15f41162b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f15e400c810 tx=0x7f15e40043f0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 shutdown_connections 2024-04-24T23:23:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f15e006e150 0x7f15e0070600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f15f4101cd0 0x7f15f4115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 --2- 172.21.15.31:0/2077667765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15f4102670 0x7f15f41162b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 >> 172.21.15.31:0/2077667765 conn(0x7f15f4071460 msgr2=0x7f15f40fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 shutdown_connections 2024-04-24T23:23:57.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.615+0000 7f15fbc37700 1 -- 172.21.15.31:0/2077667765 wait complete. 2024-04-24T23:23:57.626 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:23:57.627 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:57 smithi031 ceph-mon[111404]: pgmap v209: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 8.8 MiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-24T23:23:57.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.722+0000 7fe4ded99700 1 -- 172.21.15.31:0/224417155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 msgr2=0x7fe4d8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.722+0000 7fe4ded99700 1 --2- 172.21.15.31:0/224417155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 0x7fe4d8109350 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4004940 tx=0x7fe4c4009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:57.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.722+0000 7fe4ded99700 1 -- 172.21.15.31:0/224417155 shutdown_connections 2024-04-24T23:23:57.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.722+0000 7fe4ded99700 1 --2- 172.21.15.31:0/224417155 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 0x7fe4d8109350 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.722+0000 7fe4ded99700 1 --2- 172.21.15.31:0/224417155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80fc790 0x7fe4d80fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.722+0000 7fe4ded99700 1 -- 172.21.15.31:0/224417155 >> 172.21.15.31:0/224417155 conn(0x7fe4d8071d00 msgr2=0x7fe4d8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.722+0000 7fe4ded99700 1 -- 172.21.15.31:0/224417155 shutdown_connections 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.722+0000 7fe4ded99700 1 -- 172.21.15.31:0/224417155 wait complete. 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4ded99700 1 Processor -- start 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4ded99700 1 -- start start 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4ded99700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80fc790 0x7fe4d8115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4ded99700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 0x7fe4d8116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4ded99700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d8116990 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4ded99700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d810fdb0 con 0x7fe4d80fc790 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 0x7fe4d8116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4d7fff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 0x7fe4d8116340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51742/0 (socket says 172.21.15.31:51742) 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4d7fff700 1 -- 172.21.15.31:0/2659674084 learned_addr learned my addr 172.21.15.31:0/2659674084 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.723+0000 7fe4dcb35700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80fc790 0x7fe4d8115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4d7fff700 1 -- 172.21.15.31:0/2659674084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80fc790 msgr2=0x7fe4d8115e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4d7fff700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80fc790 0x7fe4d8115e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4d7fff700 1 -- 172.21.15.31:0/2659674084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c4005040 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4d7fff700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 0x7fe4d8116340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009150 tx=0x7fe4c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4d57fa700 1 -- 172.21.15.31:0/2659674084 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c400e040 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d8110030 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4d57fa700 1 -- 172.21.15.31:0/2659674084 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4c4013420 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4d57fa700 1 -- 172.21.15.31:0/2659674084 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4c401c610 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.724+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d8110570 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.725+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d80fc330 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.726+0000 7fe4d57fa700 1 -- 172.21.15.31:0/2659674084 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe4c400baf0 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.727+0000 7fe4d57fa700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe4c806e150 0x7fe4c8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:57.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.727+0000 7fe4d57fa700 1 -- 172.21.15.31:0/2659674084 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe4c408f7c0 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.727+0000 7fe4dcb35700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe4c806e150 0x7fe4c8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:57.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.728+0000 7fe4dcb35700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe4c806e150 0x7fe4c8070600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc004f40 tx=0x7fe4cc00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:57.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.735+0000 7fe4d57fa700 1 -- 172.21.15.31:0/2659674084 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe4c405c6e0 con 0x7fe4d80fd0a0 2024-04-24T23:23:57.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:57 smithi040 ceph-mon[83047]: pgmap v209: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 8.8 MiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-04-24T23:23:57.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.963+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4d810c170 con 0x7fe4c806e150 2024-04-24T23:23:57.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.964+0000 7fe4d57fa700 1 -- 172.21.15.31:0/2659674084 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe4d810c170 con 0x7fe4c806e150 2024-04-24T23:23:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe4c806e150 msgr2=0x7fe4c8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe4c806e150 0x7fe4c8070600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc004f40 tx=0x7fe4cc00b040 comp rx=0 tx=0).stop 2024-04-24T23:23:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 msgr2=0x7fe4d8116340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 0x7fe4d8116340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4009150 tx=0x7fe4c4009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 shutdown_connections 2024-04-24T23:23:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe4c806e150 0x7fe4c8070600 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4d80fc790 0x7fe4d8115e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 --2- 172.21.15.31:0/2659674084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4d80fd0a0 0x7fe4d8116340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 >> 172.21.15.31:0/2659674084 conn(0x7fe4d8071d00 msgr2=0x7fe4d80f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 shutdown_connections 2024-04-24T23:23:57.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:57.967+0000 7fe4ded99700 1 -- 172.21.15.31:0/2659674084 wait complete. 2024-04-24T23:23:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.072+0000 7f4a29490700 1 -- 172.21.15.31:0/963404370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 msgr2=0x7f4a24103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.072+0000 7f4a29490700 1 --2- 172.21.15.31:0/963404370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 0x7f4a24103220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4a14004940 tx=0x7f4a14009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.073+0000 7f4a29490700 1 -- 172.21.15.31:0/963404370 shutdown_connections 2024-04-24T23:23:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.073+0000 7f4a29490700 1 --2- 172.21.15.31:0/963404370 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a24065640 0x7f4a24065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.073+0000 7f4a29490700 1 --2- 172.21.15.31:0/963404370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 0x7f4a24103220 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.073+0000 7f4a29490700 1 -- 172.21.15.31:0/963404370 >> 172.21.15.31:0/963404370 conn(0x7f4a24071d00 msgr2=0x7f4a24072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.073+0000 7f4a29490700 1 -- 172.21.15.31:0/963404370 shutdown_connections 2024-04-24T23:23:58.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.073+0000 7f4a29490700 1 -- 172.21.15.31:0/963404370 wait complete. 2024-04-24T23:23:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a29490700 1 Processor -- start 2024-04-24T23:23:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a29490700 1 -- start start 2024-04-24T23:23:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a29490700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a24065640 0x7f4a24113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a29490700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 0x7f4a241140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a29490700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24114710 con 0x7f4a24102e50 2024-04-24T23:23:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a29490700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2410db00 con 0x7f4a24065640 2024-04-24T23:23:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a227fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 0x7f4a241140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a227fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 0x7f4a241140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51768/0 (socket says 172.21.15.31:51768) 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a227fc700 1 -- 172.21.15.31:0/3543727797 learned_addr learned my addr 172.21.15.31:0/3543727797 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a22ffd700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a24065640 0x7f4a24113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a227fc700 1 -- 172.21.15.31:0/3543727797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a24065640 msgr2=0x7f4a24113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.074+0000 7f4a227fc700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a24065640 0x7f4a24113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.075+0000 7f4a227fc700 1 -- 172.21.15.31:0/3543727797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14005040 con 0x7f4a24102e50 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.075+0000 7f4a227fc700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 0x7f4a241140c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c8c0 tx=0x7f4a180044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.075+0000 7f4a0b7fe700 1 -- 172.21.15.31:0/3543727797 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a1800a760 con 0x7f4a24102e50 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.075+0000 7f4a0b7fe700 1 -- 172.21.15.31:0/3543727797 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a1800ad90 con 0x7f4a24102e50 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.075+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2410dde0 con 0x7f4a24102e50 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.075+0000 7f4a0b7fe700 1 -- 172.21.15.31:0/3543727797 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a180113a0 con 0x7f4a24102e50 2024-04-24T23:23:58.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.075+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2410e3a0 con 0x7f4a24102e50 2024-04-24T23:23:58.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.076+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a24100100 con 0x7f4a24102e50 2024-04-24T23:23:58.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.077+0000 7f4a0b7fe700 1 -- 172.21.15.31:0/3543727797 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4a18012400 con 0x7f4a24102e50 2024-04-24T23:23:58.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.077+0000 7f4a0b7fe700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4a0c06e140 0x7f4a0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.077+0000 7f4a0b7fe700 1 -- 172.21.15.31:0/3543727797 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4a1808dc10 con 0x7f4a24102e50 2024-04-24T23:23:58.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.077+0000 7f4a22ffd700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4a0c06e140 0x7f4a0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.078+0000 7f4a22ffd700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4a0c06e140 0x7f4a0c0705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4a14004f90 tx=0x7f4a14009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.087 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.086+0000 7f4a0b7fe700 1 -- 172.21.15.31:0/3543727797 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4a1805aa90 con 0x7f4a24102e50 2024-04-24T23:23:58.311 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.310+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a2410f1d0 con 0x7f4a0c06e140 2024-04-24T23:23:58.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.330+0000 7f4a0b7fe700 1 -- 172.21.15.31:0/3543727797 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f4a2410f1d0 con 0x7f4a0c06e140 2024-04-24T23:23:58.331 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:23:58.331 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (7m) 38s ago 18m 25.6M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (4m) 38s ago 18m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (4m) 15s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (6m) 38s ago 18m 79.7M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (43s) 38s ago 13m 11.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 956ce7ff5df1 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (65s) 38s ago 13m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1d7ff0e06219 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (22s) 15s ago 13m 13.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 4ee160da943d 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (13m) 15s ago 13m 107M - 17.2.7-902-gc61472c8 0de407c1b3af a1025ab17f85 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (9m) 38s ago 20m 507M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (8m) 15s ago 17m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:23:58.332 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (5m) 38s ago 20m 52.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (5m) 15s ago 17m 41.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 38s ago 18m 16.8M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (7m) 15s ago 17m 14.9M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (4m) 38s ago 16m 96.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (3m) 38s ago 16m 111M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (3m) 38s ago 15m 91.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 271a0764ef5c 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (2m) 15s ago 15m 128M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 97c6f05632f6 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (117s) 15s ago 15m 102M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 85d9bab7b352 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (83s) 15s ago 14m 95.2M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 32b6ff45fea8 2024-04-24T23:23:58.333 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (7m) 38s ago 18m 60.1M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:23:58.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4a0c06e140 msgr2=0x7f4a0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4a0c06e140 0x7f4a0c0705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4a14004f90 tx=0x7f4a14009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:58.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 msgr2=0x7f4a241140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 0x7f4a241140c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c8c0 tx=0x7f4a180044c0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 shutdown_connections 2024-04-24T23:23:58.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4a0c06e140 0x7f4a0c0705f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4a24065640 0x7f4a24113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 --2- 172.21.15.31:0/3543727797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a24102e50 0x7f4a241140c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 >> 172.21.15.31:0/3543727797 conn(0x7f4a24071d00 msgr2=0x7f4a240fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 shutdown_connections 2024-04-24T23:23:58.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.332+0000 7f4a29490700 1 -- 172.21.15.31:0/3543727797 wait complete. 2024-04-24T23:23:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.436+0000 7f88c023d700 1 -- 172.21.15.31:0/986899189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 msgr2=0x7f88b8102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.436+0000 7f88c023d700 1 --2- 172.21.15.31:0/986899189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 0x7f88b8102160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f88b4004940 tx=0x7f88b4009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.436+0000 7f88c023d700 1 -- 172.21.15.31:0/986899189 shutdown_connections 2024-04-24T23:23:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.436+0000 7f88c023d700 1 --2- 172.21.15.31:0/986899189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b80fbd40 0x7f88b80fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.436+0000 7f88c023d700 1 --2- 172.21.15.31:0/986899189 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 0x7f88b8102160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.436+0000 7f88c023d700 1 -- 172.21.15.31:0/986899189 >> 172.21.15.31:0/986899189 conn(0x7f88b8071d00 msgr2=0x7f88b8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.437+0000 7f88c023d700 1 -- 172.21.15.31:0/986899189 shutdown_connections 2024-04-24T23:23:58.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.437+0000 7f88c023d700 1 -- 172.21.15.31:0/986899189 wait complete. 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.437+0000 7f88c023d700 1 Processor -- start 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.437+0000 7f88c023d700 1 -- start start 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.437+0000 7f88c023d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b80fbd40 0x7f88b806f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.437+0000 7f88c023d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 0x7f88b806fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.437+0000 7f88c023d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b806a460 con 0x7f88b8101d90 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.437+0000 7f88c023d700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b806a5a0 con 0x7f88b80fbd40 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88bd7d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 0x7f88b806fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88bd7d8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 0x7f88b806fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51784/0 (socket says 172.21.15.31:51784) 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88bd7d8700 1 -- 172.21.15.31:0/4143652491 learned_addr learned my addr 172.21.15.31:0/4143652491 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:58.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88bdfd9700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b80fbd40 0x7f88b806f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88bd7d8700 1 -- 172.21.15.31:0/4143652491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b80fbd40 msgr2=0x7f88b806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88bd7d8700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b80fbd40 0x7f88b806f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88bd7d8700 1 -- 172.21.15.31:0/4143652491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4005040 con 0x7f88b8101d90 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88bd7d8700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 0x7f88b806fa30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f88a8002a50 tx=0x7f88a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88ae7fc700 1 -- 172.21.15.31:0/4143652491 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88a8007cc0 con 0x7f88b8101d90 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88ae7fc700 1 -- 172.21.15.31:0/4143652491 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88a800f040 con 0x7f88b8101d90 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b806a880 con 0x7f88b8101d90 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88ae7fc700 1 -- 172.21.15.31:0/4143652491 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88a800da30 con 0x7f88b8101d90 2024-04-24T23:23:58.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.438+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b806ae40 con 0x7f88b8101d90 2024-04-24T23:23:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.440+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b80fd430 con 0x7f88b8101d90 2024-04-24T23:23:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.440+0000 7f88ae7fc700 1 -- 172.21.15.31:0/4143652491 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f88a800dbb0 con 0x7f88b8101d90 2024-04-24T23:23:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.441+0000 7f88ae7fc700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f88a406e140 0x7f88a40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.441+0000 7f88ae7fc700 1 -- 172.21.15.31:0/4143652491 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f88a808d970 con 0x7f88b8101d90 2024-04-24T23:23:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.441+0000 7f88bdfd9700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f88a406e140 0x7f88a40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.441+0000 7f88bdfd9700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f88a406e140 0x7f88a40705f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88b4004560 tx=0x7f88b4009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.446+0000 7f88ae7fc700 1 -- 172.21.15.31:0/4143652491 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f88a8056c40 con 0x7f88b8101d90 2024-04-24T23:23:58.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:58 smithi031 ceph-mon[111404]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:58.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.762+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f88b806bb80 con 0x7f88b8101d90 2024-04-24T23:23:58.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.762+0000 7f88ae7fc700 1 -- 172.21.15.31:0/4143652491 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f88a805a300 con 0x7f88b8101d90 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T23:23:58.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:23:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:23:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-24T23:23:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:23:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:23:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-24T23:23:58.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:23:58.767 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:23:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f88a406e140 msgr2=0x7f88a40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f88a406e140 0x7f88a40705f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88b4004560 tx=0x7f88b4009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 msgr2=0x7f88b806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 0x7f88b806fa30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f88a8002a50 tx=0x7f88a80046b0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 shutdown_connections 2024-04-24T23:23:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f88a406e140 0x7f88a40705f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f88b80fbd40 0x7f88b806f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 --2- 172.21.15.31:0/4143652491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88b8101d90 0x7f88b806fa30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 >> 172.21.15.31:0/4143652491 conn(0x7f88b8071d00 msgr2=0x7f88b80fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 shutdown_connections 2024-04-24T23:23:58.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.766+0000 7f88c023d700 1 -- 172.21.15.31:0/4143652491 wait complete. 2024-04-24T23:23:58.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.873+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1614661201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 msgr2=0x7f7a48102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.873+0000 7f7a4fa98700 1 --2- 172.21.15.31:0/1614661201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 0x7f7a48102160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7a44004940 tx=0x7f7a44009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.874+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1614661201 shutdown_connections 2024-04-24T23:23:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.874+0000 7f7a4fa98700 1 --2- 172.21.15.31:0/1614661201 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a480fbd40 0x7f7a480fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.874+0000 7f7a4fa98700 1 --2- 172.21.15.31:0/1614661201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 0x7f7a48102160 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.874+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1614661201 >> 172.21.15.31:0/1614661201 conn(0x7f7a48071d00 msgr2=0x7f7a48072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:58.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.874+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1614661201 shutdown_connections 2024-04-24T23:23:58.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.874+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1614661201 wait complete. 2024-04-24T23:23:58.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.874+0000 7f7a4fa98700 1 Processor -- start 2024-04-24T23:23:58.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.874+0000 7f7a4fa98700 1 -- start start 2024-04-24T23:23:58.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4fa98700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a480fbd40 0x7f7a48113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4fa98700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 0x7f7a48114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4fa98700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48114770 con 0x7f7a48101d90 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4fa98700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4810dad0 con 0x7f7a480fbd40 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d033700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 0x7f7a48114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d033700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 0x7f7a48114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42088/0 (socket says 172.21.15.31:42088) 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d033700 1 -- 172.21.15.31:0/1187441799 learned_addr learned my addr 172.21.15.31:0/1187441799 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d834700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a480fbd40 0x7f7a48113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d033700 1 -- 172.21.15.31:0/1187441799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a480fbd40 msgr2=0x7f7a48113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d033700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a480fbd40 0x7f7a48113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d033700 1 -- 172.21.15.31:0/1187441799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a44005040 con 0x7f7a48101d90 2024-04-24T23:23:58.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d834700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a480fbd40 0x7f7a48113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:23:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.875+0000 7f7a4d033700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 0x7f7a48114090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800b790 tx=0x7f7a38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.876+0000 7f7a3e7fc700 1 -- 172.21.15.31:0/1187441799 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3800f770 con 0x7f7a48101d90 2024-04-24T23:23:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.876+0000 7f7a3e7fc700 1 -- 172.21.15.31:0/1187441799 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a3800fda0 con 0x7f7a48101d90 2024-04-24T23:23:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.876+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4810ddb0 con 0x7f7a48101d90 2024-04-24T23:23:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.876+0000 7f7a3e7fc700 1 -- 172.21.15.31:0/1187441799 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3800d310 con 0x7f7a48101d90 2024-04-24T23:23:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.876+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4810e370 con 0x7f7a48101d90 2024-04-24T23:23:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.877+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a480fd430 con 0x7f7a48101d90 2024-04-24T23:23:58.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.877+0000 7f7a3e7fc700 1 -- 172.21.15.31:0/1187441799 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7a38010420 con 0x7f7a48101d90 2024-04-24T23:23:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.878+0000 7f7a3e7fc700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7a3406e140 0x7f7a340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.878+0000 7f7a3e7fc700 1 -- 172.21.15.31:0/1187441799 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7a3808db20 con 0x7f7a48101d90 2024-04-24T23:23:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.878+0000 7f7a4d834700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7a3406e140 0x7f7a340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:58.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.879+0000 7f7a4d834700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7a3406e140 0x7f7a340705f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009fd0 tx=0x7f7a44009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:58.884 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:58.883+0000 7f7a3e7fc700 1 -- 172.21.15.31:0/1187441799 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7a3805aa40 con 0x7f7a48101d90 2024-04-24T23:23:58.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:58 smithi040 ceph-mon[83047]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:59.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.136+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7a4805f5c0 con 0x7f7a48101d90 2024-04-24T23:23:59.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.136+0000 7f7a3e7fc700 1 -- 172.21.15.31:0/1187441799 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1749 (secure 0 0 0) 0x7f7a3805a5d0 con 0x7f7a48101d90 2024-04-24T23:23:59.138 INFO:teuthology.orchestra.run.smithi031.stdout:e30 2024-04-24T23:23:59.138 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:23:59.138 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:23:59.138 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 30 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:23:51.327504+0000 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:23:59.139 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 84 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34386} 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:23:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{0:34386} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:23:59.141 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:23:59.142 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/3258483612,v1:172.21.15.31:6829/3258483612] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:23:59.142 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:44317} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6824/3497670820,v1:172.21.15.40:6825/3497670820] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7a3406e140 msgr2=0x7f7a340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7a3406e140 0x7f7a340705f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a44009fd0 tx=0x7f7a44009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 msgr2=0x7f7a48114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 0x7f7a48114090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800b790 tx=0x7f7a38009250 comp rx=0 tx=0).stop 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 shutdown_connections 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f7a3406e140 0x7f7a340705f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a480fbd40 0x7f7a48113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 --2- 172.21.15.31:0/1187441799 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48101d90 0x7f7a48114090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 >> 172.21.15.31:0/1187441799 conn(0x7f7a48071d00 msgr2=0x7f7a480fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.145+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 shutdown_connections 2024-04-24T23:23:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.146+0000 7f7a4fa98700 1 -- 172.21.15.31:0/1187441799 wait complete. 2024-04-24T23:23:59.148 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 30 2024-04-24T23:23:59.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.251+0000 7f74ed118700 1 -- 172.21.15.31:0/1053792962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 msgr2=0x7f74e8065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.251+0000 7f74ed118700 1 --2- 172.21.15.31:0/1053792962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 0x7f74e8065ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f74dc004940 tx=0x7f74dc009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:59.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.251+0000 7f74ed118700 1 -- 172.21.15.31:0/1053792962 shutdown_connections 2024-04-24T23:23:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.251+0000 7f74ed118700 1 --2- 172.21.15.31:0/1053792962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 0x7f74e8065ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.251+0000 7f74ed118700 1 --2- 172.21.15.31:0/1053792962 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74e8102e50 0x7f74e8103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.251+0000 7f74ed118700 1 -- 172.21.15.31:0/1053792962 >> 172.21.15.31:0/1053792962 conn(0x7f74e8071d00 msgr2=0x7f74e8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.251+0000 7f74ed118700 1 -- 172.21.15.31:0/1053792962 shutdown_connections 2024-04-24T23:23:59.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.251+0000 7f74ed118700 1 -- 172.21.15.31:0/1053792962 wait complete. 2024-04-24T23:23:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74ed118700 1 Processor -- start 2024-04-24T23:23:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74ed118700 1 -- start start 2024-04-24T23:23:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74ed118700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 0x7f74e8113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74ed118700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74e8102e50 0x7f74e81140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74ed118700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e8114700 con 0x7f74e8065640 2024-04-24T23:23:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74ed118700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e810daf0 con 0x7f74e8102e50 2024-04-24T23:23:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74e6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 0x7f74e8113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.254 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74e6d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 0x7f74e8113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42116/0 (socket says 172.21.15.31:42116) 2024-04-24T23:23:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74e6d9d700 1 -- 172.21.15.31:0/1719794672 learned_addr learned my addr 172.21.15.31:0/1719794672 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.252+0000 7f74e659c700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74e8102e50 0x7f74e81140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74e6d9d700 1 -- 172.21.15.31:0/1719794672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74e8102e50 msgr2=0x7f74e81140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74e6d9d700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74e8102e50 0x7f74e81140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74e6d9d700 1 -- 172.21.15.31:0/1719794672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc005040 con 0x7f74e8065640 2024-04-24T23:23:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74e6d9d700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 0x7f74e8113b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74d800c830 tx=0x7f74d8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74cf7fe700 1 -- 172.21.15.31:0/1719794672 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d80106f0 con 0x7f74e8065640 2024-04-24T23:23:59.255 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e810ddd0 con 0x7f74e8065640 2024-04-24T23:23:59.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74cf7fe700 1 -- 172.21.15.31:0/1719794672 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74d8010d20 con 0x7f74e8065640 2024-04-24T23:23:59.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74cf7fe700 1 -- 172.21.15.31:0/1719794672 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74d80113a0 con 0x7f74e8065640 2024-04-24T23:23:59.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.253+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e810e200 con 0x7f74e8065640 2024-04-24T23:23:59.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.254+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e8100100 con 0x7f74e8065640 2024-04-24T23:23:59.256 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.255+0000 7f74cf7fe700 1 -- 172.21.15.31:0/1719794672 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f74d8012400 con 0x7f74e8065640 2024-04-24T23:23:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.256+0000 7f74cf7fe700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f74d006e140 0x7f74d00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.256+0000 7f74cf7fe700 1 -- 172.21.15.31:0/1719794672 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f74d808db20 con 0x7f74e8065640 2024-04-24T23:23:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.256+0000 7f74e659c700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f74d006e140 0x7f74d00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.256+0000 7f74e659c700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f74d006e140 0x7f74d00705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f74e810edd0 tx=0x7f74dc009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.261+0000 7f74cf7fe700 1 -- 172.21.15.31:0/1719794672 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f74d805a9a0 con 0x7f74e8065640 2024-04-24T23:23:59.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.488+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74e8109f70 con 0x7f74d006e140 2024-04-24T23:23:59.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.490+0000 7f74cf7fe700 1 -- 172.21.15.31:0/1719794672 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f74e8109f70 con 0x7f74d006e140 2024-04-24T23:23:59.491 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:23:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-24T23:23:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-04-24T23:23:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-24T23:23:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-04-24T23:23:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "osd", 2024-04-24T23:23:59.491 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-04-24T23:23:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-04-24T23:23:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "mon" 2024-04-24T23:23:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-04-24T23:23:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "15/21 daemons upgraded", 2024-04-24T23:23:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mds daemons", 2024-04-24T23:23:59.492 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:23:59.492 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f74d006e140 msgr2=0x7f74d00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f74d006e140 0x7f74d00705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f74e810edd0 tx=0x7f74dc009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 msgr2=0x7f74e8113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 0x7f74e8113b70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74d800c830 tx=0x7f74d8004470 comp rx=0 tx=0).stop 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 shutdown_connections 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74e8065640 0x7f74e8113b70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f74d006e140 0x7f74d00705f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 --2- 172.21.15.31:0/1719794672 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f74e8102e50 0x7f74e81140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 >> 172.21.15.31:0/1719794672 conn(0x7f74e8071d00 msgr2=0x7f74e80fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 shutdown_connections 2024-04-24T23:23:59.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.492+0000 7f74ed118700 1 -- 172.21.15.31:0/1719794672 wait complete. 2024-04-24T23:23:59.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.596+0000 7fc905c21700 1 -- 172.21.15.31:0/706068861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900065640 msgr2=0x7fc900065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.596+0000 7fc905c21700 1 --2- 172.21.15.31:0/706068861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900065640 0x7fc900065ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4004940 tx=0x7fc8f4009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:59.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.597+0000 7fc905c21700 1 -- 172.21.15.31:0/706068861 shutdown_connections 2024-04-24T23:23:59.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.597+0000 7fc905c21700 1 --2- 172.21.15.31:0/706068861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900065640 0x7fc900065ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.597+0000 7fc905c21700 1 --2- 172.21.15.31:0/706068861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900102e50 0x7fc900103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.597+0000 7fc905c21700 1 -- 172.21.15.31:0/706068861 >> 172.21.15.31:0/706068861 conn(0x7fc900071d00 msgr2=0x7fc900072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.597+0000 7fc905c21700 1 -- 172.21.15.31:0/706068861 shutdown_connections 2024-04-24T23:23:59.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.597+0000 7fc905c21700 1 -- 172.21.15.31:0/706068861 wait complete. 2024-04-24T23:23:59.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.597+0000 7fc905c21700 1 Processor -- start 2024-04-24T23:23:59.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc905c21700 1 -- start start 2024-04-24T23:23:59.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc905c21700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900065640 0x7fc900113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc905c21700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900102e50 0x7fc9001140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc905c21700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc900114700 con 0x7fc900102e50 2024-04-24T23:23:59.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc905c21700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90010daf0 con 0x7fc900065640 2024-04-24T23:23:59.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc8feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900102e50 0x7fc9001140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc8feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900102e50 0x7fc9001140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42134/0 (socket says 172.21.15.31:42134) 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc8feffd700 1 -- 172.21.15.31:0/3075609842 learned_addr learned my addr 172.21.15.31:0/3075609842 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc8ff7fe700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900065640 0x7fc900113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc8feffd700 1 -- 172.21.15.31:0/3075609842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900065640 msgr2=0x7fc900113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc8feffd700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900065640 0x7fc900113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.598+0000 7fc8feffd700 1 -- 172.21.15.31:0/3075609842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4005040 con 0x7fc900102e50 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.599+0000 7fc8feffd700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900102e50 0x7fc9001140b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009150 tx=0x7fc8f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.599+0000 7fc8e7fff700 1 -- 172.21.15.31:0/3075609842 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f400e040 con 0x7fc900102e50 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.599+0000 7fc8e7fff700 1 -- 172.21.15.31:0/3075609842 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f4013420 con 0x7fc900102e50 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.599+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90010dd70 con 0x7fc900102e50 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.599+0000 7fc8e7fff700 1 -- 172.21.15.31:0/3075609842 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f401c610 con 0x7fc900102e50 2024-04-24T23:23:59.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.599+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90010e330 con 0x7fc900102e50 2024-04-24T23:23:59.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.600+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc900100100 con 0x7fc900102e50 2024-04-24T23:23:59.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.601+0000 7fc8e7fff700 1 -- 172.21.15.31:0/3075609842 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc8f400baf0 con 0x7fc900102e50 2024-04-24T23:23:59.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.601+0000 7fc8e7fff700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc8e806e150 0x7fc8e8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:23:59.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.602+0000 7fc8e7fff700 1 -- 172.21.15.31:0/3075609842 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc8f408edf0 con 0x7fc900102e50 2024-04-24T23:23:59.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.602+0000 7fc8ff7fe700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc8e806e150 0x7fc8e8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:23:59.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.602+0000 7fc8ff7fe700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc8e806e150 0x7fc8e8070600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0004f40 tx=0x7fc8f000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:23:59.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.606+0000 7fc8e7fff700 1 -- 172.21.15.31:0/3075609842 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc8f405bc70 con 0x7fc900102e50 2024-04-24T23:23:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:59 smithi031 ceph-mon[111404]: pgmap v210: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-24T23:23:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:59 smithi031 ceph-mon[111404]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:59.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:59 smithi031 ceph-mon[111404]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:59.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:59 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4143652491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:59.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:59.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:59.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:23:59 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1187441799' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:23:59.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:59 smithi040 ceph-mon[83047]: pgmap v210: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-24T23:23:59.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:59 smithi040 ceph-mon[83047]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:59.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:59 smithi040 ceph-mon[83047]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:23:59.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:59 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4143652491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:23:59.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:23:59.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:23:59.813 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:23:59 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1187441799' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:23:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.930+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc90010f1c0 con 0x7fc900102e50 2024-04-24T23:23:59.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.930+0000 7fc8e7fff700 1 -- 172.21.15.31:0/3075609842 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7fc8f405b800 con 0x7fc900102e50 2024-04-24T23:23:59.932 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem is degraded 2024-04-24T23:23:59.932 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-24T23:23:59.932 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs is degraded 2024-04-24T23:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc8e806e150 msgr2=0x7fc8e8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc8e806e150 0x7fc8e8070600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0004f40 tx=0x7fc8f000b040 comp rx=0 tx=0).stop 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900102e50 msgr2=0x7fc9001140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900102e50 0x7fc9001140b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009150 tx=0x7fc8f4009e30 comp rx=0 tx=0).stop 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 shutdown_connections 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc8e806e150 0x7fc8e8070600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc900065640 0x7fc900113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 --2- 172.21.15.31:0/3075609842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc900102e50 0x7fc9001140b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 >> 172.21.15.31:0/3075609842 conn(0x7fc900071d00 msgr2=0x7fc9000fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 shutdown_connections 2024-04-24T23:23:59.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:23:59.932+0000 7fc905c21700 1 -- 172.21.15.31:0/3075609842 wait complete. 2024-04-24T23:24:00.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:00 smithi031 ceph-mon[111404]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:00.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:00 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3075609842' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:24:00.790 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:00 smithi040 ceph-mon[83047]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:00.790 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:00 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3075609842' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:24:01.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:01 smithi040 ceph-mon[83047]: pgmap v211: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-24T23:24:01.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:01 smithi031 ceph-mon[111404]: pgmap v211: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-04-24T23:24:02.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:02 smithi031 ceph-mon[111404]: reconnect by client.24327 192.168.0.1:0/3971968853 after 0.000999984 2024-04-24T23:24:02.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:02 smithi031 ceph-mon[111404]: reconnect by client.14518 192.168.0.1:0/2806865259 after 0.000999984 2024-04-24T23:24:02.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:02 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] up:reconnect 2024-04-24T23:24:02.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:02 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi031.xxejxr=up:reconnect} 2 up:standby 2024-04-24T23:24:02.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:02 smithi040 ceph-mon[83047]: reconnect by client.24327 192.168.0.1:0/3971968853 after 0.000999984 2024-04-24T23:24:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:02 smithi040 ceph-mon[83047]: reconnect by client.14518 192.168.0.1:0/2806865259 after 0.000999984 2024-04-24T23:24:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:02 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] up:reconnect 2024-04-24T23:24:02.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:02 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi031.xxejxr=up:reconnect} 2 up:standby 2024-04-24T23:24:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:03 smithi031 ceph-mon[111404]: pgmap v212: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-24T23:24:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:03 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] up:rejoin 2024-04-24T23:24:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:03 smithi031 ceph-mon[111404]: fsmap cephfs:1/1 {0=cephfs.smithi031.xxejxr=up:rejoin} 2 up:standby 2024-04-24T23:24:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:03 smithi031 ceph-mon[111404]: daemon mds.cephfs.smithi031.xxejxr is now active in filesystem cephfs as rank 0 2024-04-24T23:24:03.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:03 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:03.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:03 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:03.750 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:03 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:03.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:03 smithi040 ceph-mon[83047]: pgmap v212: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-04-24T23:24:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:03 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] up:rejoin 2024-04-24T23:24:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:03 smithi040 ceph-mon[83047]: fsmap cephfs:1/1 {0=cephfs.smithi031.xxejxr=up:rejoin} 2 up:standby 2024-04-24T23:24:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:03 smithi040 ceph-mon[83047]: daemon mds.cephfs.smithi031.xxejxr is now active in filesystem cephfs as rank 0 2024-04-24T23:24:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:03 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:03 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:03.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:03 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:04.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:04 smithi031 ceph-mon[111404]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:24:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:04 smithi031 ceph-mon[111404]: Cluster is now healthy 2024-04-24T23:24:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:04 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.40:6826/342930745,v1:172.21.15.40:6827/342930745] up:boot 2024-04-24T23:24:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:04 smithi031 ceph-mon[111404]: mds.? [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] up:active 2024-04-24T23:24:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:04 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 3 up:standby 2024-04-24T23:24:04.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:04 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:24:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:04 smithi040 ceph-mon[83047]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-24T23:24:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:04 smithi040 ceph-mon[83047]: Cluster is now healthy 2024-04-24T23:24:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:04 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.40:6826/342930745,v1:172.21.15.40:6827/342930745] up:boot 2024-04-24T23:24:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:04 smithi040 ceph-mon[83047]: mds.? [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] up:active 2024-04-24T23:24:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:04 smithi040 ceph-mon[83047]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 3 up:standby 2024-04-24T23:24:04.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:04 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:24:05.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:05 smithi031 ceph-mon[111404]: pgmap v213: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:24:05.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:05.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:05 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:05.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:05 smithi040 ceph-mon[83047]: pgmap v213: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:24:05.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:05.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:05 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:07.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:07 smithi040 ceph-mon[83047]: pgmap v214: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:24:07.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:07.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:07 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:07.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:07 smithi031 ceph-mon[111404]: pgmap v214: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 85 B/s wr, 5 op/s 2024-04-24T23:24:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:07.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:07 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:09.341 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:09 smithi040 ceph-mon[83047]: pgmap v215: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 9.5 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T23:24:09.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:09 smithi031 ceph-mon[111404]: pgmap v215: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 9.5 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-24T23:24:10.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: pgmap v216: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all mds 2024-04-24T23:24:10.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.acqiox"}]': finished 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.xxejxr"}]': finished 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.dicevu"}]': finished 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.osunbp"}]': finished 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: Upgrade: Setting filesystem cephfs Joinable 2024-04-24T23:24:11.000 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:10 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: pgmap v216: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 5.5 MiB/s rd, 170 B/s wr, 4 op/s 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:11.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all mds 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.acqiox"}]: dispatch 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.acqiox"}]': finished 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.xxejxr"}]: dispatch 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.xxejxr"}]': finished 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.dicevu"}]: dispatch 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.dicevu"}]': finished 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.osunbp"}]: dispatch 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi040.osunbp"}]': finished 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: Upgrade: Setting filesystem cephfs Joinable 2024-04-24T23:24:11.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:10 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-24T23:24:12.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 3 up:standby 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all rgw 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.163 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all iscsi 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all nfs 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: Upgrade: Setting container_image for all nvmeof 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.164 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: Upgrade: Finalizing container_image settings 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T23:24:12.165 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.166 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: Upgrade: Complete! 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:12.167 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: fsmap cephfs:1 {0=cephfs.smithi031.xxejxr=up:active} 3 up:standby 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all rgw 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all rbd-mirror 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all ceph-exporter 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all iscsi 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all nfs 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: Upgrade: Setting container_image for all nvmeof 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: Upgrade: Finalizing container_image settings 2024-04-24T23:24:12.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-24T23:24:12.251 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-24T23:24:12.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: Upgrade: Complete! 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:24:12.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:13.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:13 smithi040 ceph-mon[83047]: pgmap v217: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-24T23:24:13.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:13 smithi031 ceph-mon[111404]: pgmap v217: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-24T23:24:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:24:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:14 smithi040 ceph-mon[83047]: pgmap v218: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-24T23:24:15.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:15.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:15.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:24:15.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:14 smithi031 ceph-mon[111404]: pgmap v218: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-04-24T23:24:15.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:24:17.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:16 smithi040 ceph-mon[83047]: pgmap v219: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-24T23:24:17.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:16 smithi031 ceph-mon[111404]: pgmap v219: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-24T23:24:19.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:18 smithi040 ceph-mon[83047]: pgmap v220: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-04-24T23:24:19.249 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:18 smithi031 ceph-mon[111404]: pgmap v220: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 4.2 KiB/s wr, 3 op/s 2024-04-24T23:24:21.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:20 smithi040 ceph-mon[83047]: pgmap v221: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s wr, 1 op/s 2024-04-24T23:24:21.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:20 smithi031 ceph-mon[111404]: pgmap v221: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s wr, 1 op/s 2024-04-24T23:24:23.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:22 smithi040 ceph-mon[83047]: pgmap v222: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s wr, 1 op/s 2024-04-24T23:24:23.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:22 smithi031 ceph-mon[111404]: pgmap v222: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s wr, 1 op/s 2024-04-24T23:24:25.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:24 smithi040 ceph-mon[83047]: pgmap v223: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:25.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:24 smithi031 ceph-mon[111404]: pgmap v223: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:27.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:26 smithi040 ceph-mon[83047]: pgmap v224: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:27.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:26 smithi031 ceph-mon[111404]: pgmap v224: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:29.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:28 smithi040 ceph-mon[83047]: pgmap v225: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:29.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:24:29.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:28 smithi031 ceph-mon[111404]: pgmap v225: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:29.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:24:30.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.052+0000 7feded3f1700 1 -- 172.21.15.31:0/1700269511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 msgr2=0x7fede80fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.052+0000 7feded3f1700 1 --2- 172.21.15.31:0/1700269511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 0x7fede80fce00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feddc004940 tx=0x7feddc009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:30.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.052+0000 7feded3f1700 1 -- 172.21.15.31:0/1700269511 shutdown_connections 2024-04-24T23:24:30.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.052+0000 7feded3f1700 1 --2- 172.21.15.31:0/1700269511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 0x7fede80fce00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.052+0000 7feded3f1700 1 --2- 172.21.15.31:0/1700269511 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8102df0 0x7fede81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.052+0000 7feded3f1700 1 -- 172.21.15.31:0/1700269511 >> 172.21.15.31:0/1700269511 conn(0x7fede80f8980 msgr2=0x7fede80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:30.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.052+0000 7feded3f1700 1 -- 172.21.15.31:0/1700269511 shutdown_connections 2024-04-24T23:24:30.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.052+0000 7feded3f1700 1 -- 172.21.15.31:0/1700269511 wait complete. 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.053+0000 7feded3f1700 1 Processor -- start 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.053+0000 7feded3f1700 1 -- start start 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.053+0000 7feded3f1700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8102df0 0x7fede8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.053+0000 7feded3f1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 0x7fede80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.053+0000 7feded3f1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede8070d30 con 0x7fede8103790 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.053+0000 7feded3f1700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede8070e70 con 0x7fede8102df0 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fede67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 0x7fede80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fede67fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 0x7fede80707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47102/0 (socket says 172.21.15.31:47102) 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fede67fc700 1 -- 172.21.15.31:0/785738613 learned_addr learned my addr 172.21.15.31:0/785738613 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fede6ffd700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8102df0 0x7fede8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fede67fc700 1 -- 172.21.15.31:0/785738613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8102df0 msgr2=0x7fede8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fede67fc700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8102df0 0x7fede8070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fede67fc700 1 -- 172.21.15.31:0/785738613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc005040 con 0x7fede8103790 2024-04-24T23:24:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fede67fc700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 0x7fede80707a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feddc000c00 tx=0x7feddc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fedcf7fe700 1 -- 172.21.15.31:0/785738613 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feddc021070 con 0x7fede8103790 2024-04-24T23:24:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede80710c0 con 0x7fede8103790 2024-04-24T23:24:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fedcf7fe700 1 -- 172.21.15.31:0/785738613 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feddc013cd0 con 0x7fede8103790 2024-04-24T23:24:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7fedcf7fe700 1 -- 172.21.15.31:0/785738613 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feddc00e070 con 0x7fede8103790 2024-04-24T23:24:30.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.054+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede811a170 con 0x7fede8103790 2024-04-24T23:24:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.055+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede80fe540 con 0x7fede8103790 2024-04-24T23:24:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.056+0000 7fedcf7fe700 1 -- 172.21.15.31:0/785738613 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feddc01caf0 con 0x7fede8103790 2024-04-24T23:24:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.056+0000 7fedcf7fe700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fedd006e140 0x7fedd00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:30.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.056+0000 7fedcf7fe700 1 -- 172.21.15.31:0/785738613 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7feddc08f0d0 con 0x7fede8103790 2024-04-24T23:24:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.056+0000 7fede6ffd700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fedd006e140 0x7fedd00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:30.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.057+0000 7fede6ffd700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fedd006e140 0x7fedd00705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedd8004600 tx=0x7fedd800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:30.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.061+0000 7fedcf7fe700 1 -- 172.21.15.31:0/785738613 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feddc058420 con 0x7fede8103790 2024-04-24T23:24:30.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.290+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fede8109f70 con 0x7fedd006e140 2024-04-24T23:24:30.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.291+0000 7fedcf7fe700 1 -- 172.21.15.31:0/785738613 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fedd800f040 con 0x7fedd006e140 2024-04-24T23:24:30.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fedd006e140 msgr2=0x7fedd00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fedd006e140 0x7fedd00705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedd8004600 tx=0x7fedd800b040 comp rx=0 tx=0).stop 2024-04-24T23:24:30.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 msgr2=0x7fede80707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 0x7fede80707a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7feddc000c00 tx=0x7feddc009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 shutdown_connections 2024-04-24T23:24:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fedd006e140 0x7fedd00705f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fede8102df0 0x7fede8070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 --2- 172.21.15.31:0/785738613 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fede8103790 0x7fede80707a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 >> 172.21.15.31:0/785738613 conn(0x7fede80f8980 msgr2=0x7fede80f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 shutdown_connections 2024-04-24T23:24:30.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:30.293+0000 7feded3f1700 1 -- 172.21.15.31:0/785738613 wait complete. 2024-04-24T23:24:31.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-24T23:24:31.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:30 smithi040 ceph-mon[83047]: pgmap v226: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:31.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:30 smithi040 ceph-mon[83047]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:31.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:30 smithi031 ceph-mon[111404]: pgmap v226: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:31.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:30 smithi031 ceph-mon[111404]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:31.334 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:24:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.998+0000 7fb199bd5700 1 -- 172.21.15.31:0/2052527407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fbb70 msgr2=0x7fb1940fbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.998+0000 7fb199bd5700 1 --2- 172.21.15.31:0/2052527407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fbb70 0x7fb1940fbf40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb184004940 tx=0x7fb184009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:33.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.998+0000 7fb199bd5700 1 -- 172.21.15.31:0/2052527407 shutdown_connections 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.998+0000 7fb199bd5700 1 --2- 172.21.15.31:0/2052527407 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1940fc480 0x7fb1941091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.998+0000 7fb199bd5700 1 --2- 172.21.15.31:0/2052527407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fbb70 0x7fb1940fbf40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.998+0000 7fb199bd5700 1 -- 172.21.15.31:0/2052527407 >> 172.21.15.31:0/2052527407 conn(0x7fb1940f7860 msgr2=0x7fb1940f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.998+0000 7fb199bd5700 1 -- 172.21.15.31:0/2052527407 shutdown_connections 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.998+0000 7fb199bd5700 1 -- 172.21.15.31:0/2052527407 wait complete. 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb199bd5700 1 Processor -- start 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb199bd5700 1 -- start start 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb199bd5700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1940fbb70 0x7fb194115af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb199bd5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fc480 0x7fb194116030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb199bd5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb194116680 con 0x7fb1940fc480 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb199bd5700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19410fc60 con 0x7fb1940fbb70 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb192ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fc480 0x7fb194116030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb192ffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fc480 0x7fb194116030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47126/0 (socket says 172.21.15.31:47126) 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:32.999+0000 7fb192ffd700 1 -- 172.21.15.31:0/2174285770 learned_addr learned my addr 172.21.15.31:0/2174285770 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb1937fe700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1940fbb70 0x7fb194115af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb192ffd700 1 -- 172.21.15.31:0/2174285770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1940fbb70 msgr2=0x7fb194115af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb192ffd700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1940fbb70 0x7fb194115af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb192ffd700 1 -- 172.21.15.31:0/2174285770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb184005040 con 0x7fb1940fc480 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb192ffd700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fc480 0x7fb194116030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00c8c0 tx=0x7fb17c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb18bfff700 1 -- 172.21.15.31:0/2174285770 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17c00a760 con 0x7fb1940fc480 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb18bfff700 1 -- 172.21.15.31:0/2174285770 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb17c00ad90 con 0x7fb1940fc480 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb19410ff40 con 0x7fb1940fc480 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb18bfff700 1 -- 172.21.15.31:0/2174285770 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17c0113a0 con 0x7fb1940fc480 2024-04-24T23:24:33.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.000+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb194110500 con 0x7fb1940fc480 2024-04-24T23:24:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.001+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb194066b50 con 0x7fb1940fc480 2024-04-24T23:24:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.002+0000 7fb18bfff700 1 -- 172.21.15.31:0/2174285770 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb17c012400 con 0x7fb1940fc480 2024-04-24T23:24:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.002+0000 7fb18bfff700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb18006e140 0x7fb1800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:33.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.002+0000 7fb18bfff700 1 -- 172.21.15.31:0/2174285770 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb17c08dbc0 con 0x7fb1940fc480 2024-04-24T23:24:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.002+0000 7fb1937fe700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb18006e140 0x7fb1800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:33.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.003+0000 7fb1937fe700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb18006e140 0x7fb1800705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb184004f30 tx=0x7fb184009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:33.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.007+0000 7fb18bfff700 1 -- 172.21.15.31:0/2174285770 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb17c05aa40 con 0x7fb1940fc480 2024-04-24T23:24:33.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:32 smithi040 ceph-mon[83047]: pgmap v227: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:33.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.223+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb19410c090 con 0x7fb18006e140 2024-04-24T23:24:33.224 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:32 smithi031 ceph-mon[111404]: pgmap v227: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:33.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.234+0000 7fb18bfff700 1 -- 172.21.15.31:0/2174285770 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fb19410c090 con 0x7fb18006e140 2024-04-24T23:24:33.235 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (7m) 73s ago 19m 25.6M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (5m) 73s ago 19m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (5m) 28s ago 18m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 73s ago 19m 79.7M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (78s) 73s ago 13m 11.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 956ce7ff5df1 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (100s) 73s ago 13m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1d7ff0e06219 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (57s) 28s ago 13m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 4ee160da943d 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (31s) 28s ago 13m 11.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 924dcede7c05 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (9m) 73s ago 20m 507M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (8m) 28s ago 18m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (6m) 73s ago 20m 52.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:24:33.236 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (6m) 28s ago 18m 42.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (8m) 73s ago 19m 16.8M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (8m) 28s ago 18m 14.7M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (4m) 73s ago 17m 96.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 73s ago 17m 111M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (3m) 73s ago 16m 91.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 271a0764ef5c 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (3m) 28s ago 16m 129M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 97c6f05632f6 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (2m) 28s ago 15m 103M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 85d9bab7b352 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (118s) 28s ago 15m 95.4M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 32b6ff45fea8 2024-04-24T23:24:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (8m) 73s ago 18m 60.1M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:24:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb18006e140 msgr2=0x7fb1800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb18006e140 0x7fb1800705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb184004f30 tx=0x7fb184009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fc480 msgr2=0x7fb194116030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fc480 0x7fb194116030 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00c8c0 tx=0x7fb17c0044c0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 shutdown_connections 2024-04-24T23:24:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb18006e140 0x7fb1800705f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb1940fbb70 0x7fb194115af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 --2- 172.21.15.31:0/2174285770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1940fc480 0x7fb194116030 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 >> 172.21.15.31:0/2174285770 conn(0x7fb1940f7860 msgr2=0x7fb1940f7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 shutdown_connections 2024-04-24T23:24:33.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:33.236+0000 7fb199bd5700 1 -- 172.21.15.31:0/2174285770 wait complete. 2024-04-24T23:24:33.985 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade status' 2024-04-24T23:24:34.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:33 smithi040 ceph-mon[83047]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:34.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:33 smithi031 ceph-mon[111404]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:34.255 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:24:34.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:34 smithi031 ceph-mon[111404]: pgmap v228: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:35.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:34 smithi040 ceph-mon[83047]: pgmap v228: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:36.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.062+0000 7f90c11b6700 1 -- 172.21.15.31:0/412890190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fdde0 msgr2=0x7f90bc0fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.062+0000 7f90c11b6700 1 --2- 172.21.15.31:0/412890190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fdde0 0x7f90bc0fe1b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f90b0004940 tx=0x7f90b0009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.062+0000 7f90c11b6700 1 -- 172.21.15.31:0/412890190 shutdown_connections 2024-04-24T23:24:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.062+0000 7f90c11b6700 1 --2- 172.21.15.31:0/412890190 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90bc0fe780 0x7f90bc106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.062+0000 7f90c11b6700 1 --2- 172.21.15.31:0/412890190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fdde0 0x7f90bc0fe1b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.062+0000 7f90c11b6700 1 -- 172.21.15.31:0/412890190 >> 172.21.15.31:0/412890190 conn(0x7f90bc0f7850 msgr2=0x7f90bc0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.062+0000 7f90c11b6700 1 -- 172.21.15.31:0/412890190 shutdown_connections 2024-04-24T23:24:36.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.062+0000 7f90c11b6700 1 -- 172.21.15.31:0/412890190 wait complete. 2024-04-24T23:24:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90c11b6700 1 Processor -- start 2024-04-24T23:24:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90c11b6700 1 -- start start 2024-04-24T23:24:36.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90c11b6700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90bc0fdde0 0x7f90bc113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90c11b6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fe780 0x7f90bc1140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90c11b6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc114780 con 0x7f90bc0fe780 2024-04-24T23:24:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90c11b6700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc10dae0 con 0x7f90bc0fdde0 2024-04-24T23:24:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90ba59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fe780 0x7f90bc1140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90bad9d700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90bc0fdde0 0x7f90bc113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90ba59c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fe780 0x7f90bc1140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:47154/0 (socket says 172.21.15.31:47154) 2024-04-24T23:24:36.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90ba59c700 1 -- 172.21.15.31:0/223069253 learned_addr learned my addr 172.21.15.31:0/223069253 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:24:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90ba59c700 1 -- 172.21.15.31:0/223069253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90bc0fdde0 msgr2=0x7f90bc113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90ba59c700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90bc0fdde0 0x7f90bc113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.063+0000 7f90ba59c700 1 -- 172.21.15.31:0/223069253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0005040 con 0x7f90bc0fe780 2024-04-24T23:24:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.064+0000 7f90ba59c700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fe780 0x7f90bc1140a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b790 tx=0x7f90ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.064+0000 7f90a37fe700 1 -- 172.21.15.31:0/223069253 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ac00f6f0 con 0x7f90bc0fe780 2024-04-24T23:24:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.064+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc10ddc0 con 0x7f90bc0fe780 2024-04-24T23:24:36.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.064+0000 7f90a37fe700 1 -- 172.21.15.31:0/223069253 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90ac00fd20 con 0x7f90bc0fe780 2024-04-24T23:24:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.064+0000 7f90a37fe700 1 -- 172.21.15.31:0/223069253 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ac0103a0 con 0x7f90bc0fe780 2024-04-24T23:24:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.064+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc10e300 con 0x7f90bc0fe780 2024-04-24T23:24:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.065+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc066750 con 0x7f90bc0fe780 2024-04-24T23:24:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.065+0000 7f90a37fe700 1 -- 172.21.15.31:0/223069253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f90ac011400 con 0x7f90bc0fe780 2024-04-24T23:24:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.066+0000 7f90a37fe700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90a406e140 0x7f90a40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.066+0000 7f90a37fe700 1 -- 172.21.15.31:0/223069253 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f90ac08d9c0 con 0x7f90bc0fe780 2024-04-24T23:24:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.066+0000 7f90bad9d700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90a406e140 0x7f90a40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:36.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.066+0000 7f90bad9d700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90a406e140 0x7f90a40705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f90b000b3b0 tx=0x7f90b0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:36.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.071+0000 7f90a37fe700 1 -- 172.21.15.31:0/223069253 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f90ac05a840 con 0x7f90bc0fe780 2024-04-24T23:24:36.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.296+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90bc109ef0 con 0x7f90a406e140 2024-04-24T23:24:36.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.297+0000 7f90a37fe700 1 -- 172.21.15.31:0/223069253 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f90bc109ef0 con 0x7f90a406e140 2024-04-24T23:24:36.299 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:24:36.299 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": null, 2024-04-24T23:24:36.299 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": false, 2024-04-24T23:24:36.299 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "", 2024-04-24T23:24:36.299 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-04-24T23:24:36.299 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": null, 2024-04-24T23:24:36.300 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-04-24T23:24:36.300 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-04-24T23:24:36.300 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.299+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90a406e140 msgr2=0x7f90a40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.299+0000 7f90c11b6700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90a406e140 0x7f90a40705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f90b000b3b0 tx=0x7f90b0009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.299+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fe780 msgr2=0x7f90bc1140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.300+0000 7f90c11b6700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fe780 0x7f90bc1140a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b790 tx=0x7f90ac009250 comp rx=0 tx=0).stop 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.300+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 shutdown_connections 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.300+0000 7f90c11b6700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90a406e140 0x7f90a40705f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.300+0000 7f90c11b6700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90bc0fdde0 0x7f90bc113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.300+0000 7f90c11b6700 1 --2- 172.21.15.31:0/223069253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90bc0fe780 0x7f90bc1140a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:36.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.300+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 >> 172.21.15.31:0/223069253 conn(0x7f90bc0f7850 msgr2=0x7f90bc103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:36.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.300+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 shutdown_connections 2024-04-24T23:24:36.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:36.300+0000 7f90c11b6700 1 -- 172.21.15.31:0/223069253 wait complete. 2024-04-24T23:24:36.929 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph health detail' 2024-04-24T23:24:37.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:36 smithi040 ceph-mon[83047]: pgmap v229: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:37.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:36 smithi040 ceph-mon[83047]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:37.200 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:24:37.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:36 smithi031 ceph-mon[111404]: pgmap v229: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:37.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:36 smithi031 ceph-mon[111404]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.909+0000 7ffbe9a40700 1 -- 172.21.15.31:0/4000204434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 msgr2=0x7ffbe4106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.909+0000 7ffbe9a40700 1 --2- 172.21.15.31:0/4000204434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 0x7ffbe4106de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8004940 tx=0x7ffbd8009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.909+0000 7ffbe9a40700 1 -- 172.21.15.31:0/4000204434 shutdown_connections 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.909+0000 7ffbe9a40700 1 --2- 172.21.15.31:0/4000204434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 0x7ffbe4106de0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.909+0000 7ffbe9a40700 1 --2- 172.21.15.31:0/4000204434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbe40fde00 0x7ffbe40fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.909+0000 7ffbe9a40700 1 -- 172.21.15.31:0/4000204434 >> 172.21.15.31:0/4000204434 conn(0x7ffbe40f7870 msgr2=0x7ffbe40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.909+0000 7ffbe9a40700 1 -- 172.21.15.31:0/4000204434 shutdown_connections 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.909+0000 7ffbe9a40700 1 -- 172.21.15.31:0/4000204434 wait complete. 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe9a40700 1 Processor -- start 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe9a40700 1 -- start start 2024-04-24T23:24:38.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe9a40700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbe40fde00 0x7ffbe4113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe9a40700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 0x7ffbe4113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe9a40700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe4114500 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe9a40700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe410db00 con 0x7ffbe40fde00 2024-04-24T23:24:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 0x7ffbe4113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 0x7ffbe4113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41648/0 (socket says 172.21.15.31:41648) 2024-04-24T23:24:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe27fc700 1 -- 172.21.15.31:0/330128266 learned_addr learned my addr 172.21.15.31:0/330128266 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:24:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.910+0000 7ffbe2ffd700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbe40fde00 0x7ffbe4113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbe27fc700 1 -- 172.21.15.31:0/330128266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbe40fde00 msgr2=0x7ffbe4113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbe27fc700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbe40fde00 0x7ffbe4113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbe27fc700 1 -- 172.21.15.31:0/330128266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8005040 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbe2ffd700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbe40fde00 0x7ffbe4113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:24:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbe27fc700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 0x7ffbe4113eb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009380 tx=0x7ffbd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbcb7fe700 1 -- 172.21.15.31:0/330128266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbd800e040 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbcb7fe700 1 -- 172.21.15.31:0/330128266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbd8009af0 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe410dd80 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbcb7fe700 1 -- 172.21.15.31:0/330128266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbd801c630 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.915 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.911+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe410e2c0 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.912+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe4066b30 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.912+0000 7ffbcb7fe700 1 -- 172.21.15.31:0/330128266 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffbd800bbc0 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.913+0000 7ffbcb7fe700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffbcc06e140 0x7ffbcc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.913+0000 7ffbcb7fe700 1 -- 172.21.15.31:0/330128266 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ffbd808ef10 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.913+0000 7ffbe2ffd700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffbcc06e140 0x7ffbcc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.913+0000 7ffbe2ffd700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffbcc06e140 0x7ffbcc0705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009fd0 tx=0x7ffbd4009ee0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:38.918+0000 7ffbcb7fe700 1 -- 172.21.15.31:0/330128266 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ffbd805bd90 con 0x7ffbe40fe7a0 2024-04-24T23:24:38.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:38 smithi031 ceph-mon[111404]: pgmap v230: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:39.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:38 smithi040 ceph-mon[83047]: pgmap v230: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.210+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffbe404c710 con 0x7ffbe40fe7a0 2024-04-24T23:24:39.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.211+0000 7ffbcb7fe700 1 -- 172.21.15.31:0/330128266 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ffbd805b920 con 0x7ffbe40fe7a0 2024-04-24T23:24:39.212 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffbcc06e140 msgr2=0x7ffbcc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffbcc06e140 0x7ffbcc0705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009fd0 tx=0x7ffbd4009ee0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 msgr2=0x7ffbe4113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 0x7ffbe4113eb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009380 tx=0x7ffbd8009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 shutdown_connections 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ffbcc06e140 0x7ffbcc0705f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbe40fde00 0x7ffbe4113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 --2- 172.21.15.31:0/330128266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbe40fe7a0 0x7ffbe4113eb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 >> 172.21.15.31:0/330128266 conn(0x7ffbe40f7870 msgr2=0x7ffbe40f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 shutdown_connections 2024-04-24T23:24:39.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:39.213+0000 7ffbe9a40700 1 -- 172.21.15.31:0/330128266 wait complete. 2024-04-24T23:24:39.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions' 2024-04-24T23:24:40.142 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:24:40.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:39 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/330128266' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:24:40.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:39 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/330128266' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-24T23:24:41.142 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:40 smithi031 ceph-mon[111404]: pgmap v231: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:41.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:40 smithi040 ceph-mon[83047]: pgmap v231: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.662+0000 7fa2a0c29700 1 -- 172.21.15.31:0/4237227531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c0fbd40 msgr2=0x7fa29c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.662+0000 7fa2a0c29700 1 --2- 172.21.15.31:0/4237227531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c0fbd40 0x7fa29c0fc1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa284004940 tx=0x7fa284009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 -- 172.21.15.31:0/4237227531 shutdown_connections 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 --2- 172.21.15.31:0/4237227531 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c0fbd40 0x7fa29c0fc1b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 --2- 172.21.15.31:0/4237227531 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c101d90 0x7fa29c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 -- 172.21.15.31:0/4237227531 >> 172.21.15.31:0/4237227531 conn(0x7fa29c071d00 msgr2=0x7fa29c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 -- 172.21.15.31:0/4237227531 shutdown_connections 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 -- 172.21.15.31:0/4237227531 wait complete. 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 Processor -- start 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 -- start start 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0fbd40 0x7fa29c10ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:41.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c101d90 0x7fa29c10f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c10f750 con 0x7fa29c101d90 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa2a0c29700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c10f890 con 0x7fa29c0fbd40 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.663+0000 7fa299d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c101d90 0x7fa29c10f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa299d9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c101d90 0x7fa29c10f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41668/0 (socket says 172.21.15.31:41668) 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa299d9b700 1 -- 172.21.15.31:0/3094181004 learned_addr learned my addr 172.21.15.31:0/3094181004 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa29a59c700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0fbd40 0x7fa29c10ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa299d9b700 1 -- 172.21.15.31:0/3094181004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0fbd40 msgr2=0x7fa29c10ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa299d9b700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0fbd40 0x7fa29c10ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa299d9b700 1 -- 172.21.15.31:0/3094181004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa284005040 con 0x7fa29c101d90 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa29a59c700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0fbd40 0x7fa29c10ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa299d9b700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c101d90 0x7fa29c10f210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa28400b8b0 tx=0x7fa284009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa292ffd700 1 -- 172.21.15.31:0/3094181004 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa28400e040 con 0x7fa29c101d90 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29c117b80 con 0x7fa29c101d90 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa292ffd700 1 -- 172.21.15.31:0/3094181004 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa284013420 con 0x7fa29c101d90 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa292ffd700 1 -- 172.21.15.31:0/3094181004 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa28401c630 con 0x7fa29c101d90 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.664+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29c118110 con 0x7fa29c101d90 2024-04-24T23:24:41.671 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.665+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c0fd430 con 0x7fa29c101d90 2024-04-24T23:24:41.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.666+0000 7fa292ffd700 1 -- 172.21.15.31:0/3094181004 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa284013a00 con 0x7fa29c101d90 2024-04-24T23:24:41.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.666+0000 7fa292ffd700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa28806e140 0x7fa2880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:24:41.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.666+0000 7fa292ffd700 1 -- 172.21.15.31:0/3094181004 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa28408ee60 con 0x7fa29c101d90 2024-04-24T23:24:41.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.666+0000 7fa29a59c700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa28806e140 0x7fa2880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:24:41.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.667+0000 7fa29a59c700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa28806e140 0x7fa2880705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa28c007fd0 tx=0x7fa28c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:24:41.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.671+0000 7fa292ffd700 1 -- 172.21.15.31:0/3094181004 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa2840581b0 con 0x7fa29c101d90 2024-04-24T23:24:41.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.971+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa29c04bed0 con 0x7fa29c101d90 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.972+0000 7fa292ffd700 1 -- 172.21.15.31:0/3094181004 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa28405b870 con 0x7fa29c101d90 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T23:24:41.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:24:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:24:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T23:24:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:24:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:24:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-24T23:24:41.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:24:41.974 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:24:41.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa28806e140 msgr2=0x7fa2880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:41.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa28806e140 0x7fa2880705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa28c007fd0 tx=0x7fa28c009040 comp rx=0 tx=0).stop 2024-04-24T23:24:41.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c101d90 msgr2=0x7fa29c10f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:24:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c101d90 0x7fa29c10f210 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa28400b8b0 tx=0x7fa284009e30 comp rx=0 tx=0).stop 2024-04-24T23:24:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 shutdown_connections 2024-04-24T23:24:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa28806e140 0x7fa2880705f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa29c0fbd40 0x7fa29c10ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 --2- 172.21.15.31:0/3094181004 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa29c101d90 0x7fa29c10f210 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:24:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 >> 172.21.15.31:0/3094181004 conn(0x7fa29c071d00 msgr2=0x7fa29c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:24:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 shutdown_connections 2024-04-24T23:24:41.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:24:41.974+0000 7fa2a0c29700 1 -- 172.21.15.31:0/3094181004 wait complete. 2024-04-24T23:24:42.593 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-24T23:24:42.863 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:24:43.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:42 smithi040 ceph-mon[83047]: pgmap v232: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:43.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:42 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3094181004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:43.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:42 smithi031 ceph-mon[111404]: pgmap v232: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:43.194 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:42 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3094181004' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:24:44.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:43 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:24:44.207 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:43 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:24:44.410 INFO:teuthology.orchestra.run.smithi031.stdout:wait for servicemap items w/ changing names to refresh 2024-04-24T23:24:45.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'sleep 60' 2024-04-24T23:24:45.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:44 smithi040 ceph-mon[83047]: pgmap v233: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:45.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:44 smithi031 ceph-mon[111404]: pgmap v233: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:45.301 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:24:46.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:46 smithi031 ceph-mon[111404]: pgmap v234: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:47.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:46 smithi040 ceph-mon[83047]: pgmap v234: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:49.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:48 smithi040 ceph-mon[83047]: pgmap v235: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:49.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:48 smithi031 ceph-mon[111404]: pgmap v235: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:51.162 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:50 smithi040 ceph-mon[83047]: pgmap v236: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:51.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:50 smithi031 ceph-mon[111404]: pgmap v236: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:53.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:52 smithi031 ceph-mon[111404]: pgmap v237: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:53.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:52 smithi040 ceph-mon[83047]: pgmap v237: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:55.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:54 smithi031 ceph-mon[111404]: pgmap v238: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:55.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:54 smithi040 ceph-mon[83047]: pgmap v238: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:57.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:56 smithi031 ceph-mon[111404]: pgmap v239: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:57.414 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:56 smithi040 ceph-mon[83047]: pgmap v239: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:59.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:58 smithi031 ceph-mon[111404]: pgmap v240: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:59.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:24:58 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:24:59.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:58 smithi040 ceph-mon[83047]: pgmap v240: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:24:59.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:24:58 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:25:01.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:00 smithi040 ceph-mon[83047]: pgmap v241: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:01.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:00 smithi031 ceph-mon[111404]: pgmap v241: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:03.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:02 smithi031 ceph-mon[111404]: pgmap v242: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:03.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:02 smithi040 ceph-mon[83047]: pgmap v242: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:05.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:04 smithi031 ceph-mon[111404]: pgmap v243: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:05.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:04 smithi040 ceph-mon[83047]: pgmap v243: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:07.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:06 smithi031 ceph-mon[111404]: pgmap v244: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:07.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:06 smithi040 ceph-mon[83047]: pgmap v244: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:09.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:08 smithi031 ceph-mon[111404]: pgmap v245: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:09.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:08 smithi040 ceph-mon[83047]: pgmap v245: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:11.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:10 smithi031 ceph-mon[111404]: pgmap v246: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:11.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:10 smithi040 ceph-mon[83047]: pgmap v246: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:12.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:12.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:25:12.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:11 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:25:12.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:25:12.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:25:12.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:11 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:25:13.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:12 smithi031 ceph-mon[111404]: pgmap v247: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:13.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:12 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:13.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:12 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:13.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:12 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:25:13.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:12 smithi040 ceph-mon[83047]: pgmap v247: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:13.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:12 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:25:13.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:12 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:25:13.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:12 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:25:14.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:25:14.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:13 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:25:15.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:14 smithi031 ceph-mon[111404]: pgmap v248: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:15.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:14 smithi040 ceph-mon[83047]: pgmap v248: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:17.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:16 smithi031 ceph-mon[111404]: pgmap v249: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:17.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:16 smithi040 ceph-mon[83047]: pgmap v249: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:19.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:18 smithi031 ceph-mon[111404]: pgmap v250: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:19.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:18 smithi040 ceph-mon[83047]: pgmap v250: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:21.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:20 smithi031 ceph-mon[111404]: pgmap v251: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:21.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:20 smithi040 ceph-mon[83047]: pgmap v251: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:23.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:22 smithi031 ceph-mon[111404]: pgmap v252: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:23.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:22 smithi040 ceph-mon[83047]: pgmap v252: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:25.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:24 smithi031 ceph-mon[111404]: pgmap v253: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:25.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:24 smithi040 ceph-mon[83047]: pgmap v253: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:27.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:26 smithi031 ceph-mon[111404]: pgmap v254: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:27.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:26 smithi040 ceph-mon[83047]: pgmap v254: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:29.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:28 smithi031 ceph-mon[111404]: pgmap v255: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:29.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:25:29.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:28 smithi040 ceph-mon[83047]: pgmap v255: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:29.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:25:31.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:30 smithi031 ceph-mon[111404]: pgmap v256: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:31.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:30 smithi040 ceph-mon[83047]: pgmap v256: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:33.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:32 smithi031 ceph-mon[111404]: pgmap v257: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:33.412 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:32 smithi040 ceph-mon[83047]: pgmap v257: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:35.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:34 smithi031 ceph-mon[111404]: pgmap v258: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:35.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:34 smithi040 ceph-mon[83047]: pgmap v258: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:37.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:36 smithi031 ceph-mon[111404]: pgmap v259: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:37.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:36 smithi040 ceph-mon[83047]: pgmap v259: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:39.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:38 smithi040 ceph-mon[83047]: pgmap v260: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:39.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:38 smithi031 ceph-mon[111404]: pgmap v260: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:41.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:41 smithi031 ceph-mon[111404]: pgmap v261: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:41.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:41 smithi040 ceph-mon[83047]: pgmap v261: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:43.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:43 smithi040 ceph-mon[83047]: pgmap v262: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:43.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:43 smithi031 ceph-mon[111404]: pgmap v262: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:44.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:25:44.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:25:45.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:45 smithi040 ceph-mon[83047]: pgmap v263: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:45.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:45 smithi031 ceph-mon[111404]: pgmap v263: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:47.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:47 smithi031 ceph-mon[111404]: pgmap v264: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:47.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:47 smithi040 ceph-mon[83047]: pgmap v264: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:47.676 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-24T23:25:47.952 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:25:49.396 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:49 smithi031 ceph-mon[111404]: pgmap v265: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:49.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 -- 172.21.15.31:0/2900402620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 msgr2=0x7fdf941031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 --2- 172.21.15.31:0/2900402620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 0x7fdf941031b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf88004940 tx=0x7fdf88009e30 comp rx=0 tx=0).stop 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 -- 172.21.15.31:0/2900402620 shutdown_connections 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 --2- 172.21.15.31:0/2900402620 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf94103780 0x7fdf940fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 --2- 172.21.15.31:0/2900402620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 0x7fdf941031b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 -- 172.21.15.31:0/2900402620 >> 172.21.15.31:0/2900402620 conn(0x7fdf940f8990 msgr2=0x7fdf940fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 -- 172.21.15.31:0/2900402620 shutdown_connections 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 -- 172.21.15.31:0/2900402620 wait complete. 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 Processor -- start 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.632+0000 7fdf9ab62700 1 -- start start 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf9ab62700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 0x7fdf94070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf9ab62700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf94103780 0x7fdf940707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:49.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf9ab62700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94072160 con 0x7fdf94102de0 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf9ab62700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94070ce0 con 0x7fdf94103780 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf988fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 0x7fdf94070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf988fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 0x7fdf94070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59548/0 (socket says 172.21.15.31:59548) 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf988fe700 1 -- 172.21.15.31:0/3964982148 learned_addr learned my addr 172.21.15.31:0/3964982148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf93fff700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf94103780 0x7fdf940707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf988fe700 1 -- 172.21.15.31:0/3964982148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf94103780 msgr2=0x7fdf940707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf988fe700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf94103780 0x7fdf940707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf988fe700 1 -- 172.21.15.31:0/3964982148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88005040 con 0x7fdf94102de0 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf93fff700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf94103780 0x7fdf940707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.633+0000 7fdf988fe700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 0x7fdf94070260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009180 tx=0x7fdf88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.634+0000 7fdf917fa700 1 -- 172.21.15.31:0/3964982148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf8800e040 con 0x7fdf94102de0 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.634+0000 7fdf917fa700 1 -- 172.21.15.31:0/3964982148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf8800bbc0 con 0x7fdf94102de0 2024-04-24T23:25:49.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.634+0000 7fdf917fa700 1 -- 172.21.15.31:0/3964982148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf8801c630 con 0x7fdf94102de0 2024-04-24T23:25:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.634+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf94070f60 con 0x7fdf94102de0 2024-04-24T23:25:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.634+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9411a130 con 0x7fdf94102de0 2024-04-24T23:25:49.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.635+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf941000f0 con 0x7fdf94102de0 2024-04-24T23:25:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.635+0000 7fdf917fa700 1 -- 172.21.15.31:0/3964982148 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdf8800bd30 con 0x7fdf94102de0 2024-04-24T23:25:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.636+0000 7fdf917fa700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf8406e140 0x7fdf840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.636+0000 7fdf917fa700 1 -- 172.21.15.31:0/3964982148 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdf8808eeb0 con 0x7fdf94102de0 2024-04-24T23:25:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.636+0000 7fdf93fff700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf8406e140 0x7fdf840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:49.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.636+0000 7fdf93fff700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf8406e140 0x7fdf840705f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009fd0 tx=0x7fdf8000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:49.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.641+0000 7fdf917fa700 1 -- 172.21.15.31:0/3964982148 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdf880582a0 con 0x7fdf94102de0 2024-04-24T23:25:49.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:49 smithi040 ceph-mon[83047]: pgmap v265: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:49.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.857+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 --> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf94071760 con 0x7fdf8406e140 2024-04-24T23:25:49.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.867+0000 7fdf917fa700 1 -- 172.21.15.31:0/3964982148 <== mgr.34102 v2:172.21.15.31:6800/4072072744 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fdf94071760 con 0x7fdf8406e140 2024-04-24T23:25:49.868 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-24T23:25:49.868 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (8m) 2m ago 20m 25.6M - 0.25.0 c8568f914cd2 0432e46810d0 2024-04-24T23:25:49.868 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (6m) 2m ago 20m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 112ad4802884 2024-04-24T23:25:49.868 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi040 smithi040 running (6m) 104s ago 19m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef d16356242200 2024-04-24T23:25:49.868 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (8m) 2m ago 20m 79.7M - 9.4.12 2bacad6d85d8 26a838d0d320 2024-04-24T23:25:49.868 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.acqiox smithi031 running (2m) 2m ago 14m 11.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef 956ce7ff5df1 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.xxejxr smithi031 running (2m) 2m ago 15m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 1d7ff0e06219 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.dicevu smithi040 running (2m) 104s ago 15m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 4ee160da943d 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi040.osunbp smithi040 running (107s) 104s ago 14m 11.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 924dcede7c05 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.rrnmwr smithi031 *:8443,9283,8765 running (10m) 2m ago 22m 507M - 19.0.0-2445-gd2c45d97 f68e37b807ef 34e599df5cec 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi040.qtdwuq smithi040 *:8443,9283,8765 running (9m) 104s ago 19m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 959d9ca4f645 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (7m) 2m ago 22m 52.3M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 78d60c8cc2f3 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi040 smithi040 running (7m) 104s ago 19m 42.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 62515ef0bf53 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (9m) 2m ago 20m 16.8M - 1.5.0 0da6a335fe13 dd406755cd74 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi040 smithi040 *:9100 running (9m) 104s ago 19m 14.7M - 1.5.0 0da6a335fe13 d35669818ef7 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 2m ago 18m 96.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef d216357cdbb6 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 2m ago 18m 111M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef ad59872a4b02 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 2m ago 17m 91.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 271a0764ef5c 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi040 running (4m) 104s ago 17m 129M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 97c6f05632f6 2024-04-24T23:25:49.869 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi040 running (3m) 104s ago 16m 103M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 85d9bab7b352 2024-04-24T23:25:49.870 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi040 running (3m) 104s ago 16m 95.4M 2583M 19.0.0-2445-gd2c45d97 f68e37b807ef 32b6ff45fea8 2024-04-24T23:25:49.870 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (9m) 2m ago 20m 60.1M - 2.43.0 a07b618ecd1d 7f87d3f2e48d 2024-04-24T23:25:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf8406e140 msgr2=0x7fdf840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf8406e140 0x7fdf840705f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009fd0 tx=0x7fdf8000b040 comp rx=0 tx=0).stop 2024-04-24T23:25:49.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 msgr2=0x7fdf94070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 0x7fdf94070260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009180 tx=0x7fdf88009e30 comp rx=0 tx=0).stop 2024-04-24T23:25:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 shutdown_connections 2024-04-24T23:25:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94102de0 0x7fdf94070260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fdf8406e140 0x7fdf840705f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 --2- 172.21.15.31:0/3964982148 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdf94103780 0x7fdf940707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 >> 172.21.15.31:0/3964982148 conn(0x7fdf940f8990 msgr2=0x7fdf94107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 shutdown_connections 2024-04-24T23:25:49.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:49.869+0000 7fdf9ab62700 1 -- 172.21.15.31:0/3964982148 wait complete. 2024-04-24T23:25:50.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions' 2024-04-24T23:25:50.837 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:25:51.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:51 smithi031 ceph-mon[111404]: pgmap v266: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:51.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:51 smithi031 ceph-mon[111404]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:25:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:51 smithi040 ceph-mon[83047]: pgmap v266: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:51.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:51 smithi040 ceph-mon[83047]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.563+0000 7fb44c06c700 1 -- 172.21.15.31:0/3095937167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444102e50 msgr2=0x7fb444103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.563+0000 7fb44c06c700 1 --2- 172.21.15.31:0/3095937167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444102e50 0x7fb444103220 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb440004940 tx=0x7fb440009e30 comp rx=0 tx=0).stop 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.563+0000 7fb44c06c700 1 -- 172.21.15.31:0/3095937167 shutdown_connections 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.563+0000 7fb44c06c700 1 --2- 172.21.15.31:0/3095937167 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb444065640 0x7fb444065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.563+0000 7fb44c06c700 1 --2- 172.21.15.31:0/3095937167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444102e50 0x7fb444103220 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.563+0000 7fb44c06c700 1 -- 172.21.15.31:0/3095937167 >> 172.21.15.31:0/3095937167 conn(0x7fb444071d00 msgr2=0x7fb444072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.563+0000 7fb44c06c700 1 -- 172.21.15.31:0/3095937167 shutdown_connections 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.563+0000 7fb44c06c700 1 -- 172.21.15.31:0/3095937167 wait complete. 2024-04-24T23:25:52.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb44c06c700 1 Processor -- start 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb44c06c700 1 -- start start 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb44c06c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444065640 0x7fb4440fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb44c06c700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb444102e50 0x7fb4440fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb44c06c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440fc910 con 0x7fb444065640 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb44c06c700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440fca50 con 0x7fb444102e50 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb449e08700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444065640 0x7fb4440fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb449e08700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444065640 0x7fb4440fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59572/0 (socket says 172.21.15.31:59572) 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb449e08700 1 -- 172.21.15.31:0/595660717 learned_addr learned my addr 172.21.15.31:0/595660717 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.564+0000 7fb449607700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb444102e50 0x7fb4440fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb449e08700 1 -- 172.21.15.31:0/595660717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb444102e50 msgr2=0x7fb4440fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb449e08700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb444102e50 0x7fb4440fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb449e08700 1 -- 172.21.15.31:0/595660717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb440005040 con 0x7fb444065640 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb449e08700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444065640 0x7fb4440fbdb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb44000b3b0 tx=0x7fb440009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb43a7fc700 1 -- 172.21.15.31:0/595660717 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb44000e040 con 0x7fb444065640 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb43a7fc700 1 -- 172.21.15.31:0/595660717 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb440013420 con 0x7fb444065640 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb444119ea0 con 0x7fb444065640 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb43a7fc700 1 -- 172.21.15.31:0/595660717 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb44001c630 con 0x7fb444065640 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.565+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44411a380 con 0x7fb444065640 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.566+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4440fe540 con 0x7fb444065640 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.567+0000 7fb43a7fc700 1 -- 172.21.15.31:0/595660717 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb440013590 con 0x7fb444065640 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.567+0000 7fb43a7fc700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb43006e140 0x7fb4300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.567+0000 7fb43a7fc700 1 -- 172.21.15.31:0/595660717 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb44008ecb0 con 0x7fb444065640 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.567+0000 7fb449607700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb43006e140 0x7fb4300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.567+0000 7fb449607700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb43006e140 0x7fb4300705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb4440fd6f0 tx=0x7fb4340091a0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.572+0000 7fb43a7fc700 1 -- 172.21.15.31:0/595660717 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb4400580a0 con 0x7fb444065640 2024-04-24T23:25:52.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.867+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb44404bf00 con 0x7fb444065640 2024-04-24T23:25:52.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.868+0000 7fb43a7fc700 1 -- 172.21.15.31:0/595660717 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb44005b760 con 0x7fb444065640 2024-04-24T23:25:52.869 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-04-24T23:25:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-04-24T23:25:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:25:52.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-04-24T23:25:52.870 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-04-24T23:25:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb43006e140 msgr2=0x7fb4300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb43006e140 0x7fb4300705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb4440fd6f0 tx=0x7fb4340091a0 comp rx=0 tx=0).stop 2024-04-24T23:25:52.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444065640 msgr2=0x7fb4440fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444065640 0x7fb4440fbdb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb44000b3b0 tx=0x7fb440009e30 comp rx=0 tx=0).stop 2024-04-24T23:25:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 shutdown_connections 2024-04-24T23:25:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb444065640 0x7fb4440fbdb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb43006e140 0x7fb4300705f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 --2- 172.21.15.31:0/595660717 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb444102e50 0x7fb4440fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 >> 172.21.15.31:0/595660717 conn(0x7fb444071d00 msgr2=0x7fb4440fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 shutdown_connections 2024-04-24T23:25:52.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:52.870+0000 7fb44c06c700 1 -- 172.21.15.31:0/595660717 wait complete. 2024-04-24T23:25:53.466 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:53 smithi031 ceph-mon[111404]: pgmap v267: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:53.467 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:53 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/595660717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:25:53.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-24T23:25:53.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:53 smithi040 ceph-mon[83047]: pgmap v267: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:53.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:53 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/595660717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:25:53.762 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:25:55.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.353+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/3291393376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc102df0 msgr2=0x7f4bcc1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:55.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.353+0000 7f4bd2a9b700 1 --2- 172.21.15.31:0/3291393376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc102df0 0x7f4bcc1031c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0004940 tx=0x7f4bc0009e30 comp rx=0 tx=0).stop 2024-04-24T23:25:55.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.353+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/3291393376 shutdown_connections 2024-04-24T23:25:55.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.353+0000 7f4bd2a9b700 1 --2- 172.21.15.31:0/3291393376 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc103790 0x7f4bcc0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.353+0000 7f4bd2a9b700 1 --2- 172.21.15.31:0/3291393376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc102df0 0x7f4bcc1031c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.353+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/3291393376 >> 172.21.15.31:0/3291393376 conn(0x7f4bcc0f8980 msgr2=0x7f4bcc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.353+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/3291393376 shutdown_connections 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.353+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/3291393376 wait complete. 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bd2a9b700 1 Processor -- start 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bd2a9b700 1 -- start start 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bd2a9b700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc102df0 0x7f4bcc115db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bd2a9b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103790 0x7f4bcc1162f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bd2a9b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc116940 con 0x7f4bcc103790 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bd2a9b700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bcc10fd10 con 0x7f4bcc102df0 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bcbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103790 0x7f4bcc1162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bcbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103790 0x7f4bcc1162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59588/0 (socket says 172.21.15.31:59588) 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bcbfff700 1 -- 172.21.15.31:0/67473086 learned_addr learned my addr 172.21.15.31:0/67473086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:25:55.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.354+0000 7f4bd0837700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc102df0 0x7f4bcc115db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bcbfff700 1 -- 172.21.15.31:0/67473086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc102df0 msgr2=0x7f4bcc115db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bcbfff700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc102df0 0x7f4bcc115db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bcbfff700 1 -- 172.21.15.31:0/67473086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc0005040 con 0x7f4bcc103790 2024-04-24T23:25:55.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bcbfff700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103790 0x7f4bcc1162f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8002a50 tx=0x7f4bb8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:55.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bc97fa700 1 -- 172.21.15.31:0/67473086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bb8007cc0 con 0x7f4bcc103790 2024-04-24T23:25:55.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bcc10fff0 con 0x7f4bcc103790 2024-04-24T23:25:55.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bc97fa700 1 -- 172.21.15.31:0/67473086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bb800f040 con 0x7f4bcc103790 2024-04-24T23:25:55.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bc97fa700 1 -- 172.21.15.31:0/67473086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bb800d9c0 con 0x7f4bcc103790 2024-04-24T23:25:55.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.355+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bcc1105b0 con 0x7f4bcc103790 2024-04-24T23:25:55.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.356+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bcc106a90 con 0x7f4bcc103790 2024-04-24T23:25:55.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.356+0000 7f4bc97fa700 1 -- 172.21.15.31:0/67473086 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4bb8009730 con 0x7f4bcc103790 2024-04-24T23:25:55.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.357+0000 7f4bc97fa700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4bbc06e140 0x7f4bbc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:55.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.357+0000 7f4bc97fa700 1 -- 172.21.15.31:0/67473086 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4bb808d950 con 0x7f4bcc103790 2024-04-24T23:25:55.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.357+0000 7f4bd0837700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4bbc06e140 0x7f4bbc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:55.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.357+0000 7f4bd0837700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4bbc06e140 0x7f4bbc0705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0004f90 tx=0x7f4bc0009e30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:55.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.362+0000 7f4bc97fa700 1 -- 172.21.15.31:0/67473086 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4bb805a750 con 0x7f4bcc103790 2024-04-24T23:25:55.363 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:55 smithi031 ceph-mon[111404]: pgmap v268: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:55.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.660+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4bcc111400 con 0x7f4bcc103790 2024-04-24T23:25:55.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.660+0000 7f4bc97fa700 1 -- 172.21.15.31:0/67473086 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4bb805a2e0 con 0x7f4bcc103790 2024-04-24T23:25:55.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:55 smithi040 ceph-mon[83047]: pgmap v268: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4bbc06e140 msgr2=0x7f4bbc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4bbc06e140 0x7f4bbc0705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0004f90 tx=0x7f4bc0009e30 comp rx=0 tx=0).stop 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103790 msgr2=0x7f4bcc1162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103790 0x7f4bcc1162f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8002a50 tx=0x7f4bb8004600 comp rx=0 tx=0).stop 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 shutdown_connections 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4bbc06e140 0x7f4bbc0705f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4bcc102df0 0x7f4bcc115db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 --2- 172.21.15.31:0/67473086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bcc103790 0x7f4bcc1162f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 >> 172.21.15.31:0/67473086 conn(0x7f4bcc0f8980 msgr2=0x7f4bcc1073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 shutdown_connections 2024-04-24T23:25:55.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:55.663+0000 7f4bd2a9b700 1 -- 172.21.15.31:0/67473086 wait complete. 2024-04-24T23:25:55.674 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-04-24T23:25:56.250 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-24T23:25:56.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:56 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/67473086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:25:56.515 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:25:56.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:56 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/67473086' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:25:57.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:57 smithi031 ceph-mon[111404]: pgmap v269: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:57.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:57 smithi040 ceph-mon[83047]: pgmap v269: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.134+0000 7f8ad0468700 1 -- 172.21.15.31:0/945022196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 msgr2=0x7f8ac81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.134+0000 7f8ad0468700 1 --2- 172.21.15.31:0/945022196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 0x7f8ac81031b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4004940 tx=0x7f8ac4009e30 comp rx=0 tx=0).stop 2024-04-24T23:25:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.134+0000 7f8ad0468700 1 -- 172.21.15.31:0/945022196 shutdown_connections 2024-04-24T23:25:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.134+0000 7f8ad0468700 1 --2- 172.21.15.31:0/945022196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8103780 0x7f8ac80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.134+0000 7f8ad0468700 1 --2- 172.21.15.31:0/945022196 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 0x7f8ac81031b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.134+0000 7f8ad0468700 1 -- 172.21.15.31:0/945022196 >> 172.21.15.31:0/945022196 conn(0x7f8ac80f8990 msgr2=0x7f8ac80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:58.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.134+0000 7f8ad0468700 1 -- 172.21.15.31:0/945022196 shutdown_connections 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.134+0000 7f8ad0468700 1 -- 172.21.15.31:0/945022196 wait complete. 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ad0468700 1 Processor -- start 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ad0468700 1 -- start start 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ad0468700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 0x7f8ac8115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ad0468700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8103780 0x7f8ac81160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ad0468700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac8116720 con 0x7f8ac8102de0 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ad0468700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac810fd00 con 0x7f8ac8103780 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ace204700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 0x7f8ac8115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ace204700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 0x7f8ac8115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59604/0 (socket says 172.21.15.31:59604) 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ace204700 1 -- 172.21.15.31:0/4278357214 learned_addr learned my addr 172.21.15.31:0/4278357214 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8acda03700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8103780 0x7f8ac81160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:58.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ace204700 1 -- 172.21.15.31:0/4278357214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8103780 msgr2=0x7f8ac81160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ace204700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8103780 0x7f8ac81160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.135+0000 7f8ace204700 1 -- 172.21.15.31:0/4278357214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4005040 con 0x7f8ac8102de0 2024-04-24T23:25:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.136+0000 7f8ace204700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 0x7f8ac8115b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400b8b0 tx=0x7f8ac4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.136+0000 7f8abeffd700 1 -- 172.21.15.31:0/4278357214 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac400e040 con 0x7f8ac8102de0 2024-04-24T23:25:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.136+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac810ff80 con 0x7f8ac8102de0 2024-04-24T23:25:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.136+0000 7f8abeffd700 1 -- 172.21.15.31:0/4278357214 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ac4004290 con 0x7f8ac8102de0 2024-04-24T23:25:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.136+0000 7f8abeffd700 1 -- 172.21.15.31:0/4278357214 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ac401c3f0 con 0x7f8ac8102de0 2024-04-24T23:25:58.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.136+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac8110540 con 0x7f8ac8102de0 2024-04-24T23:25:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.137+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac8106a30 con 0x7f8ac8102de0 2024-04-24T23:25:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.138+0000 7f8abeffd700 1 -- 172.21.15.31:0/4278357214 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8ac4013420 con 0x7f8ac8102de0 2024-04-24T23:25:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.138+0000 7f8abeffd700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8ab406e140 0x7f8ab40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:25:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.138+0000 7f8abeffd700 1 -- 172.21.15.31:0/4278357214 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f8ac408edf0 con 0x7f8ac8102de0 2024-04-24T23:25:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.138+0000 7f8acda03700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8ab406e140 0x7f8ab40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:25:58.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.139+0000 7f8acda03700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8ab406e140 0x7f8ab40705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8111430 tx=0x7f8ab800b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:25:58.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.143+0000 7f8abeffd700 1 -- 172.21.15.31:0/4278357214 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8ac405bc70 con 0x7f8ac8102de0 2024-04-24T23:25:58.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.441+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8ac8109ee0 con 0x7f8ac8102de0 2024-04-24T23:25:58.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.442+0000 7f8abeffd700 1 -- 172.21.15.31:0/4278357214 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8ac405b800 con 0x7f8ac8102de0 2024-04-24T23:25:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8ab406e140 msgr2=0x7f8ab40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8ab406e140 0x7f8ab40705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8111430 tx=0x7f8ab800b040 comp rx=0 tx=0).stop 2024-04-24T23:25:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 msgr2=0x7f8ac8115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:25:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 0x7f8ac8115b90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400b8b0 tx=0x7f8ac4009e30 comp rx=0 tx=0).stop 2024-04-24T23:25:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 shutdown_connections 2024-04-24T23:25:58.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ac8102de0 0x7f8ac8115b90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:58.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f8ab406e140 0x7f8ab40705f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:58.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 --2- 172.21.15.31:0/4278357214 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8ac8103780 0x7f8ac81160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:25:58.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 >> 172.21.15.31:0/4278357214 conn(0x7f8ac80f8990 msgr2=0x7f8ac8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:25:58.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 shutdown_connections 2024-04-24T23:25:58.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:25:58.444+0000 7f8ad0468700 1 -- 172.21.15.31:0/4278357214 wait complete. 2024-04-24T23:25:58.456 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)" 2024-04-24T23:25:59.144 DEBUG:teuthology.parallel:result is None 2024-04-24T23:25:59.144 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-24T23:25:59.156 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:25:59.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-24T23:25:59.424 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/mon.smithi031/config 2024-04-24T23:25:59.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:59 smithi031 ceph-mon[111404]: pgmap v270: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:59.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:59 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4278357214' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:25:59.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:25:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:25:59.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:59 smithi040 ceph-mon[83047]: pgmap v270: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:25:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:59 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4278357214' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-24T23:25:59.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:25:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:26:01.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 -- 172.21.15.31:0/3170975530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 msgr2=0x7f3280106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 --2- 172.21.15.31:0/3170975530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 0x7f3280106e90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3270004970 tx=0x7f3270009e30 comp rx=0 tx=0).stop 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 -- 172.21.15.31:0/3170975530 shutdown_connections 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 --2- 172.21.15.31:0/3170975530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 0x7f3280106e90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 --2- 172.21.15.31:0/3170975530 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32800fde20 0x7f32800fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 -- 172.21.15.31:0/3170975530 >> 172.21.15.31:0/3170975530 conn(0x7f32800f7860 msgr2=0x7f32800f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 -- 172.21.15.31:0/3170975530 shutdown_connections 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 -- 172.21.15.31:0/3170975530 wait complete. 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.071+0000 7f32863d4700 1 Processor -- start 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f32863d4700 1 -- start start 2024-04-24T23:26:01.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f32863d4700 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32800fde20 0x7f3280115a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f32863d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 0x7f3280115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f32863d4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3280116600 con 0x7f32800fe7c0 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f32863d4700 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328010fc90 con 0x7f32800fde20 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f327f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 0x7f3280115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f327f7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 0x7f3280115fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43838/0 (socket says 172.21.15.31:43838) 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f327f7fe700 1 -- 172.21.15.31:0/3774312527 learned_addr learned my addr 172.21.15.31:0/3774312527 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f327ffff700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32800fde20 0x7f3280115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f327f7fe700 1 -- 172.21.15.31:0/3774312527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32800fde20 msgr2=0x7f3280115a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f327f7fe700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32800fde20 0x7f3280115a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.072+0000 7f327f7fe700 1 -- 172.21.15.31:0/3774312527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3270005040 con 0x7f32800fe7c0 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.073+0000 7f327f7fe700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 0x7f3280115fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3270000c00 tx=0x7f3270009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.073+0000 7f327cff9700 1 -- 172.21.15.31:0/3774312527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f327000e040 con 0x7f32800fe7c0 2024-04-24T23:26:01.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.073+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328010ff10 con 0x7f32800fe7c0 2024-04-24T23:26:01.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.073+0000 7f327cff9700 1 -- 172.21.15.31:0/3774312527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3270009400 con 0x7f32800fe7c0 2024-04-24T23:26:01.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.073+0000 7f327cff9700 1 -- 172.21.15.31:0/3774312527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f327001c3f0 con 0x7f32800fe7c0 2024-04-24T23:26:01.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.073+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3280110450 con 0x7f32800fe7c0 2024-04-24T23:26:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.074+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32801069b0 con 0x7f32800fe7c0 2024-04-24T23:26:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.074+0000 7f327cff9700 1 -- 172.21.15.31:0/3774312527 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f327000bc10 con 0x7f32800fe7c0 2024-04-24T23:26:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.075+0000 7f327cff9700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f326c06e140 0x7f326c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.075+0000 7f327cff9700 1 -- 172.21.15.31:0/3774312527 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f327008edf0 con 0x7f32800fe7c0 2024-04-24T23:26:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.075+0000 7f327ffff700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f326c06e140 0x7f326c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:01.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.076+0000 7f327ffff700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f326c06e140 0x7f326c0705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32680041d0 tx=0x7f32680043c0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:01.081 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.080+0000 7f327cff9700 1 -- 172.21.15.31:0/3774312527 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3270058140 con 0x7f32800fe7c0 2024-04-24T23:26:01.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.335+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3280111170 con 0x7f32800fe7c0 2024-04-24T23:26:01.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.335+0000 7f327cff9700 1 -- 172.21.15.31:0/3774312527 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1934 (secure 0 0 0) 0x7f327005b800 con 0x7f32800fe7c0 2024-04-24T23:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout:e34 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 34 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-04-24T23:10:39.902781+0000 2024-04-24T23:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-04-24T23:24:09.885870+0000 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 84 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-04-24T23:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34386} 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 34386 members: 34386 2024-04-24T23:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.xxejxr{0:34386} state up:active seq 20 join_fscid=1 addr [v2:172.21.15.31:6826/2304914531,v1:172.21.15.31:6827/2304914531] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-04-24T23:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.acqiox{-1:34410} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/3258483612,v1:172.21.15.31:6829/3258483612] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.dicevu{-1:44317} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6824/3497670820,v1:172.21.15.40:6825/3497670820] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi040.osunbp{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.40:6826/342930745,v1:172.21.15.40:6827/342930745] compat {c=[1],r=[1],i=[1fff]}] 2024-04-24T23:26:01.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f326c06e140 msgr2=0x7f326c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f326c06e140 0x7f326c0705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32680041d0 tx=0x7f32680043c0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 msgr2=0x7f3280115fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:01.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 0x7f3280115fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3270000c00 tx=0x7f3270009e30 comp rx=0 tx=0).stop 2024-04-24T23:26:01.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 shutdown_connections 2024-04-24T23:26:01.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f326c06e140 0x7f326c0705f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f32800fde20 0x7f3280115a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 --2- 172.21.15.31:0/3774312527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32800fe7c0 0x7f3280115fb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:01.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 >> 172.21.15.31:0/3774312527 conn(0x7f32800f7860 msgr2=0x7f32800fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:01.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 shutdown_connections 2024-04-24T23:26:01.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:01.338+0000 7f32863d4700 1 -- 172.21.15.31:0/3774312527 wait complete. 2024-04-24T23:26:01.342 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2024-04-24T23:26:01.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:01 smithi031 ceph-mon[111404]: pgmap v271: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:01.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:01 smithi040 ceph-mon[83047]: pgmap v271: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:02.073 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-24T23:26:02.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.185+0000 7f873dd54640 1 -- 172.21.15.31:0/575369337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102b20 msgr2=0x7f87380fa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.185+0000 7f873dd54640 1 --2- 172.21.15.31:0/575369337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102b20 0x7f87380fa7f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f87280048d0 tx=0x7f8728009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.185+0000 7f873dd54640 1 -- 172.21.15.31:0/575369337 shutdown_connections 2024-04-24T23:26:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.185+0000 7f873dd54640 1 --2- 172.21.15.31:0/575369337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102b20 0x7f87380fa7f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.185+0000 7f873dd54640 1 --2- 172.21.15.31:0/575369337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8738102210 0x7f87381025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.185+0000 7f873dd54640 1 -- 172.21.15.31:0/575369337 >> 172.21.15.31:0/575369337 conn(0x7f8738071770 msgr2=0x7f8738071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.185+0000 7f873dd54640 1 -- 172.21.15.31:0/575369337 shutdown_connections 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.185+0000 7f873dd54640 1 -- 172.21.15.31:0/575369337 wait complete. 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f873dd54640 1 Processor -- start 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f873dd54640 1 -- start start 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f873dd54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102210 0x7f87380fb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f873dd54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8738102b20 0x7f87380fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f873dd54640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87380fd8b0 con 0x7f8738102210 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f873dd54640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87380fc450 con 0x7f8738102b20 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f87377fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102210 0x7f87380fb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f87377fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102210 0x7f87380fb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43870/0 (socket says 172.21.15.31:43870) 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.186+0000 7f87377fe640 1 -- 172.21.15.31:0/4194828225 learned_addr learned my addr 172.21.15.31:0/4194828225 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f8736ffd640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8738102b20 0x7f87380fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f87377fe640 1 -- 172.21.15.31:0/4194828225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8738102b20 msgr2=0x7f87380fbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f87377fe640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8738102b20 0x7f87380fbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f87377fe640 1 -- 172.21.15.31:0/4194828225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8728005040 con 0x7f8738102210 2024-04-24T23:26:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f8736ffd640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8738102b20 0x7f87380fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f87377fe640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102210 0x7f87380fb9d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f872c002a50 tx=0x7f872c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f871ffff640 1 -- 172.21.15.31:0/4194828225 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872c007c30 con 0x7f8738102210 2024-04-24T23:26:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f871ffff640 1 -- 172.21.15.31:0/4194828225 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f872c00f040 con 0x7f8738102210 2024-04-24T23:26:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87380fc730 con 0x7f8738102210 2024-04-24T23:26:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f871ffff640 1 -- 172.21.15.31:0/4194828225 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f872c00db50 con 0x7f8738102210 2024-04-24T23:26:02.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.187+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87381194a0 con 0x7f8738102210 2024-04-24T23:26:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.188+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87380fdb10 con 0x7f8738102210 2024-04-24T23:26:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.189+0000 7f871ffff640 1 -- 172.21.15.31:0/4194828225 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f872c01c400 con 0x7f8738102210 2024-04-24T23:26:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.189+0000 7f871ffff640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f872006e0b0 0x7f8720070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.189+0000 7f871ffff640 1 -- 172.21.15.31:0/4194828225 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f872c08d4d0 con 0x7f8738102210 2024-04-24T23:26:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.189+0000 7f8736ffd640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f872006e0b0 0x7f8720070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.190+0000 7f8736ffd640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f872006e0b0 0x7f8720070560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f87380fd100 tx=0x7f8728009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.194+0000 7f871ffff640 1 -- 172.21.15.31:0/4194828225 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f872c0567a0 con 0x7f8738102210 2024-04-24T23:26:02.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.380+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f873804bcc0 con 0x7f8738102210 2024-04-24T23:26:02.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.381+0000 7f871ffff640 1 -- 172.21.15.31:0/4194828225 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16758 (secure 0 0 0) 0x7f872c059e60 con 0x7f8738102210 2024-04-24T23:26:02.382 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:02.383 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":84,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:03:41.065781+0000","modified":"2024-04-24T23:23:51.321472+0000","last_up_change":"2024-04-24T23:22:41.379468+0000","last_in_change":"2024-04-24T23:09:04.280370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:08:05.473853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T23:10:38.874551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T23:10:39.325676+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f1ff679-6b2e-4059-a299-2d2356bbe91c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6803","nonce":4093691970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6805","nonce":4093691970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6809","nonce":4093691970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6807","nonce":4093691970}]},"public_addr":"172.21.15.31:6803/4093691970","cluster_addr":"172.21.15.31:6805/4093691970","heartbeat_back_addr":"172.21.15.31:6809/4093691970","heartbeat_front_addr":"172.21.15.31:6807/4093691970","state":["exists","up"]},{"osd":1,"uuid":"6885f0d2-70e8-4c54-a381-0b3683a0bc49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6811","nonce":1890497972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6813","nonce":1890497972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6817","nonce":1890497972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6815","nonce":1890497972}]},"public_addr":"172.21.15.31:6811/1890497972","cluster_addr":"172.21.15.31:6813/1890497972","heartbeat_back_addr":"172.21.15.31:6817/1890497972","heartbeat_front_addr":"172.21.15.31:6815/1890497972","state":["exists","up"]},{"osd":2,"uuid":"9b57c743-5826-4bf9-ac00-0cb539b30224","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6819","nonce":1599239694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6821","nonce":1599239694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6825","nonce":1599239694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6823","nonce":1599239694}]},"public_addr":"172.21.15.31:6819/1599239694","cluster_addr":"172.21.15.31:6821/1599239694","heartbeat_back_addr":"172.21.15.31:6825/1599239694","heartbeat_front_addr":"172.21.15.31:6823/1599239694","state":["exists","up"]},{"osd":3,"uuid":"1dc47b54-65da-4579-bcac-2258be1f4cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6801","nonce":4020191995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6803","nonce":4020191995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6807","nonce":4020191995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6805","nonce":4020191995}]},"public_addr":"172.21.15.40:6801/4020191995","cluster_addr":"172.21.15.40:6803/4020191995","heartbeat_back_addr":"172.21.15.40:6807/4020191995","heartbeat_front_addr":"172.21.15.40:6805/4020191995","state":["exists","up"]},{"osd":4,"uuid":"e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6809","nonce":3032298464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6811","nonce":3032298464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6815","nonce":3032298464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6813","nonce":3032298464}]},"public_addr":"172.21.15.40:6809/3032298464","cluster_addr":"172.21.15.40:6811/3032298464","heartbeat_back_addr":"172.21.15.40:6815/3032298464","heartbeat_front_addr":"172.21.15.40:6813/3032298464","state":["exists","up"]},{"osd":5,"uuid":"027b7b7a-afb6-4507-8cb2-274ebadbb2ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6817","nonce":3395166888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6819","nonce":3395166888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6823","nonce":3395166888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6821","nonce":3395166888}]},"public_addr":"172.21.15.40:6817/3395166888","cluster_addr":"172.21.15.40:6819/3395166888","heartbeat_back_addr":"172.21.15.40:6823/3395166888","heartbeat_front_addr":"172.21.15.40:6821/3395166888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:08.221856+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:36.407326+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:03.612047+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:29.069233+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:55.660200+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:21.507297+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6825/1413423535":"2024-04-25T23:23:28.676954+0000","172.21.15.31:6829/1027887614":"2024-04-25T23:23:07.318594+0000","172.21.15.31:0/1654686211":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/823375198":"2024-04-25T23:12:59.389091+0000","172.21.15.40:6828/2171597925":"2024-04-25T23:15:14.419837+0000","172.21.15.31:0/1340983696":"2024-04-25T23:04:46.066984+0000","172.21.15.40:6826/1172521773":"2024-04-25T23:23:51.321440+0000","172.21.15.31:6826/2779330105":"2024-04-25T23:10:55.127017+0000","172.21.15.31:0/2308793407":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6800/1306034850":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/2457576751":"2024-04-25T23:04:46.066984+0000","172.21.15.40:0/2449703740":"2024-04-25T23:15:14.419837+0000","172.21.15.31:0/611668999":"2024-04-25T23:04:46.066984+0000","172.21.15.40:0/1074123018":"2024-04-25T23:15:14.419837+0000","172.21.15.31:6800/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6801/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/1805904019":"2024-04-25T23:05:48.958372+0000","172.21.15.40:6824/1413423535":"2024-04-25T23:23:28.676954+0000","172.21.15.31:0/917746224":"2024-04-25T23:18:13.679561+0000","172.21.15.31:6801/1306034850":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/582471448":"2024-04-25T23:12:59.389091+0000","172.21.15.31:6800/1061697388":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/2443214115":"2024-04-25T23:12:59.389091+0000","172.21.15.31:6801/1061697388":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/271213718":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/39610504":"2024-04-25T23:05:48.958372+0000","172.21.15.40:6829/2171597925":"2024-04-25T23:15:14.419837+0000","172.21.15.40:0/3498771068":"2024-04-25T23:15:14.419837+0000","172.21.15.31:6800/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6801/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6801/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6800/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6828/1027887614":"2024-04-25T23:23:07.318594+0000","172.21.15.40:0/3435188664":"2024-04-25T23:15:14.419837+0000","172.21.15.40:6827/1172521773":"2024-04-25T23:23:51.321440+0000","172.21.15.31:0/1480561026":"2024-04-25T23:18:13.679561+0000","172.21.15.31:6827/2779330105":"2024-04-25T23:10:55.127017+0000","172.21.15.31:0/382933206":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/3645410271":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/3145670559":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/3776080305":"2024-04-25T23:18:13.679561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f872006e0b0 msgr2=0x7f8720070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f872006e0b0 0x7f8720070560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f87380fd100 tx=0x7f8728009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102210 msgr2=0x7f87380fb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102210 0x7f87380fb9d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f872c002a50 tx=0x7f872c004600 comp rx=0 tx=0).stop 2024-04-24T23:26:02.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 shutdown_connections 2024-04-24T23:26:02.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f872006e0b0 0x7f8720070560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8738102b20 0x7f87380fbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 --2- 172.21.15.31:0/4194828225 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8738102210 0x7f87380fb9d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 >> 172.21.15.31:0/4194828225 conn(0x7f8738071770 msgr2=0x7f87380f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 shutdown_connections 2024-04-24T23:26:02.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.383+0000 7f873dd54640 1 -- 172.21.15.31:0/4194828225 wait complete. 2024-04-24T23:26:02.394 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:02 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3774312527' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:26:02.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:08:05.473853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T23:10:38.874551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T23:10:39.325676+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:02.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.496+0000 7f925a31d640 1 -- 172.21.15.31:0/2937395899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102220 msgr2=0x7f92541025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.496+0000 7f925a31d640 1 --2- 172.21.15.31:0/2937395899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102220 0x7f92541025f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f92440048d0 tx=0x7f9244009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.496+0000 7f925a31d640 1 -- 172.21.15.31:0/2937395899 shutdown_connections 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.496+0000 7f925a31d640 1 --2- 172.21.15.31:0/2937395899 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254102b30 0x7f92540fa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.496+0000 7f925a31d640 1 --2- 172.21.15.31:0/2937395899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102220 0x7f92541025f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.496+0000 7f925a31d640 1 -- 172.21.15.31:0/2937395899 >> 172.21.15.31:0/2937395899 conn(0x7f9254070890 msgr2=0x7f9254070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.496+0000 7f925a31d640 1 -- 172.21.15.31:0/2937395899 shutdown_connections 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.496+0000 7f925a31d640 1 -- 172.21.15.31:0/2937395899 wait complete. 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.497+0000 7f925a31d640 1 Processor -- start 2024-04-24T23:26:02.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.497+0000 7f925a31d640 1 -- start start 2024-04-24T23:26:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.497+0000 7f925a31d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254102220 0x7f9254112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.497+0000 7f925a31d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102b30 0x7f9254113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.497+0000 7f925a31d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9254113870 con 0x7f9254102b30 2024-04-24T23:26:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.497+0000 7f925a31d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925410cec0 con 0x7f9254102220 2024-04-24T23:26:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.497+0000 7f92537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102b30 0x7f9254113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f92537fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102b30 0x7f9254113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43882/0 (socket says 172.21.15.31:43882) 2024-04-24T23:26:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f92537fe640 1 -- 172.21.15.31:0/310595612 learned_addr learned my addr 172.21.15.31:0/310595612 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:02.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f9253fff640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254102220 0x7f9254112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f92537fe640 1 -- 172.21.15.31:0/310595612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254102220 msgr2=0x7f9254112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f92537fe640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254102220 0x7f9254112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f92537fe640 1 -- 172.21.15.31:0/310595612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9244005040 con 0x7f9254102b30 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f92537fe640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102b30 0x7f9254113270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f924800c760 tx=0x7f92480043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f9250ff9640 1 -- 172.21.15.31:0/310595612 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f924800a790 con 0x7f9254102b30 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925410d1a0 con 0x7f9254102b30 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f9250ff9640 1 -- 172.21.15.31:0/310595612 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f924800adc0 con 0x7f9254102b30 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f9250ff9640 1 -- 172.21.15.31:0/310595612 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92480115e0 con 0x7f9254102b30 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.498+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925410d6c0 con 0x7f9254102b30 2024-04-24T23:26:02.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.499+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9254065300 con 0x7f9254102b30 2024-04-24T23:26:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.500+0000 7f9250ff9640 1 -- 172.21.15.31:0/310595612 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9248012400 con 0x7f9254102b30 2024-04-24T23:26:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.500+0000 7f9250ff9640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f923c06e0b0 0x7f923c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.500+0000 7f9250ff9640 1 -- 172.21.15.31:0/310595612 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f924808dda0 con 0x7f9254102b30 2024-04-24T23:26:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.500+0000 7f9253fff640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f923c06e0b0 0x7f923c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.501+0000 7f9253fff640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f923c06e0b0 0x7f923c070560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92440043d0 tx=0x7f9244009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.505+0000 7f9250ff9640 1 -- 172.21.15.31:0/310595612 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f92480570f0 con 0x7f9254102b30 2024-04-24T23:26:02.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:02 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3774312527' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-24T23:26:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.688+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9254109340 con 0x7f9254102b30 2024-04-24T23:26:02.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.688+0000 7f9250ff9640 1 -- 172.21.15.31:0/310595612 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f924805a7b0 con 0x7f9254102b30 2024-04-24T23:26:02.689 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-24T23:26:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f923c06e0b0 msgr2=0x7f923c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f923c06e0b0 0x7f923c070560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92440043d0 tx=0x7f9244009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102b30 msgr2=0x7f9254113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102b30 0x7f9254113270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f924800c760 tx=0x7f92480043c0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 shutdown_connections 2024-04-24T23:26:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f923c06e0b0 0x7f923c070560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9254102b30 0x7f9254113270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 --2- 172.21.15.31:0/310595612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9254102220 0x7f9254112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.690+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 >> 172.21.15.31:0/310595612 conn(0x7f9254070890 msgr2=0x7f9254104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.691+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 shutdown_connections 2024-04-24T23:26:02.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.691+0000 7f925a31d640 1 -- 172.21.15.31:0/310595612 wait complete. 2024-04-24T23:26:02.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T23:26:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.802+0000 7fd9bb75d640 1 -- 172.21.15.31:0/2190436157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 msgr2=0x7fd9b40fd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.802+0000 7fd9bb75d640 1 --2- 172.21.15.31:0/2190436157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 0x7fd9b40fd570 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00048d0 tx=0x7fd9b0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.802+0000 7fd9bb75d640 1 -- 172.21.15.31:0/2190436157 shutdown_connections 2024-04-24T23:26:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.802+0000 7fd9bb75d640 1 --2- 172.21.15.31:0/2190436157 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b40fdb40 0x7fd9b4106180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.802+0000 7fd9bb75d640 1 --2- 172.21.15.31:0/2190436157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 0x7fd9b40fd570 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.802+0000 7fd9bb75d640 1 -- 172.21.15.31:0/2190436157 >> 172.21.15.31:0/2190436157 conn(0x7fd9b40f6cb0 msgr2=0x7fd9b40f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.802+0000 7fd9bb75d640 1 -- 172.21.15.31:0/2190436157 shutdown_connections 2024-04-24T23:26:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.802+0000 7fd9bb75d640 1 -- 172.21.15.31:0/2190436157 wait complete. 2024-04-24T23:26:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9bb75d640 1 Processor -- start 2024-04-24T23:26:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9bb75d640 1 -- start start 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9bb75d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 0x7fd9b4112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9bb75d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b40fdb40 0x7fd9b4113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9bb75d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b4113860 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9bb75d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b410ceb0 con 0x7fd9b40fdb40 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9b94f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 0x7fd9b4112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9b94f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 0x7fd9b4112d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43904/0 (socket says 172.21.15.31:43904) 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9b94f9640 1 -- 172.21.15.31:0/1642709465 learned_addr learned my addr 172.21.15.31:0/1642709465 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9b8cf8640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b40fdb40 0x7fd9b4113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9b94f9640 1 -- 172.21.15.31:0/1642709465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b40fdb40 msgr2=0x7fd9b4113260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9b94f9640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b40fdb40 0x7fd9b4113260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.803+0000 7fd9b94f9640 1 -- 172.21.15.31:0/1642709465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b0005040 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.804+0000 7fd9b94f9640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 0x7fd9b4112d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0004930 tx=0x7fd9b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.804+0000 7fd9a9ffb640 1 -- 172.21.15.31:0/1642709465 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9b00094c0 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.804+0000 7fd9a9ffb640 1 -- 172.21.15.31:0/1642709465 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9b0032e50 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.804+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b410d130 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.804+0000 7fd9a9ffb640 1 -- 172.21.15.31:0/1642709465 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9b00433b0 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.804+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b410d5f0 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.805+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b4065860 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.806+0000 7fd9a9ffb640 1 -- 172.21.15.31:0/1642709465 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd9b002f070 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.806+0000 7fd9a9ffb640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd9a006e0b0 0x7fd9a0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:02.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.806+0000 7fd9a9ffb640 1 -- 172.21.15.31:0/1642709465 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fd9b00afb00 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.806+0000 7fd9b8cf8640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd9a006e0b0 0x7fd9a0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:02.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.807+0000 7fd9b8cf8640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd9a006e0b0 0x7fd9a0070560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd9b410e5e0 tx=0x7fd9a4004330 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:02.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.811+0000 7fd9a9ffb640 1 -- 172.21.15.31:0/1642709465 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd9b0078e80 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.995+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fd9b410e390 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.996+0000 7fd9a9ffb640 1 -- 172.21.15.31:0/1642709465 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fd9b007c540 con 0x7fd9b40fd1a0 2024-04-24T23:26:02.997 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd9a006e0b0 msgr2=0x7fd9a0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd9a006e0b0 0x7fd9a0070560 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd9b410e5e0 tx=0x7fd9a4004330 comp rx=0 tx=0).stop 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 msgr2=0x7fd9b4112d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 0x7fd9b4112d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0004930 tx=0x7fd9b0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 shutdown_connections 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fd9a006e0b0 0x7fd9a0070560 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd9b40fdb40 0x7fd9b4113260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 --2- 172.21.15.31:0/1642709465 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9b40fd1a0 0x7fd9b4112d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 >> 172.21.15.31:0/1642709465 conn(0x7fd9b40f6cb0 msgr2=0x7fd9b40f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 shutdown_connections 2024-04-24T23:26:02.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:02.998+0000 7fd9bb75d640 1 -- 172.21.15.31:0/1642709465 wait complete. 2024-04-24T23:26:03.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.109+0000 7ff6db4ae640 1 -- 172.21.15.31:0/3849424491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102b20 msgr2=0x7ff6d40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.109+0000 7ff6db4ae640 1 --2- 172.21.15.31:0/3849424491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102b20 0x7ff6d40fc1b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40048d0 tx=0x7ff6c4009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.109+0000 7ff6db4ae640 1 -- 172.21.15.31:0/3849424491 shutdown_connections 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.109+0000 7ff6db4ae640 1 --2- 172.21.15.31:0/3849424491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102b20 0x7ff6d40fc1b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.109+0000 7ff6db4ae640 1 --2- 172.21.15.31:0/3849424491 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d4102180 0x7ff6d4102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.109+0000 7ff6db4ae640 1 -- 172.21.15.31:0/3849424491 >> 172.21.15.31:0/3849424491 conn(0x7ff6d40f7db0 msgr2=0x7ff6d40fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.109+0000 7ff6db4ae640 1 -- 172.21.15.31:0/3849424491 shutdown_connections 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.109+0000 7ff6db4ae640 1 -- 172.21.15.31:0/3849424491 wait complete. 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6db4ae640 1 Processor -- start 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6db4ae640 1 -- start start 2024-04-24T23:26:03.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6db4ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102180 0x7ff6d4112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6db4ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d4102b20 0x7ff6d4113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6db4ae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4113770 con 0x7ff6d4102180 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6db4ae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d41138e0 con 0x7ff6d4102b20 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d924a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102180 0x7ff6d4112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d924a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102180 0x7ff6d4112c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43918/0 (socket says 172.21.15.31:43918) 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d924a640 1 -- 172.21.15.31:0/1021876318 learned_addr learned my addr 172.21.15.31:0/1021876318 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d8a49640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d4102b20 0x7ff6d4113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d924a640 1 -- 172.21.15.31:0/1021876318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d4102b20 msgr2=0x7ff6d4113170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d924a640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d4102b20 0x7ff6d4113170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d924a640 1 -- 172.21.15.31:0/1021876318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4005040 con 0x7ff6d4102180 2024-04-24T23:26:03.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d8a49640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d4102b20 0x7ff6d4113170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.110+0000 7ff6d924a640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102180 0x7ff6d4112c30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000c810 tx=0x7ff6d00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.111+0000 7ff6c9ffb640 1 -- 172.21.15.31:0/1021876318 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6d000a820 con 0x7ff6d4102180 2024-04-24T23:26:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.111+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d410cf70 con 0x7ff6d4102180 2024-04-24T23:26:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.111+0000 7ff6c9ffb640 1 -- 172.21.15.31:0/1021876318 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6d000ae50 con 0x7ff6d4102180 2024-04-24T23:26:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.111+0000 7ff6c9ffb640 1 -- 172.21.15.31:0/1021876318 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6d00115c0 con 0x7ff6d4102180 2024-04-24T23:26:03.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.111+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d410d490 con 0x7ff6d4102180 2024-04-24T23:26:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.112+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d40fd8f0 con 0x7ff6d4102180 2024-04-24T23:26:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.112+0000 7ff6c9ffb640 1 -- 172.21.15.31:0/1021876318 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff6d000a980 con 0x7ff6d4102180 2024-04-24T23:26:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.113+0000 7ff6c9ffb640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff6c006e0b0 0x7ff6c0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.113+0000 7ff6c9ffb640 1 -- 172.21.15.31:0/1021876318 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff6d008e1d0 con 0x7ff6d4102180 2024-04-24T23:26:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.113+0000 7ff6d8a49640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff6c006e0b0 0x7ff6c0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.113+0000 7ff6d8a49640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff6c006e0b0 0x7ff6c0070560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4000c00 tx=0x7ff6c4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.117+0000 7ff6c9ffb640 1 -- 172.21.15.31:0/1021876318 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff6d00574a0 con 0x7ff6d4102180 2024-04-24T23:26:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.301+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff6d404bcc0 con 0x7ff6d4102180 2024-04-24T23:26:03.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.302+0000 7ff6c9ffb640 1 -- 172.21.15.31:0/1021876318 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff6d005ab60 con 0x7ff6d4102180 2024-04-24T23:26:03.303 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-24T23:26:03.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff6c006e0b0 msgr2=0x7ff6c0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff6c006e0b0 0x7ff6c0070560 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4000c00 tx=0x7ff6c4009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102180 msgr2=0x7ff6d4112c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102180 0x7ff6d4112c30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000c810 tx=0x7ff6d00043c0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 shutdown_connections 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff6c006e0b0 0x7ff6c0070560 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff6d4102b20 0x7ff6d4113170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 --2- 172.21.15.31:0/1021876318 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff6d4102180 0x7ff6d4112c30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 >> 172.21.15.31:0/1021876318 conn(0x7ff6d40f7db0 msgr2=0x7ff6d40f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 shutdown_connections 2024-04-24T23:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.304+0000 7ff6db4ae640 1 -- 172.21.15.31:0/1021876318 wait complete. 2024-04-24T23:26:03.314 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:03 smithi031 ceph-mon[111404]: pgmap v272: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:03.314 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:03 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4194828225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:03.314 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:03 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/310595612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:03.314 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:03 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1642709465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T23:26:03.316 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-24T23:26:03.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.413+0000 7f442a59d640 1 -- 172.21.15.31:0/2238071168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 msgr2=0x7f4424064350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.413+0000 7f442a59d640 1 --2- 172.21.15.31:0/2238071168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 0x7f4424064350 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f44140048d0 tx=0x7f4414009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.413+0000 7f442a59d640 1 -- 172.21.15.31:0/2238071168 shutdown_connections 2024-04-24T23:26:03.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.413+0000 7f442a59d640 1 --2- 172.21.15.31:0/2238071168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 0x7f4424064350 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.413+0000 7f442a59d640 1 --2- 172.21.15.31:0/2238071168 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44241021f0 0x7f44241025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.413+0000 7f442a59d640 1 -- 172.21.15.31:0/2238071168 >> 172.21.15.31:0/2238071168 conn(0x7f44240711c0 msgr2=0x7f44240715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.413+0000 7f442a59d640 1 -- 172.21.15.31:0/2238071168 shutdown_connections 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.413+0000 7f442a59d640 1 -- 172.21.15.31:0/2238071168 wait complete. 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f442a59d640 1 Processor -- start 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f442a59d640 1 -- start start 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f442a59d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44241021f0 0x7f4424112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f442a59d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 0x7f4424113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f442a59d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4424113860 con 0x7f4424102b90 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f442a59d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442410cf00 con 0x7f44241021f0 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f44237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 0x7f4424113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f44237fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 0x7f4424113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43932/0 (socket says 172.21.15.31:43932) 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.414+0000 7f44237fe640 1 -- 172.21.15.31:0/888155931 learned_addr learned my addr 172.21.15.31:0/888155931 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:03.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f4423fff640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44241021f0 0x7f4424112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f44237fe640 1 -- 172.21.15.31:0/888155931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44241021f0 msgr2=0x7f4424112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f44237fe640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44241021f0 0x7f4424112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f44237fe640 1 -- 172.21.15.31:0/888155931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414005040 con 0x7f4424102b90 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f44237fe640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 0x7f4424113260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f441402bda0 tx=0x7f4414009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f4420ff9640 1 -- 172.21.15.31:0/888155931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44140094c0 con 0x7f4424102b90 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442410d180 con 0x7f4424102b90 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442410d6a0 con 0x7f4424102b90 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f4420ff9640 1 -- 172.21.15.31:0/888155931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4414032e50 con 0x7f4424102b90 2024-04-24T23:26:03.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.415+0000 7f4420ff9640 1 -- 172.21.15.31:0/888155931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441403b400 con 0x7f4424102b90 2024-04-24T23:26:03.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.416+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44240fd990 con 0x7f4424102b90 2024-04-24T23:26:03.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.417+0000 7f4420ff9640 1 -- 172.21.15.31:0/888155931 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f441402f070 con 0x7f4424102b90 2024-04-24T23:26:03.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.417+0000 7f4420ff9640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f441006e0b0 0x7f4410070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.417+0000 7f4420ff9640 1 -- 172.21.15.31:0/888155931 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f44140affa0 con 0x7f4424102b90 2024-04-24T23:26:03.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.417+0000 7f4423fff640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f441006e0b0 0x7f4410070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.418+0000 7f4423fff640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f441006e0b0 0x7f4410070560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f440c009470 tx=0x7f440c009ed0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.423 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.422+0000 7f4420ff9640 1 -- 172.21.15.31:0/888155931 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4414079270 con 0x7f4424102b90 2024-04-24T23:26:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.646+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4424109350 con 0x7f4424102b90 2024-04-24T23:26:03.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.647+0000 7f4420ff9640 1 -- 172.21.15.31:0/888155931 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5204 (secure 0 0 0) 0x7f441402d040 con 0x7f4424102b90 2024-04-24T23:26:03.648 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:03.648 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44317,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6825/3497670820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3497670820},{"type":"v1","addr":"172.21.15.40:6825","nonce":3497670820}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44339,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/342930745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":342930745},{"type":"v1","addr":"172.21.15.40:6827","nonce":342930745}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:24:09.885870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34386":{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":0,"incarnation":30,"state":"up:active","state_seq":20,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34386,"qdb_cluster":[34386]},"id":1}]} 2024-04-24T23:26:03.648 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2024-04-24T23:26:03.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f441006e0b0 msgr2=0x7f4410070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f441006e0b0 0x7f4410070560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f440c009470 tx=0x7f440c009ed0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 msgr2=0x7f4424113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 0x7f4424113260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f441402bda0 tx=0x7f4414009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 shutdown_connections 2024-04-24T23:26:03.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f441006e0b0 0x7f4410070560 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4424102b90 0x7f4424113260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 --2- 172.21.15.31:0/888155931 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f44241021f0 0x7f4424112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 >> 172.21.15.31:0/888155931 conn(0x7f44240711c0 msgr2=0x7f44240fa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 shutdown_connections 2024-04-24T23:26:03.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.649+0000 7f442a59d640 1 -- 172.21.15.31:0/888155931 wait complete. 2024-04-24T23:26:03.661 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 1, 'flags': 18} 2024-04-24T23:26:03.661 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-24T23:26:03.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:03 smithi040 ceph-mon[83047]: pgmap v272: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:03.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:03 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4194828225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:03.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:03 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/310595612' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:03.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:03 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1642709465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T23:26:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.762+0000 7f989235d640 1 -- 172.21.15.31:0/1261943036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c102b80 msgr2=0x7f988c0fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.762+0000 7f989235d640 1 --2- 172.21.15.31:0/1261943036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c102b80 0x7f988c0fc210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f987c0048d0 tx=0x7f987c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.762+0000 7f989235d640 1 -- 172.21.15.31:0/1261943036 shutdown_connections 2024-04-24T23:26:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.762+0000 7f989235d640 1 --2- 172.21.15.31:0/1261943036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c102b80 0x7f988c0fc210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.762+0000 7f989235d640 1 --2- 172.21.15.31:0/1261943036 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988c1021e0 0x7f988c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.762+0000 7f989235d640 1 -- 172.21.15.31:0/1261943036 >> 172.21.15.31:0/1261943036 conn(0x7f988c0f7db0 msgr2=0x7f988c0fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:03.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.762+0000 7f989235d640 1 -- 172.21.15.31:0/1261943036 shutdown_connections 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.762+0000 7f989235d640 1 -- 172.21.15.31:0/1261943036 wait complete. 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f989235d640 1 Processor -- start 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f989235d640 1 -- start start 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f989235d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c1021e0 0x7f988c06f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f989235d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988c102b80 0x7f988c06fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f989235d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c071650 con 0x7f988c1021e0 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f989235d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988c0701f0 con 0x7f988c102b80 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f988bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c1021e0 0x7f988c06f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f988bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c1021e0 0x7f988c06f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43950/0 (socket says 172.21.15.31:43950) 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.763+0000 7f988bfff640 1 -- 172.21.15.31:0/3334941378 learned_addr learned my addr 172.21.15.31:0/3334941378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f988b7fe640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988c102b80 0x7f988c06fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f988bfff640 1 -- 172.21.15.31:0/3334941378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988c102b80 msgr2=0x7f988c06fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f988bfff640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988c102b80 0x7f988c06fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f988bfff640 1 -- 172.21.15.31:0/3334941378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c005040 con 0x7f988c1021e0 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f988b7fe640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988c102b80 0x7f988c06fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f988bfff640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c1021e0 0x7f988c06f770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f987400c810 tx=0x7f98740043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f9888ff9640 1 -- 172.21.15.31:0/3334941378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f987400a820 con 0x7f988c1021e0 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f9888ff9640 1 -- 172.21.15.31:0/3334941378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f987400ae50 con 0x7f988c1021e0 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.764+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988c0704d0 con 0x7f988c1021e0 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.765+0000 7f9888ff9640 1 -- 172.21.15.31:0/3334941378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98740115c0 con 0x7f988c1021e0 2024-04-24T23:26:03.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.765+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988c119380 con 0x7f988c1021e0 2024-04-24T23:26:03.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.766+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988c0fd950 con 0x7f988c1021e0 2024-04-24T23:26:03.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.766+0000 7f9888ff9640 1 -- 172.21.15.31:0/3334941378 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9874011720 con 0x7f988c1021e0 2024-04-24T23:26:03.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.766+0000 7f9888ff9640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f987806e0b0 0x7f9878070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:03.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.766+0000 7f9888ff9640 1 -- 172.21.15.31:0/3334941378 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f987408e110 con 0x7f988c1021e0 2024-04-24T23:26:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.766+0000 7f988b7fe640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f987806e0b0 0x7f9878070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:03.770 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.767+0000 7f988b7fe640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f987806e0b0 0x7f9878070560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f987c000c00 tx=0x7f987c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.771+0000 7f9888ff9640 1 -- 172.21.15.31:0/3334941378 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f987405b000 con 0x7f988c1021e0 2024-04-24T23:26:03.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.996+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f988c104620 con 0x7f988c1021e0 2024-04-24T23:26:03.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.996+0000 7f9888ff9640 1 -- 172.21.15.31:0/3334941378 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v34) v1 ==== 107+0+4078 (secure 0 0 0) 0x7f987405ab90 con 0x7f988c1021e0 2024-04-24T23:26:03.997 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:03.998 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:10:58.131406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14464":{"gid":14464,"name":"cephfs.smithi031.acqiox","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6829/1027887614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1027887614},{"type":"v1","addr":"172.21.15.31:6829","nonce":1027887614}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:03.998 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 16 2024-04-24T23:26:03.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.998+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f987806e0b0 msgr2=0x7f9878070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.998+0000 7f989235d640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f987806e0b0 0x7f9878070560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f987c000c00 tx=0x7f987c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.998+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c1021e0 msgr2=0x7f988c06f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.998+0000 7f989235d640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c1021e0 0x7f988c06f770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f987400c810 tx=0x7f98740043c0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.999+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 shutdown_connections 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.999+0000 7f989235d640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f987806e0b0 0x7f9878070560 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.999+0000 7f989235d640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f988c102b80 0x7f988c06fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.999+0000 7f989235d640 1 --2- 172.21.15.31:0/3334941378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f988c1021e0 0x7f988c06f770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.999+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 >> 172.21.15.31:0/3334941378 conn(0x7f988c0f7db0 msgr2=0x7f988c0f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.999+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 shutdown_connections 2024-04-24T23:26:04.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:03.999+0000 7f989235d640 1 -- 172.21.15.31:0/3334941378 wait complete. 2024-04-24T23:26:04.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-24T23:26:04.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.110+0000 7fe22d99d640 1 -- 172.21.15.31:0/2015572273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101a10 msgr2=0x7fe2280fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.110+0000 7fe22d99d640 1 --2- 172.21.15.31:0/2015572273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101a10 0x7fe2280fb0a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe2180048d0 tx=0x7fe218009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.110+0000 7fe22d99d640 1 -- 172.21.15.31:0/2015572273 shutdown_connections 2024-04-24T23:26:04.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.110+0000 7fe22d99d640 1 --2- 172.21.15.31:0/2015572273 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101a10 0x7fe2280fb0a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.110+0000 7fe22d99d640 1 --2- 172.21.15.31:0/2015572273 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe228101070 0x7fe228101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.110+0000 7fe22d99d640 1 -- 172.21.15.31:0/2015572273 >> 172.21.15.31:0/2015572273 conn(0x7fe2280f6be0 msgr2=0x7fe2280f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.110+0000 7fe22d99d640 1 -- 172.21.15.31:0/2015572273 shutdown_connections 2024-04-24T23:26:04.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.110+0000 7fe22d99d640 1 -- 172.21.15.31:0/2015572273 wait complete. 2024-04-24T23:26:04.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe22d99d640 1 Processor -- start 2024-04-24T23:26:04.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe22d99d640 1 -- start start 2024-04-24T23:26:04.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe22d99d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101070 0x7fe2280fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe22d99d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe228101a10 0x7fe2280fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe22d99d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280fec70 con 0x7fe228101070 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe22d99d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2280fd790 con 0x7fe228101a10 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe226ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101070 0x7fe2280fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe226ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101070 0x7fe2280fcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43966/0 (socket says 172.21.15.31:43966) 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe226ffd640 1 -- 172.21.15.31:0/3083532967 learned_addr learned my addr 172.21.15.31:0/3083532967 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe2267fc640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe228101a10 0x7fe2280fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe226ffd640 1 -- 172.21.15.31:0/3083532967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe228101a10 msgr2=0x7fe2280fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe226ffd640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe228101a10 0x7fe2280fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe226ffd640 1 -- 172.21.15.31:0/3083532967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218005040 con 0x7fe228101070 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.111+0000 7fe226ffd640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101070 0x7fe2280fcd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c790 tx=0x7fe21c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.112+0000 7fe20f7fe640 1 -- 172.21.15.31:0/3083532967 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe21c00a7a0 con 0x7fe228101070 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.112+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2280fda70 con 0x7fe228101070 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.112+0000 7fe20f7fe640 1 -- 172.21.15.31:0/3083532967 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe21c00add0 con 0x7fe228101070 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.112+0000 7fe20f7fe640 1 -- 172.21.15.31:0/3083532967 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe21c0115e0 con 0x7fe228101070 2024-04-24T23:26:04.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.112+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe228119460 con 0x7fe228101070 2024-04-24T23:26:04.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.113+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2280fc7e0 con 0x7fe228101070 2024-04-24T23:26:04.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.113+0000 7fe20f7fe640 1 -- 172.21.15.31:0/3083532967 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe21c012400 con 0x7fe228101070 2024-04-24T23:26:04.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.114+0000 7fe20f7fe640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe21006e0b0 0x7fe210070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.114+0000 7fe2267fc640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe21006e0b0 0x7fe210070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.114+0000 7fe20f7fe640 1 -- 172.21.15.31:0/3083532967 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe21c08dd80 con 0x7fe228101070 2024-04-24T23:26:04.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.114+0000 7fe2267fc640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe21006e0b0 0x7fe210070560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2180048a0 tx=0x7fe218009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.118+0000 7fe20f7fe640 1 -- 172.21.15.31:0/3083532967 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe21c091030 con 0x7fe228101070 2024-04-24T23:26:04.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.342+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fe228064350 con 0x7fe228101070 2024-04-24T23:26:04.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.343+0000 7fe20f7fe640 1 -- 172.21.15.31:0/3083532967 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v34) v1 ==== 107+0+4925 (secure 0 0 0) 0x7fe21c057070 con 0x7fe228101070 2024-04-24T23:26:04.344 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:04.344 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:10:58.131406+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14464},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14464":{"gid":14464,"name":"cephfs.smithi031.acqiox","rank":0,"incarnation":12,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6829/1027887614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1027887614},{"type":"v1","addr":"172.21.15.31:6829","nonce":1027887614}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:04.344 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-04-24T23:26:04.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe21006e0b0 msgr2=0x7fe210070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe21006e0b0 0x7fe210070560 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe2180048a0 tx=0x7fe218009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101070 msgr2=0x7fe2280fcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101070 0x7fe2280fcd10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00c790 tx=0x7fe21c0043c0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 shutdown_connections 2024-04-24T23:26:04.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe21006e0b0 0x7fe210070560 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe228101a10 0x7fe2280fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 --2- 172.21.15.31:0/3083532967 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe228101070 0x7fe2280fcd10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 >> 172.21.15.31:0/3083532967 conn(0x7fe2280f6be0 msgr2=0x7fe2280f7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 shutdown_connections 2024-04-24T23:26:04.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.345+0000 7fe22d99d640 1 -- 172.21.15.31:0/3083532967 wait complete. 2024-04-24T23:26:04.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:04 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1021876318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T23:26:04.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:04 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/888155931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:04.356 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:04 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3334941378' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-24T23:26:04.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.455+0000 7fba4702e640 1 -- 172.21.15.31:0/105848177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40064390 msgr2=0x7fba40064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.455+0000 7fba4702e640 1 --2- 172.21.15.31:0/105848177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40064390 0x7fba40064800 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0048d0 tx=0x7fba2c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.455+0000 7fba4702e640 1 -- 172.21.15.31:0/105848177 shutdown_connections 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.455+0000 7fba4702e640 1 --2- 172.21.15.31:0/105848177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40064390 0x7fba40064800 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.455+0000 7fba4702e640 1 --2- 172.21.15.31:0/105848177 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba40102270 0x7fba40102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.455+0000 7fba4702e640 1 -- 172.21.15.31:0/105848177 >> 172.21.15.31:0/105848177 conn(0x7fba40071fd0 msgr2=0x7fba400723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.455+0000 7fba4702e640 1 -- 172.21.15.31:0/105848177 shutdown_connections 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.455+0000 7fba4702e640 1 -- 172.21.15.31:0/105848177 wait complete. 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.456+0000 7fba4702e640 1 Processor -- start 2024-04-24T23:26:04.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.456+0000 7fba4702e640 1 -- start start 2024-04-24T23:26:04.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.456+0000 7fba4702e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba40064390 0x7fba40112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.456+0000 7fba4702e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40102270 0x7fba401131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.456+0000 7fba4702e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba401137c0 con 0x7fba40102270 2024-04-24T23:26:04.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.456+0000 7fba4702e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4010ce10 con 0x7fba40064390 2024-04-24T23:26:04.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40102270 0x7fba401131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40102270 0x7fba401131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43982/0 (socket says 172.21.15.31:43982) 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3ffff640 1 -- 172.21.15.31:0/4121945023 learned_addr learned my addr 172.21.15.31:0/4121945023 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba44dca640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba40064390 0x7fba40112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3ffff640 1 -- 172.21.15.31:0/4121945023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba40064390 msgr2=0x7fba40112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3ffff640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba40064390 0x7fba40112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3ffff640 1 -- 172.21.15.31:0/4121945023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c005040 con 0x7fba40102270 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3ffff640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40102270 0x7fba401131c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fba2c004930 tx=0x7fba2c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3d7fa640 1 -- 172.21.15.31:0/4121945023 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2c0094c0 con 0x7fba40102270 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3d7fa640 1 -- 172.21.15.31:0/4121945023 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba2c032e50 con 0x7fba40102270 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4010d090 con 0x7fba40102270 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba3d7fa640 1 -- 172.21.15.31:0/4121945023 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2c0433b0 con 0x7fba40102270 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.457+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4010d550 con 0x7fba40102270 2024-04-24T23:26:04.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.459+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba400fd970 con 0x7fba40102270 2024-04-24T23:26:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.459+0000 7fba3d7fa640 1 -- 172.21.15.31:0/4121945023 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fba2c02f070 con 0x7fba40102270 2024-04-24T23:26:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.459+0000 7fba3d7fa640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fba3006e0b0 0x7fba30070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.459+0000 7fba3d7fa640 1 -- 172.21.15.31:0/4121945023 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fba2c0afaa0 con 0x7fba40102270 2024-04-24T23:26:04.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.459+0000 7fba44dca640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fba3006e0b0 0x7fba30070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.460+0000 7fba44dca640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fba3006e0b0 0x7fba30070560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba34007fd0 tx=0x7fba34009e90 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.465 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.464+0000 7fba3d7fa640 1 -- 172.21.15.31:0/4121945023 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fba2c078d70 con 0x7fba40102270 2024-04-24T23:26:04.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:04 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1021876318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T23:26:04.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:04 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/888155931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-24T23:26:04.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:04 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3334941378' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-24T23:26:04.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.688+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7fba401093a0 con 0x7fba40102270 2024-04-24T23:26:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.688+0000 7fba3d7fa640 1 -- 172.21.15.31:0/4121945023 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v34) v1 ==== 107+0+4135 (secure 0 0 0) 0x7fba2c07c430 con 0x7fba40102270 2024-04-24T23:26:04.689 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:04.689 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:07.318607+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:04.689 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.690+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fba3006e0b0 msgr2=0x7fba30070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.690+0000 7fba4702e640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fba3006e0b0 0x7fba30070560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba34007fd0 tx=0x7fba34009e90 comp rx=0 tx=0).stop 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.690+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40102270 msgr2=0x7fba401131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.690+0000 7fba4702e640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40102270 0x7fba401131c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fba2c004930 tx=0x7fba2c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.691+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 shutdown_connections 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.691+0000 7fba4702e640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fba3006e0b0 0x7fba30070560 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.691+0000 7fba4702e640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba40102270 0x7fba401131c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.691+0000 7fba4702e640 1 --2- 172.21.15.31:0/4121945023 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fba40064390 0x7fba40112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.691+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 >> 172.21.15.31:0/4121945023 conn(0x7fba40071fd0 msgr2=0x7fba400fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.691+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 shutdown_connections 2024-04-24T23:26:04.692 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.691+0000 7fba4702e640 1 -- 172.21.15.31:0/4121945023 wait complete. 2024-04-24T23:26:04.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 -- 172.21.15.31:0/879986065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7341021d0 msgr2=0x7fb7341025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 --2- 172.21.15.31:0/879986065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7341021d0 0x7fb7341025a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb7280048d0 tx=0x7fb728009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 -- 172.21.15.31:0/879986065 shutdown_connections 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 --2- 172.21.15.31:0/879986065 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb734102b70 0x7fb7340fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 --2- 172.21.15.31:0/879986065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7341021d0 0x7fb7341025a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 -- 172.21.15.31:0/879986065 >> 172.21.15.31:0/879986065 conn(0x7fb7340f7d60 msgr2=0x7fb7340fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 -- 172.21.15.31:0/879986065 shutdown_connections 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 -- 172.21.15.31:0/879986065 wait complete. 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.800+0000 7fb73ab1d640 1 Processor -- start 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb73ab1d640 1 -- start start 2024-04-24T23:26:04.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb73ab1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7341021d0 0x7fb734112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb73ab1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb734102b70 0x7fb734113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb73ab1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734113800 con 0x7fb734102b70 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb73ab1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73410ce50 con 0x7fb7341021d0 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb733fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb734102b70 0x7fb734113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb733fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb734102b70 0x7fb734113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43996/0 (socket says 172.21.15.31:43996) 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb733fff640 1 -- 172.21.15.31:0/1321585555 learned_addr learned my addr 172.21.15.31:0/1321585555 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb7388b9640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7341021d0 0x7fb734112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb733fff640 1 -- 172.21.15.31:0/1321585555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7341021d0 msgr2=0x7fb734112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb733fff640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7341021d0 0x7fb734112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.801+0000 7fb733fff640 1 -- 172.21.15.31:0/1321585555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728005040 con 0x7fb734102b70 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.802+0000 7fb733fff640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb734102b70 0x7fb734113200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb72000c790 tx=0x7fb7200043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.802+0000 7fb7317fa640 1 -- 172.21.15.31:0/1321585555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb72000a820 con 0x7fb734102b70 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.802+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73410d130 con 0x7fb734102b70 2024-04-24T23:26:04.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.802+0000 7fb7317fa640 1 -- 172.21.15.31:0/1321585555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb72000ae50 con 0x7fb734102b70 2024-04-24T23:26:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.802+0000 7fb7317fa640 1 -- 172.21.15.31:0/1321585555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7200115e0 con 0x7fb734102b70 2024-04-24T23:26:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.802+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73410d650 con 0x7fb734102b70 2024-04-24T23:26:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.803+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7340fd940 con 0x7fb734102b70 2024-04-24T23:26:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.803+0000 7fb7317fa640 1 -- 172.21.15.31:0/1321585555 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb720012400 con 0x7fb734102b70 2024-04-24T23:26:04.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.804+0000 7fb7317fa640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb72406e0b0 0x7fb724070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.804+0000 7fb7317fa640 1 -- 172.21.15.31:0/1321585555 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fb72008dce0 con 0x7fb734102b70 2024-04-24T23:26:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.804+0000 7fb7388b9640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb72406e0b0 0x7fb724070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:04.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.804+0000 7fb7388b9640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb72406e0b0 0x7fb724070560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb728003080 tx=0x7fb728009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:04.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:04.809+0000 7fb7317fa640 1 -- 172.21.15.31:0/1321585555 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb720057030 con 0x7fb734102b70 2024-04-24T23:26:05.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.034+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fb734109330 con 0x7fb734102b70 2024-04-24T23:26:05.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.035+0000 7fb7317fa640 1 -- 172.21.15.31:0/1321585555 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v34) v1 ==== 107+0+4146 (secure 0 0 0) 0x7fb72005a6f0 con 0x7fb734102b70 2024-04-24T23:26:05.036 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:05.036 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:07.325555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":0,"incarnation":19,"state":"up:replay","state_seq":3,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:05.036 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-04-24T23:26:05.038 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb72406e0b0 msgr2=0x7fb724070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb72406e0b0 0x7fb724070560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb728003080 tx=0x7fb728009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb734102b70 msgr2=0x7fb734113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb734102b70 0x7fb734113200 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb72000c790 tx=0x7fb7200043c0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 shutdown_connections 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fb72406e0b0 0x7fb724070560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb734102b70 0x7fb734113200 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 --2- 172.21.15.31:0/1321585555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7341021d0 0x7fb734112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 >> 172.21.15.31:0/1321585555 conn(0x7fb7340f7d60 msgr2=0x7fb7340f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 shutdown_connections 2024-04-24T23:26:05.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.037+0000 7fb73ab1d640 1 -- 172.21.15.31:0/1321585555 wait complete. 2024-04-24T23:26:05.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-24T23:26:05.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 -- 172.21.15.31:0/2914793924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 msgr2=0x7f89a8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 --2- 172.21.15.31:0/2914793924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 0x7f89a8101460 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f89980048d0 tx=0x7f8998009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 -- 172.21.15.31:0/2914793924 shutdown_connections 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 --2- 172.21.15.31:0/2914793924 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89a8101a30 0x7f89a80fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 --2- 172.21.15.31:0/2914793924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 0x7f89a8101460 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 -- 172.21.15.31:0/2914793924 >> 172.21.15.31:0/2914793924 conn(0x7f89a80f6c20 msgr2=0x7f89a80f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 -- 172.21.15.31:0/2914793924 shutdown_connections 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 -- 172.21.15.31:0/2914793924 wait complete. 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 Processor -- start 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.149+0000 7f89aeb5d640 1 -- start start 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89aeb5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 0x7f89a8112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89aeb5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89a8101a30 0x7f89a81131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89aeb5d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a81137f0 con 0x7f89a8101090 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89aeb5d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a810ce40 con 0x7f89a8101a30 2024-04-24T23:26:05.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89ac8f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 0x7f89a8112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89ac8f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 0x7f89a8112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44022/0 (socket says 172.21.15.31:44022) 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89ac8f9640 1 -- 172.21.15.31:0/67221861 learned_addr learned my addr 172.21.15.31:0/67221861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89ac8f9640 1 -- 172.21.15.31:0/67221861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89a8101a30 msgr2=0x7f89a81131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89ac8f9640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89a8101a30 0x7f89a81131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89ac8f9640 1 -- 172.21.15.31:0/67221861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998005040 con 0x7f89a8101090 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89ac8f9640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 0x7f89a8112cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8998004930 tx=0x7f8998009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89a57fa640 1 -- 172.21.15.31:0/67221861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89980094c0 con 0x7f89a8101090 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89a57fa640 1 -- 172.21.15.31:0/67221861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8998032e50 con 0x7f89a8101090 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a810d0c0 con 0x7f89a8101090 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89a57fa640 1 -- 172.21.15.31:0/67221861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89980433b0 con 0x7f89a8101090 2024-04-24T23:26:05.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.150+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a810d580 con 0x7f89a8101090 2024-04-24T23:26:05.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.151+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a80fc800 con 0x7f89a8101090 2024-04-24T23:26:05.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.152+0000 7f89a57fa640 1 -- 172.21.15.31:0/67221861 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f899802f070 con 0x7f89a8101090 2024-04-24T23:26:05.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.152+0000 7f89a57fa640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f899006e0b0 0x7f8990070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.152+0000 7f89a57fa640 1 -- 172.21.15.31:0/67221861 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f89980afbe0 con 0x7f89a8101090 2024-04-24T23:26:05.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.152+0000 7f89a7fff640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f899006e0b0 0x7f8990070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.153+0000 7f89a7fff640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f899006e0b0 0x7f8990070560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f89a810e570 tx=0x7f899c007c30 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.157+0000 7f89a57fa640 1 -- 172.21.15.31:0/67221861 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8998078e10 con 0x7f89a8101090 2024-04-24T23:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.383+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f89a81081d0 con 0x7f89a8101090 2024-04-24T23:26:05.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.384+0000 7f89a57fa640 1 -- 172.21.15.31:0/67221861 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v34) v1 ==== 107+0+4993 (secure 0 0 0) 0x7f899807c4d0 con 0x7f89a8101090 2024-04-24T23:26:05.385 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:05.385 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:07.325555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":0,"incarnation":19,"state":"up:replay","state_seq":3,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:05.385 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f899006e0b0 msgr2=0x7f8990070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f899006e0b0 0x7f8990070560 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f89a810e570 tx=0x7f899c007c30 comp rx=0 tx=0).stop 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 msgr2=0x7f89a8112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 0x7f89a8112cb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8998004930 tx=0x7f8998009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 shutdown_connections 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f899006e0b0 0x7f8990070560 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f89a8101a30 0x7f89a81131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 --2- 172.21.15.31:0/67221861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89a8101090 0x7f89a8112cb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 >> 172.21.15.31:0/67221861 conn(0x7f89a80f6c20 msgr2=0x7f89a80f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 shutdown_connections 2024-04-24T23:26:05.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.386+0000 7f89aeb5d640 1 -- 172.21.15.31:0/67221861 wait complete. 2024-04-24T23:26:05.396 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:05 smithi031 ceph-mon[111404]: pgmap v273: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:05.396 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:05 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3083532967' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-24T23:26:05.396 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:05 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4121945023' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-24T23:26:05.396 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:05 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1321585555' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-24T23:26:05.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.495+0000 7f119162e640 1 -- 172.21.15.31:0/1172222734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c102b80 msgr2=0x7f118c0fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.495+0000 7f119162e640 1 --2- 172.21.15.31:0/1172222734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c102b80 0x7f118c0fc210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f11740048d0 tx=0x7f1174009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.495+0000 7f119162e640 1 -- 172.21.15.31:0/1172222734 shutdown_connections 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.495+0000 7f119162e640 1 --2- 172.21.15.31:0/1172222734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c102b80 0x7f118c0fc210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.495+0000 7f119162e640 1 --2- 172.21.15.31:0/1172222734 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118c1021e0 0x7f118c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.495+0000 7f119162e640 1 -- 172.21.15.31:0/1172222734 >> 172.21.15.31:0/1172222734 conn(0x7f118c0f7d90 msgr2=0x7f118c0fa1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.495+0000 7f119162e640 1 -- 172.21.15.31:0/1172222734 shutdown_connections 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.495+0000 7f119162e640 1 -- 172.21.15.31:0/1172222734 wait complete. 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.496+0000 7f119162e640 1 Processor -- start 2024-04-24T23:26:05.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.496+0000 7f119162e640 1 -- start start 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.496+0000 7f119162e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1021e0 0x7f118c06f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.496+0000 7f119162e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118c102b80 0x7f118c06fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.496+0000 7f119162e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c071650 con 0x7f118c1021e0 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.496+0000 7f119162e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c0701f0 con 0x7f118c102b80 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f118affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1021e0 0x7f118c06f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f118affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1021e0 0x7f118c06f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44040/0 (socket says 172.21.15.31:44040) 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f118affd640 1 -- 172.21.15.31:0/1686941925 learned_addr learned my addr 172.21.15.31:0/1686941925 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f118a7fc640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118c102b80 0x7f118c06fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f118affd640 1 -- 172.21.15.31:0/1686941925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118c102b80 msgr2=0x7f118c06fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f118affd640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118c102b80 0x7f118c06fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f118affd640 1 -- 172.21.15.31:0/1686941925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1174005040 con 0x7f118c1021e0 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f118affd640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1021e0 0x7f118c06f770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1184002a50 tx=0x7f1184004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f11837fe640 1 -- 172.21.15.31:0/1686941925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1184007c30 con 0x7f118c1021e0 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c0704d0 con 0x7f118c1021e0 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f11837fe640 1 -- 172.21.15.31:0/1686941925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f118400f040 con 0x7f118c1021e0 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f11837fe640 1 -- 172.21.15.31:0/1686941925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118400dbe0 con 0x7f118c1021e0 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.497+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c1193c0 con 0x7f118c1021e0 2024-04-24T23:26:05.500 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.498+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118c0fd950 con 0x7f118c1021e0 2024-04-24T23:26:05.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.499+0000 7f11837fe640 1 -- 172.21.15.31:0/1686941925 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f118400dd40 con 0x7f118c1021e0 2024-04-24T23:26:05.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.499+0000 7f11837fe640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f117806e0b0 0x7f1178070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.499+0000 7f11837fe640 1 -- 172.21.15.31:0/1686941925 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f118408dbd0 con 0x7f118c1021e0 2024-04-24T23:26:05.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.499+0000 7f118a7fc640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f117806e0b0 0x7f1178070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.500+0000 7f118a7fc640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f117806e0b0 0x7f1178070560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1174003080 tx=0x7f1174009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.504+0000 7f11837fe640 1 -- 172.21.15.31:0/1686941925 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1184056ea0 con 0x7f118c1021e0 2024-04-24T23:26:05.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:05 smithi040 ceph-mon[83047]: pgmap v273: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:05.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:05 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3083532967' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-24T23:26:05.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:05 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4121945023' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-24T23:26:05.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:05 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1321585555' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-24T23:26:05.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.725+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f118c1195b0 con 0x7f118c1021e0 2024-04-24T23:26:05.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.725+0000 7f11837fe640 1 -- 172.21.15.31:0/1686941925 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v34) v1 ==== 107+0+4998 (secure 0 0 0) 0x7f118405a560 con 0x7f118c1021e0 2024-04-24T23:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:16.406919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":0,"incarnation":19,"state":"up:reconnect","state_seq":189,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:05.727 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-04-24T23:26:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f117806e0b0 msgr2=0x7f1178070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f117806e0b0 0x7f1178070560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1174003080 tx=0x7f1174009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1021e0 msgr2=0x7f118c06f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1021e0 0x7f118c06f770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1184002a50 tx=0x7f1184004600 comp rx=0 tx=0).stop 2024-04-24T23:26:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 shutdown_connections 2024-04-24T23:26:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f117806e0b0 0x7f1178070560 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f118c102b80 0x7f118c06fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 --2- 172.21.15.31:0/1686941925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f118c1021e0 0x7f118c06f770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.727+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 >> 172.21.15.31:0/1686941925 conn(0x7f118c0f7d90 msgr2=0x7f118c0f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.728+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 shutdown_connections 2024-04-24T23:26:05.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.728+0000 7f119162e640 1 -- 172.21.15.31:0/1686941925 wait complete. 2024-04-24T23:26:05.739 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-24T23:26:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 -- 172.21.15.31:0/2478568536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d41021d0 msgr2=0x7f08d41025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 --2- 172.21.15.31:0/2478568536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d41021d0 0x7f08d41025a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f08c40048d0 tx=0x7f08c4009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 -- 172.21.15.31:0/2478568536 shutdown_connections 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 --2- 172.21.15.31:0/2478568536 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d4102b70 0x7f08d40fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 --2- 172.21.15.31:0/2478568536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d41021d0 0x7f08d41025a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 -- 172.21.15.31:0/2478568536 >> 172.21.15.31:0/2478568536 conn(0x7f08d40f7d60 msgr2=0x7f08d40fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 -- 172.21.15.31:0/2478568536 shutdown_connections 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 -- 172.21.15.31:0/2478568536 wait complete. 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 Processor -- start 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 -- start start 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d41021d0 0x7f08d4112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d4102b70 0x7f08d4113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d4113800 con 0x7f08d4102b70 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.838+0000 7f08da91d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d410ce50 con 0x7f08d41021d0 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d4102b70 0x7f08d4113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d4102b70 0x7f08d4113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44060/0 (socket says 172.21.15.31:44060) 2024-04-24T23:26:05.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d37fe640 1 -- 172.21.15.31:0/3684096608 learned_addr learned my addr 172.21.15.31:0/3684096608 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d3fff640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d41021d0 0x7f08d4112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d37fe640 1 -- 172.21.15.31:0/3684096608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d41021d0 msgr2=0x7f08d4112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d37fe640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d41021d0 0x7f08d4112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d37fe640 1 -- 172.21.15.31:0/3684096608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c4005040 con 0x7f08d4102b70 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d37fe640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d4102b70 0x7f08d4113200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08c8002a50 tx=0x7f08c8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d0ff9640 1 -- 172.21.15.31:0/3684096608 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c8007c30 con 0x7f08d4102b70 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d410d130 con 0x7f08d4102b70 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d0ff9640 1 -- 172.21.15.31:0/3684096608 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08c800f040 con 0x7f08d4102b70 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08d0ff9640 1 -- 172.21.15.31:0/3684096608 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c800dad0 con 0x7f08d4102b70 2024-04-24T23:26:05.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.839+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d410d650 con 0x7f08d4102b70 2024-04-24T23:26:05.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.840+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d40fd940 con 0x7f08d4102b70 2024-04-24T23:26:05.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.841+0000 7f08d0ff9640 1 -- 172.21.15.31:0/3684096608 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f08c800dd40 con 0x7f08d4102b70 2024-04-24T23:26:05.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.841+0000 7f08d0ff9640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f08bc06e0b0 0x7f08bc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:05.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.842+0000 7f08d0ff9640 1 -- 172.21.15.31:0/3684096608 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f08c808d490 con 0x7f08d4102b70 2024-04-24T23:26:05.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.842+0000 7f08d3fff640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f08bc06e0b0 0x7f08bc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:05.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.842+0000 7f08d3fff640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f08bc06e0b0 0x7f08bc070560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08c4000c00 tx=0x7f08c4009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:05.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:05.846+0000 7f08d0ff9640 1 -- 172.21.15.31:0/3684096608 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f08c8056770 con 0x7f08d4102b70 2024-04-24T23:26:06.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.067+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f08d4109330 con 0x7f08d4102b70 2024-04-24T23:26:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.068+0000 7f08d0ff9640 1 -- 172.21.15.31:0/3684096608 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v34) v1 ==== 107+0+4995 (secure 0 0 0) 0x7f08c8059e30 con 0x7f08d4102b70 2024-04-24T23:26:06.069 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:06.069 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:16.973829+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":0,"incarnation":19,"state":"up:rejoin","state_seq":190,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:06.069 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f08bc06e0b0 msgr2=0x7f08bc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f08bc06e0b0 0x7f08bc070560 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08c4000c00 tx=0x7f08c4009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d4102b70 msgr2=0x7f08d4113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d4102b70 0x7f08d4113200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08c8002a50 tx=0x7f08c8004600 comp rx=0 tx=0).stop 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 shutdown_connections 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f08bc06e0b0 0x7f08bc070560 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08d4102b70 0x7f08d4113200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 --2- 172.21.15.31:0/3684096608 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f08d41021d0 0x7f08d4112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 >> 172.21.15.31:0/3684096608 conn(0x7f08d40f7d60 msgr2=0x7f08d40f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:06.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 shutdown_connections 2024-04-24T23:26:06.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.070+0000 7f08da91d640 1 -- 172.21.15.31:0/3684096608 wait complete. 2024-04-24T23:26:06.082 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-24T23:26:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.179+0000 7f12ebe2e640 1 -- 172.21.15.31:0/4086809137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 msgr2=0x7f12e4101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.179+0000 7f12ebe2e640 1 --2- 172.21.15.31:0/4086809137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 0x7f12e4101540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f12e00048d0 tx=0x7f12e0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 -- 172.21.15.31:0/4086809137 shutdown_connections 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 --2- 172.21.15.31:0/4086809137 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e40fb150 0x7f12e40fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 --2- 172.21.15.31:0/4086809137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 0x7f12e4101540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 -- 172.21.15.31:0/4086809137 >> 172.21.15.31:0/4086809137 conn(0x7f12e4071fd0 msgr2=0x7f12e40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 -- 172.21.15.31:0/4086809137 shutdown_connections 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 -- 172.21.15.31:0/4086809137 wait complete. 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 Processor -- start 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 -- start start 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.180+0000 7f12ebe2e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e40fb150 0x7f12e410bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12ebe2e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 0x7f12e410c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12ebe2e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4065220 con 0x7f12e4101170 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12ebe2e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e410c690 con 0x7f12e40fb150 2024-04-24T23:26:06.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12e93c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 0x7f12e410c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12e93c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 0x7f12e410c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44082/0 (socket says 172.21.15.31:44082) 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12e93c9640 1 -- 172.21.15.31:0/287797564 learned_addr learned my addr 172.21.15.31:0/287797564 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12e9bca640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e40fb150 0x7f12e410bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12e93c9640 1 -- 172.21.15.31:0/287797564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e40fb150 msgr2=0x7f12e410bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12e93c9640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e40fb150 0x7f12e410bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12e93c9640 1 -- 172.21.15.31:0/287797564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e0005040 con 0x7f12e4101170 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12e93c9640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 0x7f12e410c150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12d4002a50 tx=0x7f12d4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12da7fc640 1 -- 172.21.15.31:0/287797564 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d4007c30 con 0x7f12e4101170 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12da7fc640 1 -- 172.21.15.31:0/287797564 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12d400f040 con 0x7f12e4101170 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e410c970 con 0x7f12e4101170 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12da7fc640 1 -- 172.21.15.31:0/287797564 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d400db50 con 0x7f12e4101170 2024-04-24T23:26:06.184 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.181+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e4119420 con 0x7f12e4101170 2024-04-24T23:26:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.182+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e40fc870 con 0x7f12e4101170 2024-04-24T23:26:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.183+0000 7f12da7fc640 1 -- 172.21.15.31:0/287797564 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f12d401c400 con 0x7f12e4101170 2024-04-24T23:26:06.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.183+0000 7f12da7fc640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f12d006e0b0 0x7f12d0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.183+0000 7f12da7fc640 1 -- 172.21.15.31:0/287797564 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f12d408d420 con 0x7f12e4101170 2024-04-24T23:26:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.183+0000 7f12e9bca640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f12d006e0b0 0x7f12d0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.184+0000 7f12e9bca640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f12d006e0b0 0x7f12d0070560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12e0003080 tx=0x7f12e0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.188+0000 7f12da7fc640 1 -- 172.21.15.31:0/287797564 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f12d40567a0 con 0x7f12e4101170 2024-04-24T23:26:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.416+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f12e40fe3a0 con 0x7f12e4101170 2024-04-24T23:26:06.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.417+0000 7f12da7fc640 1 -- 172.21.15.31:0/287797564 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v34) v1 ==== 107+0+5004 (secure 0 0 0) 0x7f12d4059e60 con 0x7f12e4101170 2024-04-24T23:26:06.418 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:06.418 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:18.976584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24285":{"gid":24285,"name":"cephfs.smithi040.dicevu","rank":0,"incarnation":19,"state":"up:active","state_seq":191,"addr":"172.21.15.40:6825/1413423535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":1413423535},{"type":"v1","addr":"172.21.15.40:6825","nonce":1413423535}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24285,"qdb_cluster":[24285]},"id":1}]} 2024-04-24T23:26:06.418 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-04-24T23:26:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.418+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f12d006e0b0 msgr2=0x7f12d0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.418+0000 7f12ebe2e640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f12d006e0b0 0x7f12d0070560 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12e0003080 tx=0x7f12e0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.418+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 msgr2=0x7f12e410c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.418+0000 7f12ebe2e640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 0x7f12e410c150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12d4002a50 tx=0x7f12d4004600 comp rx=0 tx=0).stop 2024-04-24T23:26:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.419+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 shutdown_connections 2024-04-24T23:26:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.419+0000 7f12ebe2e640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f12d006e0b0 0x7f12d0070560 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.419+0000 7f12ebe2e640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e4101170 0x7f12e410c150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.419+0000 7f12ebe2e640 1 --2- 172.21.15.31:0/287797564 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f12e40fb150 0x7f12e410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.419+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 >> 172.21.15.31:0/287797564 conn(0x7f12e4071fd0 msgr2=0x7f12e40fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.419+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 shutdown_connections 2024-04-24T23:26:06.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.419+0000 7f12ebe2e640 1 -- 172.21.15.31:0/287797564 wait complete. 2024-04-24T23:26:06.429 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:06 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/67221861' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-24T23:26:06.429 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:06 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1686941925' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-24T23:26:06.429 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:06 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3684096608' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-24T23:26:06.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-24T23:26:06.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.527+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2019664842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fd1d0 msgr2=0x7f3eb00fd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.527+0000 7f3eb5e65640 1 --2- 172.21.15.31:0/2019664842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fd1d0 0x7f3eb00fd5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00048d0 tx=0x7f3ea0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.527+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2019664842 shutdown_connections 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.527+0000 7f3eb5e65640 1 --2- 172.21.15.31:0/2019664842 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb00fdb70 0x7f3eb01061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.527+0000 7f3eb5e65640 1 --2- 172.21.15.31:0/2019664842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fd1d0 0x7f3eb00fd5a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.527+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2019664842 >> 172.21.15.31:0/2019664842 conn(0x7f3eb00f6c80 msgr2=0x7f3eb00f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.527+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2019664842 shutdown_connections 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.527+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2019664842 wait complete. 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eb5e65640 1 Processor -- start 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eb5e65640 1 -- start start 2024-04-24T23:26:06.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eb5e65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb00fd1d0 0x7f3eb0112d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eb5e65640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fdb70 0x7f3eb0113240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eb5e65640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0113840 con 0x7f3eb00fdb70 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eb5e65640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb010ce90 con 0x7f3eb00fd1d0 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eaeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fdb70 0x7f3eb0113240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eaeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fdb70 0x7f3eb0113240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44094/0 (socket says 172.21.15.31:44094) 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eaeffd640 1 -- 172.21.15.31:0/2962060017 learned_addr learned my addr 172.21.15.31:0/2962060017 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eaf7fe640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb00fd1d0 0x7f3eb0112d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eaeffd640 1 -- 172.21.15.31:0/2962060017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb00fd1d0 msgr2=0x7f3eb0112d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eaeffd640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb00fd1d0 0x7f3eb0112d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.528+0000 7f3eaeffd640 1 -- 172.21.15.31:0/2962060017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea0005040 con 0x7f3eb00fdb70 2024-04-24T23:26:06.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.529+0000 7f3eaeffd640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fdb70 0x7f3eb0113240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400eb70 tx=0x7f3ea400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.529+0000 7f3e97fff640 1 -- 172.21.15.31:0/2962060017 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea4007c30 con 0x7f3eb00fdb70 2024-04-24T23:26:06.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.529+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb010d110 con 0x7f3eb00fdb70 2024-04-24T23:26:06.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.529+0000 7f3e97fff640 1 -- 172.21.15.31:0/2962060017 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ea400f040 con 0x7f3eb00fdb70 2024-04-24T23:26:06.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.529+0000 7f3e97fff640 1 -- 172.21.15.31:0/2962060017 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea40095b0 con 0x7f3eb00fdb70 2024-04-24T23:26:06.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.529+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb010d630 con 0x7f3eb00fdb70 2024-04-24T23:26:06.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.530+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb0066040 con 0x7f3eb00fdb70 2024-04-24T23:26:06.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.530+0000 7f3e97fff640 1 -- 172.21.15.31:0/2962060017 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3ea4009730 con 0x7f3eb00fdb70 2024-04-24T23:26:06.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.531+0000 7f3e97fff640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3e9806e0b0 0x7f3e98070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.531+0000 7f3e97fff640 1 -- 172.21.15.31:0/2962060017 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f3ea408d960 con 0x7f3eb00fdb70 2024-04-24T23:26:06.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.531+0000 7f3eaf7fe640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3e9806e0b0 0x7f3e98070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.531+0000 7f3eaf7fe640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3e9806e0b0 0x7f3e98070560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0003080 tx=0x7f3ea0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.535+0000 7f3e97fff640 1 -- 172.21.15.31:0/2962060017 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3ea4056c40 con 0x7f3eb00fdb70 2024-04-24T23:26:06.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:06 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/67221861' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-24T23:26:06.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:06 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1686941925' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-24T23:26:06.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:06 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3684096608' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-24T23:26:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.761+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f3eb0109330 con 0x7f3eb00fdb70 2024-04-24T23:26:06.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.761+0000 7f3e97fff640 1 -- 172.21.15.31:0/2962060017 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v34) v1 ==== 107+0+4203 (secure 0 0 0) 0x7f3ea405a300 con 0x7f3eb00fdb70 2024-04-24T23:26:06.762 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:06.763 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:28.677266+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:06.763 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2024-04-24T23:26:06.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3e9806e0b0 msgr2=0x7f3e98070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3e9806e0b0 0x7f3e98070560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0003080 tx=0x7f3ea0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fdb70 msgr2=0x7f3eb0113240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fdb70 0x7f3eb0113240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400eb70 tx=0x7f3ea400b040 comp rx=0 tx=0).stop 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 shutdown_connections 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f3e9806e0b0 0x7f3e98070560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3eb00fdb70 0x7f3eb0113240 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 --2- 172.21.15.31:0/2962060017 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eb00fd1d0 0x7f3eb0112d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.763+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 >> 172.21.15.31:0/2962060017 conn(0x7f3eb00f6c80 msgr2=0x7f3eb00f8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.764+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 shutdown_connections 2024-04-24T23:26:06.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.764+0000 7f3eb5e65640 1 -- 172.21.15.31:0/2962060017 wait complete. 2024-04-24T23:26:06.775 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-24T23:26:06.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.872+0000 7f71c895d640 1 -- 172.21.15.31:0/2393355925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 msgr2=0x7f71c4064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.872+0000 7f71c895d640 1 --2- 172.21.15.31:0/2393355925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 0x7f71c4064800 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f71b80048d0 tx=0x7f71b8009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 -- 172.21.15.31:0/2393355925 shutdown_connections 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 --2- 172.21.15.31:0/2393355925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 0x7f71c4064800 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 --2- 172.21.15.31:0/2393355925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4102270 0x7f71c4102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 -- 172.21.15.31:0/2393355925 >> 172.21.15.31:0/2393355925 conn(0x7f71c4071fd0 msgr2=0x7f71c40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 -- 172.21.15.31:0/2393355925 shutdown_connections 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 -- 172.21.15.31:0/2393355925 wait complete. 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 Processor -- start 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 -- start start 2024-04-24T23:26:06.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 0x7f71c4112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4102270 0x7f71c41131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c41137c0 con 0x7f71c4064390 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c895d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c410ce10 con 0x7f71c4102270 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 0x7f71c4112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 0x7f71c4112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44120/0 (socket says 172.21.15.31:44120) 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.873+0000 7f71c259c640 1 -- 172.21.15.31:0/676002758 learned_addr learned my addr 172.21.15.31:0/676002758 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71c1d9b640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4102270 0x7f71c41131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71c259c640 1 -- 172.21.15.31:0/676002758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4102270 msgr2=0x7f71c41131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71c259c640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4102270 0x7f71c41131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71c259c640 1 -- 172.21.15.31:0/676002758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b8005040 con 0x7f71c4064390 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71c259c640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 0x7f71c4112c80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c790 tx=0x7f71b40043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71b2ffd640 1 -- 172.21.15.31:0/676002758 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71b400a820 con 0x7f71c4064390 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71b2ffd640 1 -- 172.21.15.31:0/676002758 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71b400ae50 con 0x7f71c4064390 2024-04-24T23:26:06.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71b2ffd640 1 -- 172.21.15.31:0/676002758 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71b40115e0 con 0x7f71c4064390 2024-04-24T23:26:06.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c410d0f0 con 0x7f71c4064390 2024-04-24T23:26:06.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.874+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c410d5e0 con 0x7f71c4064390 2024-04-24T23:26:06.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.875+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c40fd970 con 0x7f71c4064390 2024-04-24T23:26:06.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.875+0000 7f71b2ffd640 1 -- 172.21.15.31:0/676002758 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f71b4012400 con 0x7f71c4064390 2024-04-24T23:26:06.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.876+0000 7f71b2ffd640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f71ac06e0b0 0x7f71ac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:06.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.876+0000 7f71b2ffd640 1 -- 172.21.15.31:0/676002758 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f71b408dd90 con 0x7f71c4064390 2024-04-24T23:26:06.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.876+0000 7f71c1d9b640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f71ac06e0b0 0x7f71ac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:06.879 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.876+0000 7f71c1d9b640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f71ac06e0b0 0x7f71ac070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f71c40f9610 tx=0x7f71b8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:06.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:06.881+0000 7f71b2ffd640 1 -- 172.21.15.31:0/676002758 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f71b40570e0 con 0x7f71c4064390 2024-04-24T23:26:07.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.104+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f71c40ff4a0 con 0x7f71c4064390 2024-04-24T23:26:07.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.105+0000 7f71b2ffd640 1 -- 172.21.15.31:0/676002758 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v34) v1 ==== 107+0+4214 (secure 0 0 0) 0x7f71b405a7a0 con 0x7f71c4064390 2024-04-24T23:26:07.106 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:07.106 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:28.682848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24291":{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":0,"incarnation":25,"state":"up:replay","state_seq":2,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:07.106 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-04-24T23:26:07.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.106+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f71ac06e0b0 msgr2=0x7f71ac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.106+0000 7f71c895d640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f71ac06e0b0 0x7f71ac070560 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f71c40f9610 tx=0x7f71b8009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.106+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 msgr2=0x7f71c4112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.106+0000 7f71c895d640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 0x7f71c4112c80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f71b400c790 tx=0x7f71b40043c0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.107+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 shutdown_connections 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.107+0000 7f71c895d640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f71ac06e0b0 0x7f71ac070560 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.107+0000 7f71c895d640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f71c4102270 0x7f71c41131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.107+0000 7f71c895d640 1 --2- 172.21.15.31:0/676002758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c4064390 0x7f71c4112c80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.107+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 >> 172.21.15.31:0/676002758 conn(0x7f71c4071fd0 msgr2=0x7f71c40fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.107+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 shutdown_connections 2024-04-24T23:26:07.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.107+0000 7f71c895d640 1 -- 172.21.15.31:0/676002758 wait complete. 2024-04-24T23:26:07.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-24T23:26:07.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.215+0000 7fa487b5d640 1 -- 172.21.15.31:0/1916200925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4800fb9d0 msgr2=0x7fa4800fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.215+0000 7fa487b5d640 1 --2- 172.21.15.31:0/1916200925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4800fb9d0 0x7fa4800fbe40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa4700048d0 tx=0x7fa470009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.215+0000 7fa487b5d640 1 -- 172.21.15.31:0/1916200925 shutdown_connections 2024-04-24T23:26:07.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.215+0000 7fa487b5d640 1 --2- 172.21.15.31:0/1916200925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4800fb9d0 0x7fa4800fbe40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.215+0000 7fa487b5d640 1 --2- 172.21.15.31:0/1916200925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa480100160 0x7fa480100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.215+0000 7fa487b5d640 1 -- 172.21.15.31:0/1916200925 >> 172.21.15.31:0/1916200925 conn(0x7fa4800717e0 msgr2=0x7fa480071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:07.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.215+0000 7fa487b5d640 1 -- 172.21.15.31:0/1916200925 shutdown_connections 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.215+0000 7fa487b5d640 1 -- 172.21.15.31:0/1916200925 wait complete. 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa487b5d640 1 Processor -- start 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa487b5d640 1 -- start start 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa487b5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4800fb9d0 0x7fa480112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa487b5d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa480100160 0x7fa480113320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa487b5d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa480113920 con 0x7fa480100160 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa487b5d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48010cf70 con 0x7fa4800fb9d0 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa4850f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa480100160 0x7fa480113320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa4850f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa480100160 0x7fa480113320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44146/0 (socket says 172.21.15.31:44146) 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.216+0000 7fa4850f8640 1 -- 172.21.15.31:0/2272531058 learned_addr learned my addr 172.21.15.31:0/2272531058 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa4850f8640 1 -- 172.21.15.31:0/2272531058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4800fb9d0 msgr2=0x7fa480112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa4858f9640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4800fb9d0 0x7fa480112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa4850f8640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4800fb9d0 0x7fa480112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa4850f8640 1 -- 172.21.15.31:0/2272531058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470005040 con 0x7fa480100160 2024-04-24T23:26:07.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa4850f8640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa480100160 0x7fa480113320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa470004930 tx=0x7fa470009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa4767fc640 1 -- 172.21.15.31:0/2272531058 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4700094c0 con 0x7fa480100160 2024-04-24T23:26:07.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa4767fc640 1 -- 172.21.15.31:0/2272531058 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa470032e50 con 0x7fa480100160 2024-04-24T23:26:07.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48010d1f0 con 0x7fa480100160 2024-04-24T23:26:07.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa4767fc640 1 -- 172.21.15.31:0/2272531058 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4700433b0 con 0x7fa480100160 2024-04-24T23:26:07.220 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.217+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48010d6b0 con 0x7fa480100160 2024-04-24T23:26:07.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.218+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4800fc9e0 con 0x7fa480100160 2024-04-24T23:26:07.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.219+0000 7fa4767fc640 1 -- 172.21.15.31:0/2272531058 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa47002f070 con 0x7fa480100160 2024-04-24T23:26:07.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.219+0000 7fa4767fc640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa46c06e0b0 0x7fa46c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.219+0000 7fa4767fc640 1 -- 172.21.15.31:0/2272531058 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa4700af910 con 0x7fa480100160 2024-04-24T23:26:07.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.219+0000 7fa4858f9640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa46c06e0b0 0x7fa46c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.219+0000 7fa4858f9640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa46c06e0b0 0x7fa46c070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa47c004ce0 tx=0x7fa47c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.224+0000 7fa4767fc640 1 -- 172.21.15.31:0/2272531058 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa470078be0 con 0x7fa480100160 2024-04-24T23:26:07.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.446+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fa480064350 con 0x7fa480100160 2024-04-24T23:26:07.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.447+0000 7fa4767fc640 1 -- 172.21.15.31:0/2272531058 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v34) v1 ==== 107+0+4219 (secure 0 0 0) 0x7fa47007c2a0 con 0x7fa480100160 2024-04-24T23:26:07.448 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:07.448 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:38.170679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24291":{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":0,"incarnation":25,"state":"up:reconnect","state_seq":193,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:07.448 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa46c06e0b0 msgr2=0x7fa46c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa46c06e0b0 0x7fa46c070560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa47c004ce0 tx=0x7fa47c00b040 comp rx=0 tx=0).stop 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa480100160 msgr2=0x7fa480113320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa480100160 0x7fa480113320 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa470004930 tx=0x7fa470009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 shutdown_connections 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fa46c06e0b0 0x7fa46c070560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa480100160 0x7fa480113320 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 --2- 172.21.15.31:0/2272531058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4800fb9d0 0x7fa480112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 >> 172.21.15.31:0/2272531058 conn(0x7fa4800717e0 msgr2=0x7fa4800fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:07.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 shutdown_connections 2024-04-24T23:26:07.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.449+0000 7fa487b5d640 1 -- 172.21.15.31:0/2272531058 wait complete. 2024-04-24T23:26:07.459 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:07 smithi031 ceph-mon[111404]: pgmap v274: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:07.459 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:07 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/287797564' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-24T23:26:07.459 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:07 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/2962060017' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-24T23:26:07.459 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:07 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/676002758' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-24T23:26:07.460 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-24T23:26:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.557+0000 7f688951d640 1 -- 172.21.15.31:0/3060889525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 msgr2=0x7f6884101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.557+0000 7f688951d640 1 --2- 172.21.15.31:0/3060889525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 0x7f6884101540 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f687c0048d0 tx=0x7f687c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.557+0000 7f688951d640 1 -- 172.21.15.31:0/3060889525 shutdown_connections 2024-04-24T23:26:07.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.557+0000 7f688951d640 1 --2- 172.21.15.31:0/3060889525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68840fb150 0x7f68840fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.557+0000 7f688951d640 1 --2- 172.21.15.31:0/3060889525 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 0x7f6884101540 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.557+0000 7f688951d640 1 -- 172.21.15.31:0/3060889525 >> 172.21.15.31:0/3060889525 conn(0x7f6884071fd0 msgr2=0x7f68840723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.558+0000 7f688951d640 1 -- 172.21.15.31:0/3060889525 shutdown_connections 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.558+0000 7f688951d640 1 -- 172.21.15.31:0/3060889525 wait complete. 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.558+0000 7f688951d640 1 Processor -- start 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.558+0000 7f688951d640 1 -- start start 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.558+0000 7f688951d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68840fb150 0x7f6884112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.558+0000 7f688951d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 0x7f6884113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.558+0000 7f688951d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6884113770 con 0x7f6884101170 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.558+0000 7f688951d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688410cdc0 con 0x7f68840fb150 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f68827fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 0x7f6884113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f68827fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 0x7f6884113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44160/0 (socket says 172.21.15.31:44160) 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f68827fc640 1 -- 172.21.15.31:0/3069545848 learned_addr learned my addr 172.21.15.31:0/3069545848 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:07.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f6882ffd640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68840fb150 0x7f6884112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f68827fc640 1 -- 172.21.15.31:0/3069545848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68840fb150 msgr2=0x7f6884112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f68827fc640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68840fb150 0x7f6884112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f68827fc640 1 -- 172.21.15.31:0/3069545848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c005040 con 0x7f6884101170 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f68827fc640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 0x7f6884113170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f686c002a50 tx=0x7f686c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f687b7fe640 1 -- 172.21.15.31:0/3069545848 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686c007c30 con 0x7f6884101170 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f687b7fe640 1 -- 172.21.15.31:0/3069545848 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f686c00f040 con 0x7f6884101170 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688410d0a0 con 0x7f6884101170 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f687b7fe640 1 -- 172.21.15.31:0/3069545848 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686c00dbe0 con 0x7f6884101170 2024-04-24T23:26:07.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.559+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688410d5c0 con 0x7f6884101170 2024-04-24T23:26:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.560+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68840fc870 con 0x7f6884101170 2024-04-24T23:26:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.561+0000 7f687b7fe640 1 -- 172.21.15.31:0/3069545848 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f686c00dd40 con 0x7f6884101170 2024-04-24T23:26:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.561+0000 7f687b7fe640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f687006e0b0 0x7f6870070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.561+0000 7f687b7fe640 1 -- 172.21.15.31:0/3069545848 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f686c08dbd0 con 0x7f6884101170 2024-04-24T23:26:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.561+0000 7f6882ffd640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f687006e0b0 0x7f6870070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.562+0000 7f6882ffd640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f687006e0b0 0x7f6870070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f687c003080 tx=0x7f687c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.566+0000 7f687b7fe640 1 -- 172.21.15.31:0/3069545848 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f686c056ea0 con 0x7f6884101170 2024-04-24T23:26:07.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:07 smithi040 ceph-mon[83047]: pgmap v274: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:07.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:07 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/287797564' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-24T23:26:07.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:07 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/2962060017' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-24T23:26:07.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:07 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/676002758' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-24T23:26:07.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.788+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f6884063f00 con 0x7f6884101170 2024-04-24T23:26:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.789+0000 7f687b7fe640 1 -- 172.21.15.31:0/3069545848 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v34) v1 ==== 107+0+5063 (secure 0 0 0) 0x7f686c05a560 con 0x7f6884101170 2024-04-24T23:26:07.790 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:07.790 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44317,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6825/3497670820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3497670820},{"type":"v1","addr":"172.21.15.40:6825","nonce":3497670820}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:38.601711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24291":{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":0,"incarnation":25,"state":"up:rejoin","state_seq":194,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:07.790 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-04-24T23:26:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f687006e0b0 msgr2=0x7f6870070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f687006e0b0 0x7f6870070560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f687c003080 tx=0x7f687c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 msgr2=0x7f6884113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 0x7f6884113170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f686c002a50 tx=0x7f686c004600 comp rx=0 tx=0).stop 2024-04-24T23:26:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 shutdown_connections 2024-04-24T23:26:07.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f687006e0b0 0x7f6870070560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6884101170 0x7f6884113170 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 --2- 172.21.15.31:0/3069545848 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68840fb150 0x7f6884112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 >> 172.21.15.31:0/3069545848 conn(0x7f6884071fd0 msgr2=0x7f6884104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 shutdown_connections 2024-04-24T23:26:07.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.791+0000 7f688951d640 1 -- 172.21.15.31:0/3069545848 wait complete. 2024-04-24T23:26:07.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 -- 172.21.15.31:0/4026761593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 msgr2=0x7f423c102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 --2- 172.21.15.31:0/4026761593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 0x7f423c102580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f422c0048d0 tx=0x7f422c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 -- 172.21.15.31:0/4026761593 shutdown_connections 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 --2- 172.21.15.31:0/4026761593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f423c102b50 0x7f423c0fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 --2- 172.21.15.31:0/4026761593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 0x7f423c102580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 -- 172.21.15.31:0/4026761593 >> 172.21.15.31:0/4026761593 conn(0x7f423c0f7d80 msgr2=0x7f423c0fa190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 -- 172.21.15.31:0/4026761593 shutdown_connections 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 -- 172.21.15.31:0/4026761593 wait complete. 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 Processor -- start 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.901+0000 7f4242407640 1 -- start start 2024-04-24T23:26:07.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4242407640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 0x7f423c06f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4242407640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f423c102b50 0x7f423c06fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4242407640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c071650 con 0x7f423c1021b0 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4242407640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c0701f0 con 0x7f423c102b50 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f423bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 0x7f423c06f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f423bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 0x7f423c06f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44178/0 (socket says 172.21.15.31:44178) 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f423bfff640 1 -- 172.21.15.31:0/1912688680 learned_addr learned my addr 172.21.15.31:0/1912688680 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f423b7fe640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f423c102b50 0x7f423c06fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f423bfff640 1 -- 172.21.15.31:0/1912688680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f423c102b50 msgr2=0x7f423c06fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f423bfff640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f423c102b50 0x7f423c06fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f423bfff640 1 -- 172.21.15.31:0/1912688680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c005040 con 0x7f423c1021b0 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f423bfff640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 0x7f423c06f770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f422c004930 tx=0x7f422c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4238ff9640 1 -- 172.21.15.31:0/1912688680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f422c0094c0 con 0x7f423c1021b0 2024-04-24T23:26:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4238ff9640 1 -- 172.21.15.31:0/1912688680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f422c032e50 con 0x7f423c1021b0 2024-04-24T23:26:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c070470 con 0x7f423c1021b0 2024-04-24T23:26:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4238ff9640 1 -- 172.21.15.31:0/1912688680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f422c0433b0 con 0x7f423c1021b0 2024-04-24T23:26:07.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.902+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c119390 con 0x7f423c1021b0 2024-04-24T23:26:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.903+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c0fd920 con 0x7f423c1021b0 2024-04-24T23:26:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.904+0000 7f4238ff9640 1 -- 172.21.15.31:0/1912688680 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f422c02f070 con 0x7f423c1021b0 2024-04-24T23:26:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.904+0000 7f4238ff9640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f422406e0b0 0x7f4224070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.905+0000 7f4238ff9640 1 -- 172.21.15.31:0/1912688680 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f422c0af9d0 con 0x7f423c1021b0 2024-04-24T23:26:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.905+0000 7f423b7fe640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f422406e0b0 0x7f4224070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:07.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.905+0000 7f423b7fe640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f422406e0b0 0x7f4224070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f423c070ea0 tx=0x7f423000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:07.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:07.909+0000 7f4238ff9640 1 -- 172.21.15.31:0/1912688680 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f422c078d50 con 0x7f423c1021b0 2024-04-24T23:26:08.130 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.130+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f423c119520 con 0x7f423c1021b0 2024-04-24T23:26:08.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.130+0000 7f4238ff9640 1 -- 172.21.15.31:0/1912688680 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v34) v1 ==== 107+0+5072 (secure 0 0 0) 0x7f422c07c410 con 0x7f423c1021b0 2024-04-24T23:26:08.131 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:08.131 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44317,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6825/3497670820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3497670820},{"type":"v1","addr":"172.21.15.40:6825","nonce":3497670820}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:40.605790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24291":{"gid":24291,"name":"cephfs.smithi040.osunbp","rank":0,"incarnation":25,"state":"up:active","state_seq":195,"addr":"172.21.15.40:6827/1172521773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":1172521773},{"type":"v1","addr":"172.21.15.40:6827","nonce":1172521773}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24291,"qdb_cluster":[24291]},"id":1}]} 2024-04-24T23:26:08.132 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2024-04-24T23:26:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f422406e0b0 msgr2=0x7f4224070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f422406e0b0 0x7f4224070560 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f423c070ea0 tx=0x7f423000b040 comp rx=0 tx=0).stop 2024-04-24T23:26:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 msgr2=0x7f423c06f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 0x7f423c06f770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f422c004930 tx=0x7f422c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 shutdown_connections 2024-04-24T23:26:08.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f422406e0b0 0x7f4224070560 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f423c102b50 0x7f423c06fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 --2- 172.21.15.31:0/1912688680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f423c1021b0 0x7f423c06f770 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 >> 172.21.15.31:0/1912688680 conn(0x7f423c0f7d80 msgr2=0x7f423c0f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 shutdown_connections 2024-04-24T23:26:08.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.132+0000 7f4242407640 1 -- 172.21.15.31:0/1912688680 wait complete. 2024-04-24T23:26:08.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.241+0000 7f4d0779d640 1 -- 172.21.15.31:0/1689075384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 msgr2=0x7f4d00106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.241+0000 7f4d0779d640 1 --2- 172.21.15.31:0/1689075384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 0x7f4d00106170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00048d0 tx=0x7f4cf0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.241+0000 7f4d0779d640 1 -- 172.21.15.31:0/1689075384 shutdown_connections 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.241+0000 7f4d0779d640 1 --2- 172.21.15.31:0/1689075384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 0x7f4d00106170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.241+0000 7f4d0779d640 1 --2- 172.21.15.31:0/1689075384 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000fd190 0x7f4d000fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.241+0000 7f4d0779d640 1 -- 172.21.15.31:0/1689075384 >> 172.21.15.31:0/1689075384 conn(0x7f4d000f6be0 msgr2=0x7f4d000f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.241+0000 7f4d0779d640 1 -- 172.21.15.31:0/1689075384 shutdown_connections 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.241+0000 7f4d0779d640 1 -- 172.21.15.31:0/1689075384 wait complete. 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d0779d640 1 Processor -- start 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d0779d640 1 -- start start 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d0779d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000fd190 0x7f4d000fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d0779d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 0x7f4d000fb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d0779d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000fcea0 con 0x7f4d000fdb30 2024-04-24T23:26:08.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d0779d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000fba40 con 0x7f4d000fd190 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d04d38640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 0x7f4d000fb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d04d38640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 0x7f4d000fb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44202/0 (socket says 172.21.15.31:44202) 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d04d38640 1 -- 172.21.15.31:0/2974712038 learned_addr learned my addr 172.21.15.31:0/2974712038 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d05539640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000fd190 0x7f4d000fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d04d38640 1 -- 172.21.15.31:0/2974712038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000fd190 msgr2=0x7f4d000fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d04d38640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000fd190 0x7f4d000fafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.242+0000 7f4d04d38640 1 -- 172.21.15.31:0/2974712038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf0005040 con 0x7f4d000fdb30 2024-04-24T23:26:08.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.243+0000 7f4d04d38640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 0x7f4d000fb500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0004930 tx=0x7f4cf0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.245+0000 7f4cf5ffb640 1 -- 172.21.15.31:0/2974712038 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf00094c0 con 0x7f4d000fdb30 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.245+0000 7f4cf5ffb640 1 -- 172.21.15.31:0/2974712038 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cf0032e50 con 0x7f4d000fdb30 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.245+0000 7f4cf5ffb640 1 -- 172.21.15.31:0/2974712038 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cf00433b0 con 0x7f4d000fdb30 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.245+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d000fbcc0 con 0x7f4d000fdb30 2024-04-24T23:26:08.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.245+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d001193a0 con 0x7f4d000fdb30 2024-04-24T23:26:08.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.246+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d00066040 con 0x7f4d000fdb30 2024-04-24T23:26:08.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.246+0000 7f4cf5ffb640 1 -- 172.21.15.31:0/2974712038 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4cf002f070 con 0x7f4d000fdb30 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.247+0000 7f4cf5ffb640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4cec06e0b0 0x7f4cec070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.247+0000 7f4cf5ffb640 1 -- 172.21.15.31:0/2974712038 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f4cf00af9e0 con 0x7f4d000fdb30 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.247+0000 7f4d05539640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4cec06e0b0 0x7f4cec070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.247+0000 7f4d05539640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4cec06e0b0 0x7f4cec070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc007fd0 tx=0x7f4cfc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.252 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.251+0000 7f4cf5ffb640 1 -- 172.21.15.31:0/2974712038 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4cf0078d60 con 0x7f4d000fdb30 2024-04-24T23:26:08.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.478+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f4d00064350 con 0x7f4d000fdb30 2024-04-24T23:26:08.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.479+0000 7f4cf5ffb640 1 -- 172.21.15.31:0/2974712038 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v34) v1 ==== 107+0+4271 (secure 0 0 0) 0x7f4cf007c420 con 0x7f4d000fdb30 2024-04-24T23:26:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:08.480 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44317,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6825/3497670820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3497670820},{"type":"v1","addr":"172.21.15.40:6825","nonce":3497670820}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:51.321724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:08.480 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 29 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4cec06e0b0 msgr2=0x7f4cec070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4cec06e0b0 0x7f4cec070560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc007fd0 tx=0x7f4cfc009040 comp rx=0 tx=0).stop 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 msgr2=0x7f4d000fb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 0x7f4d000fb500 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0004930 tx=0x7f4cf0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 shutdown_connections 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f4cec06e0b0 0x7f4cec070560 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d000fdb30 0x7f4d000fb500 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 --2- 172.21.15.31:0/2974712038 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4d000fd190 0x7f4d000fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 >> 172.21.15.31:0/2974712038 conn(0x7f4d000f6be0 msgr2=0x7f4d000f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 shutdown_connections 2024-04-24T23:26:08.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.481+0000 7f4d0779d640 1 -- 172.21.15.31:0/2974712038 wait complete. 2024-04-24T23:26:08.491 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:08 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/2272531058' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-24T23:26:08.491 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:08 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3069545848' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-24T23:26:08.491 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:08 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1912688680' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-24T23:26:08.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-24T23:26:08.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.590+0000 7fe0d559d640 1 -- 172.21.15.31:0/2293894810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 msgr2=0x7fe0d0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.590+0000 7fe0d559d640 1 --2- 172.21.15.31:0/2293894810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 0x7fe0d0101480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00048d0 tx=0x7fe0c0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.590+0000 7fe0d559d640 1 -- 172.21.15.31:0/2293894810 shutdown_connections 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.590+0000 7fe0d559d640 1 --2- 172.21.15.31:0/2293894810 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0d0101a50 0x7fe0d00fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.590+0000 7fe0d559d640 1 --2- 172.21.15.31:0/2293894810 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 0x7fe0d0101480 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.590+0000 7fe0d559d640 1 -- 172.21.15.31:0/2293894810 >> 172.21.15.31:0/2293894810 conn(0x7fe0d00f6c80 msgr2=0x7fe0d00f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.590+0000 7fe0d559d640 1 -- 172.21.15.31:0/2293894810 shutdown_connections 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.590+0000 7fe0d559d640 1 -- 172.21.15.31:0/2293894810 wait complete. 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0d559d640 1 Processor -- start 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0d559d640 1 -- start start 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0d559d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 0x7fe0d010de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0d559d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0d0101a50 0x7fe0d010e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0d559d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d010e8c0 con 0x7fe0d01010b0 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0d559d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d010ea30 con 0x7fe0d0101a50 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 0x7fe0d010de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 0x7fe0d010de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44214/0 (socket says 172.21.15.31:44214) 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0ceffd640 1 -- 172.21.15.31:0/3383736111 learned_addr learned my addr 172.21.15.31:0/3383736111 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0c7fff640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0d0101a50 0x7fe0d010e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0ceffd640 1 -- 172.21.15.31:0/3383736111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0d0101a50 msgr2=0x7fe0d010e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0ceffd640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0d0101a50 0x7fe0d010e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0ceffd640 1 -- 172.21.15.31:0/3383736111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c0005040 con 0x7fe0d01010b0 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.591+0000 7fe0ceffd640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 0x7fe0d010de40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0004930 tx=0x7fe0c0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.592+0000 7fe0c77fe640 1 -- 172.21.15.31:0/3383736111 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0c00094c0 con 0x7fe0d01010b0 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.592+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d010ecb0 con 0x7fe0d01010b0 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.592+0000 7fe0c77fe640 1 -- 172.21.15.31:0/3383736111 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0c0032e50 con 0x7fe0d01010b0 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.592+0000 7fe0c77fe640 1 -- 172.21.15.31:0/3383736111 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0c00433b0 con 0x7fe0d01010b0 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.592+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d0117100 con 0x7fe0d01010b0 2024-04-24T23:26:08.595 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.593+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d00fc820 con 0x7fe0d01010b0 2024-04-24T23:26:08.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.593+0000 7fe0c77fe640 1 -- 172.21.15.31:0/3383736111 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe0c002f070 con 0x7fe0d01010b0 2024-04-24T23:26:08.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.593+0000 7fe0c77fe640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe0bc06e0b0 0x7fe0bc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.594+0000 7fe0c77fe640 1 -- 172.21.15.31:0/3383736111 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fe0c00afa30 con 0x7fe0d01010b0 2024-04-24T23:26:08.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.594+0000 7fe0c7fff640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe0bc06e0b0 0x7fe0bc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.594+0000 7fe0c7fff640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe0bc06e0b0 0x7fe0bc070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0065fb0 tx=0x7fe0b8009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.598+0000 7fe0c77fe640 1 -- 172.21.15.31:0/3383736111 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe0c0078d00 con 0x7fe0d01010b0 2024-04-24T23:26:08.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:08 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/2272531058' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-24T23:26:08.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:08 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3069545848' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-24T23:26:08.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:08 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1912688680' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-24T23:26:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.838+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fe0d0064350 con 0x7fe0d01010b0 2024-04-24T23:26:08.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.839+0000 7fe0c77fe640 1 -- 172.21.15.31:0/3383736111 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v34) v1 ==== 107+0+4350 (secure 0 0 0) 0x7fe0c007c3c0 con 0x7fe0d01010b0 2024-04-24T23:26:08.840 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:08.840 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44317,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6825/3497670820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3497670820},{"type":"v1","addr":"172.21.15.40:6825","nonce":3497670820}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:23:51.327504+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34386":{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":0,"incarnation":30,"state":"up:replay","state_seq":1,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:08.840 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 30 2024-04-24T23:26:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe0bc06e0b0 msgr2=0x7fe0bc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe0bc06e0b0 0x7fe0bc070560 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0065fb0 tx=0x7fe0b8009040 comp rx=0 tx=0).stop 2024-04-24T23:26:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 msgr2=0x7fe0d010de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 0x7fe0d010de40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0004930 tx=0x7fe0c0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 shutdown_connections 2024-04-24T23:26:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fe0bc06e0b0 0x7fe0bc070560 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0d0101a50 0x7fe0d010e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 --2- 172.21.15.31:0/3383736111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0d01010b0 0x7fe0d010de40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 >> 172.21.15.31:0/3383736111 conn(0x7fe0d00f6c80 msgr2=0x7fe0d00f7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 shutdown_connections 2024-04-24T23:26:08.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.841+0000 7fe0d559d640 1 -- 172.21.15.31:0/3383736111 wait complete. 2024-04-24T23:26:08.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-24T23:26:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.950+0000 7f85e2cae640 1 -- 172.21.15.31:0/1333236356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 msgr2=0x7f85dc106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.950+0000 7f85e2cae640 1 --2- 172.21.15.31:0/1333236356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 0x7f85dc106220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f85c8004900 tx=0x7f85c8009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.950+0000 7f85e2cae640 1 -- 172.21.15.31:0/1333236356 shutdown_connections 2024-04-24T23:26:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.950+0000 7f85e2cae640 1 --2- 172.21.15.31:0/1333236356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 0x7f85dc106220 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.950+0000 7f85e2cae640 1 --2- 172.21.15.31:0/1333236356 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0fd1b0 0x7f85dc0fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.950+0000 7f85e2cae640 1 -- 172.21.15.31:0/1333236356 >> 172.21.15.31:0/1333236356 conn(0x7f85dc0f6c50 msgr2=0x7f85dc0f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:08.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.950+0000 7f85e2cae640 1 -- 172.21.15.31:0/1333236356 shutdown_connections 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.950+0000 7f85e2cae640 1 -- 172.21.15.31:0/1333236356 wait complete. 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85e2cae640 1 Processor -- start 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85e2cae640 1 -- start start 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85e2cae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0fd1b0 0x7f85dc0fb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85e2cae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 0x7f85dc0fb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85e2cae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0fcf50 con 0x7f85dc0fdb50 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85e2cae640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0fbaf0 con 0x7f85dc0fd1b0 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 0x7f85dc0fb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 0x7f85dc0fb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33044/0 (socket says 172.21.15.31:33044) 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85dbfff640 1 -- 172.21.15.31:0/1774468708 learned_addr learned my addr 172.21.15.31:0/1774468708 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:08.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.951+0000 7f85e0a4a640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0fd1b0 0x7f85dc0fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85dbfff640 1 -- 172.21.15.31:0/1774468708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0fd1b0 msgr2=0x7f85dc0fb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85dbfff640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0fd1b0 0x7f85dc0fb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85dbfff640 1 -- 172.21.15.31:0/1774468708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8005040 con 0x7f85dc0fdb50 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85dbfff640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 0x7f85dc0fb5b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f85c8034040 tx=0x7f85c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85d97fa640 1 -- 172.21.15.31:0/1774468708 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85c80094c0 con 0x7f85dc0fdb50 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc0fbd70 con 0x7f85dc0fdb50 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85d97fa640 1 -- 172.21.15.31:0/1774468708 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85c803d920 con 0x7f85dc0fdb50 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85d97fa640 1 -- 172.21.15.31:0/1774468708 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85c803b870 con 0x7f85dc0fdb50 2024-04-24T23:26:08.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.952+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc119410 con 0x7f85dc0fdb50 2024-04-24T23:26:08.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.953+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc065fc0 con 0x7f85dc0fdb50 2024-04-24T23:26:08.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.953+0000 7f85d97fa640 1 -- 172.21.15.31:0/1774468708 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f85c803da90 con 0x7f85dc0fdb50 2024-04-24T23:26:08.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.954+0000 7f85d97fa640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f85cc06e0b0 0x7f85cc070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:08.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.954+0000 7f85d97fa640 1 -- 172.21.15.31:0/1774468708 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f85c80b07d0 con 0x7f85dc0fdb50 2024-04-24T23:26:08.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.954+0000 7f85e0a4a640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f85cc06e0b0 0x7f85cc070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:08.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.954+0000 7f85e0a4a640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f85cc06e0b0 0x7f85cc070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f85d0004ed0 tx=0x7f85d000b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:08.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:08.959+0000 7f85d97fa640 1 -- 172.21.15.31:0/1774468708 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f85c8079aa0 con 0x7f85dc0fdb50 2024-04-24T23:26:09.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.181+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f85dc064350 con 0x7f85dc0fdb50 2024-04-24T23:26:09.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.182+0000 7f85d97fa640 1 -- 172.21.15.31:0/1774468708 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v34) v1 ==== 107+0+4354 (secure 0 0 0) 0x7f85c8001050 con 0x7f85dc0fdb50 2024-04-24T23:26:09.183 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:09.183 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44317,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6825/3497670820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3497670820},{"type":"v1","addr":"172.21.15.40:6825","nonce":3497670820}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:24:00.666497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34386":{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":0,"incarnation":30,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:09.183 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 31 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f85cc06e0b0 msgr2=0x7f85cc070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f85cc06e0b0 0x7f85cc070560 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f85d0004ed0 tx=0x7f85d000b040 comp rx=0 tx=0).stop 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 msgr2=0x7f85dc0fb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 0x7f85dc0fb5b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f85c8034040 tx=0x7f85c8009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 shutdown_connections 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f85cc06e0b0 0x7f85cc070560 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85dc0fdb50 0x7f85dc0fb5b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 --2- 172.21.15.31:0/1774468708 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f85dc0fd1b0 0x7f85dc0fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 >> 172.21.15.31:0/1774468708 conn(0x7f85dc0f6c50 msgr2=0x7f85dc0f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 shutdown_connections 2024-04-24T23:26:09.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.184+0000 7f85e2cae640 1 -- 172.21.15.31:0/1774468708 wait complete. 2024-04-24T23:26:09.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-24T23:26:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.293+0000 7f979719d640 1 -- 172.21.15.31:0/2774830204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102ba0 msgr2=0x7f97900fc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.293+0000 7f979719d640 1 --2- 172.21.15.31:0/2774830204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102ba0 0x7f97900fc230 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f97840048d0 tx=0x7f9784009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.293+0000 7f979719d640 1 -- 172.21.15.31:0/2774830204 shutdown_connections 2024-04-24T23:26:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.293+0000 7f979719d640 1 --2- 172.21.15.31:0/2774830204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102ba0 0x7f97900fc230 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.293+0000 7f979719d640 1 --2- 172.21.15.31:0/2774830204 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9790102200 0x7f97901025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.293+0000 7f979719d640 1 -- 172.21.15.31:0/2774830204 >> 172.21.15.31:0/2774830204 conn(0x7f97900f7e10 msgr2=0x7f97900fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.293+0000 7f979719d640 1 -- 172.21.15.31:0/2774830204 shutdown_connections 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.293+0000 7f979719d640 1 -- 172.21.15.31:0/2774830204 wait complete. 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f979719d640 1 Processor -- start 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f979719d640 1 -- start start 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f979719d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102200 0x7f9790112c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f979719d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9790102ba0 0x7f9790113190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f979719d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9790113790 con 0x7f9790102200 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f979719d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f979010ce30 con 0x7f9790102ba0 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f9794f39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102200 0x7f9790112c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f9794f39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102200 0x7f9790112c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33066/0 (socket says 172.21.15.31:33066) 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.294+0000 7f9794f39640 1 -- 172.21.15.31:0/2283019539 learned_addr learned my addr 172.21.15.31:0/2283019539 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f978ffff640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9790102ba0 0x7f9790113190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f9794f39640 1 -- 172.21.15.31:0/2283019539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9790102ba0 msgr2=0x7f9790113190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f9794f39640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9790102ba0 0x7f9790113190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f9794f39640 1 -- 172.21.15.31:0/2283019539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9784005040 con 0x7f9790102200 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f978ffff640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9790102ba0 0x7f9790113190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f9794f39640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102200 0x7f9790112c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f978800cb10 tx=0x7f9788007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f978d7fa640 1 -- 172.21.15.31:0/2283019539 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9788004200 con 0x7f9790102200 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f978d7fa640 1 -- 172.21.15.31:0/2283019539 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f978800acf0 con 0x7f9790102200 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979010d110 con 0x7f9790102200 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f978d7fa640 1 -- 172.21.15.31:0/2283019539 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9788003e40 con 0x7f9790102200 2024-04-24T23:26:09.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.295+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f979010d600 con 0x7f9790102200 2024-04-24T23:26:09.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.296+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97900fd970 con 0x7f9790102200 2024-04-24T23:26:09.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.297+0000 7f978d7fa640 1 -- 172.21.15.31:0/2283019539 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f978800a820 con 0x7f9790102200 2024-04-24T23:26:09.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.297+0000 7f978d7fa640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f977806e050 0x7f9778070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.297+0000 7f978d7fa640 1 -- 172.21.15.31:0/2283019539 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f978808e320 con 0x7f9790102200 2024-04-24T23:26:09.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.297+0000 7f978ffff640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f977806e050 0x7f9778070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.298+0000 7f978ffff640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f977806e050 0x7f9778070500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f97900f94a0 tx=0x7f9784009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.302+0000 7f978d7fa640 1 -- 172.21.15.31:0/2283019539 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f97880575f0 con 0x7f9790102200 2024-04-24T23:26:09.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:09 smithi031 ceph-mon[111404]: pgmap v275: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:09.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:09 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/2974712038' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-24T23:26:09.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:09 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3383736111' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-24T23:26:09.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:09 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1774468708' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-24T23:26:09.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.524+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f9790104640 con 0x7f9790102200 2024-04-24T23:26:09.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.524+0000 7f978d7fa640 1 -- 172.21.15.31:0/2283019539 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v34) v1 ==== 107+0+4351 (secure 0 0 0) 0x7f978805acb0 con 0x7f9790102200 2024-04-24T23:26:09.525 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:09.525 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44317,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6825/3497670820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3497670820},{"type":"v1","addr":"172.21.15.40:6825","nonce":3497670820}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:24:01.465888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34386":{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":0,"incarnation":30,"state":"up:rejoin","state_seq":19,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-24T23:26:09.526 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 32 2024-04-24T23:26:09.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f977806e050 msgr2=0x7f9778070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f977806e050 0x7f9778070500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f97900f94a0 tx=0x7f9784009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102200 msgr2=0x7f9790112c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102200 0x7f9790112c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f978800cb10 tx=0x7f9788007ee0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 shutdown_connections 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f977806e050 0x7f9778070500 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9790102ba0 0x7f9790113190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 --2- 172.21.15.31:0/2283019539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9790102200 0x7f9790112c50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 >> 172.21.15.31:0/2283019539 conn(0x7f97900f7e10 msgr2=0x7f97900f9b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 shutdown_connections 2024-04-24T23:26:09.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.526+0000 7f979719d640 1 -- 172.21.15.31:0/2283019539 wait complete. 2024-04-24T23:26:09.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-24T23:26:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.636+0000 7fcd3042e640 1 -- 172.21.15.31:0/391765309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102c00 msgr2=0x7fcd280fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.636+0000 7fcd3042e640 1 --2- 172.21.15.31:0/391765309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102c00 0x7fcd280fa8c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0048d0 tx=0x7fcd1c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.636+0000 7fcd3042e640 1 -- 172.21.15.31:0/391765309 shutdown_connections 2024-04-24T23:26:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.636+0000 7fcd3042e640 1 --2- 172.21.15.31:0/391765309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102c00 0x7fcd280fa8c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.636+0000 7fcd3042e640 1 --2- 172.21.15.31:0/391765309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd28102260 0x7fcd28102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.636+0000 7fcd3042e640 1 -- 172.21.15.31:0/391765309 >> 172.21.15.31:0/391765309 conn(0x7fcd280717e0 msgr2=0x7fcd28071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd3042e640 1 -- 172.21.15.31:0/391765309 shutdown_connections 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd3042e640 1 -- 172.21.15.31:0/391765309 wait complete. 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd3042e640 1 Processor -- start 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd3042e640 1 -- start start 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd3042e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102260 0x7fcd28112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd3042e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd28102c00 0x7fcd281131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd3042e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd281137c0 con 0x7fcd28102260 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd3042e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2810ce10 con 0x7fcd28102c00 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd2e1ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102260 0x7fcd28112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd2e1ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102260 0x7fcd28112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33078/0 (socket says 172.21.15.31:33078) 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.637+0000 7fcd2e1ca640 1 -- 172.21.15.31:0/1522378149 learned_addr learned my addr 172.21.15.31:0/1522378149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd2d9c9640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd28102c00 0x7fcd281131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd2e1ca640 1 -- 172.21.15.31:0/1522378149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd28102c00 msgr2=0x7fcd281131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd2e1ca640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd28102c00 0x7fcd281131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd2e1ca640 1 -- 172.21.15.31:0/1522378149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c005040 con 0x7fcd28102260 2024-04-24T23:26:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd2e1ca640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102260 0x7fcd28112c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcd20002a50 tx=0x7fcd20004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd26ffd640 1 -- 172.21.15.31:0/1522378149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd20007c30 con 0x7fcd28102260 2024-04-24T23:26:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd26ffd640 1 -- 172.21.15.31:0/1522378149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd2000f040 con 0x7fcd28102260 2024-04-24T23:26:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2810d0f0 con 0x7fcd28102260 2024-04-24T23:26:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd26ffd640 1 -- 172.21.15.31:0/1522378149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd2000db50 con 0x7fcd28102260 2024-04-24T23:26:09.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.638+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2810d610 con 0x7fcd28102260 2024-04-24T23:26:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.639+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd28065530 con 0x7fcd28102260 2024-04-24T23:26:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.640+0000 7fcd26ffd640 1 -- 172.21.15.31:0/1522378149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcd2001c400 con 0x7fcd28102260 2024-04-24T23:26:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.640+0000 7fcd26ffd640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcd1006e0b0 0x7fcd10070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.640+0000 7fcd2d9c9640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcd1006e0b0 0x7fcd10070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.640+0000 7fcd26ffd640 1 -- 172.21.15.31:0/1522378149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fcd2008d4d0 con 0x7fcd28102260 2024-04-24T23:26:09.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.641+0000 7fcd2d9c9640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcd1006e0b0 0x7fcd10070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c000c00 tx=0x7fcd1c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:09.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.645+0000 7fcd26ffd640 1 -- 172.21.15.31:0/1522378149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcd200567a0 con 0x7fcd28102260 2024-04-24T23:26:09.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:09 smithi040 ceph-mon[83047]: pgmap v275: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:09.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:09 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/2974712038' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-24T23:26:09.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:09 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3383736111' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-24T23:26:09.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:09 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1774468708' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-24T23:26:09.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.869+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fcd28064350 con 0x7fcd28102260 2024-04-24T23:26:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.870+0000 7fcd26ffd640 1 -- 172.21.15.31:0/1522378149 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v34) v1 ==== 107+0+5204 (secure 0 0 0) 0x7fcd20059e60 con 0x7fcd28102260 2024-04-24T23:26:09.871 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:09.871 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34410,"name":"cephfs.smithi031.acqiox","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/3258483612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3258483612},{"type":"v1","addr":"172.21.15.31:6829","nonce":3258483612}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":44317,"name":"cephfs.smithi040.dicevu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6825/3497670820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3497670820},{"type":"v1","addr":"172.21.15.40:6825","nonce":3497670820}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44339,"name":"cephfs.smithi040.osunbp","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6827/342930745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6826","nonce":342930745},{"type":"v1","addr":"172.21.15.40:6827","nonce":342930745}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-24T23:10:39.902781+0000","modified":"2024-04-24T23:24:03.468633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34386":{"gid":34386,"name":"cephfs.smithi031.xxejxr","rank":0,"incarnation":30,"state":"up:active","state_seq":20,"addr":"172.21.15.31:6827/2304914531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2304914531},{"type":"v1","addr":"172.21.15.31:6827","nonce":2304914531}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34386,"qdb_cluster":[34386]},"id":1}]} 2024-04-24T23:26:09.871 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 33 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcd1006e0b0 msgr2=0x7fcd10070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcd1006e0b0 0x7fcd10070560 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c000c00 tx=0x7fcd1c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102260 msgr2=0x7fcd28112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102260 0x7fcd28112c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcd20002a50 tx=0x7fcd20004600 comp rx=0 tx=0).stop 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 shutdown_connections 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fcd1006e0b0 0x7fcd10070560 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd28102c00 0x7fcd281131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 --2- 172.21.15.31:0/1522378149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd28102260 0x7fcd28112c80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:09.873 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 >> 172.21.15.31:0/1522378149 conn(0x7fcd280717e0 msgr2=0x7fcd280f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:09.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 shutdown_connections 2024-04-24T23:26:09.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:09.872+0000 7fcd3042e640 1 -- 172.21.15.31:0/1522378149 wait complete. 2024-04-24T23:26:09.884 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-24T23:26:09.894 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-24T23:26:09.894 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:26:09.894 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:09.911 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:26:09.911 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:09.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.105+0000 7f533b39d640 1 -- 172.21.15.31:0/521876517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 msgr2=0x7f53340fa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.105+0000 7f533b39d640 1 --2- 172.21.15.31:0/521876517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 0x7f53340fa7f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53240048d0 tx=0x7f5324009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.105+0000 7f533b39d640 1 -- 172.21.15.31:0/521876517 shutdown_connections 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.105+0000 7f533b39d640 1 --2- 172.21.15.31:0/521876517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 0x7f53340fa7f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.105+0000 7f533b39d640 1 --2- 172.21.15.31:0/521876517 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334102210 0x7f53341025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.105+0000 7f533b39d640 1 -- 172.21.15.31:0/521876517 >> 172.21.15.31:0/521876517 conn(0x7f5334071770 msgr2=0x7f5334071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.105+0000 7f533b39d640 1 -- 172.21.15.31:0/521876517 shutdown_connections 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.105+0000 7f533b39d640 1 -- 172.21.15.31:0/521876517 wait complete. 2024-04-24T23:26:10.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f533b39d640 1 Processor -- start 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f533b39d640 1 -- start start 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f533b39d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334102210 0x7f53340fb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f533b39d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 0x7f53340fbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f533b39d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53340fd8b0 con 0x7f5334102b20 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f533b39d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53340fc450 con 0x7f5334102210 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f5338938640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 0x7f53340fbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f5338938640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 0x7f53340fbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33104/0 (socket says 172.21.15.31:33104) 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f5338938640 1 -- 172.21.15.31:0/2505770885 learned_addr learned my addr 172.21.15.31:0/2505770885 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f5339139640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334102210 0x7f53340fb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f5338938640 1 -- 172.21.15.31:0/2505770885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334102210 msgr2=0x7f53340fb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f5338938640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334102210 0x7f53340fb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f5338938640 1 -- 172.21.15.31:0/2505770885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5324005040 con 0x7f5334102b20 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.106+0000 7f5338938640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 0x7f53340fbf10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5324009fa0 tx=0x7f5324009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.107+0000 7f5329ffb640 1 -- 172.21.15.31:0/2505770885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5324030870 con 0x7f5334102b20 2024-04-24T23:26:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.107+0000 7f5329ffb640 1 -- 172.21.15.31:0/2505770885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53240092e0 con 0x7f5334102b20 2024-04-24T23:26:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.107+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53340fc6d0 con 0x7f5334102b20 2024-04-24T23:26:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.107+0000 7f5329ffb640 1 -- 172.21.15.31:0/2505770885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5324043640 con 0x7f5334102b20 2024-04-24T23:26:10.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.107+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5334119450 con 0x7f5334102b20 2024-04-24T23:26:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.108+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533404bcc0 con 0x7f5334102b20 2024-04-24T23:26:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.108+0000 7f5329ffb640 1 -- 172.21.15.31:0/2505770885 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f532403b420 con 0x7f5334102b20 2024-04-24T23:26:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.109+0000 7f5329ffb640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f532006e0b0 0x7f5320070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.109+0000 7f5329ffb640 1 -- 172.21.15.31:0/2505770885 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f53240b07f0 con 0x7f5334102b20 2024-04-24T23:26:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.109+0000 7f5339139640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f532006e0b0 0x7f5320070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.109+0000 7f5339139640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f532006e0b0 0x7f5320070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53300041d0 tx=0x7f5330004330 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.113+0000 7f5329ffb640 1 -- 172.21.15.31:0/2505770885 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5324079be0 con 0x7f5334102b20 2024-04-24T23:26:10.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.299+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53340fceb0 con 0x7f5334102b20 2024-04-24T23:26:10.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.301+0000 7f5329ffb640 1 -- 172.21.15.31:0/2505770885 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16758 (secure 0 0 0) 0x7f532407d2a0 con 0x7f5334102b20 2024-04-24T23:26:10.302 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":84,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:03:41.065781+0000","modified":"2024-04-24T23:23:51.321472+0000","last_up_change":"2024-04-24T23:22:41.379468+0000","last_in_change":"2024-04-24T23:09:04.280370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:08:05.473853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T23:10:38.874551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T23:10:39.325676+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f1ff679-6b2e-4059-a299-2d2356bbe91c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6803","nonce":4093691970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6805","nonce":4093691970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6809","nonce":4093691970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6807","nonce":4093691970}]},"public_addr":"172.21.15.31:6803/4093691970","cluster_addr":"172.21.15.31:6805/4093691970","heartbeat_back_addr":"172.21.15.31:6809/4093691970","heartbeat_front_addr":"172.21.15.31:6807/4093691970","state":["exists","up"]},{"osd":1,"uuid":"6885f0d2-70e8-4c54-a381-0b3683a0bc49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6811","nonce":1890497972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6813","nonce":1890497972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6817","nonce":1890497972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6815","nonce":1890497972}]},"public_addr":"172.21.15.31:6811/1890497972","cluster_addr":"172.21.15.31:6813/1890497972","heartbeat_back_addr":"172.21.15.31:6817/1890497972","heartbeat_front_addr":"172.21.15.31:6815/1890497972","state":["exists","up"]},{"osd":2,"uuid":"9b57c743-5826-4bf9-ac00-0cb539b30224","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6819","nonce":1599239694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6821","nonce":1599239694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6825","nonce":1599239694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6823","nonce":1599239694}]},"public_addr":"172.21.15.31:6819/1599239694","cluster_addr":"172.21.15.31:6821/1599239694","heartbeat_back_addr":"172.21.15.31:6825/1599239694","heartbeat_front_addr":"172.21.15.31:6823/1599239694","state":["exists","up"]},{"osd":3,"uuid":"1dc47b54-65da-4579-bcac-2258be1f4cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6801","nonce":4020191995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6803","nonce":4020191995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6807","nonce":4020191995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6805","nonce":4020191995}]},"public_addr":"172.21.15.40:6801/4020191995","cluster_addr":"172.21.15.40:6803/4020191995","heartbeat_back_addr":"172.21.15.40:6807/4020191995","heartbeat_front_addr":"172.21.15.40:6805/4020191995","state":["exists","up"]},{"osd":4,"uuid":"e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6809","nonce":3032298464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6811","nonce":3032298464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6815","nonce":3032298464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6813","nonce":3032298464}]},"public_addr":"172.21.15.40:6809/3032298464","cluster_addr":"172.21.15.40:6811/3032298464","heartbeat_back_addr":"172.21.15.40:6815/3032298464","heartbeat_front_addr":"172.21.15.40:6813/3032298464","state":["exists","up"]},{"osd":5,"uuid":"027b7b7a-afb6-4507-8cb2-274ebadbb2ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6817","nonce":3395166888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6819","nonce":3395166888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6823","nonce":3395166888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6821","nonce":3395166888}]},"public_addr":"172.21.15.40:6817/3395166888","cluster_addr":"172.21.15.40:6819/3395166888","heartbeat_back_addr":"172.21.15.40:6823/3395166888","heartbeat_front_addr":"172.21.15.40:6821/3395166888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:08.221856+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:36.407326+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:03.612047+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:29.069233+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:55.660200+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:21.507297+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6825/1413423535":"2024-04-25T23:23:28.676954+0000","172.21.15.31:6829/1027887614":"2024-04-25T23:23:07.318594+0000","172.21.15.31:0/1654686211":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/823375198":"2024-04-25T23:12:59.389091+0000","172.21.15.40:6828/2171597925":"2024-04-25T23:15:14.419837+0000","172.21.15.31:0/1340983696":"2024-04-25T23:04:46.066984+0000","172.21.15.40:6826/1172521773":"2024-04-25T23:23:51.321440+0000","172.21.15.31:6826/2779330105":"2024-04-25T23:10:55.127017+0000","172.21.15.31:0/2308793407":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6800/1306034850":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/2457576751":"2024-04-25T23:04:46.066984+0000","172.21.15.40:0/2449703740":"2024-04-25T23:15:14.419837+0000","172.21.15.31:0/611668999":"2024-04-25T23:04:46.066984+0000","172.21.15.40:0/1074123018":"2024-04-25T23:15:14.419837+0000","172.21.15.31:6800/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6801/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/1805904019":"2024-04-25T23:05:48.958372+0000","172.21.15.40:6824/1413423535":"2024-04-25T23:23:28.676954+0000","172.21.15.31:0/917746224":"2024-04-25T23:18:13.679561+0000","172.21.15.31:6801/1306034850":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/582471448":"2024-04-25T23:12:59.389091+0000","172.21.15.31:6800/1061697388":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/2443214115":"2024-04-25T23:12:59.389091+0000","172.21.15.31:6801/1061697388":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/271213718":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/39610504":"2024-04-25T23:05:48.958372+0000","172.21.15.40:6829/2171597925":"2024-04-25T23:15:14.419837+0000","172.21.15.40:0/3498771068":"2024-04-25T23:15:14.419837+0000","172.21.15.31:6800/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6801/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6801/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6800/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6828/1027887614":"2024-04-25T23:23:07.318594+0000","172.21.15.40:0/3435188664":"2024-04-25T23:15:14.419837+0000","172.21.15.40:6827/1172521773":"2024-04-25T23:23:51.321440+0000","172.21.15.31:0/1480561026":"2024-04-25T23:18:13.679561+0000","172.21.15.31:6827/2779330105":"2024-04-25T23:10:55.127017+0000","172.21.15.31:0/382933206":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/3645410271":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/3145670559":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/3776080305":"2024-04-25T23:18:13.679561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f532006e0b0 msgr2=0x7f5320070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f532006e0b0 0x7f5320070560 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f53300041d0 tx=0x7f5330004330 comp rx=0 tx=0).stop 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 msgr2=0x7f53340fbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 0x7f53340fbf10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5324009fa0 tx=0x7f5324009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 shutdown_connections 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f532006e0b0 0x7f5320070560 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5334102b20 0x7f53340fbf10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 --2- 172.21.15.31:0/2505770885 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5334102210 0x7f53340fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 >> 172.21.15.31:0/2505770885 conn(0x7f5334071770 msgr2=0x7f53340f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 shutdown_connections 2024-04-24T23:26:10.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.303+0000 7f533b39d640 1 -- 172.21.15.31:0/2505770885 wait complete. 2024-04-24T23:26:10.314 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:08:05.473853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T23:10:38.874551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T23:10:39.325676+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:10.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:26:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.415+0000 7f2bebe6e640 1 -- 172.21.15.31:0/2008929560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be41010a0 msgr2=0x7f2be4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.415+0000 7f2bebe6e640 1 --2- 172.21.15.31:0/2008929560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be41010a0 0x7f2be4101470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2be00048d0 tx=0x7f2be0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.415+0000 7f2bebe6e640 1 -- 172.21.15.31:0/2008929560 shutdown_connections 2024-04-24T23:26:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.415+0000 7f2bebe6e640 1 --2- 172.21.15.31:0/2008929560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be4101a40 0x7f2be40fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.415+0000 7f2bebe6e640 1 --2- 172.21.15.31:0/2008929560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be41010a0 0x7f2be4101470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.415+0000 7f2bebe6e640 1 -- 172.21.15.31:0/2008929560 >> 172.21.15.31:0/2008929560 conn(0x7f2be40f6cb0 msgr2=0x7f2be40f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.415+0000 7f2bebe6e640 1 -- 172.21.15.31:0/2008929560 shutdown_connections 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.415+0000 7f2bebe6e640 1 -- 172.21.15.31:0/2008929560 wait complete. 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2bebe6e640 1 Processor -- start 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2bebe6e640 1 -- start start 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2bebe6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be41010a0 0x7f2be410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2bebe6e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4101a40 0x7f2be410e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2bebe6e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be410e910 con 0x7f2be4101a40 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2bebe6e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be410ea80 con 0x7f2be41010a0 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2be9409640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4101a40 0x7f2be410e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2be9409640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4101a40 0x7f2be410e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33134/0 (socket says 172.21.15.31:33134) 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2be9409640 1 -- 172.21.15.31:0/4165507733 learned_addr learned my addr 172.21.15.31:0/4165507733 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2be9c0a640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be41010a0 0x7f2be410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2be9409640 1 -- 172.21.15.31:0/4165507733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be41010a0 msgr2=0x7f2be410de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2be9409640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be41010a0 0x7f2be410de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2be9409640 1 -- 172.21.15.31:0/4165507733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0005040 con 0x7f2be4101a40 2024-04-24T23:26:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.416+0000 7f2be9409640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4101a40 0x7f2be410e3d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400eb70 tx=0x7f2bd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.417+0000 7f2bda7fc640 1 -- 172.21.15.31:0/4165507733 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd4007c30 con 0x7f2be4101a40 2024-04-24T23:26:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.417+0000 7f2bda7fc640 1 -- 172.21.15.31:0/4165507733 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bd400f040 con 0x7f2be4101a40 2024-04-24T23:26:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.417+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be410ed00 con 0x7f2be4101a40 2024-04-24T23:26:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.417+0000 7f2bda7fc640 1 -- 172.21.15.31:0/4165507733 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bd40095b0 con 0x7f2be4101a40 2024-04-24T23:26:10.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.417+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be41171b0 con 0x7f2be4101a40 2024-04-24T23:26:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.418+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be40fc810 con 0x7f2be4101a40 2024-04-24T23:26:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.418+0000 7f2bda7fc640 1 -- 172.21.15.31:0/4165507733 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2bd4009730 con 0x7f2be4101a40 2024-04-24T23:26:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.419+0000 7f2bda7fc640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2bd006e0b0 0x7f2bd0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.419+0000 7f2be9c0a640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2bd006e0b0 0x7f2bd0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.419+0000 7f2bda7fc640 1 -- 172.21.15.31:0/4165507733 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2bd408d710 con 0x7f2be4101a40 2024-04-24T23:26:10.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.419+0000 7f2be9c0a640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2bd006e0b0 0x7f2bd0070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2be0003080 tx=0x7f2be0009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.424+0000 7f2bda7fc640 1 -- 172.21.15.31:0/4165507733 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2bd4056a90 con 0x7f2be4101a40 2024-04-24T23:26:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.607+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2be4108260 con 0x7f2be4101a40 2024-04-24T23:26:10.608 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.607+0000 7f2bda7fc640 1 -- 172.21.15.31:0/4165507733 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2bd405a150 con 0x7f2be4101a40 2024-04-24T23:26:10.608 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-24T23:26:10.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2bd006e0b0 msgr2=0x7f2bd0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2bd006e0b0 0x7f2bd0070560 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2be0003080 tx=0x7f2be0009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4101a40 msgr2=0x7f2be410e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4101a40 0x7f2be410e3d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400eb70 tx=0x7f2bd400b040 comp rx=0 tx=0).stop 2024-04-24T23:26:10.610 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 shutdown_connections 2024-04-24T23:26:10.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f2bd006e0b0 0x7f2bd0070560 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be4101a40 0x7f2be410e3d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 --2- 172.21.15.31:0/4165507733 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2be41010a0 0x7f2be410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 >> 172.21.15.31:0/4165507733 conn(0x7f2be40f6cb0 msgr2=0x7f2be40f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 shutdown_connections 2024-04-24T23:26:10.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.609+0000 7f2bebe6e640 1 -- 172.21.15.31:0/4165507733 wait complete. 2024-04-24T23:26:10.619 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:10 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/2283019539' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-24T23:26:10.619 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:10 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1522378149' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-24T23:26:10.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T23:26:10.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:10 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/2283019539' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-24T23:26:10.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:10 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1522378149' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-24T23:26:10.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 -- 172.21.15.31:0/926154664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c0fb0e0 msgr2=0x7f536c0fb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 --2- 172.21.15.31:0/926154664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c0fb0e0 0x7f536c0fb530 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f53600048d0 tx=0x7f5360009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 -- 172.21.15.31:0/926154664 shutdown_connections 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 --2- 172.21.15.31:0/926154664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c0fb0e0 0x7f536c0fb530 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 --2- 172.21.15.31:0/926154664 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f536c101110 0x7f536c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 -- 172.21.15.31:0/926154664 >> 172.21.15.31:0/926154664 conn(0x7f536c0710c0 msgr2=0x7f536c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 -- 172.21.15.31:0/926154664 shutdown_connections 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 -- 172.21.15.31:0/926154664 wait complete. 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 Processor -- start 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.720+0000 7f5370f1d640 1 -- start start 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5370f1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f536c0fb0e0 0x7f536c0fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5370f1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c101110 0x7f536c0fd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5370f1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0febe0 con 0x7f536c101110 2024-04-24T23:26:10.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5370f1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0fd780 con 0x7f536c0fb0e0 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5369d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c101110 0x7f536c0fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5369d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c101110 0x7f536c0fd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33138/0 (socket says 172.21.15.31:33138) 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5369d9b640 1 -- 172.21.15.31:0/845730687 learned_addr learned my addr 172.21.15.31:0/845730687 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f536a59c640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f536c0fb0e0 0x7f536c0fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5369d9b640 1 -- 172.21.15.31:0/845730687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f536c0fb0e0 msgr2=0x7f536c0fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5369d9b640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f536c0fb0e0 0x7f536c0fcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5369d9b640 1 -- 172.21.15.31:0/845730687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5360005040 con 0x7f536c101110 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5369d9b640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c101110 0x7f536c0fd240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5360030640 tx=0x7f5360009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f535affd640 1 -- 172.21.15.31:0/845730687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5360030820 con 0x7f536c101110 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f535affd640 1 -- 172.21.15.31:0/845730687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53600092e0 con 0x7f536c101110 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c0fda00 con 0x7f536c101110 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f535affd640 1 -- 172.21.15.31:0/845730687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5360043400 con 0x7f536c101110 2024-04-24T23:26:10.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.721+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c119450 con 0x7f536c101110 2024-04-24T23:26:10.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.722+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c0fc7d0 con 0x7f536c101110 2024-04-24T23:26:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.723+0000 7f535affd640 1 -- 172.21.15.31:0/845730687 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f536003b420 con 0x7f536c101110 2024-04-24T23:26:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.723+0000 7f535affd640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f535406e0b0 0x7f5354070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.723+0000 7f535affd640 1 -- 172.21.15.31:0/845730687 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f53600b0940 con 0x7f536c101110 2024-04-24T23:26:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.723+0000 7f536a59c640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f535406e0b0 0x7f5354070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:10.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.724+0000 7f536a59c640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f535406e0b0 0x7f5354070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f535c004a70 tx=0x7f535c00b040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:10.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.728+0000 7f535affd640 1 -- 172.21.15.31:0/845730687 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5360079cb0 con 0x7f536c101110 2024-04-24T23:26:10.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.910+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f536c04bcc0 con 0x7f536c101110 2024-04-24T23:26:10.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.911+0000 7f535affd640 1 -- 172.21.15.31:0/845730687 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f536007d370 con 0x7f536c101110 2024-04-24T23:26:10.912 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-24T23:26:10.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.912+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f535406e0b0 msgr2=0x7f5354070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.912+0000 7f5370f1d640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f535406e0b0 0x7f5354070560 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f535c004a70 tx=0x7f535c00b040 comp rx=0 tx=0).stop 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.912+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c101110 msgr2=0x7f536c0fd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.912+0000 7f5370f1d640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c101110 0x7f536c0fd240 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5360030640 tx=0x7f5360009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.913+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 shutdown_connections 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.913+0000 7f5370f1d640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f535406e0b0 0x7f5354070560 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.913+0000 7f5370f1d640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f536c101110 0x7f536c0fd240 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.913+0000 7f5370f1d640 1 --2- 172.21.15.31:0/845730687 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f536c0fb0e0 0x7f536c0fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.913+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 >> 172.21.15.31:0/845730687 conn(0x7f536c0710c0 msgr2=0x7f536c064350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.913+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 shutdown_connections 2024-04-24T23:26:10.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:10.913+0000 7f5370f1d640 1 -- 172.21.15.31:0/845730687 wait complete. 2024-04-24T23:26:10.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T23:26:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.023+0000 7ff969114640 1 -- 172.21.15.31:0/3978351391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 msgr2=0x7ff964064720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.023+0000 7ff969114640 1 --2- 172.21.15.31:0/3978351391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 0x7ff964064720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c50 tx=0x7ff95c004d10 comp rx=0 tx=0).stop 2024-04-24T23:26:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.023+0000 7ff969114640 1 -- 172.21.15.31:0/3978351391 shutdown_connections 2024-04-24T23:26:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.023+0000 7ff969114640 1 --2- 172.21.15.31:0/3978351391 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff964064c60 0x7ff964108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.023+0000 7ff969114640 1 --2- 172.21.15.31:0/3978351391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 0x7ff964064720 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.023+0000 7ff969114640 1 -- 172.21.15.31:0/3978351391 >> 172.21.15.31:0/3978351391 conn(0x7ff9640710c0 msgr2=0x7ff9640714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.023+0000 7ff969114640 1 -- 172.21.15.31:0/3978351391 shutdown_connections 2024-04-24T23:26:11.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.023+0000 7ff969114640 1 -- 172.21.15.31:0/3978351391 wait complete. 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff969114640 1 Processor -- start 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff969114640 1 -- start start 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff969114640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 0x7ff964114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff969114640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff964064c60 0x7ff964115440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff969114640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff964115a40 con 0x7ff964064350 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff969114640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96410f090 con 0x7ff964064c60 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff962d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 0x7ff964114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff962d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 0x7ff964114f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33156/0 (socket says 172.21.15.31:33156) 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff962d9d640 1 -- 172.21.15.31:0/461340015 learned_addr learned my addr 172.21.15.31:0/461340015 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.024+0000 7ff96259c640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff964064c60 0x7ff964115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff962d9d640 1 -- 172.21.15.31:0/461340015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff964064c60 msgr2=0x7ff964115440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff962d9d640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff964064c60 0x7ff964115440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff962d9d640 1 -- 172.21.15.31:0/461340015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0098b0 con 0x7ff964064350 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff96259c640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff964064c60 0x7ff964115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff962d9d640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 0x7ff964114f00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c20 tx=0x7ff95c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff95b7fe640 1 -- 172.21.15.31:0/461340015 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff95c03a070 con 0x7ff964064350 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff95b7fe640 1 -- 172.21.15.31:0/461340015 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff95c035b20 con 0x7ff964064350 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96410f310 con 0x7ff964064350 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff95b7fe640 1 -- 172.21.15.31:0/461340015 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff95c03eb70 con 0x7ff964064350 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.025+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96410f8b0 con 0x7ff964064350 2024-04-24T23:26:11.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.026+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9640fb6c0 con 0x7ff964064350 2024-04-24T23:26:11.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.027+0000 7ff95b7fe640 1 -- 172.21.15.31:0/461340015 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff95c032070 con 0x7ff964064350 2024-04-24T23:26:11.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.027+0000 7ff95b7fe640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff95006e0b0 0x7ff950070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.027+0000 7ff95b7fe640 1 -- 172.21.15.31:0/461340015 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff95c0b0190 con 0x7ff964064350 2024-04-24T23:26:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.027+0000 7ff96259c640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff95006e0b0 0x7ff950070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.028+0000 7ff96259c640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff95006e0b0 0x7ff950070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff9641107c0 tx=0x7ff94c009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.032+0000 7ff95b7fe640 1 -- 172.21.15.31:0/461340015 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff95c079510 con 0x7ff964064350 2024-04-24T23:26:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.218+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7ff964110570 con 0x7ff964064350 2024-04-24T23:26:11.219 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.219+0000 7ff95b7fe640 1 -- 172.21.15.31:0/461340015 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff95c07cbd0 con 0x7ff964064350 2024-04-24T23:26:11.220 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.220+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff95006e0b0 msgr2=0x7ff950070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.220+0000 7ff969114640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff95006e0b0 0x7ff950070560 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff9641107c0 tx=0x7ff94c009040 comp rx=0 tx=0).stop 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.220+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 msgr2=0x7ff964114f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.220+0000 7ff969114640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 0x7ff964114f00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009c20 tx=0x7ff95c004d10 comp rx=0 tx=0).stop 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.221+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 shutdown_connections 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.221+0000 7ff969114640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7ff95006e0b0 0x7ff950070560 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.221+0000 7ff969114640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff964064c60 0x7ff964115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.221+0000 7ff969114640 1 --2- 172.21.15.31:0/461340015 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff964064350 0x7ff964114f00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.221+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 >> 172.21.15.31:0/461340015 conn(0x7ff9640710c0 msgr2=0x7ff9640f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.221+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 shutdown_connections 2024-04-24T23:26:11.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.221+0000 7ff969114640 1 -- 172.21.15.31:0/461340015 wait complete. 2024-04-24T23:26:11.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 -- 172.21.15.31:0/953426486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c01012a0 msgr2=0x7f03c0101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 --2- 172.21.15.31:0/953426486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c01012a0 0x7f03c0101670 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009c50 tx=0x7f03b0004d10 comp rx=0 tx=0).stop 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 -- 172.21.15.31:0/953426486 shutdown_connections 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 --2- 172.21.15.31:0/953426486 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c0101c40 0x7f03c00fb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 --2- 172.21.15.31:0/953426486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c01012a0 0x7f03c0101670 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 -- 172.21.15.31:0/953426486 >> 172.21.15.31:0/953426486 conn(0x7f03c00f6e70 msgr2=0x7f03c00f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 -- 172.21.15.31:0/953426486 shutdown_connections 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 -- 172.21.15.31:0/953426486 wait complete. 2024-04-24T23:26:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 Processor -- start 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.333+0000 7f03c842e640 1 -- start start 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c842e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c01012a0 0x7f03c010e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c842e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c0101c40 0x7f03c010e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c842e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c00664f0 con 0x7f03c0101c40 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c842e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c010eb30 con 0x7f03c01012a0 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c59c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c0101c40 0x7f03c010e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c59c9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c0101c40 0x7f03c010e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33170/0 (socket says 172.21.15.31:33170) 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c59c9640 1 -- 172.21.15.31:0/736287182 learned_addr learned my addr 172.21.15.31:0/736287182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c61ca640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c01012a0 0x7f03c010e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c59c9640 1 -- 172.21.15.31:0/736287182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c01012a0 msgr2=0x7f03c010e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c59c9640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c01012a0 0x7f03c010e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c59c9640 1 -- 172.21.15.31:0/736287182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00098b0 con 0x7f03c0101c40 2024-04-24T23:26:11.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.334+0000 7f03c59c9640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c0101c40 0x7f03c010e5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f03b4002a50 tx=0x7f03b4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.335+0000 7f03beffd640 1 -- 172.21.15.31:0/736287182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b4007c30 con 0x7f03c0101c40 2024-04-24T23:26:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.335+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c010ee10 con 0x7f03c0101c40 2024-04-24T23:26:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.335+0000 7f03beffd640 1 -- 172.21.15.31:0/736287182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03b400f040 con 0x7f03c0101c40 2024-04-24T23:26:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.335+0000 7f03beffd640 1 -- 172.21.15.31:0/736287182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b400db50 con 0x7f03c0101c40 2024-04-24T23:26:11.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.335+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c0117450 con 0x7f03c0101c40 2024-04-24T23:26:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.336+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c00fca10 con 0x7f03c0101c40 2024-04-24T23:26:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.336+0000 7f03beffd640 1 -- 172.21.15.31:0/736287182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f03b401c400 con 0x7f03c0101c40 2024-04-24T23:26:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.336+0000 7f03beffd640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f03a806e0b0 0x7f03a8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.337+0000 7f03beffd640 1 -- 172.21.15.31:0/736287182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f03b408d4d0 con 0x7f03c0101c40 2024-04-24T23:26:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.337+0000 7f03c61ca640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f03a806e0b0 0x7f03a8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.337+0000 7f03c61ca640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f03a806e0b0 0x7f03a8070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f03b0004a30 tx=0x7f03b002e040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.341+0000 7f03beffd640 1 -- 172.21.15.31:0/736287182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f03b40567a0 con 0x7f03c0101c40 2024-04-24T23:26:11.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:11 smithi031 ceph-mon[111404]: pgmap v276: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:11.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:11 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/2505770885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:11.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:11 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/4165507733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:11.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:11 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/845730687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T23:26:11.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:11 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/461340015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T23:26:11.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.541+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f03c004bcc0 con 0x7f03c0101c40 2024-04-24T23:26:11.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.541+0000 7f03beffd640 1 -- 172.21.15.31:0/736287182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2430 (secure 0 0 0) 0x7f03b4059e60 con 0x7f03c0101c40 2024-04-24T23:26:11.542 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6825/1413423535 2024-04-25T23:23:28.676954+0000 2024-04-24T23:26:11.542 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6829/1027887614 2024-04-25T23:23:07.318594+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1654686211 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/823375198 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6828/2171597925 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1340983696 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6826/1172521773 2024-04-25T23:23:51.321440+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6826/2779330105 2024-04-25T23:10:55.127017+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2308793407 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1306034850 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2457576751 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:0/2449703740 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/611668999 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:0/1074123018 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:11.543 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1555170645 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2092175909 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1805904019 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6824/1413423535 2024-04-25T23:23:28.676954+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/917746224 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1306034850 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/582471448 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1061697388 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2443214115 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1061697388 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/271213718 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/39610504 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6829/2171597925 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:0/3498771068 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2092175909 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:11.544 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/4106752127 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1555170645 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/4106752127 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6828/1027887614 2024-04-25T23:23:07.318594+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:0/3435188664 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6827/1172521773 2024-04-25T23:23:51.321440+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1480561026 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6827/2779330105 2024-04-25T23:10:55.127017+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/382933206 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3645410271 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3145670559 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3776080305 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:11.546 INFO:teuthology.orchestra.run.smithi031.stderr:listed 41 entries 2024-04-24T23:26:11.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f03a806e0b0 msgr2=0x7f03a8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f03a806e0b0 0x7f03a8070560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f03b0004a30 tx=0x7f03b002e040 comp rx=0 tx=0).stop 2024-04-24T23:26:11.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c0101c40 msgr2=0x7f03c010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c0101c40 0x7f03c010e5f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f03b4002a50 tx=0x7f03b4004600 comp rx=0 tx=0).stop 2024-04-24T23:26:11.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 shutdown_connections 2024-04-24T23:26:11.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f03a806e0b0 0x7f03a8070560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03c0101c40 0x7f03c010e5f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 --2- 172.21.15.31:0/736287182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f03c01012a0 0x7f03c010e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 >> 172.21.15.31:0/736287182 conn(0x7f03c00f6e70 msgr2=0x7f03c00f7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 shutdown_connections 2024-04-24T23:26:11.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.544+0000 7f03c842e640 1 -- 172.21.15.31:0/736287182 wait complete. 2024-04-24T23:26:11.555 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-04-24T23:26:11.555 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-24T23:26:11.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T23:26:11.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:11 smithi040 ceph-mon[83047]: pgmap v276: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:11.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:11 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/2505770885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:11 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/4165507733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:11 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/845730687' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T23:26:11.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:11 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/461340015' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.708+0000 7f90cee6e640 1 -- 172.21.15.31:0/286167213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c81021f0 msgr2=0x7f90c81025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.708+0000 7f90cee6e640 1 --2- 172.21.15.31:0/286167213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c81021f0 0x7f90c81025c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f90b80048d0 tx=0x7f90b8009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.708+0000 7f90cee6e640 1 -- 172.21.15.31:0/286167213 shutdown_connections 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.708+0000 7f90cee6e640 1 --2- 172.21.15.31:0/286167213 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c8102b90 0x7f90c8064350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.708+0000 7f90cee6e640 1 --2- 172.21.15.31:0/286167213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c81021f0 0x7f90c81025c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.708+0000 7f90cee6e640 1 -- 172.21.15.31:0/286167213 >> 172.21.15.31:0/286167213 conn(0x7f90c80711c0 msgr2=0x7f90c80715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.708+0000 7f90cee6e640 1 -- 172.21.15.31:0/286167213 shutdown_connections 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.708+0000 7f90cee6e640 1 -- 172.21.15.31:0/286167213 wait complete. 2024-04-24T23:26:11.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90cee6e640 1 Processor -- start 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90cee6e640 1 -- start start 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90cee6e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c81021f0 0x7f90c8112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90cee6e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c8102b90 0x7f90c8113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90cee6e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c8113920 con 0x7f90c8102b90 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90cee6e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c810cee0 con 0x7f90c81021f0 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c8102b90 0x7f90c8113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c8102b90 0x7f90c8113290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33196/0 (socket says 172.21.15.31:33196) 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90c7fff640 1 -- 172.21.15.31:0/1341031555 learned_addr learned my addr 172.21.15.31:0/1341031555 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.709+0000 7f90ccc0a640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c81021f0 0x7f90c8112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90c7fff640 1 -- 172.21.15.31:0/1341031555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c81021f0 msgr2=0x7f90c8112d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90c7fff640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c81021f0 0x7f90c8112d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90c7fff640 1 -- 172.21.15.31:0/1341031555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b8005040 con 0x7f90c8102b90 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90c7fff640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c8102b90 0x7f90c8113290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00b540 tx=0x7f90bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90c57fa640 1 -- 172.21.15.31:0/1341031555 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90bc00f6f0 con 0x7f90c8102b90 2024-04-24T23:26:11.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c810d1c0 con 0x7f90c8102b90 2024-04-24T23:26:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90c57fa640 1 -- 172.21.15.31:0/1341031555 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90bc00fd20 con 0x7f90c8102b90 2024-04-24T23:26:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90c57fa640 1 -- 172.21.15.31:0/1341031555 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90bc0103a0 con 0x7f90c8102b90 2024-04-24T23:26:11.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.710+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c810d6e0 con 0x7f90c8102b90 2024-04-24T23:26:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.711+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c80fd990 con 0x7f90c8102b90 2024-04-24T23:26:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.712+0000 7f90c57fa640 1 -- 172.21.15.31:0/1341031555 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f90bc011400 con 0x7f90c8102b90 2024-04-24T23:26:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.712+0000 7f90c57fa640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90b006e0b0 0x7f90b0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.712+0000 7f90c57fa640 1 -- 172.21.15.31:0/1341031555 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f90bc08db20 con 0x7f90c8102b90 2024-04-24T23:26:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.712+0000 7f90ccc0a640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90b006e0b0 0x7f90b0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:11.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.713+0000 7f90ccc0a640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90b006e0b0 0x7f90b0070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90b80048a0 tx=0x7f90b8009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.717+0000 7f90c57fa640 1 -- 172.21.15.31:0/1341031555 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f90bc056e70 con 0x7f90c8102b90 2024-04-24T23:26:11.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.903+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90c804bcc0 con 0x7f90c8102b90 2024-04-24T23:26:11.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.904+0000 7f90c57fa640 1 -- 172.21.15.31:0/1341031555 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16758 (secure 0 0 0) 0x7f90bc05a530 con 0x7f90c8102b90 2024-04-24T23:26:11.906 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-04-24T23:26:11.906 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":84,"fsid":"ca7b26e4-028e-11ef-bc93-c7b262605968","created":"2024-04-24T23:03:41.065781+0000","modified":"2024-04-24T23:23:51.321472+0000","last_up_change":"2024-04-24T23:22:41.379468+0000","last_in_change":"2024-04-24T23:09:04.280370+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-24T23:08:05.473853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-24T23:10:38.874551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-24T23:10:39.325676+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f1ff679-6b2e-4059-a299-2d2356bbe91c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6803","nonce":4093691970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6805","nonce":4093691970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6809","nonce":4093691970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":4093691970},{"type":"v1","addr":"172.21.15.31:6807","nonce":4093691970}]},"public_addr":"172.21.15.31:6803/4093691970","cluster_addr":"172.21.15.31:6805/4093691970","heartbeat_back_addr":"172.21.15.31:6809/4093691970","heartbeat_front_addr":"172.21.15.31:6807/4093691970","state":["exists","up"]},{"osd":1,"uuid":"6885f0d2-70e8-4c54-a381-0b3683a0bc49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6811","nonce":1890497972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6813","nonce":1890497972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6817","nonce":1890497972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1890497972},{"type":"v1","addr":"172.21.15.31:6815","nonce":1890497972}]},"public_addr":"172.21.15.31:6811/1890497972","cluster_addr":"172.21.15.31:6813/1890497972","heartbeat_back_addr":"172.21.15.31:6817/1890497972","heartbeat_front_addr":"172.21.15.31:6815/1890497972","state":["exists","up"]},{"osd":2,"uuid":"9b57c743-5826-4bf9-ac00-0cb539b30224","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6819","nonce":1599239694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6821","nonce":1599239694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6825","nonce":1599239694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1599239694},{"type":"v1","addr":"172.21.15.31:6823","nonce":1599239694}]},"public_addr":"172.21.15.31:6819/1599239694","cluster_addr":"172.21.15.31:6821/1599239694","heartbeat_back_addr":"172.21.15.31:6825/1599239694","heartbeat_front_addr":"172.21.15.31:6823/1599239694","state":["exists","up"]},{"osd":3,"uuid":"1dc47b54-65da-4579-bcac-2258be1f4cf8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6801","nonce":4020191995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6803","nonce":4020191995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6807","nonce":4020191995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":4020191995},{"type":"v1","addr":"172.21.15.40:6805","nonce":4020191995}]},"public_addr":"172.21.15.40:6801/4020191995","cluster_addr":"172.21.15.40:6803/4020191995","heartbeat_back_addr":"172.21.15.40:6807/4020191995","heartbeat_front_addr":"172.21.15.40:6805/4020191995","state":["exists","up"]},{"osd":4,"uuid":"e09ebdb8-c1ac-4bcb-a542-c2e6bf0f03b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6809","nonce":3032298464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6811","nonce":3032298464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6815","nonce":3032298464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":3032298464},{"type":"v1","addr":"172.21.15.40:6813","nonce":3032298464}]},"public_addr":"172.21.15.40:6809/3032298464","cluster_addr":"172.21.15.40:6811/3032298464","heartbeat_back_addr":"172.21.15.40:6815/3032298464","heartbeat_front_addr":"172.21.15.40:6813/3032298464","state":["exists","up"]},{"osd":5,"uuid":"027b7b7a-afb6-4507-8cb2-274ebadbb2ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6817","nonce":3395166888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6818","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6819","nonce":3395166888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6822","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6823","nonce":3395166888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3395166888},{"type":"v1","addr":"172.21.15.40:6821","nonce":3395166888}]},"public_addr":"172.21.15.40:6817/3395166888","cluster_addr":"172.21.15.40:6819/3395166888","heartbeat_back_addr":"172.21.15.40:6823/3395166888","heartbeat_front_addr":"172.21.15.40:6821/3395166888","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:08.221856+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:07:36.407326+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:03.612047+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:29.069233+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:08:55.660200+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T23:09:21.507297+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.40:6825/1413423535":"2024-04-25T23:23:28.676954+0000","172.21.15.31:6829/1027887614":"2024-04-25T23:23:07.318594+0000","172.21.15.31:0/1654686211":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/823375198":"2024-04-25T23:12:59.389091+0000","172.21.15.40:6828/2171597925":"2024-04-25T23:15:14.419837+0000","172.21.15.31:0/1340983696":"2024-04-25T23:04:46.066984+0000","172.21.15.40:6826/1172521773":"2024-04-25T23:23:51.321440+0000","172.21.15.31:6826/2779330105":"2024-04-25T23:10:55.127017+0000","172.21.15.31:0/2308793407":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6800/1306034850":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/2457576751":"2024-04-25T23:04:46.066984+0000","172.21.15.40:0/2449703740":"2024-04-25T23:15:14.419837+0000","172.21.15.31:0/611668999":"2024-04-25T23:04:46.066984+0000","172.21.15.40:0/1074123018":"2024-04-25T23:15:14.419837+0000","172.21.15.31:6800/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6801/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:0/1805904019":"2024-04-25T23:05:48.958372+0000","172.21.15.40:6824/1413423535":"2024-04-25T23:23:28.676954+0000","172.21.15.31:0/917746224":"2024-04-25T23:18:13.679561+0000","172.21.15.31:6801/1306034850":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/582471448":"2024-04-25T23:12:59.389091+0000","172.21.15.31:6800/1061697388":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/2443214115":"2024-04-25T23:12:59.389091+0000","172.21.15.31:6801/1061697388":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/271213718":"2024-04-25T23:12:59.389091+0000","172.21.15.31:0/39610504":"2024-04-25T23:05:48.958372+0000","172.21.15.40:6829/2171597925":"2024-04-25T23:15:14.419837+0000","172.21.15.40:0/3498771068":"2024-04-25T23:15:14.419837+0000","172.21.15.31:6800/2092175909":"2024-04-25T23:04:46.066984+0000","172.21.15.31:6801/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6801/1555170645":"2024-04-25T23:05:48.958372+0000","172.21.15.31:6800/4106752127":"2024-04-25T23:04:15.327046+0000","172.21.15.31:6828/1027887614":"2024-04-25T23:23:07.318594+0000","172.21.15.40:0/3435188664":"2024-04-25T23:15:14.419837+0000","172.21.15.40:6827/1172521773":"2024-04-25T23:23:51.321440+0000","172.21.15.31:0/1480561026":"2024-04-25T23:18:13.679561+0000","172.21.15.31:6827/2779330105":"2024-04-25T23:10:55.127017+0000","172.21.15.31:0/382933206":"2024-04-25T23:05:48.958372+0000","172.21.15.31:0/3645410271":"2024-04-25T23:04:15.327046+0000","172.21.15.31:0/3145670559":"2024-04-25T23:18:13.679561+0000","172.21.15.31:0/3776080305":"2024-04-25T23:18:13.679561+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T23:26:11.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.906+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90b006e0b0 msgr2=0x7f90b0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.906+0000 7f90cee6e640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90b006e0b0 0x7f90b0070560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90b80048a0 tx=0x7f90b8009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.906+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c8102b90 msgr2=0x7f90c8113290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.906+0000 7f90cee6e640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c8102b90 0x7f90c8113290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00b540 tx=0x7f90bc0091a0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.907+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 shutdown_connections 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.907+0000 7f90cee6e640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f90b006e0b0 0x7f90b0070560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.907+0000 7f90cee6e640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90c8102b90 0x7f90c8113290 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.907+0000 7f90cee6e640 1 --2- 172.21.15.31:0/1341031555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f90c81021f0 0x7f90c8112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.907+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 >> 172.21.15.31:0/1341031555 conn(0x7f90c80711c0 msgr2=0x7f90c80fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.907+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 shutdown_connections 2024-04-24T23:26:11.908 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:11.907+0000 7f90cee6e640 1 -- 172.21.15.31:0/1341031555 wait complete. 2024-04-24T23:26:11.918 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-24T23:08:05.473853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-24T23:10:38.874551+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-24T23:10:39.325676+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-24T23:26:11.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-24T23:26:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 -- 172.21.15.31:0/1092252000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9980102240 msgr2=0x7f9980102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 --2- 172.21.15.31:0/1092252000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9980102240 0x7f9980102610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f99700048d0 tx=0x7f9970009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 -- 172.21.15.31:0/1092252000 shutdown_connections 2024-04-24T23:26:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 --2- 172.21.15.31:0/1092252000 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99800fa840 0x7f99800facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 --2- 172.21.15.31:0/1092252000 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9980102240 0x7f9980102610 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 -- 172.21.15.31:0/1092252000 >> 172.21.15.31:0/1092252000 conn(0x7f9980071700 msgr2=0x7f9980071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 -- 172.21.15.31:0/1092252000 shutdown_connections 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 -- 172.21.15.31:0/1092252000 wait complete. 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 Processor -- start 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 -- start start 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.019+0000 7f9986b1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99800fa840 0x7f9980112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f9986b1d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9980102240 0x7f9980113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f9986b1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9980113820 con 0x7f99800fa840 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f9986b1d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998010ce70 con 0x7f9980102240 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f99848b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99800fa840 0x7f9980112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f99848b9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99800fa840 0x7f9980112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33212/0 (socket says 172.21.15.31:33212) 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f99848b9640 1 -- 172.21.15.31:0/1336946260 learned_addr learned my addr 172.21.15.31:0/1336946260 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f997ffff640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9980102240 0x7f9980113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f99848b9640 1 -- 172.21.15.31:0/1336946260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9980102240 msgr2=0x7f9980113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f99848b9640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9980102240 0x7f9980113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f99848b9640 1 -- 172.21.15.31:0/1336946260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970005040 con 0x7f99800fa840 2024-04-24T23:26:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f99848b9640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99800fa840 0x7f9980112ce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f997002be70 tx=0x7f9970009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f997d7fa640 1 -- 172.21.15.31:0/1336946260 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99700094c0 con 0x7f99800fa840 2024-04-24T23:26:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f997d7fa640 1 -- 172.21.15.31:0/1336946260 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9970009620 con 0x7f99800fa840 2024-04-24T23:26:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998010d0f0 con 0x7f99800fa840 2024-04-24T23:26:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f997d7fa640 1 -- 172.21.15.31:0/1336946260 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99700435f0 con 0x7f99800fa840 2024-04-24T23:26:12.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.020+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998010d5b0 con 0x7f99800fa840 2024-04-24T23:26:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.021+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99800652b0 con 0x7f99800fa840 2024-04-24T23:26:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.022+0000 7f997d7fa640 1 -- 172.21.15.31:0/1336946260 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f997002f070 con 0x7f99800fa840 2024-04-24T23:26:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.022+0000 7f997d7fa640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f996c06e0b0 0x7f996c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.022+0000 7f997d7fa640 1 -- 172.21.15.31:0/1336946260 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f99700afa50 con 0x7f99800fa840 2024-04-24T23:26:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.022+0000 7f997ffff640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f996c06e0b0 0x7f996c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.023+0000 7f997ffff640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f996c06e0b0 0x7f996c070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f998010e5a0 tx=0x7f9968009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.027+0000 7f997d7fa640 1 -- 172.21.15.31:0/1336946260 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9970078dd0 con 0x7f99800fa840 2024-04-24T23:26:12.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.211+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f998010e350 con 0x7f99800fa840 2024-04-24T23:26:12.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.211+0000 7f997d7fa640 1 -- 172.21.15.31:0/1336946260 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f997007c490 con 0x7f99800fa840 2024-04-24T23:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-04-24T23:26:12.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f996c06e0b0 msgr2=0x7f996c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f996c06e0b0 0x7f996c070560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f998010e5a0 tx=0x7f9968009040 comp rx=0 tx=0).stop 2024-04-24T23:26:12.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99800fa840 msgr2=0x7f9980112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99800fa840 0x7f9980112ce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f997002be70 tx=0x7f9970009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 shutdown_connections 2024-04-24T23:26:12.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f996c06e0b0 0x7f996c070560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9980102240 0x7f9980113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 --2- 172.21.15.31:0/1336946260 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99800fa840 0x7f9980112ce0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 >> 172.21.15.31:0/1336946260 conn(0x7f9980071700 msgr2=0x7f99800f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 shutdown_connections 2024-04-24T23:26:12.215 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.213+0000 7f9986b1d640 1 -- 172.21.15.31:0/1336946260 wait complete. 2024-04-24T23:26:12.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-24T23:26:12.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 -- 172.21.15.31:0/3748658085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0058101a40 msgr2=0x7f00580fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 --2- 172.21.15.31:0/3748658085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0058101a40 0x7f00580fb0d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f004c0048d0 tx=0x7f004c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 -- 172.21.15.31:0/3748658085 shutdown_connections 2024-04-24T23:26:12.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 --2- 172.21.15.31:0/3748658085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0058101a40 0x7f00580fb0d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 --2- 172.21.15.31:0/3748658085 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00581010a0 0x7f0058101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 -- 172.21.15.31:0/3748658085 >> 172.21.15.31:0/3748658085 conn(0x7f00580f6c90 msgr2=0x7f00580f90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 -- 172.21.15.31:0/3748658085 shutdown_connections 2024-04-24T23:26:12.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 -- 172.21.15.31:0/3748658085 wait complete. 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.326+0000 7f005dd9d640 1 Processor -- start 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f005dd9d640 1 -- start start 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f005dd9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00581010a0 0x7f0058105f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f005dd9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0058101a40 0x7f00581064a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f005dd9d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0058107e40 con 0x7f00581010a0 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f005dd9d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00581069e0 con 0x7f0058101a40 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f00577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00581010a0 0x7f0058105f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f00577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00581010a0 0x7f0058105f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33234/0 (socket says 172.21.15.31:33234) 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f00577fe640 1 -- 172.21.15.31:0/3921743231 learned_addr learned my addr 172.21.15.31:0/3921743231 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f0056ffd640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0058101a40 0x7f00581064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f00577fe640 1 -- 172.21.15.31:0/3921743231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0058101a40 msgr2=0x7f00581064a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f00577fe640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0058101a40 0x7f00581064a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f00577fe640 1 -- 172.21.15.31:0/3921743231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c005040 con 0x7f00581010a0 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.327+0000 7f00577fe640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00581010a0 0x7f0058105f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0048002a50 tx=0x7f0048004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.328+0000 7f003ffff640 1 -- 172.21.15.31:0/3921743231 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0048007c30 con 0x7f00581010a0 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.328+0000 7f003ffff640 1 -- 172.21.15.31:0/3921743231 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f004800f040 con 0x7f00581010a0 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.328+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0058106cc0 con 0x7f00581010a0 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.328+0000 7f003ffff640 1 -- 172.21.15.31:0/3921743231 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f004800db50 con 0x7f00581010a0 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.328+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0058119430 con 0x7f00581010a0 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.329+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00580fc810 con 0x7f00581010a0 2024-04-24T23:26:12.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.329+0000 7f003ffff640 1 -- 172.21.15.31:0/3921743231 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f004801c400 con 0x7f00581010a0 2024-04-24T23:26:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.330+0000 7f003ffff640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f004006e0b0 0x7f0040070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.330+0000 7f003ffff640 1 -- 172.21.15.31:0/3921743231 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f004808d570 con 0x7f00581010a0 2024-04-24T23:26:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.330+0000 7f0056ffd640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f004006e0b0 0x7f0040070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.330+0000 7f0056ffd640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f004006e0b0 0x7f0040070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f004c003080 tx=0x7f004c009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.334+0000 7f003ffff640 1 -- 172.21.15.31:0/3921743231 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f00480567a0 con 0x7f00581010a0 2024-04-24T23:26:12.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:12 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/736287182' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:26:12.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:12 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1341031555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:12.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:12 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1336946260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.520+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f005804bcc0 con 0x7f00581010a0 2024-04-24T23:26:12.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.521+0000 7f003ffff640 1 -- 172.21.15.31:0/3921743231 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0048059e60 con 0x7f00581010a0 2024-04-24T23:26:12.522 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.522+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f004006e0b0 msgr2=0x7f0040070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.522+0000 7f005dd9d640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f004006e0b0 0x7f0040070560 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f004c003080 tx=0x7f004c009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.522+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00581010a0 msgr2=0x7f0058105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.522+0000 7f005dd9d640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00581010a0 0x7f0058105f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0048002a50 tx=0x7f0048004600 comp rx=0 tx=0).stop 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.523+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 shutdown_connections 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.523+0000 7f005dd9d640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f004006e0b0 0x7f0040070560 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.523+0000 7f005dd9d640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0058101a40 0x7f00581064a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.523+0000 7f005dd9d640 1 --2- 172.21.15.31:0/3921743231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00581010a0 0x7f0058105f60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.523+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 >> 172.21.15.31:0/3921743231 conn(0x7f00580f6c90 msgr2=0x7f00580f7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.523+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 shutdown_connections 2024-04-24T23:26:12.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.523+0000 7f005dd9d640 1 -- 172.21.15.31:0/3921743231 wait complete. 2024-04-24T23:26:12.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-24T23:26:12.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.633+0000 7fc0cd39d640 1 -- 172.21.15.31:0/4100778425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c81021e0 msgr2=0x7fc0c81025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.633+0000 7fc0cd39d640 1 --2- 172.21.15.31:0/4100778425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c81021e0 0x7fc0c81025b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80048d0 tx=0x7fc0b8009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 -- 172.21.15.31:0/4100778425 shutdown_connections 2024-04-24T23:26:12.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 --2- 172.21.15.31:0/4100778425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c8064380 0x7fc0c80647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 --2- 172.21.15.31:0/4100778425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c81021e0 0x7fc0c81025b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 -- 172.21.15.31:0/4100778425 >> 172.21.15.31:0/4100778425 conn(0x7fc0c80710c0 msgr2=0x7fc0c80714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 -- 172.21.15.31:0/4100778425 shutdown_connections 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 -- 172.21.15.31:0/4100778425 wait complete. 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 Processor -- start 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 -- start start 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c8064380 0x7fc0c8112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.634+0000 7fc0cd39d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c81021e0 0x7fc0c81131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0cd39d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c81137d0 con 0x7fc0c8064380 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0cd39d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c810ce20 con 0x7fc0c81021e0 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c8064380 0x7fc0c8112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c8064380 0x7fc0c8112c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33254/0 (socket says 172.21.15.31:33254) 2024-04-24T23:26:12.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0c6ffd640 1 -- 172.21.15.31:0/1634036113 learned_addr learned my addr 172.21.15.31:0/1634036113 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0c6ffd640 1 -- 172.21.15.31:0/1634036113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c81021e0 msgr2=0x7fc0c81131d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0c67fc640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c81021e0 0x7fc0c81131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0c6ffd640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c81021e0 0x7fc0c81131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0c6ffd640 1 -- 172.21.15.31:0/1634036113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b8005040 con 0x7fc0c8064380 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0c6ffd640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c8064380 0x7fc0c8112c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8004930 tx=0x7fc0b8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0af7fe640 1 -- 172.21.15.31:0/1634036113 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0b80094c0 con 0x7fc0c8064380 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0af7fe640 1 -- 172.21.15.31:0/1634036113 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0b8032e50 con 0x7fc0c8064380 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c810d0a0 con 0x7fc0c8064380 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0af7fe640 1 -- 172.21.15.31:0/1634036113 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0b80433b0 con 0x7fc0c8064380 2024-04-24T23:26:12.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.635+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c810d560 con 0x7fc0c8064380 2024-04-24T23:26:12.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.636+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c80fd8e0 con 0x7fc0c8064380 2024-04-24T23:26:12.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.637+0000 7fc0af7fe640 1 -- 172.21.15.31:0/1634036113 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc0b802f070 con 0x7fc0c8064380 2024-04-24T23:26:12.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.637+0000 7fc0af7fe640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc0b006e0b0 0x7fc0b0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.637+0000 7fc0af7fe640 1 -- 172.21.15.31:0/1634036113 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fc0b80afbf0 con 0x7fc0c8064380 2024-04-24T23:26:12.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.637+0000 7fc0c67fc640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc0b006e0b0 0x7fc0b0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.638+0000 7fc0c67fc640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc0b006e0b0 0x7fc0b0070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0c810e550 tx=0x7fc0bc009040 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.642 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.642+0000 7fc0af7fe640 1 -- 172.21.15.31:0/1634036113 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc0b8078ec0 con 0x7fc0c8064380 2024-04-24T23:26:12.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:12 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/736287182' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:26:12.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:12 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1341031555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-24T23:26:12.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:12 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1336946260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-24T23:26:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.828+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fc0c810e300 con 0x7fc0c8064380 2024-04-24T23:26:12.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.828+0000 7fc0af7fe640 1 -- 172.21.15.31:0/1634036113 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc0b807c580 con 0x7fc0c8064380 2024-04-24T23:26:12.830 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 32 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc0b006e0b0 msgr2=0x7fc0b0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc0b006e0b0 0x7fc0b0070560 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc0c810e550 tx=0x7fc0bc009040 comp rx=0 tx=0).stop 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c8064380 msgr2=0x7fc0c8112c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c8064380 0x7fc0c8112c90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8004930 tx=0x7fc0b8009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 shutdown_connections 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7fc0b006e0b0 0x7fc0b0070560 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc0c81021e0 0x7fc0c81131d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 --2- 172.21.15.31:0/1634036113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0c8064380 0x7fc0c8112c90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.830+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 >> 172.21.15.31:0/1634036113 conn(0x7fc0c80710c0 msgr2=0x7fc0c80f9490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.831+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 shutdown_connections 2024-04-24T23:26:12.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.831+0000 7fc0cd39d640 1 -- 172.21.15.31:0/1634036113 wait complete. 2024-04-24T23:26:12.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-24T23:26:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 -- 172.21.15.31:0/439004875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 msgr2=0x7f87ec1025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 --2- 172.21.15.31:0/439004875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 0x7f87ec1025f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0048d0 tx=0x7f87dc009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 -- 172.21.15.31:0/439004875 shutdown_connections 2024-04-24T23:26:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 --2- 172.21.15.31:0/439004875 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec064390 0x7f87ec0647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 --2- 172.21.15.31:0/439004875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 0x7f87ec1025f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 -- 172.21.15.31:0/439004875 >> 172.21.15.31:0/439004875 conn(0x7f87ec0710c0 msgr2=0x7f87ec0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 -- 172.21.15.31:0/439004875 shutdown_connections 2024-04-24T23:26:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 -- 172.21.15.31:0/439004875 wait complete. 2024-04-24T23:26:12.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.941+0000 7f87f2425640 1 Processor -- start 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87f2425640 1 -- start start 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87f2425640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec064390 0x7f87ec112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87f2425640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 0x7f87ec113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87f2425640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec113800 con 0x7f87ec102220 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87f2425640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec10ce50 con 0x7f87ec064390 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 0x7f87ec113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87eb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 0x7f87ec113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33266/0 (socket says 172.21.15.31:33266) 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec064390 0x7f87ec112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87eb7fe640 1 -- 172.21.15.31:0/3142868240 learned_addr learned my addr 172.21.15.31:0/3142868240 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87eb7fe640 1 -- 172.21.15.31:0/3142868240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec064390 msgr2=0x7f87ec112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87eb7fe640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec064390 0x7f87ec112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.942+0000 7f87eb7fe640 1 -- 172.21.15.31:0/3142868240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc005040 con 0x7f87ec102220 2024-04-24T23:26:12.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.943+0000 7f87eb7fe640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 0x7f87ec113200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f87d400eb70 tx=0x7f87d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.943+0000 7f87e8ff9640 1 -- 172.21.15.31:0/3142868240 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87d4007c30 con 0x7f87ec102220 2024-04-24T23:26:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.943+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec10d0d0 con 0x7f87ec102220 2024-04-24T23:26:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.943+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec10d5f0 con 0x7f87ec102220 2024-04-24T23:26:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.943+0000 7f87e8ff9640 1 -- 172.21.15.31:0/3142868240 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87d400f040 con 0x7f87ec102220 2024-04-24T23:26:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.943+0000 7f87e8ff9640 1 -- 172.21.15.31:0/3142868240 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87d4009590 con 0x7f87ec102220 2024-04-24T23:26:12.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.944+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87cc005210 con 0x7f87ec102220 2024-04-24T23:26:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.944+0000 7f87e8ff9640 1 -- 172.21.15.31:0/3142868240 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f87d4009730 con 0x7f87ec102220 2024-04-24T23:26:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.945+0000 7f87e8ff9640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f87d806e0c0 0x7f87d8070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-24T23:26:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.945+0000 7f87e8ff9640 1 -- 172.21.15.31:0/3142868240 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f87d408d650 con 0x7f87ec102220 2024-04-24T23:26:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.945+0000 7f87ebfff640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f87d806e0c0 0x7f87d8070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-24T23:26:12.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.945+0000 7f87ebfff640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f87d806e0c0 0x7f87d8070570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000c00 tx=0x7f87dc009da0 comp rx=0 tx=0).ready entity=mgr.34102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-24T23:26:12.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:12.954+0000 7f87e8ff9640 1 -- 172.21.15.31:0/3142868240 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f87d40569d0 con 0x7f87ec102220 2024-04-24T23:26:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.154+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f87cc005a40 con 0x7f87ec102220 2024-04-24T23:26:13.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.155+0000 7f87e8ff9640 1 -- 172.21.15.31:0/3142868240 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v84) v1 ==== 81+0+2430 (secure 0 0 0) 0x7f87d405a090 con 0x7f87ec102220 2024-04-24T23:26:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6825/1413423535 2024-04-25T23:23:28.676954+0000 2024-04-24T23:26:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6829/1027887614 2024-04-25T23:23:07.318594+0000 2024-04-24T23:26:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1654686211 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/823375198 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6828/2171597925 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:13.156 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1340983696 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6826/1172521773 2024-04-25T23:23:51.321440+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6826/2779330105 2024-04-25T23:10:55.127017+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2308793407 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1306034850 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2457576751 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:0/2449703740 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/611668999 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:0/1074123018 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1555170645 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2092175909 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1805904019 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6824/1413423535 2024-04-25T23:23:28.676954+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/917746224 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:13.157 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1306034850 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/582471448 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/1061697388 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2443214115 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1061697388 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/271213718 2024-04-25T23:12:59.389091+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/39610504 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6829/2171597925 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:0/3498771068 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2092175909 2024-04-25T23:04:46.066984+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/4106752127 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/1555170645 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/4106752127 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6828/1027887614 2024-04-25T23:23:07.318594+0000 2024-04-24T23:26:13.158 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:0/3435188664 2024-04-25T23:15:14.419837+0000 2024-04-24T23:26:13.159 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.40:6827/1172521773 2024-04-25T23:23:51.321440+0000 2024-04-24T23:26:13.159 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1480561026 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:13.159 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6827/2779330105 2024-04-25T23:10:55.127017+0000 2024-04-24T23:26:13.159 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/382933206 2024-04-25T23:05:48.958372+0000 2024-04-24T23:26:13.159 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3645410271 2024-04-25T23:04:15.327046+0000 2024-04-24T23:26:13.159 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3145670559 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:13.159 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3776080305 2024-04-25T23:18:13.679561+0000 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:listed 41 entries 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f87d806e0c0 msgr2=0x7f87d8070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f87d806e0c0 0x7f87d8070570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000c00 tx=0x7f87dc009da0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 msgr2=0x7f87ec113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 0x7f87ec113200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f87d400eb70 tx=0x7f87d400b040 comp rx=0 tx=0).stop 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 shutdown_connections 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:6800/4072072744,v1:172.21.15.31:6801/4072072744] conn(0x7f87d806e0c0 0x7f87d8070570 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87ec102220 0x7f87ec113200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 --2- 172.21.15.31:0/3142868240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f87ec064390 0x7f87ec112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-24T23:26:13.160 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 >> 172.21.15.31:0/3142868240 conn(0x7f87ec0710c0 msgr2=0x7f87ec0f9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-24T23:26:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 shutdown_connections 2024-04-24T23:26:13.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-04-24T23:26:13.157+0000 7f87f2425640 1 -- 172.21.15.31:0/3142868240 wait complete. 2024-04-24T23:26:13.169 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi031.front.sepia.ceph.com... 2024-04-24T23:26:13.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-24T23:26:13.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-24T23:26:13.196 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T23:26:13.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:13 smithi031 ceph-mon[111404]: pgmap v277: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:13.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:13 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3921743231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T23:26:13.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:13.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:13 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/1634036113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T23:26:13.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:13 smithi031 ceph-mon[111404]: from='client.? 172.21.15.31:0/3142868240' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:26:13.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:26:13.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:13 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:26:13.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:13 smithi040 ceph-mon[83047]: pgmap v277: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:13.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:13 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3921743231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-24T23:26:13.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:13 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:26:13.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:13 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/1634036113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-24T23:26:13.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:13 smithi040 ceph-mon[83047]: from='client.? 172.21.15.31:0/3142868240' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-24T23:26:13.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:13 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:26:13.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:13 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:26:14.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:26:14.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:14.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:14.498 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:26:14.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:26:14.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:26:14.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:26:14.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:26:15.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:15 smithi040 ceph-mon[83047]: pgmap v278: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:15.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:15 smithi031 ceph-mon[111404]: pgmap v278: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:17.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:17 smithi040 ceph-mon[83047]: pgmap v279: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:17.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:17 smithi031 ceph-mon[111404]: pgmap v279: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:19.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:19 smithi040 ceph-mon[83047]: pgmap v280: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:19.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:19 smithi031 ceph-mon[111404]: pgmap v280: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:21.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:21 smithi040 ceph-mon[83047]: pgmap v281: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:21.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:21 smithi031 ceph-mon[111404]: pgmap v281: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:23.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:23 smithi040 ceph-mon[83047]: pgmap v282: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:23.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:23 smithi031 ceph-mon[111404]: pgmap v282: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:25.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:25 smithi040 ceph-mon[83047]: pgmap v283: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:25.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:25 smithi031 ceph-mon[111404]: pgmap v283: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:27.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:27 smithi040 ceph-mon[83047]: pgmap v284: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:27.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:27 smithi031 ceph-mon[111404]: pgmap v284: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:29.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:29 smithi040 ceph-mon[83047]: pgmap v285: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:29.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:26:29.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:29 smithi031 ceph-mon[111404]: pgmap v285: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:29.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:26:31.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:31 smithi040 ceph-mon[83047]: pgmap v286: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:31.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:31 smithi031 ceph-mon[111404]: pgmap v286: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:33.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:33 smithi040 ceph-mon[83047]: pgmap v287: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:33.749 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:33 smithi031 ceph-mon[111404]: pgmap v287: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:35.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:35 smithi040 ceph-mon[83047]: pgmap v288: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:35.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:35 smithi031 ceph-mon[111404]: pgmap v288: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:37.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:37 smithi040 ceph-mon[83047]: pgmap v289: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:37.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:37 smithi031 ceph-mon[111404]: pgmap v289: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:39.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:39 smithi040 ceph-mon[83047]: pgmap v290: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:39.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:39 smithi031 ceph-mon[111404]: pgmap v290: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:41.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:41 smithi040 ceph-mon[83047]: pgmap v291: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:41.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:41 smithi031 ceph-mon[111404]: pgmap v291: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:43.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:43 smithi040 ceph-mon[83047]: pgmap v292: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:43.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:43 smithi031 ceph-mon[111404]: pgmap v292: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:44.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:26:44.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:26:45.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:45 smithi040 ceph-mon[83047]: pgmap v293: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:45.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:45 smithi031 ceph-mon[111404]: pgmap v293: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:47.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:47 smithi040 ceph-mon[83047]: pgmap v294: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:47.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:47 smithi031 ceph-mon[111404]: pgmap v294: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:49.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:49 smithi040 ceph-mon[83047]: pgmap v295: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:49.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:49 smithi031 ceph-mon[111404]: pgmap v295: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:51.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:51 smithi040 ceph-mon[83047]: pgmap v296: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:51.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:51 smithi031 ceph-mon[111404]: pgmap v296: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:53.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:53 smithi040 ceph-mon[83047]: pgmap v297: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:53.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:53 smithi031 ceph-mon[111404]: pgmap v297: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:55.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:55 smithi040 ceph-mon[83047]: pgmap v298: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:55.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:55 smithi031 ceph-mon[111404]: pgmap v298: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:57.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:57 smithi040 ceph-mon[83047]: pgmap v299: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:57.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:57 smithi031 ceph-mon[111404]: pgmap v299: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:59.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:59 smithi040 ceph-mon[83047]: pgmap v300: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:59.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:26:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:26:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:59 smithi031 ceph-mon[111404]: pgmap v300: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:26:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:26:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:27:01.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:01 smithi040 ceph-mon[83047]: pgmap v301: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:01.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:01 smithi031 ceph-mon[111404]: pgmap v301: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:03.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:03 smithi040 ceph-mon[83047]: pgmap v302: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:03.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:03 smithi031 ceph-mon[111404]: pgmap v302: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:05.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:05 smithi040 ceph-mon[83047]: pgmap v303: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:05.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:05 smithi031 ceph-mon[111404]: pgmap v303: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:07.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:07 smithi040 ceph-mon[83047]: pgmap v304: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:07.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:07 smithi031 ceph-mon[111404]: pgmap v304: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:09.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:09 smithi040 ceph-mon[83047]: pgmap v305: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:09.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:09 smithi031 ceph-mon[111404]: pgmap v305: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:11.663 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:11 smithi040 ceph-mon[83047]: pgmap v306: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:11.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:11 smithi031 ceph-mon[111404]: pgmap v306: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:13.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:13 smithi040 ceph-mon[83047]: pgmap v307: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:13.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:13 smithi031 ceph-mon[111404]: pgmap v307: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:14.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:27:14.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:14.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:27:14.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:27:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:15 smithi040 ceph-mon[83047]: pgmap v308: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:15.662 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:27:15.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:15 smithi031 ceph-mon[111404]: pgmap v308: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:15.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:27:15.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:27:15.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:27:17.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:17 smithi040 ceph-mon[83047]: pgmap v309: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:17.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:17 smithi031 ceph-mon[111404]: pgmap v309: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:19.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:19 smithi040 ceph-mon[83047]: pgmap v310: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:19.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:19 smithi031 ceph-mon[111404]: pgmap v310: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:21.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:21 smithi040 ceph-mon[83047]: pgmap v311: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:21.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:21 smithi031 ceph-mon[111404]: pgmap v311: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:23.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:23 smithi040 ceph-mon[83047]: pgmap v312: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:23.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:23 smithi031 ceph-mon[111404]: pgmap v312: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:25.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:25 smithi040 ceph-mon[83047]: pgmap v313: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:25.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:25 smithi031 ceph-mon[111404]: pgmap v313: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:27.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:27 smithi040 ceph-mon[83047]: pgmap v314: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:27.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:27 smithi031 ceph-mon[111404]: pgmap v314: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:29.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:29 smithi040 ceph-mon[83047]: pgmap v315: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:29.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:27:29.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:29 smithi031 ceph-mon[111404]: pgmap v315: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:29.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:27:31.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:31 smithi040 ceph-mon[83047]: pgmap v316: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:31.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:31 smithi031 ceph-mon[111404]: pgmap v316: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:33.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:33 smithi040 ceph-mon[83047]: pgmap v317: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:33.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:33 smithi031 ceph-mon[111404]: pgmap v317: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:35.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:35 smithi040 ceph-mon[83047]: pgmap v318: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:35.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:35 smithi031 ceph-mon[111404]: pgmap v318: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:37.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:37 smithi031 ceph-mon[111404]: pgmap v319: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:37.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:37 smithi040 ceph-mon[83047]: pgmap v319: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:39.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:39 smithi031 ceph-mon[111404]: pgmap v320: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:39.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:39 smithi040 ceph-mon[83047]: pgmap v320: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:41.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:41 smithi031 ceph-mon[111404]: pgmap v321: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:41.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:41 smithi040 ceph-mon[83047]: pgmap v321: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:43.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:43 smithi031 ceph-mon[111404]: pgmap v322: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:43.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:43 smithi040 ceph-mon[83047]: pgmap v322: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:44.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:27:44.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:27:45.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:45 smithi031 ceph-mon[111404]: pgmap v323: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:45.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:45 smithi040 ceph-mon[83047]: pgmap v323: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:47.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:47 smithi031 ceph-mon[111404]: pgmap v324: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:47.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:47 smithi040 ceph-mon[83047]: pgmap v324: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:49.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:49 smithi031 ceph-mon[111404]: pgmap v325: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:49.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:49 smithi040 ceph-mon[83047]: pgmap v325: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:51.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:51 smithi031 ceph-mon[111404]: pgmap v326: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:51.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:51 smithi040 ceph-mon[83047]: pgmap v326: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:53.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:53 smithi031 ceph-mon[111404]: pgmap v327: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:53 smithi040 ceph-mon[83047]: pgmap v327: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:55.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:55 smithi031 ceph-mon[111404]: pgmap v328: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:55.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:55 smithi040 ceph-mon[83047]: pgmap v328: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:57.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:57 smithi031 ceph-mon[111404]: pgmap v329: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:57.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:57 smithi040 ceph-mon[83047]: pgmap v329: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:59 smithi031 ceph-mon[111404]: pgmap v330: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:27:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:27:59.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:59 smithi040 ceph-mon[83047]: pgmap v330: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:27:59.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:27:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:28:01.713 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:01 smithi040 ceph-mon[83047]: pgmap v331: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:01.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:01 smithi031 ceph-mon[111404]: pgmap v331: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:03.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:03 smithi031 ceph-mon[111404]: pgmap v332: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:03.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:03 smithi040 ceph-mon[83047]: pgmap v332: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:05.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:05 smithi031 ceph-mon[111404]: pgmap v333: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:05.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:05 smithi040 ceph-mon[83047]: pgmap v333: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:07.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:07 smithi031 ceph-mon[111404]: pgmap v334: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:07.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:07 smithi040 ceph-mon[83047]: pgmap v334: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:09.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:09 smithi031 ceph-mon[111404]: pgmap v335: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:09.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:09 smithi040 ceph-mon[83047]: pgmap v335: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:11.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:11 smithi031 ceph-mon[111404]: pgmap v336: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:11.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:11 smithi040 ceph-mon[83047]: pgmap v336: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:13.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:13 smithi031 ceph-mon[111404]: pgmap v337: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:13.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:13 smithi040 ceph-mon[83047]: pgmap v337: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:14.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:28:14.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:28:15.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:15 smithi031 ceph-mon[111404]: pgmap v338: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:15.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:15 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:28:15.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:15 smithi040 ceph-mon[83047]: pgmap v338: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:15.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:15 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:28:16.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:28:16.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:28:16.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:28:16.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:28:16.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:28:16.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:28:17.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:17 smithi040 ceph-mon[83047]: pgmap v339: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:17.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:17 smithi031 ceph-mon[111404]: pgmap v339: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:19.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:19 smithi040 ceph-mon[83047]: pgmap v340: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:19.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:19 smithi031 ceph-mon[111404]: pgmap v340: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:21.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:21 smithi040 ceph-mon[83047]: pgmap v341: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:21.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:21 smithi031 ceph-mon[111404]: pgmap v341: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:23.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:23 smithi040 ceph-mon[83047]: pgmap v342: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:23.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:23 smithi031 ceph-mon[111404]: pgmap v342: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:25.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:25 smithi040 ceph-mon[83047]: pgmap v343: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:25.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:25 smithi031 ceph-mon[111404]: pgmap v343: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:27.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:27 smithi040 ceph-mon[83047]: pgmap v344: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:27.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:27 smithi031 ceph-mon[111404]: pgmap v344: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:29.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:29 smithi040 ceph-mon[83047]: pgmap v345: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:29.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:28:29.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:29 smithi031 ceph-mon[111404]: pgmap v345: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:29.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:28:31.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:31 smithi040 ceph-mon[83047]: pgmap v346: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:31.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:31 smithi031 ceph-mon[111404]: pgmap v346: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:33.800 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:33 smithi031 ceph-mon[111404]: pgmap v347: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:33.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:33 smithi040 ceph-mon[83047]: pgmap v347: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:35.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:35 smithi040 ceph-mon[83047]: pgmap v348: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:35.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:35 smithi031 ceph-mon[111404]: pgmap v348: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:37.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:37 smithi040 ceph-mon[83047]: pgmap v349: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:37.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:37 smithi031 ceph-mon[111404]: pgmap v349: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:39.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:39 smithi040 ceph-mon[83047]: pgmap v350: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:39.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:39 smithi031 ceph-mon[111404]: pgmap v350: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:41.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:41 smithi040 ceph-mon[83047]: pgmap v351: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:41.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:41 smithi031 ceph-mon[111404]: pgmap v351: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:43.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:43 smithi040 ceph-mon[83047]: pgmap v352: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:43.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:43 smithi031 ceph-mon[111404]: pgmap v352: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:44.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:28:44.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:28:45.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:45 smithi040 ceph-mon[83047]: pgmap v353: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:45.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:45 smithi031 ceph-mon[111404]: pgmap v353: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:47.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:47 smithi040 ceph-mon[83047]: pgmap v354: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:47.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:47 smithi031 ceph-mon[111404]: pgmap v354: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:49.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:49 smithi040 ceph-mon[83047]: pgmap v355: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:49.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:49 smithi031 ceph-mon[111404]: pgmap v355: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:51.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:51 smithi040 ceph-mon[83047]: pgmap v356: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:51.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:51 smithi031 ceph-mon[111404]: pgmap v356: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:53.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:53 smithi040 ceph-mon[83047]: pgmap v357: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:53.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:53 smithi031 ceph-mon[111404]: pgmap v357: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:55.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:55 smithi040 ceph-mon[83047]: pgmap v358: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:55.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:55 smithi031 ceph-mon[111404]: pgmap v358: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:57.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:57 smithi040 ceph-mon[83047]: pgmap v359: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:57.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:57 smithi031 ceph-mon[111404]: pgmap v359: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:59.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:59 smithi040 ceph-mon[83047]: pgmap v360: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:59.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:28:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:28:59.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:59 smithi031 ceph-mon[111404]: pgmap v360: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:28:59.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:28:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:29:01.903 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:01 smithi040 ceph-mon[83047]: pgmap v361: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:01.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:01 smithi031 ceph-mon[111404]: pgmap v361: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:03.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:03 smithi040 ceph-mon[83047]: pgmap v362: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:03.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:03 smithi031 ceph-mon[111404]: pgmap v362: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:05.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:05 smithi040 ceph-mon[83047]: pgmap v363: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:05.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:05 smithi031 ceph-mon[111404]: pgmap v363: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:07.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:07 smithi040 ceph-mon[83047]: pgmap v364: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:07.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:07 smithi031 ceph-mon[111404]: pgmap v364: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:09.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:09 smithi040 ceph-mon[83047]: pgmap v365: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:09.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:09 smithi031 ceph-mon[111404]: pgmap v365: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:11.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:11 smithi040 ceph-mon[83047]: pgmap v366: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:11.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:11 smithi031 ceph-mon[111404]: pgmap v366: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:13.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:13 smithi040 ceph-mon[83047]: pgmap v367: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:13.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:13 smithi031 ceph-mon[111404]: pgmap v367: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:14.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:29:14.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:29:15.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:15 smithi040 ceph-mon[83047]: pgmap v368: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:15.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:15 smithi031 ceph-mon[111404]: pgmap v368: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:16.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:29:16.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:29:16.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:29:16.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:29:16.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:29:16.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:29:16.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:29:16.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:29:17.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:17 smithi040 ceph-mon[83047]: pgmap v369: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:17.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:17 smithi031 ceph-mon[111404]: pgmap v369: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:19.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:19 smithi040 ceph-mon[83047]: pgmap v370: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:19.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:19 smithi031 ceph-mon[111404]: pgmap v370: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:21.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:21 smithi040 ceph-mon[83047]: pgmap v371: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:21.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:21 smithi031 ceph-mon[111404]: pgmap v371: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:23.914 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:23 smithi040 ceph-mon[83047]: pgmap v372: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:23.999 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:23 smithi031 ceph-mon[111404]: pgmap v372: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:25.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:25 smithi040 ceph-mon[83047]: pgmap v373: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:25.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:25 smithi031 ceph-mon[111404]: pgmap v373: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:27.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:27 smithi040 ceph-mon[83047]: pgmap v374: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:27.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:27 smithi031 ceph-mon[111404]: pgmap v374: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:29.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:29 smithi040 ceph-mon[83047]: pgmap v375: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:29.912 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:29 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:29:29.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:29 smithi031 ceph-mon[111404]: pgmap v375: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:29.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:29 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:29:31.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:31 smithi040 ceph-mon[83047]: pgmap v376: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:31.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:31 smithi031 ceph-mon[111404]: pgmap v376: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:33.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:33 smithi040 ceph-mon[83047]: pgmap v377: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:33.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:33 smithi031 ceph-mon[111404]: pgmap v377: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:35.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:35 smithi040 ceph-mon[83047]: pgmap v378: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:35.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:35 smithi031 ceph-mon[111404]: pgmap v378: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:37.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:37 smithi040 ceph-mon[83047]: pgmap v379: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:37.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:37 smithi031 ceph-mon[111404]: pgmap v379: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:39.911 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:39 smithi040 ceph-mon[83047]: pgmap v380: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:39.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:39 smithi031 ceph-mon[111404]: pgmap v380: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:41.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:41 smithi031 ceph-mon[111404]: pgmap v381: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:42.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:41 smithi040 ceph-mon[83047]: pgmap v381: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:43.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:43 smithi031 ceph-mon[111404]: pgmap v382: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:44.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:43 smithi040 ceph-mon[83047]: pgmap v382: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:44.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:29:45.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:29:45.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:45 smithi031 ceph-mon[111404]: pgmap v383: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:46.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:45 smithi040 ceph-mon[83047]: pgmap v383: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:47.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:47 smithi031 ceph-mon[111404]: pgmap v384: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:48.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:47 smithi040 ceph-mon[83047]: pgmap v384: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:49.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:49 smithi031 ceph-mon[111404]: pgmap v385: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:50.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:49 smithi040 ceph-mon[83047]: pgmap v385: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:51.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:51 smithi031 ceph-mon[111404]: pgmap v386: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:52.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:51 smithi040 ceph-mon[83047]: pgmap v386: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:53.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:53 smithi031 ceph-mon[111404]: pgmap v387: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:54.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:53 smithi040 ceph-mon[83047]: pgmap v387: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:55.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:55 smithi031 ceph-mon[111404]: pgmap v388: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:56.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:55 smithi040 ceph-mon[83047]: pgmap v388: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:57.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:57 smithi031 ceph-mon[111404]: pgmap v389: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:58.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:57 smithi040 ceph-mon[83047]: pgmap v389: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:59.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:59 smithi031 ceph-mon[111404]: pgmap v390: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:29:59.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:29:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:00.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:59 smithi040 ceph-mon[83047]: pgmap v390: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:00.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:29:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:00.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:00 smithi031 ceph-mon[111404]: overall HEALTH_OK 2024-04-24T23:30:01.062 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:00 smithi040 ceph-mon[83047]: overall HEALTH_OK 2024-04-24T23:30:01.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:01 smithi031 ceph-mon[111404]: pgmap v391: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:02.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:01 smithi040 ceph-mon[83047]: pgmap v391: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:03.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:03 smithi031 ceph-mon[111404]: pgmap v392: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:04.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:03 smithi040 ceph-mon[83047]: pgmap v392: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:05.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:05 smithi031 ceph-mon[111404]: pgmap v393: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:06.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:05 smithi040 ceph-mon[83047]: pgmap v393: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:07.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:07 smithi031 ceph-mon[111404]: pgmap v394: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:08.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:07 smithi040 ceph-mon[83047]: pgmap v394: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:09.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:09 smithi031 ceph-mon[111404]: pgmap v395: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:10.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:09 smithi040 ceph-mon[83047]: pgmap v395: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:11.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:11 smithi031 ceph-mon[111404]: pgmap v396: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:12.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:11 smithi040 ceph-mon[83047]: pgmap v396: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:13.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:13 smithi031 ceph-mon[111404]: pgmap v397: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:14.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:13 smithi040 ceph-mon[83047]: pgmap v397: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:14.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:14 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:15.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:14 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:15.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:15 smithi031 ceph-mon[111404]: pgmap v398: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:16.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:15 smithi040 ceph-mon[83047]: pgmap v398: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:16.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:16 smithi031 ceph-mon[111404]: pgmap v399: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:16.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:16 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:30:17.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:16 smithi040 ceph-mon[83047]: pgmap v399: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:17.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:16 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-24T23:30:17.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:17 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:30:17.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:17 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:30:17.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:17 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:30:18.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:17 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-24T23:30:18.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:17 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-24T23:30:18.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:17 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' 2024-04-24T23:30:18.997 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:18 smithi031 ceph-mon[111404]: pgmap v400: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:19.161 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:18 smithi040 ceph-mon[83047]: pgmap v400: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:21.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:20 smithi031 ceph-mon[111404]: pgmap v401: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:21.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:20 smithi040 ceph-mon[83047]: pgmap v401: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:23.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:22 smithi031 ceph-mon[111404]: pgmap v402: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:23.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:22 smithi040 ceph-mon[83047]: pgmap v402: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:25.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:24 smithi031 ceph-mon[111404]: pgmap v403: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:25.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:24 smithi040 ceph-mon[83047]: pgmap v403: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:27.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:26 smithi031 ceph-mon[111404]: pgmap v404: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:27.413 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:26 smithi040 ceph-mon[83047]: pgmap v404: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:29.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:28 smithi031 ceph-mon[111404]: pgmap v405: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:29.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:28 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:29.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:28 smithi040 ceph-mon[83047]: pgmap v405: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:29.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:28 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:31.214 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:30 smithi040 ceph-mon[83047]: pgmap v406: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:32.006 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:30 smithi031 ceph-mon[111404]: pgmap v406: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:33.248 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:32 smithi031 ceph-mon[111404]: pgmap v407: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:33.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:32 smithi040 ceph-mon[83047]: pgmap v407: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:35.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:34 smithi031 ceph-mon[111404]: pgmap v408: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:35.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:34 smithi040 ceph-mon[83047]: pgmap v408: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:37.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:36 smithi031 ceph-mon[111404]: pgmap v409: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:37.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:36 smithi040 ceph-mon[83047]: pgmap v409: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:39.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:38 smithi031 ceph-mon[111404]: pgmap v410: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:39.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:38 smithi040 ceph-mon[83047]: pgmap v410: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:41.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:41 smithi040 ceph-mon[83047]: pgmap v411: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:41.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:41 smithi031 ceph-mon[111404]: pgmap v411: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:43.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:43 smithi040 ceph-mon[83047]: pgmap v412: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:43.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:43 smithi031 ceph-mon[111404]: pgmap v412: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:44.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:44 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:44.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:44 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:45.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:45 smithi040 ceph-mon[83047]: pgmap v413: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:45.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:45 smithi031 ceph-mon[111404]: pgmap v413: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:47.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:47 smithi040 ceph-mon[83047]: pgmap v414: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:47.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:47 smithi031 ceph-mon[111404]: pgmap v414: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:49.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:49 smithi040 ceph-mon[83047]: pgmap v415: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:49.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:49 smithi031 ceph-mon[111404]: pgmap v415: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:51.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:51 smithi040 ceph-mon[83047]: pgmap v416: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:51.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:51 smithi031 ceph-mon[111404]: pgmap v416: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:53.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:53 smithi040 ceph-mon[83047]: pgmap v417: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:53.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:53 smithi031 ceph-mon[111404]: pgmap v417: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:55.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:55 smithi040 ceph-mon[83047]: pgmap v418: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:55.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:55 smithi031 ceph-mon[111404]: pgmap v418: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:57.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:57 smithi040 ceph-mon[83047]: pgmap v419: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:57.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:57 smithi031 ceph-mon[111404]: pgmap v419: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:59.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:59 smithi040 ceph-mon[83047]: pgmap v420: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:59.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:30:59 smithi040 ceph-mon[83047]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:30:59.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:59 smithi031 ceph-mon[111404]: pgmap v420: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:30:59.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:30:59 smithi031 ceph-mon[111404]: from='mgr.34102 172.21.15.31:0/1518920315' entity='mgr.smithi031.rrnmwr' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-24T23:31:01.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:01 smithi040 ceph-mon[83047]: pgmap v421: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:01.748 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:01 smithi031 ceph-mon[111404]: pgmap v421: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:03.550 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:03 smithi031 ceph-mon[111404]: pgmap v422: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:03.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:03 smithi040 ceph-mon[83047]: pgmap v422: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:05.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:05 smithi040 ceph-mon[83047]: pgmap v423: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:05.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:05 smithi031 ceph-mon[111404]: pgmap v423: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:07.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:07 smithi040 ceph-mon[83047]: pgmap v424: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:07.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:07 smithi031 ceph-mon[111404]: pgmap v424: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:09.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:09 smithi040 ceph-mon[83047]: pgmap v425: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:09.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:09 smithi031 ceph-mon[111404]: pgmap v425: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:11.661 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:11 smithi040 ceph-mon[83047]: pgmap v426: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:11.747 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:11 smithi031 ceph-mon[111404]: pgmap v426: 65 pgs: 65 active+clean; 205 MiB data, 847 MiB used, 536 GiB / 536 GiB avail 2024-04-24T23:31:12.270 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-24T23:31:12.271 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T23:31:12.272 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-24T23:31:12.286 INFO:tasks.cephadm:Teardown begin 2024-04-24T23:31:12.286 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T23:31:12.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:31:12.321 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:31:12.357 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-24T23:31:12.357 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:31:12.381 DEBUG:teuthology.orchestra.run.smithi040:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-24T23:31:12.423 INFO:tasks.cephadm:Stopping all daemons... 2024-04-24T23:31:12.423 INFO:tasks.cephadm.mon.smithi031:Stopping mon.smithi031... 2024-04-24T23:31:12.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031 2024-04-24T23:31:12.745 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:12 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:12.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:12 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[111398]: 2024-04-24T23:31:12.744+0000 7fda7fb6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:12.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:12 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031[111398]: 2024-04-24T23:31:12.744+0000 7fda7fb6d700 -1 mon.smithi031@0(leader) e3 *** Got Signal Terminated *** 2024-04-24T23:31:12.998 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:12 smithi031 podman[152812]: 2024-04-24 23:31:12.785493596 +0000 UTC m=+0.156766784 container died 78d60c8cc2f3290b86d7685b62db4330ab347b0772a816dbd29d8a60c6167018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:31:13.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:13 smithi031 podman[152812]: 2024-04-24 23:31:13.37807234 +0000 UTC m=+0.749345535 container cleanup 78d60c8cc2f3290b86d7685b62db4330ab347b0772a816dbd29d8a60c6167018 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:31:13.499 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Apr 24 23:31:13 smithi031 bash[152812]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi031 2024-04-24T23:31:13.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi031.service' 2024-04-24T23:31:13.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:13.812 INFO:tasks.cephadm.mon.smithi031:Stopped mon.smithi031 2024-04-24T23:31:13.812 INFO:tasks.cephadm.mon.smithi040:Stopping mon.smithi040... 2024-04-24T23:31:13.812 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi040 2024-04-24T23:31:14.111 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:13 smithi040 systemd[1]: Stopping Ceph mon.smithi040 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:14.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:14 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040[83043]: 2024-04-24T23:31:14.111+0000 7f96504ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi040 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:14.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:14 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040[83043]: 2024-04-24T23:31:14.111+0000 7f96504ff700 -1 mon.smithi040@1(peon) e3 *** Got Signal Terminated *** 2024-04-24T23:31:14.411 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:14 smithi040 podman[115645]: 2024-04-24 23:31:14.121429158 +0000 UTC m=+0.106285606 container died 62515ef0bf5349fd1f895bb8ef3e442b05242ccc9057114432e7d03df3e66080 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-24T23:31:14.854 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:14 smithi040 podman[115645]: 2024-04-24 23:31:14.603154471 +0000 UTC m=+0.588010914 container cleanup 62515ef0bf5349fd1f895bb8ef3e442b05242ccc9057114432e7d03df3e66080 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-24T23:31:14.854 INFO:journalctl@ceph.mon.smithi040.smithi040.stdout:Apr 24 23:31:14 smithi040 bash[115645]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-mon-smithi040 2024-04-24T23:31:14.983 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@mon.smithi040.service' 2024-04-24T23:31:15.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:15.028 INFO:tasks.cephadm.mon.smithi040:Stopped mon.smithi040 2024-04-24T23:31:15.028 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-24T23:31:15.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.0 2024-04-24T23:31:15.349 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:15 smithi031 systemd[1]: Stopping Ceph osd.0 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:15.748 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:15 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[123202]: 2024-04-24T23:31:15.348+0000 7f0106712700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:15.748 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:15 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[123202]: 2024-04-24T23:31:15.348+0000 7f0106712700 -1 osd.0 84 *** Got signal Terminated *** 2024-04-24T23:31:15.748 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:15 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0[123202]: 2024-04-24T23:31:15.348+0000 7f0106712700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:31:21.248 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:20 smithi031 podman[152946]: 2024-04-24 23:31:20.863051974 +0000 UTC m=+5.629567867 container died d216357cdbb6e7e1b1463f83376b7d6e5fd8abdca3116e23b190bb462512cc62 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:31:21.693 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:21 smithi031 podman[152946]: 2024-04-24 23:31:21.366210526 +0000 UTC m=+6.132726427 container cleanup d216357cdbb6e7e1b1463f83376b7d6e5fd8abdca3116e23b190bb462512cc62 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-24T23:31:21.693 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:21 smithi031 bash[152946]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0 2024-04-24T23:31:21.997 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:21 smithi031 podman[152963]: 2024-04-24 23:31:21.693187321 +0000 UTC m=+0.828970115 container remove d216357cdbb6e7e1b1463f83376b7d6e5fd8abdca3116e23b190bb462512cc62 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:31:22.288 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:22 smithi031 podman[153055]: 2024-04-24 23:31:21.904836953 +0000 UTC m=+0.018997984 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:31:22.288 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:22 smithi031 podman[153055]: 2024-04-24 23:31:22.027748582 +0000 UTC m=+0.141909645 container create 5a129c1ce6b81886fd8f33d119d9f699e3d925d0a56807b4e0249ac1fe69f85d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True) 2024-04-24T23:31:22.288 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:22 smithi031 podman[153055]: 2024-04-24 23:31:22.288877382 +0000 UTC m=+0.403038401 container init 5a129c1ce6b81886fd8f33d119d9f699e3d925d0a56807b4e0249ac1fe69f85d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-24T23:31:22.601 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:22 smithi031 podman[153055]: 2024-04-24 23:31:22.292383671 +0000 UTC m=+0.406544699 container start 5a129c1ce6b81886fd8f33d119d9f699e3d925d0a56807b4e0249ac1fe69f85d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:31:22.601 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:22 smithi031 podman[153055]: 2024-04-24 23:31:22.355691539 +0000 UTC m=+0.469852572 container attach 5a129c1ce6b81886fd8f33d119d9f699e3d925d0a56807b4e0249ac1fe69f85d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:31:22.967 INFO:journalctl@ceph.osd.0.smithi031.stdout:Apr 24 23:31:22 smithi031 podman[153055]: 2024-04-24 23:31:22.601891556 +0000 UTC m=+0.716052584 container died 5a129c1ce6b81886fd8f33d119d9f699e3d925d0a56807b4e0249ac1fe69f85d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-24T23:31:23.274 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.0.service' 2024-04-24T23:31:23.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:23.319 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-24T23:31:23.319 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-24T23:31:23.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.1 2024-04-24T23:31:23.549 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:23 smithi031 systemd[1]: Stopping Ceph osd.1 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:23.997 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:23 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[129051]: 2024-04-24T23:31:23.670+0000 7f5fd2166700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:23.997 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:23 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[129051]: 2024-04-24T23:31:23.670+0000 7f5fd2166700 -1 osd.1 84 *** Got signal Terminated *** 2024-04-24T23:31:23.998 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:23 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1[129051]: 2024-04-24T23:31:23.670+0000 7f5fd2166700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:31:29.498 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:29 smithi031 podman[153161]: 2024-04-24 23:31:29.217170017 +0000 UTC m=+5.660105860 container died ad59872a4b025b23a43d958a5d3efd7ef4ffb48c80b357271487d3602b7279e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-24T23:31:29.998 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:29 smithi031 podman[153161]: 2024-04-24 23:31:29.537648882 +0000 UTC m=+5.980584731 container cleanup ad59872a4b025b23a43d958a5d3efd7ef4ffb48c80b357271487d3602b7279e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-24T23:31:29.998 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:29 smithi031 bash[153161]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1 2024-04-24T23:31:29.998 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:29 smithi031 podman[153189]: 2024-04-24 23:31:29.739191365 +0000 UTC m=+0.516212906 container remove ad59872a4b025b23a43d958a5d3efd7ef4ffb48c80b357271487d3602b7279e0 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-24T23:31:30.471 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:30 smithi031 podman[153280]: 2024-04-24 23:31:30.078466191 +0000 UTC m=+0.018357890 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:31:30.471 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:30 smithi031 podman[153280]: 2024-04-24 23:31:30.19021456 +0000 UTC m=+0.130106231 container create 4961b5e1ca2300316513e3d859d07557f1378895b628655be0f18c551ba89b68 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-24T23:31:30.471 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:30 smithi031 podman[153280]: 2024-04-24 23:31:30.468017135 +0000 UTC m=+0.407908815 container init 4961b5e1ca2300316513e3d859d07557f1378895b628655be0f18c551ba89b68 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-24T23:31:30.748 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:30 smithi031 podman[153280]: 2024-04-24 23:31:30.471273588 +0000 UTC m=+0.411165272 container start 4961b5e1ca2300316513e3d859d07557f1378895b628655be0f18c551ba89b68 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-24T23:31:30.748 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:30 smithi031 podman[153280]: 2024-04-24 23:31:30.543062445 +0000 UTC m=+0.482954131 container attach 4961b5e1ca2300316513e3d859d07557f1378895b628655be0f18c551ba89b68 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-24T23:31:31.063 INFO:journalctl@ceph.osd.1.smithi031.stdout:Apr 24 23:31:30 smithi031 podman[153280]: 2024-04-24 23:31:30.782883276 +0000 UTC m=+0.722774959 container died 4961b5e1ca2300316513e3d859d07557f1378895b628655be0f18c551ba89b68 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD) 2024-04-24T23:31:31.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.1.service' 2024-04-24T23:31:31.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:31.376 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-24T23:31:31.376 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-24T23:31:31.376 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.2 2024-04-24T23:31:31.708 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:31 smithi031 systemd[1]: Stopping Ceph osd.2 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:31.997 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:31 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[134890]: 2024-04-24T23:31:31.708+0000 7fc93f5ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:31.998 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:31 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[134890]: 2024-04-24T23:31:31.708+0000 7fc93f5ae700 -1 osd.2 84 *** Got signal Terminated *** 2024-04-24T23:31:31.998 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:31 smithi031 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2[134890]: 2024-04-24T23:31:31.708+0000 7fc93f5ae700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:31:37.352 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:37 smithi031 podman[153386]: 2024-04-24 23:31:37.084056008 +0000 UTC m=+5.480887529 container died 271a0764ef5cdd1e84a0026a687ce1005370868875dc8225bdd31748ae201d87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-24T23:31:37.717 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:37 smithi031 podman[153386]: 2024-04-24 23:31:37.449971954 +0000 UTC m=+5.846803481 container cleanup 271a0764ef5cdd1e84a0026a687ce1005370868875dc8225bdd31748ae201d87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-24T23:31:37.717 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:37 smithi031 bash[153386]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2 2024-04-24T23:31:37.997 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:37 smithi031 podman[153408]: 2024-04-24 23:31:37.717981178 +0000 UTC m=+0.628662367 container remove 271a0764ef5cdd1e84a0026a687ce1005370868875dc8225bdd31748ae201d87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T23:31:38.338 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:38 smithi031 podman[153498]: 2024-04-24 23:31:37.924642503 +0000 UTC m=+0.018602282 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:31:38.338 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:38 smithi031 podman[153498]: 2024-04-24 23:31:38.044202811 +0000 UTC m=+0.138162565 container create adc94697b5ea28a8237da373b084b7078286bd78fd68c99cd5869df75dd0a18d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:31:38.338 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:38 smithi031 podman[153498]: 2024-04-24 23:31:38.255699134 +0000 UTC m=+0.349658906 container init adc94697b5ea28a8237da373b084b7078286bd78fd68c99cd5869df75dd0a18d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:31:38.339 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:38 smithi031 podman[153498]: 2024-04-24 23:31:38.259589853 +0000 UTC m=+0.353549616 container start adc94697b5ea28a8237da373b084b7078286bd78fd68c99cd5869df75dd0a18d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-24T23:31:38.339 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:38 smithi031 podman[153498]: 2024-04-24 23:31:38.338258136 +0000 UTC m=+0.432217903 container attach adc94697b5ea28a8237da373b084b7078286bd78fd68c99cd5869df75dd0a18d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-24T23:31:38.748 INFO:journalctl@ceph.osd.2.smithi031.stdout:Apr 24 23:31:38 smithi031 podman[153498]: 2024-04-24 23:31:38.557545007 +0000 UTC m=+0.651504762 container died adc94697b5ea28a8237da373b084b7078286bd78fd68c99cd5869df75dd0a18d (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-24T23:31:39.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.2.service' 2024-04-24T23:31:39.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:39.206 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-24T23:31:39.206 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-24T23:31:39.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.3 2024-04-24T23:31:39.486 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:39 smithi040 systemd[1]: Stopping Ceph osd.3 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:39.751 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:39 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[92426]: 2024-04-24T23:31:39.623+0000 7fb002fe3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:39.751 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:39 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[92426]: 2024-04-24T23:31:39.623+0000 7fb002fe3700 -1 osd.3 84 *** Got signal Terminated *** 2024-04-24T23:31:39.751 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:39 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[92426]: 2024-04-24T23:31:39.623+0000 7fb002fe3700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:31:43.661 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:43 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:43.231+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:44.522 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:44 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3[92426]: 2024-04-24T23:31:44.291+0000 7faffbdd0700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:20.166797+0000 front 2024-04-24T23:31:20.166771+0000 (oldest deadline 2024-04-24T23:31:44.167028+0000) 2024-04-24T23:31:44.523 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:44 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:44.230+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:44.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:44 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:44.522+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:45.411 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:45 smithi040 podman[115766]: 2024-04-24 23:31:45.150989667 +0000 UTC m=+5.749474053 container died 97c6f05632f661a4a5d5f82372778f5cb7ece81cc42c5a8d0b826026bf930d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-24T23:31:45.411 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:45 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:45.195+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:45.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:45 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:45.528+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:45.911 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:45 smithi040 podman[115766]: 2024-04-24 23:31:45.665303183 +0000 UTC m=+6.263787568 container cleanup 97c6f05632f661a4a5d5f82372778f5cb7ece81cc42c5a8d0b826026bf930d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True) 2024-04-24T23:31:45.911 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:45 smithi040 bash[115766]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3 2024-04-24T23:31:46.294 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:46 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:46.159+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:46.295 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:45 smithi040 podman[115796]: 2024-04-24 23:31:45.941678681 +0000 UTC m=+0.781676821 container remove 97c6f05632f661a4a5d5f82372778f5cb7ece81cc42c5a8d0b826026bf930d9f (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-24T23:31:46.553 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:46 smithi040 podman[115885]: 2024-04-24 23:31:46.197924991 +0000 UTC m=+0.018048746 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:31:46.553 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:46 smithi040 podman[115885]: 2024-04-24 23:31:46.309284729 +0000 UTC m=+0.129408512 container create dbf5b8827f7f215a59644cb59cd65e11afeda2d62861c7d23aa39fd7f60e139e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-24T23:31:46.860 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:46 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:46.568+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:46.860 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:46 smithi040 podman[115885]: 2024-04-24 23:31:46.554036825 +0000 UTC m=+0.374160576 container init dbf5b8827f7f215a59644cb59cd65e11afeda2d62861c7d23aa39fd7f60e139e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-24T23:31:46.860 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:46 smithi040 podman[115885]: 2024-04-24 23:31:46.557943271 +0000 UTC m=+0.378067015 container start dbf5b8827f7f215a59644cb59cd65e11afeda2d62861c7d23aa39fd7f60e139e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-24T23:31:46.860 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:46 smithi040 podman[115885]: 2024-04-24 23:31:46.670555071 +0000 UTC m=+0.490678812 container attach dbf5b8827f7f215a59644cb59cd65e11afeda2d62861c7d23aa39fd7f60e139e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-24T23:31:47.161 INFO:journalctl@ceph.osd.3.smithi040.stdout:Apr 24 23:31:46 smithi040 podman[115885]: 2024-04-24 23:31:46.860763713 +0000 UTC m=+0.680887462 container died dbf5b8827f7f215a59644cb59cd65e11afeda2d62861c7d23aa39fd7f60e139e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-24T23:31:47.552 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:47 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:47.202+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:47.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.3.service' 2024-04-24T23:31:47.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:47.690 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-24T23:31:47.690 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-24T23:31:47.690 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.4 2024-04-24T23:31:47.911 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:47 smithi040 systemd[1]: Stopping Ceph osd.4 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:47.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:47 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:47.551+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:48.411 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:48 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:48.018+0000 7f82b1180700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:48.411 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:48 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:48.018+0000 7f82b1180700 -1 osd.4 84 *** Got signal Terminated *** 2024-04-24T23:31:48.411 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:48 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:48.018+0000 7f82b1180700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:31:48.411 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:48 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:48.193+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:48.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:48 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:48.504+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:49.496 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:49 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:49.225+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:49.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:49 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:49.496+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:50.544 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:50 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:50.232+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:50.794 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:50 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:50.543+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:50.795 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:50 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:50.543+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:51.557 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:51 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:51.204+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:51.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:51 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:51.555+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:51.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:51 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:51.555+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:52.539 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:52 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4[97717]: 2024-04-24T23:31:52.215+0000 7f82adf75700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.168947+0000 front 2024-04-24T23:31:17.168876+0000 (oldest deadline 2024-04-24T23:31:43.068816+0000) 2024-04-24T23:31:52.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:52 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:52.537+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:52.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:52 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:52.537+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:53.661 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:53 smithi040 podman[115991]: 2024-04-24 23:31:53.398752693 +0000 UTC m=+5.480847786 container died 85d9bab7b352c0753c8453792a8db8e1aee84756a7d60bd65af779f108d58c9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-24T23:31:53.661 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:53 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:53.564+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:53.661 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:53 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:53.564+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:54.036 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:53 smithi040 podman[115991]: 2024-04-24 23:31:53.752441661 +0000 UTC m=+5.834536794 container cleanup 85d9bab7b352c0753c8453792a8db8e1aee84756a7d60bd65af779f108d58c9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-24T23:31:54.036 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:53 smithi040 bash[115991]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4 2024-04-24T23:31:54.385 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:54 smithi040 podman[116009]: 2024-04-24 23:31:54.037248698 +0000 UTC m=+0.636211375 container remove 85d9bab7b352c0753c8453792a8db8e1aee84756a7d60bd65af779f108d58c9b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-24T23:31:54.661 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:54 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:54.523+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:54.661 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:54 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:54.523+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:54.662 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:54 smithi040 podman[116103]: 2024-04-24 23:31:54.288435451 +0000 UTC m=+0.018140418 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:31:54.662 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:54 smithi040 podman[116103]: 2024-04-24 23:31:54.39653652 +0000 UTC m=+0.126241463 container create 667ecd136cde8b6210cf3dd3988858a9bb83da80dc136e76a353444a8a4ff7ce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-24T23:31:54.967 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:54 smithi040 podman[116103]: 2024-04-24 23:31:54.665926994 +0000 UTC m=+0.395631941 container init 667ecd136cde8b6210cf3dd3988858a9bb83da80dc136e76a353444a8a4ff7ce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-24T23:31:54.967 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:54 smithi040 podman[116103]: 2024-04-24 23:31:54.669759398 +0000 UTC m=+0.399464350 container start 667ecd136cde8b6210cf3dd3988858a9bb83da80dc136e76a353444a8a4ff7ce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-24T23:31:54.967 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:54 smithi040 podman[116103]: 2024-04-24 23:31:54.765936799 +0000 UTC m=+0.495641741 container attach 667ecd136cde8b6210cf3dd3988858a9bb83da80dc136e76a353444a8a4ff7ce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-24T23:31:55.260 INFO:journalctl@ceph.osd.4.smithi040.stdout:Apr 24 23:31:54 smithi040 podman[116103]: 2024-04-24 23:31:54.967837319 +0000 UTC m=+0.697542264 container died 667ecd136cde8b6210cf3dd3988858a9bb83da80dc136e76a353444a8a4ff7ce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True) 2024-04-24T23:31:55.544 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:55.481+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:55.544 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:55.481+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:55.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.4.service' 2024-04-24T23:31:55.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:31:55.617 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-24T23:31:55.617 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-24T23:31:55.617 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.5 2024-04-24T23:31:55.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:55 smithi040 systemd[1]: Stopping Ceph osd.5 for ca7b26e4-028e-11ef-bc93-c7b262605968... 2024-04-24T23:31:56.411 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:55.939+0000 7f8adb2dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-24T23:31:56.411 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:55.939+0000 7f8adb2dc700 -1 osd.5 84 *** Got signal Terminated *** 2024-04-24T23:31:56.411 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:55 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:55.939+0000 7f8adb2dc700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-24T23:31:56.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:56 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:56.523+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:56.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:56 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:56.523+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:57.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:57 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:57.505+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:57.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:57 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:57.505+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:58.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:58 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:58.470+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:58.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:58 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:58.470+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:58.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:58 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:58.470+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6822 osd.2 since back 2024-04-24T23:31:36.160837+0000 front 2024-04-24T23:31:36.160819+0000 (oldest deadline 2024-04-24T23:31:57.860671+0000) 2024-04-24T23:31:59.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:59 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:59.520+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:31:59.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:59 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:59.520+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:31:59.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:31:59 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:31:59.520+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6822 osd.2 since back 2024-04-24T23:31:36.160837+0000 front 2024-04-24T23:31:36.160819+0000 (oldest deadline 2024-04-24T23:31:57.860671+0000) 2024-04-24T23:32:00.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:32:00.490+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-04-24T23:31:17.959032+0000 front 2024-04-24T23:31:17.958946+0000 (oldest deadline 2024-04-24T23:31:43.858844+0000) 2024-04-24T23:32:00.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:32:00.490+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-04-24T23:31:23.859320+0000 front 2024-04-24T23:31:23.859451+0000 (oldest deadline 2024-04-24T23:31:49.759477+0000) 2024-04-24T23:32:00.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:00 smithi040 ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5[102573]: 2024-04-24T23:32:00.490+0000 7f8ad80d1700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.31:6822 osd.2 since back 2024-04-24T23:31:36.160837+0000 front 2024-04-24T23:31:36.160819+0000 (oldest deadline 2024-04-24T23:31:57.860671+0000) 2024-04-24T23:32:01.598 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:01 smithi040 podman[116210]: 2024-04-24 23:32:01.334584179 +0000 UTC m=+5.519968806 container died 32b6ff45fea8ba1cd0783c226a0ce54feb2639ccfd0691c6b6a14677e867b6ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-04-24T23:32:01.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:01 smithi040 podman[116210]: 2024-04-24 23:32:01.705960639 +0000 UTC m=+5.891345261 container cleanup 32b6ff45fea8ba1cd0783c226a0ce54feb2639ccfd0691c6b6a14677e867b6ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-24T23:32:01.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:01 smithi040 bash[116210]: ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5 2024-04-24T23:32:02.327 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:01 smithi040 podman[116228]: 2024-04-24 23:32:01.982141396 +0000 UTC m=+0.644891413 container remove 32b6ff45fea8ba1cd0783c226a0ce54feb2639ccfd0691c6b6a14677e867b6ba (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-24T23:32:02.643 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:02 smithi040 podman[116318]: 2024-04-24 23:32:02.231300796 +0000 UTC m=+0.018796979 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-24T23:32:02.643 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:02 smithi040 podman[116318]: 2024-04-24 23:32:02.332860345 +0000 UTC m=+0.120356521 container create 57dde468c02f25a17950b06cee3cac21af68516651001cfde03b00df4408cf7e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-24T23:32:02.643 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:02 smithi040 podman[116318]: 2024-04-24 23:32:02.543689719 +0000 UTC m=+0.331185888 container init 57dde468c02f25a17950b06cee3cac21af68516651001cfde03b00df4408cf7e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-24T23:32:02.643 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:02 smithi040 podman[116318]: 2024-04-24 23:32:02.546950021 +0000 UTC m=+0.334446198 container start 57dde468c02f25a17950b06cee3cac21af68516651001cfde03b00df4408cf7e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-24T23:32:02.643 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:02 smithi040 podman[116318]: 2024-04-24 23:32:02.643636559 +0000 UTC m=+0.431132730 container attach 57dde468c02f25a17950b06cee3cac21af68516651001cfde03b00df4408cf7e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-24T23:32:02.911 INFO:journalctl@ceph.osd.5.smithi040.stdout:Apr 24 23:32:02 smithi040 podman[116318]: 2024-04-24 23:32:02.845136772 +0000 UTC m=+0.632632933 container died 57dde468c02f25a17950b06cee3cac21af68516651001cfde03b00df4408cf7e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-ca7b26e4-028e-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-24T23:32:03.430 DEBUG:teuthology.orchestra.run.smithi040:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ca7b26e4-028e-11ef-bc93-c7b262605968@osd.5.service' 2024-04-24T23:32:03.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-24T23:32:03.471 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-24T23:32:03.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:32:06.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi031.stderr:ceph-fuse[73971]: fuse finished with error 0 and tester_r 0 2024-04-24T23:32:24.247 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-24T23:32:36.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:32:36.099 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-24T23:32:36.126 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-24T23:32:36.129 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179/remote/smithi031/crash 2024-04-24T23:32:36.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:32:36.167 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:32:36.167 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:32:36.169 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179/remote/smithi040/crash 2024-04-24T23:32:36.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/crash -- . 2024-04-24T23:32:36.193 INFO:teuthology.orchestra.run.smithi040.stderr:tar: /var/lib/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-24T23:32:36.193 INFO:teuthology.orchestra.run.smithi040.stderr:tar: Error is not recoverable: exiting now 2024-04-24T23:32:36.194 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-24T23:32:36.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-24T23:32:36.248 INFO:teuthology.orchestra.run.smithi031.stdout:1714000024.4947894 mon.smithi031 (mon.0) 339 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-24T23:32:36.248 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-24T23:32:36.248 INFO:tasks.cephadm:Compressing logs... 2024-04-24T23:32:36.248 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:32:36.292 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T23:32:36.317 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-04-24T23:32:36.318 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph-client.0.log 2024-04-24T23:32:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mgr.smithi031.rrnmwr.log 2024-04-24T23:32:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:32:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mgr.smithi031.rrnmwr.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-24T23:32:36.318 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mon.smithi031.log 2024-04-24T23:32:36.319 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:32:36.328 INFO:teuthology.orchestra.run.smithi040.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log: No such file or directory 2024-04-24T23:32:36.328 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.log 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi040.osunbp.log 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi040.osunbp.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi040.dicevu.log 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mon.smithi040.log 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi040.dicevu.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mon.smithi040.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-24T23:32:36.329 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.3.log: 87.9% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:32:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:32:36.330 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.4.log: 91.2% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:32:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mon.smithi031.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-24T23:32:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.1.log: 87.9% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-24T23:32:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-24T23:32:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.acqiox.log 2024-04-24T23:32:36.330 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.0.log: 91.0% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-24T23:32:36.331 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-24T23:32:36.332 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:32:36.332 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:32:36.334 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mgr.smithi040.qtdwuq.log 2024-04-24T23:32:36.338 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.acqiox.log: gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-24T23:32:36.339 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-24T23:32:36.347 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:32:36.352 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.xxejxr.log 2024-04-24T23:32:36.367 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-24T23:32:36.368 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mgr.smithi040.qtdwuq.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:32:36.376 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:32:36.400 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.xxejxr.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-24T23:32:36.436 INFO:teuthology.orchestra.run.smithi040.stderr: 89.8% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mgr.smithi040.qtdwuq.log.gz 2024-04-24T23:32:36.445 INFO:teuthology.orchestra.run.smithi031.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-24T23:32:36.734 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mgr.smithi031.rrnmwr.log.gz 2024-04-24T23:32:36.801 INFO:teuthology.orchestra.run.smithi040.stderr: 92.5% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mon.smithi040.log.gz 2024-04-24T23:32:37.622 INFO:teuthology.orchestra.run.smithi031.stderr: 90.5% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mon.smithi031.log.gz 2024-04-24T23:32:41.181 INFO:teuthology.orchestra.run.smithi031.stderr: 93.4% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-24T23:32:41.825 INFO:teuthology.orchestra.run.smithi040.stderr: 93.5% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-24T23:32:42.472 INFO:teuthology.orchestra.run.smithi040.stderr: 94.7% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi040.osunbp.log.gz 2024-04-24T23:32:42.708 INFO:teuthology.orchestra.run.smithi031.stderr: 93.4% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-24T23:32:42.721 INFO:teuthology.orchestra.run.smithi040.stderr: 93.3% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-24T23:32:43.384 INFO:teuthology.orchestra.run.smithi031.stderr: 93.5% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-24T23:32:44.080 INFO:teuthology.orchestra.run.smithi031.stderr: 94.7% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.xxejxr.log.gz 2024-04-24T23:32:44.186 INFO:teuthology.orchestra.run.smithi040.stderr: 94.7% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi040.dicevu.log.gz 2024-04-24T23:32:44.769 INFO:teuthology.orchestra.run.smithi040.stderr: 93.5% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-24T23:32:52.796 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-24T23:32:52.796 INFO:teuthology.orchestra.run.smithi040.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-24T23:32:52.797 INFO:teuthology.orchestra.run.smithi040.stderr: 2024-04-24T23:32:52.798 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m16.491s 2024-04-24T23:32:52.798 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m50.085s 2024-04-24T23:32:52.798 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m1.291s 2024-04-24T23:32:53.088 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/ceph-client.0.log: file size changed while zipping 2024-04-24T23:32:53.282 INFO:teuthology.orchestra.run.smithi031.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-24T23:34:09.784 INFO:teuthology.orchestra.run.smithi031.stderr: 92.7% -- replaced with /var/log/ceph/ca7b26e4-028e-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi031.acqiox.log.gz 2024-04-24T23:34:09.785 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-04-24T23:34:09.786 INFO:teuthology.orchestra.run.smithi031.stderr:real 1m33.481s 2024-04-24T23:34:09.786 INFO:teuthology.orchestra.run.smithi031.stderr:user 2m13.678s 2024-04-24T23:34:09.786 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m3.807s 2024-04-24T23:34:09.786 INFO:tasks.cephadm:Archiving logs... 2024-04-24T23:34:09.786 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179/remote/smithi031/log 2024-04-24T23:34:09.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:34:29.230 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179/remote/smithi040/log 2024-04-24T23:34:29.231 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T23:34:35.255 INFO:tasks.cephadm:Removing cluster... 2024-04-24T23:34:35.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 --force 2024-04-24T23:34:35.897 DEBUG:teuthology.orchestra.run.smithi040:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ca7b26e4-028e-11ef-bc93-c7b262605968 --force 2024-04-24T23:34:36.417 INFO:tasks.cephadm:Removing cephadm ... 2024-04-24T23:34:36.418 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:34:36.433 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-24T23:34:36.449 INFO:tasks.cephadm:Teardown complete 2024-04-24T23:34:36.450 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T23:34:36.461 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T23:34:36.461 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T23:34:36.461 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:34:36.476 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T23:34:36.559 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T23:34:36.570 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T23:34:36.570 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:34:36.573 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T23:34:36.585 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:34:36.587 INFO:teuthology.orchestra.run.smithi040.stderr:bash: line 1: ntpq: command not found 2024-04-24T23:34:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:34:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-04-24T23:34:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:^- hv01.front.sepia.ceph.com 2 6 377 21 +996us[ +996us] +/- 572ms 2024-04-24T23:34:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 +117us[ +109us] +/- 74ms 2024-04-24T23:34:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -155us[ -164us] +/- 82ms 2024-04-24T23:34:36.589 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:34:36.591 INFO:teuthology.orchestra.run.smithi040.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T23:34:36.591 INFO:teuthology.orchestra.run.smithi040.stdout:=============================================================================== 2024-04-24T23:34:36.591 INFO:teuthology.orchestra.run.smithi040.stdout:^- hv01.front.sepia.ceph.com 2 6 377 25 +1006us[+1006us] +/- 572ms 2024-04-24T23:34:36.591 INFO:teuthology.orchestra.run.smithi040.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 +125us[ +122us] +/- 74ms 2024-04-24T23:34:36.591 INFO:teuthology.orchestra.run.smithi040.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 -147us[ -147us] +/- 82ms 2024-04-24T23:34:36.591 INFO:teuthology.orchestra.run.smithi040.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T23:34:36.592 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T23:34:36.610 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T23:34:36.611 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T23:34:36.621 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:34:36.713 DEBUG:teuthology.orchestra.run.smithi040:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T23:34:36.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:34:36.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:34:36.815 DEBUG:teuthology.orchestra.run.smithi040:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:34:36.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:34:36.843 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T23:34:36.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T23:34:36.869 INFO:teuthology.task.internal:Duration was 2252.820139 seconds 2024-04-24T23:34:36.869 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T23:34:36.880 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T23:34:36.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:34:36.883 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T23:34:36.918 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:34:36.921 INFO:teuthology.orchestra.run.smithi040.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:34:37.292 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T23:34:37.292 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-04-24T23:34:37.292 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:34:37.353 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2024-04-24T23:34:37.353 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T23:34:37.375 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T23:34:37.376 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:34:37.396 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T23:34:37.441 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T23:34:37.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:34:37.462 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T23:34:38.110 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T23:34:38.122 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T23:34:38.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:34:38.147 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T23:34:38.173 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T23:34:38.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:34:38.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:34:38.210 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-04-24T23:34:38.237 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2024-04-24T23:34:38.262 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:34:38.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:34:38.301 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T23:34:38.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:34:38.320 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T23:34:38.332 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T23:34:38.333 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179/remote/smithi031 2024-04-24T23:34:38.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:34:38.390 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179/remote/smithi040 2024-04-24T23:34:38.390 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T23:34:38.427 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T23:34:38.428 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:34:38.430 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T23:34:38.476 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T23:34:38.489 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T23:34:38.489 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T23:34:38.500 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T23:34:38.500 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:34:38.502 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T23:34:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 24 23:34 /home/ubuntu/cephtest 2024-04-24T23:34:38.515 INFO:teuthology.orchestra.run.smithi031.stdout: 550976 4 d--------- 2 ubuntu ubuntu 4096 Apr 24 23:11 /home/ubuntu/cephtest/mnt.0 2024-04-24T23:34:38.516 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-24T23:34:38.516 INFO:teuthology.orchestra.run.smithi031.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-24T23:34:38.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:34:38.519 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi031 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-24T23:34:38.519 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T23:34:38.531 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T23:34:38.544 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-24T23:34:38.545 INFO:teuthology.nuke:Checking targets against current locks 2024-04-24T23:34:38.572 DEBUG:teuthology.nuke:shortname: smithi031 2024-04-24T23:34:38.572 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:34:38.596 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:51:43.944883', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKEPt1CQtIMn+3Jrv0a3xOXikA/LbAg690wZGs99omoM+WdlNGKaFYGVS9y5IREivEiMAROXd7G2myZhSpDv16w='} 2024-04-24T23:34:38.599 DEBUG:teuthology.nuke:shortname: smithi040 2024-04-24T23:34:38.599 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:34:38.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672179', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 22:51:43.946852', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIMoj4yS0kEiJ2yYtBo/bpsisRHZUUsb3JQ9sjiZJvUWByBGJkDucqTyWlriyG+XUFJhAd2lx1OF/XYTKvf3UIw='} 2024-04-24T23:34:38.654 INFO:teuthology.orchestra.console.smithi031:Power off 2024-04-24T23:34:38.654 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T23:34:38.677 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2024-04-24T23:34:38.678 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:34:38.693 INFO:teuthology.orchestra.console.smithi040:Power off 2024-04-24T23:34:38.694 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-24T23:34:38.704 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-24T23:34:38.719 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2024-04-24T23:34:38.719 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:34:38.747 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-04-24T23:34:42.705 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:34:42.748 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:34:42.815 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-24T23:34:42.857 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2024-04-24T23:34:46.821 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:34:46.858 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:34:46.932 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2024-04-24T23:34:46.967 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2024-04-24T23:34:47.068 INFO:teuthology.orchestra.console.smithi040:Power off completed 2024-04-24T23:34:50.936 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-24T23:34:51.047 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2024-04-24T23:34:51.148 INFO:teuthology.orchestra.console.smithi031:Power off completed 2024-04-24T23:34:51.253 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2252.820139169693 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-24T23:34:51.253 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:34:51.361 INFO:teuthology.run:FAIL